MCP Server

Model Context Protocol server for AI-powered web scraping

Scrapling provides a Model Context Protocol (MCP) server that enables AI assistants to perform web scraping operations. The server exposes Scrapling’s fetching capabilities as MCP tools.

Overview

The Scrapling MCP server allows AI models to:

• Fetch web pages with stealth HTTP requests
• Extract content using CSS selectors
• Handle JavaScript-heavy sites with browser automation
• Bypass Cloudflare and other protections
• Process single or multiple URLs concurrently
• Return structured data in Markdown, HTML, or text format

Quick Start

Start the Server

scrapling mcp

This starts the server in stdio transport mode (default).

HTTP Transport

For network access, use streamable-http transport:

scrapling mcp --http --host 0.0.0.0 --port 8000

Server Configuration

The MCP server configuration is defined in server.json:

{
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
  "name": "io.github.D4Vinci/Scrapling",
  "title": "Scrapling MCP Server",
  "description": "Web scraping with stealth HTTP, real browsers, and Cloudflare bypass. CSS selectors supported.",
  "version": "0.4.1"
}

Available Tools

The MCP server exposes six main tools:

get

Make HTTP GET requests with stealth features

bulk_get

Fetch multiple URLs concurrently with GET requests

fetch

Use Playwright for JavaScript-heavy sites

bulk_fetch

Fetch multiple URLs with browser automation

stealthy_fetch

Advanced stealth with Cloudflare bypass

bulk_stealthy_fetch

Stealthy fetching for multiple URLs

Transport Modes

Stdio Transport (Default)

Used for local AI assistants:

scrapling mcp

• Standard input/output communication
• Best for Claude Desktop, Cline, and other local clients
• No network configuration needed

HTTP Transport

Used for network-accessible services:

scrapling mcp --http --host 0.0.0.0 --port 8000

• Streamable HTTP transport
• Accessible over network
• Configurable host and port

Response Format

All tools return a structured ResponseModel:

{
  "status": 200,
  "content": ["Extracted content..."],
  "url": "https://example.com"
}

status · integer

HTTP status code from the website

content · array[string]

Extracted content as Markdown, HTML, or text

url · string

The URL that was requested

Installation Methods

PyPI (Recommended)

uvx scrapling mcp

Or with pip:

pip install scrapling
scrapling mcp

Docker (OCI)

docker pull ghcr.io/d4vinci/scrapling
docker run ghcr.io/d4vinci/scrapling mcp

Use Cases

AI-powered web research

Enable AI assistants to gather information from websites, extract articles, and process web content.

Automated data collection

Let AI models collect data from multiple sources concurrently with bulk operations.

Content analysis

Extract and analyze web content using CSS selectors for targeted data extraction.

Protected site access

Use stealthy fetching to access Cloudflare-protected sites through AI workflows.

Next Steps

Setup Guide

Configure MCP server for your AI client

Capabilities

Explore available MCP tools and features