MCP Server Setup

Configure Scrapling’s MCP server for AI assistants

This guide walks you through setting up Scrapling’s MCP server for use with AI assistants like Claude Desktop, Cline, and other MCP-compatible clients.

Prerequisites

Before setting up the MCP server, ensure you have:

• Python 3.8 or higher installed
• Scrapling installed with extras: pip install scrapling[all]
• (Optional) Playwright browsers: scrapling install

Installation

Method 1: Using uvx (Recommended)

The fastest way to run the MCP server:

uvx scrapling mcp

This automatically installs and runs the server without manual installation.

Method 2: Using pip

Install Scrapling globally or in a virtual environment:

# Install with all features
pip install scrapling[all]

# Install dependencies (Playwright browsers)
scrapling install

# Run the server
scrapling mcp

Method 3: Using Docker

Run the MCP server in a container:

# Pull the image
docker pull ghcr.io/d4vinci/scrapling

# Run the server
docker run -i ghcr.io/d4vinci/scrapling mcp

Configuration for AI Clients

Claude Desktop

Add to your Claude Desktop configuration file:

MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json

Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "scrapling": {
      "command": "uvx",
      "args": ["scrapling", "mcp"]
    }
  }
}

Or using pip installation:

{
  "mcpServers": {
    "scrapling": {
      "command": "scrapling",
      "args": ["mcp"]
    }
  }
}

Cline (VS Code Extension)

Add to Cline’s MCP settings:

1. Open VS Code settings
2. Search for “Cline MCP”
3. Add server configuration:

{
  "scrapling": {
    "command": "uvx",
    "args": ["scrapling", "mcp"]
  }
}

Custom MCP Clients

For custom implementations, connect to the server using:

Stdio transport:

scrapling mcp

HTTP transport:

scrapling mcp --http --host 0.0.0.0 --port 8000

Server Options

The MCP server supports several command-line options:

--http · boolean · default: "false"

Use streamable-http transport instead of stdio

--host · string · default: "0.0.0.0"

Host address for HTTP transport

--port · integer · default: "8000"

Port number for HTTP transport

Transport Modes

Stdio Mode (Default)

Best for local AI assistants:

scrapling mcp

Characteristics:

• Standard input/output communication
• No network configuration needed
• Ideal for Claude Desktop, Cline
• Lower latency for local operations

HTTP Mode

Best for network-accessible services:

scrapling mcp --http --host 127.0.0.1 --port 8080

Characteristics:

• Streamable HTTP transport
• Network accessible
• Can be used by remote clients
• Requires port configuration

Verification

After configuration, verify the setup:

Test with uvx

uvx scrapling mcp

You should see the server start without errors.

Test HTTP Transport

scrapling mcp --http --port 8000

The server should start and listen on the specified port.

Check in AI Client

1. Restart your AI client (Claude Desktop, Cline, etc.)
2. Look for “Scrapling” in available tools/servers
3. Try a simple request: “Fetch the content from example.com”

Troubleshooting

Server not found in AI client

• Ensure the configuration file path is correct
• Verify the command is accessible in your PATH
• Restart the AI client after configuration changes
• Check for syntax errors in the JSON configuration

Playwright browsers missing

Run the install command:

scrapling install

This installs Chromium and required dependencies.

Permission errors

• Ensure Python has proper permissions
• On Unix systems, check file ownership
• Try running with appropriate user privileges

Port already in use (HTTP mode)

• Choose a different port: --port 8001
• Check for other services using the port
• Use lsof -i :8000 (Unix) or netstat -ano | findstr :8000 (Windows)

Environment Variables

Optional environment variables for advanced configuration:

# Set custom user agent
export SCRAPLING_USER_AGENT="Custom Agent/1.0"

# Set default timeout (milliseconds)
export SCRAPLING_TIMEOUT=60000

# Enable debug logging
export SCRAPLING_DEBUG=1

Docker Configuration

For Docker deployments:

Basic Docker Run

docker run -i ghcr.io/d4vinci/scrapling mcp

HTTP Transport with Port Mapping

docker run -p 8000:8000 ghcr.io/d4vinci/scrapling mcp --http --port 8000

Docker Compose

version: '3.8'
services:
  scrapling-mcp:
    image: ghcr.io/d4vinci/scrapling
    command: mcp --http --host 0.0.0.0 --port 8000
    ports:
      - "8000:8000"
    stdin_open: true
    tty: true

Next Steps

MCP Server Overview

Learn about MCP server features

Capabilities

Explore available tools and operations