Installation

Requirements

Scrapling requires Python 3.10 or higher.

Basic Installation

Install the core Scrapling library (parser engine only):

pip install scrapling

This installation includes only the parser engine and its dependencies, without any fetchers or command-line features.

Optional Dependencies

Scrapling offers several optional dependency groups depending on your needs:

Fetchers

Install fetcher dependencies for HTTP requests, browser automation, and stealth capabilities:

Install fetcher dependencies

pip install "scrapling[fetchers]"

This installs dependencies for:

Fetcher & FetcherSession (HTTP requests)
DynamicFetcher & DynamicSession (Playwright browser automation)
StealthyFetcher & StealthySession (anti-bot bypass)

Install browsers

After installing fetcher dependencies, download browsers and system dependencies:

scrapling install           # Normal install
scrapling install --force   # Force reinstall

from scrapling.cli import install

install([], standalone_mode=False)          # Normal install
install(["--force"], standalone_mode=False) # Force reinstall

This downloads:

Chromium and Chrome browsers
System dependencies for browser automation
Fingerprint manipulation dependencies

You must run scrapling install after installing scrapling[fetchers] to use browser-based fetchers (DynamicFetcher, StealthyFetcher).

AI Features

Install the MCP server for AI-assisted web scraping:

pip install "scrapling[ai]"

This enables:

Built-in MCP server integration
AI-powered content extraction
Integration with Claude, Cursor, and other AI tools

The MCP server features powerful capabilities that leverage Scrapling to extract targeted content before passing it to AI, reducing token usage and costs. See the MCP Server guide for details.

Shell Features

Install interactive shell and CLI extraction tools:

pip install "scrapling[shell]"

This enables:

Interactive Web Scraping shell with IPython
scrapling shell command
scrapling extract command for extracting content without code

All Features

Install everything at once:

pip install "scrapling[all]"

Remember to run scrapling install after installing [all] to set up browser dependencies if you plan to use fetchers.

Docker Installation

Scrapling provides pre-built Docker images with all extras and browsers included:

docker pull pyd4vinci/scrapling

docker pull ghcr.io/d4vinci/scrapling:latest

The Docker image includes:

All Scrapling features ([all])
All browsers pre-installed
System dependencies configured
Ready to use out of the box

Docker images are automatically built and pushed using GitHub Actions from the repository’s main branch.

Using the Docker Image

# Run interactive Python shell
docker run -it pyd4vinci/scrapling python

# Run a Python script
docker run -v $(pwd):/app pyd4vinci/scrapling python /app/your_script.py

# Use the interactive shell
docker run -it pyd4vinci/scrapling scrapling shell

Verify Installation

Confirm Scrapling is installed correctly:

Test core parser

from scrapling.parser import Selector

page = Selector("<html><body><h1>Hello Scrapling!</h1></body></html>")
print(page.css('h1::text').get())
# Output: Hello Scrapling!

Test HTTP fetcher (if installed)

from scrapling.fetchers import Fetcher

page = Fetcher.get('https://httpbin.org/html')
print(page.status)
# Output: 200

Test browser installation (if installed)

scrapling --version

Or test in Python:

from scrapling.fetchers import DynamicFetcher

page = DynamicFetcher.fetch('https://httpbin.org/html', headless=True)
print(page.status)
# Output: 200

Installation Summary

Here’s a quick reference for installation commands:

Use Case	Command
Parser only	`pip install scrapling`
HTTP + Browser fetchers	`pip install "scrapling[fetchers]"` + `scrapling install`
AI integration	`pip install "scrapling[ai]"`
Interactive shell	`pip install "scrapling[shell]"`
Everything	`pip install "scrapling[all]"` + `scrapling install`
Docker	`docker pull pyd4vinci/scrapling`

Troubleshooting

Browser installation fails

If scrapling install fails, try:

Force reinstall: scrapling install --force
Check system dependencies for Playwright
Ensure you have enough disk space (browsers are ~300MB each)
Check Python version: python --version (must be 3.10+)

Import errors after installation

If you get import errors:

Verify installation: pip show scrapling
Check you installed the right extras: pip install "scrapling[fetchers]"
For browser features, ensure you ran: scrapling install
Try in a fresh Python session or restart your IDE

Permission errors on Linux/Mac

If you encounter permission errors:

Use a virtual environment (recommended)
Or install with --user flag: pip install --user scrapling
For system-wide installation, use sudo (not recommended)

Next Steps

Quick Start

Start scraping in 5 minutes

Choose a Fetcher

Learn which fetcher fits your needs

Selection Methods

Master CSS, XPath, and more

Build Spiders

Scale to full crawling

Getting Started

Core Concepts

Fetching

Parsing & Selection

Spiders

CLI & Tools

AI Integration

Guides

Tutorials

Requirements

Basic Installation

Optional Dependencies

Fetchers

AI Features

Shell Features

All Features

Docker Installation

Using the Docker Image

Verify Installation

Installation Summary

Troubleshooting

Next Steps

Quick Start

Choose a Fetcher

Selection Methods

Build Spiders

​Requirements

​Basic Installation

​Optional Dependencies

​Fetchers

​AI Features

​Shell Features

​All Features

​Docker Installation

​Using the Docker Image

​Verify Installation

​Installation Summary

​Troubleshooting

​Next Steps

Quick Start

Choose a Fetcher

Selection Methods

Build Spiders

Requirements

Basic Installation

Optional Dependencies

Fetchers

AI Features

Shell Features

All Features

Docker Installation

Using the Docker Image

Verify Installation

Installation Summary

Troubleshooting

Next Steps