Skip to content
RepoQuarry

Scrapling D4Vinci/Scrapling

Python scraping and crawling framework with adaptive selectors, stealth browser fetchers, Scrapy-like spiders and Markdown output for AI.

Scrapling at a glance

Checked 16 Oct 2026

Scrapling is a BSD-3-Clause Python library (3.10+) with HTTP and browser fetchers, Scrapy-like spiders with pause/resume and optional robots.txt obeying, selectors that relocate elements after site changes, an MCP server and site-to-Markdown spiders for RAG. A main selling point is stealth fetching that the README says bypasses anti-bot systems such as Cloudflare Turnstile. Caution: users must respect site terms and robots.txt; the README itself says the library is for educational and research purposes.

Cite this pageRepoQuarry. “Scrapling.” Checked 16 Oct 2026. https://repoquarry.dev/projects/scrapling

How much of Firecrawl Cloud does it replace?

2 of 4 jobs. A library you code against, not a hosted API; no built-in LLM extraction or web search.

Partial: Crawl a site to Markdown Partial replacement
Partial: Scrape pages (JS-heavy and bot-protected) Partial replacement
No: Structured data extraction Not covered
No: Web search with page content Not covered

For structured data extraction, web search with page content, see other Firecrawl Cloud alternatives. You may also need to build an interface, an API server and hosting yourself.

Facts, with sources

Every answer links to where we found it. We read the documentation; we did not install it.

Report a correction
QuestionAnswerSourceChecked
Runs on my machine? Confirmed: Yes, Python library on your machine; Docker image also published Scrapling is an adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl. github.com · Scrapling#readme ↗ 1d ago
Code licence Confirmed: BSD-3-Clause github.com · LICENSE ↗ 1d ago
Model weights licence Confirmed: No separate model weights github.com · Scrapling#readme ↗ 1d ago
Commercial use Confirmed: Yes, BSD-3-Clause; README disclaimer frames it as for educational and research purposes This library is provided for educational and research purposes only. github.com · Scrapling#disclaimer ↗ 1d ago
Paid services needed? Confirmed: None required; proxies optional github.com · Scrapling#readme ↗ 1d ago
Hardware Unknown: Not documented; browser fetchers run Chromium github.com · Scrapling#readme ↗ 1d ago
Operating systems Confirmed: Spider documentation explicitly covers Linux/macOS (uvloop) and Windows (winloop) for the optional accelerated event loop; individual browser-fetcher dependencies still need installation scrapling.readthedocs.io · advanced.html#using-uvloop ↗ today
Setup effort Confirmed: Easy, pip install scrapling; browser fetchers need the fetchers extra and scrapling install Scrapling requires Python 3.10 or higher: github.com · Scrapling#installation ↗ 1d ago

Get the weekly top 5 and what’s new, every Monday

We’ll email you once a week. Unsubscribe any time. We never share your email.

Cookie preferences

Choose what we may store in your browser. You can change this any time from Cookie settings in the footer.

Essential

Security, and remembering this choice. rq-consent · 12 months

Always on