Skip to content
Skills Library
Data & analytics · Automation

scrapling

/scrapling

CLI-first web scraping & content extraction with optional MCP server. Use when you have target URLs and need clean, selector-based outputs (html/md/txt).

Repository
Vibe-Skills ↗
Licence
Apache-2.0
Last repository change
25.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
2
Scripts
1
HTML comments
0
URLs in SKILL.md
4
Deprecation

No signals detected. Repository changed on 25.10.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What to check before installing
  • 1 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: scripts/scrapling.ps1

What it does and when to use it

CLI-first web scraping & content extraction with optional MCP server. Use when you have target URLs and need clean, selector-based outputs (html/md/txt).

Original author description

CLI-first web scraping & content extraction with optional MCP server. Use when you have target URLs and need clean, selector-based outputs (html/md/txt).

How it works

  1. When to use
  2. Boundaries (vs Playwright / Search)
  3. Prerequisite check (required)
  4. Installation (recommended)
  5. Wrapper script (Windows convenience)
  6. Common CLI patterns
  7. MCP server relationship (optional)
  8. Safety & ops notes
Original SKILL.md headings · 140 lines, 597 words
SKILL.md preview
---
name: "scrapling"
description: "CLI-first web scraping & content extraction with optional MCP server. Use when you have target URLs and need clean, selector-based outputs (html/md/txt)."
---

# Scrapling Skill (VCO)

Scrapling is a Python-based web scraping / extraction toolkit that exposes:
- a **CLI** (`scrapling ...`) for fetching + extracting content into files
- an **optional MCP server** (`scrapling mcp`) so an agent can call structured scraping tools

This skill is **CLI-first**. Prefer it when you already have URLs and need reliable, repeatable extraction (CSS selector → file).

## When to use

Use `scrapling` when you need:

- Extract **specific parts** of a web page (CSS selector / XPath) into `.txt` / `.md` / `.html`
- Run **repeatable scraping jobs** (batch URLs with a small wrapper script)
- Reduce token usage by extracting only the relevant DOM region before passing to the LLM
- Provide a local MCP endpoint for scraping tools (agent → MCP → scrapling)

## Boundaries (vs Playwright / Search)

### vs `playwright`
- `scrapling`: best for “get URL → extract selector → write file” workflows; simpler, faster iteration
- `playwright`: best for interactive UI flows (login, multi-step navigation, downloads, complex JS actions, stateful sessions)

If you must *navigate* or *click through* a UI, use `playwright`.
If you can directly fetch the target page and just need extraction, use `scrapling`.

### vs search tools
- Search tools are for discovering sources/URLs (query → result list → choose URLs).
- `scrapling` is for acquisition + extraction once you already know the URL(s).

A common pipeline:
1) Search → find candidate URLs
2) Scrapling → extract focused content from chosen URLs
3) LLM → summarize / transform / analyze extracted outputs

## Prerequisite check (required)

1) Python version (Scrapling requires Python >= 3.10):
```powershell
python --version
```

2) Scrapling CLI availability:
```powershell
scrapling --help
```

## Installation (recommended)

Scrapling’s CLI and MCP features are enabled via extras.

Recommended (CLI + MCP + fetchers):
```powershell
python -m pip install "scrapling[ai]"
```

If you only want CLI fetch/extract without MCP:
```powershell
python -m pip install "scrapling[fetchers]"
```

If you use browser-based fetchers, you may need browser binaries:
```powershell
# Option A: via Scrapling helper (after install)
scrapling install

# Option B: directly via Playwright
python -m playwright install
```

## Wrapper script (Windows convenience)

This skill ships a thin PowerShell wrapper:
- `C:/Users/羽裳/.codex/skills/scrapling/scripts/scrapling.ps1`

It checks whether `scrapling` exists and prints install hints if missing.

## Common CLI patterns

### 1) Extract full page body (to Markdown)
```powershell
scrapling extract get "https://example.com" out.md
```

### 2) Extract a specific element (CSS selector) to text
```powershell
scrapling extract get "https://example.com" out.txt --css-selector "main article"
```

### 3) Extract HTML for downstream parsing
```powershell
scrapling extract get "https://example.com" out.html --css-selector "#content"
```

### 4) Use browser-backed fetcher mode (when simple GET is blocked / dynamic)
```powershell
scrapling extract fetch "https://example.com" out.md --css-selector "main"
```

Tip: keep outputs in files and only feed the smallest relevant snippet to the LLM.

## MCP server relationship (optional)

Scrapling can run as an MCP server. This is useful when:
- the agent needs tool-style scraping calls
- you want scraping results to be structured and deterministic

Start MCP server (stdio transport by default):
```powershell
scrapling mcp
```

Optional: run MCP server with HTTP transport:
```powershell
scrapling mcp --http --host 127.0.0.1 --port 8765
```

### Example MCP server config snippet

```json
{
  "servers": {
    "scrapling": {
      "mode": "stdio",
      "command": "scrapling",
      "args": ["mcp"],
      "required": false,
      "note": "Requires: python -m pip install \"scrapling[ai]\""
    }
  }
}
```

## Safety & ops notes

- Prefer selector-based extraction to minimize data volume.
- Treat scraping as an external dependency: handle timeouts, retries, and failures explicitly.
- For aggressive bot protection, consider switching fetchers or using `playwright`.
Files in this package (2)
  • SKILL.md 4 KB
  • scripts/scrapling.ps1 1 KB

Versions

RepositoryScoreLicence
This page · foryourhealth111-pixel/Vibe-Skills100Apache-2.0
akillness/jeo-skills100MIT
TerminalSkills/skills100Apache-2.0
cedriccmh/claude-code-skill-scrapling95MIT

Identical content appears in 1 other GitHub folders. We show one copy with the highest score.

Database snapshot: 13.11.2026 · package version 9699317cfba9 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100scrapling-skill
    Install, troubleshoot, and use Scrapling CLI to extract HTML, Markdown, or text from webpages. Use this skill whenever the user mentions Scrapling, `uv tool install scrapling`, `scrapling extract`, WeChat/mp.weixin articles, browser-backed page fetching, or needs help deciding…
  • 89scrapling-or-death
    Use for web fetch. Scrapling first, never curl or browser.
  • 100ddgs
    Use when an agent needs to search the web, find images/news/videos/books, or extract content from a URL. Covers the ddgs Python library, CLI, and MCP server integration.
  • 100scrape
    Scrape web pages via Scrapling — stealth fetching, anti-bot bypass, CSS selectors, no API key. Use when the user says 'scrape', 'pull data from this URL', 'extract from this site'. Not for meaning-based search of the user's own vault; use `enable-semantic-search`.
  • 100agentscraper
    Fetch a public web page live and get its main content as clean Markdown, plain text or HTML, through the AgentScraper API. Use when you have a specific URL and need what the page says now, for example docs, articles, product or listing pages, including pages that need a real browser to render.
  • 100web-scraping
    Scrape websites, extract structured data, and automate browsers. Use when asked to scrape, extract, crawl, parse, or pull data from web pages or any URL.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.