Przejdź do treści
Biblioteka skilli
AI i agenci · Konwersja i formaty

markdown-fetch

/markdown-fetch

Pobiera treść stron internetowych jako czysty Markdown po podaniu adresu URL, zużywając o 80 proc. mniej tokenów niż surowy HTML, w trzech metodach konwersji (automatycznej, AI i renderowania przeglądarką).

Repozytorium
claude-skills ↗
Licencja
MIT
Ostatnia zmiana repozytorium
26.09.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
3
Skrypty
1
Komentarze HTML
0
Adresy URL w SKILL.md
28
Przestarzałość

Brak sygnałów. Repozytorium zmienione 26.09.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co sprawdzić przed instalacją
  • 1 skrypt w paczce. Przejrzyj je przed instalacją, bo Claude może je uruchomić.

Skrypty: scripts/fetch.sh

Co robi i kiedy

Pobiera treść stron internetowych jako czysty Markdown po podaniu adresu URL, zużywając o 80 proc. mniej tokenów niż surowy HTML, w trzech metodach konwersji (automatycznej, AI i renderowania przeglądarką).

Oryginalny opis (EN)

Fetch and extract web content as clean Markdown when provided with URLs. Use this skill whenever a user provides a URL (http/https link) that needs to be read, analyzed, summarized, or extracted. Converts web pages to Markdown with 80% fewer tokens than raw HTML. Handles all content types including JS-heavy sites, documentation, articles, and blog posts. Supports three conversion methods (auto, AI, browser rendering). Always use this instead of web_fetch when working with URLs - it's more efficient and provides cleaner output.

Jak działa

  1. Why Use This
  2. Triggering
  3. Quick Start
  4. Typical Usage Patterns
  5. Conversion Methods
  6. Options
  7. Output
  8. When to Use
  9. Implementation Notes
  10. Common Pitfalls
Sekcje z nagłówków SKILL.md, w oryginale · 306 linii, 1 380 słów
Podgląd SKILL.md
---
name: markdown-fetch
description: Fetch and extract web content as clean Markdown when provided with URLs. Use this skill whenever a user provides a URL (http/https link) that needs to be read, analyzed, summarized, or extracted. Converts web pages to Markdown with 80% fewer tokens than raw HTML. Handles all content types including JS-heavy sites, documentation, articles, and blog posts. Supports three conversion methods (auto, AI, browser rendering). Always use this instead of web_fetch when working with URLs - it's more efficient and provides cleaner output.
---

# Markdown Fetch

Efficiently fetch web content as clean Markdown using the markdown.new service.

## Why Use This

- **80% fewer tokens** than raw HTML
- **5x more content** fits in context window  
- **No external dependencies** or parsing libraries needed
- **Three-tier conversion** (Markdown-first, AI fallback, browser rendering)

## Triggering

This skill should trigger automatically when:
- User provides a URL (e.g., "Read https://example.com")
- User asks to extract/fetch/analyze web content
- User requests summarization of a webpage
- User needs to process article/blog/documentation URLs

## Quick Start

```bash
# Fetch any URL
scripts/fetch.sh "https://example.com"

# Use browser rendering for JS-heavy sites
scripts/fetch.sh "https://example.com" --method browser

# Retain images in output
scripts/fetch.sh "https://example.com" --retain-images
```

## Typical Usage Patterns

When a user says:
- "Read this article: https://..." → Use this skill to fetch the content
- "Summarize https://..." → Fetch with this skill first, then summarize
- "What does this page say: https://..." → Fetch the content
- "Extract the text from https://..." → Use this skill

## Conversion Methods

**auto** (default) - Try Markdown-first, fall back to AI or browser as needed  
**ai** - Use Cloudflare Workers AI for conversion  
**browser** - Full browser rendering for JS-heavy content

## Options

`--method <auto|ai|browser>` - Conversion method  
`--retain-images` - Keep image references in output  
`--output <file>` - Save to file instead of stdout

## Output

Returns clean Markdown with metadata:

```markdown
---
title: Page Title
url: https://example.com
method: auto
duration_ms: 725
fetched_at: 2026-03-07T12:00:00Z
---

# Content here...
```

## When to Use

- Extracting articles, documentation, or blog posts
- Building RAG pipelines with web content
- Summarizing web pages
- Fetching content for analysis
- Converting sites to Markdown format

## Implementation Notes

The service handles:
- Content negotiation (Accept: text/markdown)
- Cloudflare Workers AI conversion
- Browser rendering for dynamic content
- Automatic fallback between methods

## Common Pitfalls

### HTML Parsing Failures

**Symptom:** Empty output, truncated content, or "Error: No content in response"

**Root Causes:**
- CSS selectors broken by page layout changes
- Dynamic content not rendered by default (use `--method browser`)
- Page structure uses unusual HTML patterns (iframes, shadow DOM, Web Components)

**Debugging & Workarounds:**
```bash
# 1. Check if the page loads at all
curl -s "https://example.com" | head -c 500

# 2. Try browser rendering (handles JS-heavy sites)
scripts/fetch.sh "https://example.com" --method browser

# 3. Check what the service is getting
scripts/fetch.sh "https://example.com" 2>&1 | head -20

# 4. If still empty, the page may use iframes or require auth
```

**Best Practice:** Always use `--method browser` for single-page apps, social platforms, or content-heavy dashboards.

---

### Rate Limiting & IP Blocks

**Symptom:** HTTP 429 (Too Many Requests), 403 (Forbidden), or connection timeouts

**Root Causes:**
- Multiple rapid requests to same domain
- Service IP address blacklisted by the target site
- Bot detection triggering (User-Agent headers, request patterns)
- Cloudflare/WAF blocks (common on high-traffic sites)

**Debugging & Workarounds:**
```bash
# 1. Check HTTP status code
curl -s -w "\n%{http_code}\n" -o /dev/null "https://example.com"

# 2. Test with curl directly (bypasses service)
curl -H "User-Agent: Mozilla/5.0" "https://example.com" | head -c 500

# 3. Add delays between requests (if fetching multiple URLs)
for url in url1 url2 url3; do
  scripts/fetch.sh "$url"
  sleep 5  # Wait 5 seconds between requests
done

# 4. If blocked, try from different IP or use residential proxy
# For markdown.new service: no built-in proxy rotation; consider mirror services
```

**Best Practice:** Respect site robots.txt and use sensible request delays when processing multiple pages. Some sites require retry-after headers.

---

### Encoding Issues

**Symptom:** Garbled text, mojibake (weird characters), missing non-ASCII content (emojis, accents, CJK)

**Root Causes:**
- Page declares wrong charset in headers vs actual content
- Service not respecting Content-Type charset declaration
- UTF-8 vs Latin-1 mismatch
- Emoji or special symbols stripped during conversion

**Debugging & Workarounds:**
```bash
# 1. Check the page's declared charset
curl -sI "https://example.com" | grep -i "charset"

# 2. Save raw HTML and inspect encoding
curl -s "https://example.com" > raw.html
file raw.html
head -c 200 raw.html | od -c  # Show raw bytes

# 3. Try the markdown.new service and check output
scripts/fetch.sh "https://example.com" | file -  # Check output encoding

# 4. If output is garbled, convert explicitly
scripts/fetch.sh "https://example.com" | iconv -f UTF-8 -t UTF-8 > fixed.md
```

**Best Practice:** Always verify output is valid UTF-8. If encountering garbled content, the page likely has a charset mismatch — notify the site owner or use `--method browser` (more robust).

---

### JavaScript-Rendered Content

**Symptom:** Page returns but content is missing, loads as empty, or shows loading spinners instead of data

**Root Causes:**
- Page uses client-side JavaScript to render content (React, Vue, Angular, etc.)
- Default `auto` method only fetches HTML skeleton without executing JS
- Content loaded after page render (lazy loading, infinite scroll)
- JavaScript requires authentication or API keys

**Debugging & Workarounds:**
```bash
# 1. Fetch with browser rendering (executes JS)
scripts/fetch.sh "https://example.com" --method browser

# 2. Check what the auto method returns
scripts/fetch.sh "https://example.com" --method auto

# 3. Inspect page source to confirm JS-rendering
curl -s "https://example.com" | grep -i "react\|vue\|angular\|<div id=\"app\""

# 4. If content still missing, page may require interaction
# (e.g., clicking buttons, scrolling) — no workaround in current service
```

**Best Practice:** Use `--method browser` by default for modern web apps, news sites, and any site built in the last 5 years. The `auto` method is faster but misses JS-rendered content.

---

### Authentication & Paywall Content

**Symptom:** Returns login page instead of content, shows "Subscribe to read more", or HTTP 401/403

**Root Causes:**
- Page requires login/authentication
- Content behind paywall (subscription, membership)
- IP-based access restrictions (geo-blocking)
- Session-based authentication (cookies) not provided

**Debugging & Workarounds:**
```bash
# 1. Check if curl can access without auth
curl -s "https://example.com/article" | head -c 300

# 2. Identify authentication method
curl -sI "https://example.com" | grep -i "auth\|set-cookie\|www-authenticate"

# 3. Try public/preview version if available
# For paywalled sites, look for preview URLs or RSS feeds
curl -s "https://example.com/rss" | head -c 500

# 4. Check for paywall indicators
curl -s "https://example.com/article" | grep -i "paywall\|subscribe\|login required"

# 5. If member-only, request your credentials in the conversation
# (Don't embed in scripts — use prompt)
```

**Best Practice:** This skill cannot bypass authentication or paywalls by design. For protected content:
- Use public preview links if available
- Ask the user for authenticated access
- Try RSS feeds (often unpaywalled summaries)
- Check archive services (archive.org, 12ft.io for paywalled content) separately

---

### Service-Side Failures

**Symptom:** HTTP error codes (5xx), timeout, or malformed JSON responses

**Root Causes:**
- markdown.new service is down or overloaded
- Request body malformed (invalid URL, missing required fields)
- Service response is corrupted or incomplete
- Cloudflare Workers timeout (pages >10MB or slow servers)

**Debugging & Workarounds:**
```bash
# 1. Check service health
curl -s "https://markdown.new/health" 2>&1 | head

# 2. Verify request format
jq -n --arg url "https://example.com" --arg method "auto" \
  '{url: $url, method: $method, retain_images: false}' | jq .

# 3. Retry with exponential backoff
for attempt in {1..3}; do
  result=$(scripts/fetch.sh "https://example.com" 2>&1)
  if [[ $? -eq 0 ]]; then
    echo "$result"
    break
  fi
  sleep $((2 ** attempt))  # 2s, 4s, 8s
done

# 4. If service is down, no workaround available
# — try again later or use alternative (curl, browser, etc.)
```

**Best Practice:** The service is stateless and generally reliable. If failures persist, fall back to `curl` directly or ask the user for an alternative source.

---

## Troubleshooting Decision Tree

```
No content returned?
  ├─ Check if site requires JavaScript
  │   └─ YES → Use --method browser
  │   └─ NO → Continue
  ├─ Check if site requires authentication
  │   └─ YES → Use authenticated request (see Paywall section)
  │   └─ NO → Continue
  ├─ Check for encoding issues
  │   └─ YES → Garbled text → likely site charset mismatch
  │   └─ NO → Continue
  └─ Service may be down → Wait and retry

Getting rate-limited (HTTP 429)?
  ├─ Add delays between requests (5-10 seconds)
  └─ If persistent, try different time or request fewer pages

Truncated or broken output?
  └─ Try --method browser (more robust but slower)

Can't access paywalled content?
  └─ No workaround — try public preview, RSS, or archive services
```
Pliki w paczce (3)
  • DOCS.md 3 KB
  • SKILL.md 10 KB
  • scripts/fetch.sh 3 KB

Wersje

W bazie nie ma innych skilli o nazwie markdown-fetch.

Migawka bazy z 13.11.2026 · wersja paczki 822d95f2d627 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100web-fetch
    Pobiera treści z sieci jako czysty markdown, preferując odpowiedzi natywnie w markdown, a przy ich braku wyciąga elementy z HTML selektorami.
  • 95firecrawl-scrape
    Pobiera czysty markdown z dowolnego adresu URL, także ze stron renderowanych w JavaScript, równolegle dla wielu adresów.
  • 959router-web-fetch
    Pobiera treść strony WWW jako markdown, tekst lub HTML przez 9Router, na przykład z artykułu pod wskazanym adresem.
  • 100mcp-server-fetch
    Instaluje serwer MCP Fetch, który pobiera treść stron i zamienia ją na Markdown, oraz obsługuje jego narzędzia.
  • 100ai-fetch-web
    Korzysta z CLI ai-fetch-web do wyszukiwania w sieci, pobierania stron jako markdown, równoległego pobierania wielu adresów, wyciągania pól selektorami CSS, metadanych i zrzutów ekranu.
  • 90uwae
    Pobiera stronę internetową i zwraca ją jako oczyszczony tekst Markdown, gdy trzeba przeczytać link lub podsumować witrynę.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.