Przejdź do treści
Biblioteka skilli
Dane i analityka · Integracja z narzędziem

data-feeds

/data-feeds

Pobiera ustrukturyzowane dane z gotowych scraperów Bright Data (e-commerce, social, oferty pracy, mapy, nieruchomości) przez Bright Data CLI i zwraca czysty JSON.

Repozytorium
skills ↗
Licencja
MIT
Ostatnia zmiana repozytorium
15.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
4
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 15.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Pobiera ustrukturyzowane dane z gotowych scraperów Bright Data (e-commerce, social, oferty pracy, mapy, nieruchomości) przez Bright Data CLI i zwraca czysty JSON.

Oryginalny opis (EN)

Extract structured data from thousands of pre-built Bright Data scrapers (e-commerce, social, jobs, maps, real estate and more) via the Bright Data CLI (`bdata pipelines search`, `info`, `run`). Use when the user wants clean JSON from a known site rather than raw HTML. Hands off to `scrape` when no scraper fits and to `search` when target URLs must be discovered first. Requires the Bright Data CLI 0.4.0+; proactively guides install + login if missing.

Jak działa

  1. Setup gate (run first)
  2. Workflow (always search, then info, then run)
  3. Pick your path
  4. Action
  5. Verification gate
  6. Red flags
  7. References
Sekcje z nagłówków SKILL.md, w oryginale · 129 linii, 1 052 słów
Podgląd SKILL.md
---
name: data-feeds
description: Extract structured data from thousands of pre-built Bright Data scrapers (e-commerce, social, jobs, maps, real estate and more) via the Bright Data CLI (`bdata pipelines search`, `info`, `run`). Use when the user wants clean JSON from a known site rather than raw HTML. Hands off to `scrape` when no scraper fits and to `search` when target URLs must be discovered first. Requires the Bright Data CLI 0.4.0+; proactively guides install + login if missing.
---

# Bright Data — Data Feeds (Pipelines)

Extract structured data via `bdata pipelines`: search for a pre-built scraper, check its inputs, run it, get clean JSON. No scraping logic. If no scraper fits, hand off to `scrape`. To find target URLs first, hand off to `search`.

## Setup gate (run first)

```bash
if ! command -v bdata >/dev/null 2>&1; then
    echo "bdata CLI not installed — see bright-data-best-practices/references/cli-setup.md"
elif ! bdata zones >/dev/null 2>&1; then
    echo "bdata not authenticated — run: bdata login  (or: bdata login --device for SSH)"
fi
```

Halt and route to `skills/bright-data-best-practices/references/cli-setup.md` if either check fails.

## Workflow (always search, then info, then run)

Requires CLI 0.4.0 or later. The fixed pipeline types (`bdata pipelines <type> <url>`), `bdata pipelines list` and top-level `bdata status` were removed. Never guess a `dataset_id`, method or field name: look them up every time.

```bash
# 1. Find a scraper by site or data type
bdata pipelines search <site or data type> [--limit n]

# 2. List its collection methods
bdata pipelines info <dataset_id>

# 3. Show one method's input fields, example input and a ready-to-run command
bdata pipelines info <dataset_id> <method>

# 4. Run it
bdata pipelines run <dataset_id> <method> <url> [--<field> <value>...]
```

- `search` returns `dataset_id`, name, domain and `collection_methods`. Two scrapers can share a name, so pick by `dataset_id` and domain.
- `collect_by_url` returns one record per input URL. `discover_by_*` methods (keyword, category URL, ...) return up to `--limit-per-input` records per input (default 10).
- Use method names exactly as listed, including misspelled ones.

Full command reference: [`../brightdata-cli/references/pipelines.md`](../brightdata-cli/references/pipelines.md).

## Pick your path

| Situation | Action |
|---|---|
| Know the site + have URL(s) | `search` for the scraper, `info`, then `run ... collect_by_url <url>...` |
| Don't know which scraper fits | `bdata pipelines search <site or data type>` and compare results |
| Input is a keyword, category or other non-URL field | use a `discover_by_*` method; `info <dataset_id> <method>` shows the fields |
| Multiple URLs on the same scraper | pass them all to one `run` (each URL = one record) |
| Different values per record or nested input | `--input '<json>'` or `--input @records.json` |
| Long job (reviews, big feeds, discover methods) | `--async`, then `status --wait` and `results`; or raise `--timeout 1800` |
| No scraper fits the site | **stop, hand off to `scrape`** |
| Need to find URLs first | **hand off to `search`** |

## Action

Core commands:

```bash
# Find the scraper and its inputs (example query; use the user's site)
bdata pipelines search "<site or data type>"
bdata pipelines info <dataset_id>
bdata pipelines info <dataset_id> collect_by_url

# One record per URL; other --<field> flags apply to every record
bdata pipelines run <dataset_id> collect_by_url "<url1>" "<url2>" \
    --format json --pretty -o out.json

# Discover method: no URL, fields as flags, keep the limit small (every record is billed)
bdata pipelines run <dataset_id> <discover_method> --<field> "<value>" \
    --limit-per-input 5 -o out.json

# Per-record values or nested data as JSON
bdata pipelines run <dataset_id> <method> --input @records.json -o out.json

# Long job: start async, come back later
bdata pipelines run <dataset_id> <method> "<url>" --async   # prints snapshot_id
bdata pipelines status <snapshot_id> --wait
bdata pipelines results <snapshot_id> --format ndjson -o out.ndjson

# Script-friendly: JSON from search when piped or with --json
bdata pipelines search "<site or data type>" --json | jq -r '.[] | "\(.dataset_id) \(.domain) \(.name)"'
```

Unknown or missing required fields fail before anything is triggered, so nothing is billed; the error lists the valid fields.

Full flag reference: [`references/flags.md`](references/flags.md).

## Verification gate

1. **JSON parses cleanly:** `jq . <output>` returns 0 (or for `--format ndjson`, each line parses).
2. **Record count matches expected.** `collect_by_url` usually returns one record per URL, *but* review, post and comment scrapers and `discover_by_*` methods return arrays sized by what the site shows (capped by `--limit-per-input`). Always check:
   ```bash
   jq 'length' out.json                       # top-level array count
   # OR
   jq 'if type == "array" then length else 1 end' out.json
   ```
3. **No top-level error:**
   ```bash
   jq -e 'if type == "object" then has("error") | not else true end' out.json \
       || { echo "pipeline reported error"; exit 1; }
   ```
4. **No per-record error:** for array results, ensure no record has an `error` field:
   ```bash
   jq -e 'if type == "array" then map(has("error")) | any | not else true end' out.json \
       || echo "WARN: one or more records have error fields"
   ```
   Partial failures are silent — this check is non-optional.
5. **Core fields present.** Fields differ per scraper; spot-check with `jq '.[0] | keys'` on the first record and compare with what the user asked for.
6. **On failure:** read any record `error` field (often a URL of the wrong type for the method), re-check `bdata pipelines info <dataset_id> <method>`, then double `--timeout` (or use `--async`) and retry once. If it still fails, `bdata scrape <url>` returns the page as Markdown as a fallback.

## Red flags

- Guessing a `dataset_id`, method or field name instead of running `search` and `info` first.
- Using removed commands: `bdata pipelines <type> <url>`, `bdata pipelines list`, `bdata status`. They exit with a hint; use `search` + `info` + `run` and `bdata pipelines status <snapshot_id>`.
- Using `bdata scrape` on a site that has a pre-built scraper. Loses structure and costs more time.
- Looping `bdata pipelines run` once per URL when one `run` accepts many URLs. If you must loop, cap parallelism at 2 to 3.
- Large `--limit-per-input` on `discover_by_*` methods. Every record is billed; start with one input and a small limit.
- Claiming success without the record-count + per-record error check. Partial failures are silent in pipeline output.
- "Fixing" a misspelled method name. Use it exactly as `info` lists it.
- Using a tight `--timeout` on runs that legitimately take 5 to 15 minutes. Default 600s suits small inputs; raise it or use `--async` for long ones.

## References

- [`../brightdata-cli/references/pipelines.md`](../brightdata-cli/references/pipelines.md): canonical `bdata pipelines` reference.
- [`references/flags.md`](references/flags.md): `search`, `info`, `run`, `status`, `results` flags and timeout guidance.
- [`references/patterns.md`](references/patterns.md): scraper resolution workflow, input patterns, async runs, batching, partial-failure detection, legacy `curl` fallback, shared verification checklist.
- [`references/examples.md`](references/examples.md): (1) single URL, (2) many URLs in one run, (3) long job with `--async`, (4) discover by keyword, (5) per-record JSON input.
Pliki w paczce (4)
  • SKILL.md 7 KB
  • references/examples.md 4 KB
  • references/flags.md 4 KB
  • references/patterns.md 5 KB

Wersje

RepozytoriumWynikLicencja
Ta strona · brightdata/skills100MIT
420company/artemis95MIT

Migawka bazy z 13.11.2026 · wersja paczki f631d96975a1 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100scrape
    Pobiera treści z sieci jako czysty Markdown, HTML lub JSON przez Bright Data CLI: pojedyncze strony, listy URL-i i paginowane listingi, z przekazaniem do data-feeds lub search.
  • 100search
    Przeszukuje sieć przez CLI Bright Data: wyniki SERP z Google, Bing i Yandex oraz semantyczne wyszukiwanie, przekazując wybrane adresy URL do scrapowania.
  • 95bright-data-mcp
    Zbiera dane z sieci przez Bright Data MCP: pobiera strony, wyszukuje w internecie i wyciąga dane z Amazon, LinkedIn i innych serwisów.
  • 100bright-data-mcp
    Obsługuje wszystkie operacje na danych z sieci przez Bright Data MCP: strony, wyszukiwanie, dane z Amazon, LinkedIn i innych serwisów oraz automatyzację przeglądarki.
  • 90scraper-builder
    Buduje produkcyjne scrapery stron przez infrastrukturę Bright Data: analizuje stronę, dobiera API, wyciąga selektory i obsługuje paginację.
  • 100web-scraper
    Wyciąga dane strukturalne ze stron i ładuje je do baz danych: HTML statyczny, strony renderowane w JavaScript, proxy antybotowe Bright Data, transformacje, deduplikacja i ETL.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.