Przejdź do treści
Biblioteka skilli
Dane i analityka · Analiza

datapackage

/datapackage

Eksploruje i odpytuje zbiory z deskryptorem Frictionless Data Package (datapackage.json): zasoby, kolumny i opisy, ostrzeżenia z metadanych oraz ładowanie z Parquet, DuckDB, SQLite i CSV.

Repozytorium
agent-skills ↗
Licencja
MIT
Ostatnia zmiana repozytorium
27.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
6
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
5
Przestarzałość

Brak sygnałów. Repozytorium zmienione 27.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Eksploruje i odpytuje zbiory z deskryptorem Frictionless Data Package (datapackage.json): zasoby, kolumny i opisy, ostrzeżenia z metadanych oraz ładowanie z Parquet, DuckDB, SQLite i CSV.

Oryginalny opis (EN)

Explore and query any dataset annotated with a Frictionless Data Package descriptor (datapackage.json). Use this skill whenever a user wants to discover what tables or resources a dataset contains, look up column names and descriptions, surface usage warnings embedded in metadata, or understand how to load data from Parquet files, DuckDB or SQLite databases, or CSV files described by a datapackage.json. Also use when the user has a datapackage.json and wants to know what's in it, how to query it efficiently, or how to connect its metadata to actual data files. Pairs well with dataset-specific

Jak działa

  1. What is a datapackage.json?
  2. Dependency check
  3. Workflow overview
  4. Reference index
  5. Community patterns and recipes
  6. Companion skills
  7. Key constraints
  8. Schema reference and version detection
Sekcje z nagłówków SKILL.md, w oryginale · 159 linii, 1 076 słów
Podgląd SKILL.md
---
name: datapackage
description: >
  Explore and query any dataset annotated with a Frictionless Data Package descriptor
  (datapackage.json). Use this skill whenever a user wants to discover what tables or
  resources a dataset contains, look up column names and descriptions, surface usage
  warnings embedded in metadata, or understand how to load data from Parquet files,
  DuckDB or SQLite databases, or CSV files described by a datapackage.json. Also use
  when the user has a datapackage.json and wants to know what's in it, how to query it
  efficiently, or how to connect its metadata to actual data files. Pairs well with
  dataset-specific skills (like `pudl`) that layer domain knowledge on top.
license: CC-BY-4.0
compatibility: |
  Required CLI tools: jq >= 1.8
  Optional CLI tools: frictionless >= 5.19 (with fastparquet for Parquet support)
  Required skills: install-duckdb, query, attach-db
  Optional Python packages: pandas, polars, duckdb (for DataFrame work)
metadata:
  - author: Catalyst Cooperative
  - email: hello@catalyst.coop
  - last-updated: 2026-08-15
---

# Frictionless Data Package Guide

This skill covers any dataset described by a
[Frictionless Data Package](https://datapackage.org/) descriptor file
(`datapackage.json`). It is intentionally generic — it works for any conforming
datapackage, regardless of who published it or what the data contains.

For PUDL-specific knowledge (S3 bucket paths, table tier conventions, data source
context, usage warnings), also use the `pudl` skill on top of this one.

## What is a datapackage.json?

A `datapackage.json` is a JSON file that describes a collection of tabular data
resources. Each resource represents one table (or file) and includes:

- `name`: machine-readable identifier
- `description`: human-readable description, often including processing notes, primary
    keys, and usage warnings
- `path`: filename or URL of the actual data file
- `schema.fields`: list of columns, each with a `name` and `description`
- `schema.primaryKey`: the field or fields that uniquely identify a row in this resource
- `schema.foreignKeys`: declared links from this resource's fields to another
    resource's primary key — check these before joining or aggregating (see
    [Metadata Querying](./references/metadata-querying.md))

The file can be large (hundreds of resources, megabytes of JSON). Always query it
selectively — never load it whole into context.

## Dependency check

Before querying metadata, verify `jq` is available:

```bash
command -v jq
```

If not found, tell the user how to install it:

- macOS: `brew install jq`
- Linux (apt): `sudo apt install jq`
- Linux (conda): `conda install jq`
- Windows: `winget install jqlang.jq`

For data loading and SQL queries, the `attach-db` and `query` skills must be
installed (optionally `install-duckdb` too). Install them from `duckdb/duckdb-skills`.

## Workflow overview

1. **Locate the descriptor** — find or download `datapackage.json` (see below).
1. **Query metadata selectively** — use jq to extract only what you need.
    See [Metadata Querying](./references/metadata-querying.md).
1. **Surface warnings** — always check for usage warnings before presenting a resource.
1. **Check keys before joining or aggregating** — if the task combines two resources,
    or rolls one up, look up `schema.primaryKey` and `schema.foreignKeys` on each
    first, rather than joining on a same-named or similar-looking column. See
    [Metadata Querying: Joining resources](./references/metadata-querying.md#joining-resources-primary-keys-and-foreign-keys).
1. **Validate** *(optional)* — if the user wants to know whether the data actually
    matches the descriptor, or if you're diagnosing a suspicious package, use
    `frictionless validate`. See [Frictionless Validate](./references/frictionless-validate.md).
1. **Load the data** *(optional)* — only if the user explicitly wants to query or
    explore the actual data. Data files can be large and remote access can be slow or
    costly. Don't initiate data loading as a follow-on to a metadata lookup without
    confirming the user wants it. See [Storage Backends](./references/storage-backends.md).

## Reference index

- [Metadata Querying](./references/metadata-querying.md) — locate the descriptor,
    query it selectively with jq, surface usage warnings
- [Storage Backends](./references/storage-backends.md) — load data from Parquet,
    DuckDB, SQLite, or CSV files referenced by the descriptor
- [Frictionless Validate](./references/frictionless-validate.md) — use the `frictionless`
    CLI to validate packages, check data quality, infer schemas, and diagnose unfamiliar
    descriptors; read when the user wants to validate a descriptor, check if data matches
    its schema, or understand what the `frictionless` tool can tell them about a package

## Community patterns and recipes

The datapackage standard is permissive: publishers frequently add non-standard fields.
Two conventions are worth knowing immediately:

- **Custom fields** — non-standard keys added by publishers are common and valid.
    The `_` prefix convention marks system-generated or platform-specific keys (e.g.
    `_cache`, `_platformVersion`). Some publishers add custom keys without the prefix
    (e.g. a package-level unit registry, or per-resource provenance metadata). Treat
    unknown fields as informational metadata, not errors.
- **Compressed resources** — a resource with a `.gz` or `.zip` path may have an
    explicit `"compression": "gz"` field. The `bytes` and `hash` fields apply to the
    compressed file, not the uncompressed original.

For other patterns (catalogs, versioning, external foreign keys, translation support,
field relationships, etc.), fetch the relevant page on demand:

- v1 patterns: <https://specs.frictionlessdata.io/patterns/>
- v2 recipes: <https://datapackage.org/recipes/caching-of-resources/> (navigate via
    sidebar or next/previous links — no index page exists)

Both pages cover largely the same set of community conventions; consult whichever
matches the descriptor version you're working with.

## Companion skills

This skill delegates actual data querying to:

- **`/attach-db`** — attach a `.duckdb` or `.sqlite` database file and
    set up a persistent session for querying
- **`/query`** — run SQL or natural language queries against attached
    databases, ad-hoc files (Parquet, CSV, remote HTTPS/S3), and JSON files including
    `datapackage.json` itself (via DuckDB's `read_json`)

These skills must be installed. See `skills-lock.json` in the project root.

## Key constraints

- **Golden rule: never load the full datapackage.json into context.** It may be
    megabytes with hundreds of resources. Always query selectively.
- **Read the full description before presenting a resource.** Descriptions often
    contain important context: processing notes, primary key conventions, data
    provenance, or caveats about known limitations. Don't skip them.
- **Use `uv` to install Python packages** — prefer `uv add <package>` over
    `pip install <package>`. `uv` is faster and installs into a virtual environment
    rather than globally. Fall back to `pip` only if `uv` is not available
    (`command -v uv` returns nothing).
- **Do not use Python to query descriptor metadata.** Python is not the right tool here
    — it loads the full JSON into memory (violating the golden rule above), adds
    unnecessary dependencies, and can't easily handle remote descriptors. Use jq for
    metadata-only tasks; use DuckDB when you need to combine metadata queries with data
    queries. Python is only appropriate for loading data (via pandas or polars) after you
    already know which table and columns you need.

## Schema reference and version detection

Two versions of the Frictionless Data Package standard are in common use. Identify the
version from the top-level descriptor before parsing:

| Field present | Version                          | Example value                                             |
| ------------- | -------------------------------- | --------------------------------------------------------- |
| `"$schema"`   | v2.0                             | `"https://datapackage.org/profiles/2.0/datapackage.json"` |
| `"profile"`   | v1.0                             | `"tabular-data-package"` or `"data-package"`              |
| neither       | ambiguous (treat as v1 baseline) | —                                                         |

Key differences between versions that affect parsing:

- **Contributors** — v1 has `"role": "author"` (singular string); v2 has
    `"roles": ["author"]` (array). Both may appear in the wild.
- **Name pattern** — v1 enforces strictly lowercase `[-a-z0-9._/]`; v2 is unrestricted.
- **`version` field** — present in v2, absent in v1.

Bundled schemas:

- [`assets/datapackage-v1.schema.json`](./assets/datapackage-v1.schema.json) — v1.0
    (JSON Schema draft-04). Used by FERC XBRL packages and many older datasets.
- [`assets/datapackage-v2.schema.json`](./assets/datapackage-v2.schema.json) — v2.0
    (JSON Schema draft-07). The current standard. Canonical version always at:
    <https://datapackage.org/profiles/2.0/datapackage.json>

Read the appropriate schema when you need to understand which fields are valid in a
descriptor or validate one programmatically.
Pliki w paczce (6)
  • SKILL.md 9 KB
  • assets/datapackage-v1.schema.json 22 KB
  • assets/datapackage-v2.schema.json 155 KB
  • references/frictionless-validate.md 11 KB
  • references/metadata-querying.md 12 KB
  • references/storage-backends.md 10 KB

Wersje

W bazie nie ma innych skilli o nazwie datapackage.

Migawka bazy z 13.11.2026 · wersja paczki f17c817092ad (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100zo-dataset-creator
    Tworzy zbiory danych Zo w poprawnym formacie dla interfejsu Zo: datapackage.json, schema.yaml i bazę DuckDB, także naprawia brakujące tabele.
  • 95duckdb-en
    Obsługuje DuckDB CLI do analizy SQL, przetwarzania danych i konwersji plików, m.in. CSV, Parquet i JSON.
  • 100exploring-data
    Wykonuje eksploracyjną analizę plików csv, xlsx, json i parquet z raportami ydata-profiling lub profilowaniem DuckDB, wykrywa duplikaty, nakładanie kluczy i dryf.
  • 100matlab-use-duckdb
    Używa DuckDB z MATLAB-a przez Database Toolbox (R2026a+) do zapytań SQL na dużych plikach CSV, Parquet i JSON, redukcji danych przed analizą oraz jako osadzonej bazy do rozwoju.
  • 100pudl
    Pomaga eksplorować dane energetyczne PUDL: wyszukuje tabele, wyjaśnia kolumny i ograniczenia danych oraz ładuje pliki Parquet z S3 lub dysku lokalnego bez pakietu Pythona PUDL.
  • 100portaljs-define-schema
    Definiuje profil metadanych zbioru danych: wnioskuje Frictionless Table Schema z danych, dodaje metadane Data Package (licencja, źródła, słowa kluczowe) i zapisuje je w datasets.json.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.