Skip to content
Skills Library
Data & analytics · Review & audit

data-and-analytics

/data-and-analytics

In an aidd repository, use when code touches pipelines, ETL, dbt models, warehouses, SQL schemas, data contracts, PII columns, event tracking, or ML training data.

Repository
aidd ↗
Licence
Apache-2.0
Last repository change
12.11.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
1
URLs in SKILL.md
10
Deprecation

No signals detected. Repository changed on 12.11.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What to check before installing
  • SKILL.md zawiera komentarze HTML (1). Nie widać ich w podglądzie Markdown, a model je czyta.

What it does and when to use it

In an aidd repository, use when code touches pipelines, ETL, dbt models, warehouses, SQL schemas, data contracts, PII columns, event tracking, or ML training data.

Original author description

In an aidd repository, use when code touches pipelines, ETL, dbt models, warehouses, SQL schemas, data contracts, PII columns, event tracking, or ML training data.

How it works

  1. Scope
  2. Standards
  3. Checklist by loop step
  4. Test commands
  5. Owned elsewhere
  6. Keep it current
Original SKILL.md headings · 71 lines, 1,073 words
SKILL.md preview
---
name: data-and-analytics
description: "In an aidd repository, use when code touches pipelines, ETL, dbt models, warehouses, SQL schemas, data contracts, PII columns, event tracking, or ML training data."
---

<!-- generated by `aidd gen-docs` from schemas/packs.json; edit the registry, not this file -->

# Data and analytics

## Scope

Applies when the change touches pipelines, warehouses, schemas, ML data. Not when the change touches one service's own tables with no schema, contract, or pipeline change. Neighbours: `aidd-team:schema-migration`, `aidd-packs:privacy-by-default`, `aidd-team:performance`.

## Standards

| # | Standard | Edition | Checked | Source |
| --- | --- | --- | --- | --- |
| S1 | ISO/IEC 25012 data quality model | ISO/IEC 25012:2008 | 2026-10-16 | <https://webstore.iec.ch/en/publication/11246> |
| S2 | Open Data Contract Standard (ODCS) (emerging) | v3.2.0 | 2026-10-16 | <https://github.com/bitol-io/open-data-contract-standard/releases> |
| S3 | Open Data Product Standard (ODPS) (emerging) | v1.1.0 | 2026-10-16 | <https://github.com/bitol-io/open-data-product-standard/releases> |
| S4 | OpenLineage | 1.53.0 | 2026-10-16 | <https://github.com/OpenLineage/OpenLineage/releases> |
| S5 | Analytics engineering: tests on every model, contracts on published models, and one definition of each metric in a semantic layer | — | 2026-10-16 | <https://docs.getdbt.com/docs/collaborate/publish/model-contracts> |
| S6 | Streaming semantics stated against the transport's own guarantee: idempotent producers and transactions where Kafka carries the data | — | 2026-10-16 | <https://kafka.apache.org/documentation/#semantics> |
| S7 | PostgreSQL full text search | PostgreSQL 18 | 2026-10-17 | <https://www.postgresql.org/docs/current/textsearch-intro.html> |
| S8 | Reproducible runs: experiment and model-registry tracking, datasets versioned, and every eval run pinned to a model and data snapshot | — | 2026-10-16 | <https://mlflow.org/docs/latest/> |
| S9 | Data and model drift monitored in production, with eval-score and cost signals wired to the rollback trigger | — | 2026-10-16 | <https://research.google/pubs/pub46555/> |
| S10 | Model cards and CycloneDX ML-BOM for models and datasets | CycloneDX 1.7.2 | 2026-10-16 | <https://cyclonedx.org/capabilities/mlbom/> |

### Open choices

The team decides these. aidd's default holds until a `standards` entry fills one (`aidd standard list`).

| Choice | Question | aidd's default | Options |
| --- | --- | --- | --- |
| `transformation` | How are models and transformations built and tested in the warehouse? | dbt Core: each model is a SELECT in the repository, with data tests on every model, contracts on published ones, and SQLFluff linting the SQL, all run by `dbt build` in CI. | dbt Core, SQLMesh, plain SQL files run by the orchestrator |
| `orchestrator` | What schedules and runs the pipelines? | Apache Airflow for scheduled pipelines; Dagster when the team wants assets declared in Python with lineage and tests in the repository. One orchestrator per project. | Apache Airflow, Dagster, Prefect, the warehouse's or cloud's own scheduler |
| `ingestion` | How does data from APIs, databases, and files reach the warehouse? | dlt, a Python library, so each pipeline lives and is tested in the repository; Airbyte when the team runs a connector platform with many ready-made sources, under a source-available license (its entry says which). | dlt, Airbyte, Fivetran, hand-written loaders |
| `warehouse` | Where do the models run and the analysts query? | DuckDB, marked emerging, while the data fits one machine and CI runs the models with no server. Past that, the warehouse the team already runs, such as PostgreSQL, BigQuery, or Snowflake. | the team's warehouse, DuckDB, a query engine over the data lake, such as Trino |

## Checklist by loop step

| Step | Item | Standard |
| --- | --- | --- |
| Before the first edit (`aidd risk --paths`) | A schema, contract, or pipeline change names its producers and consumers; a migration is major (the `migrations` area). | S2 |
| Specify | The data contract change (ODCS) is written before Spec approval and listed as a `design-files` doc; quality rules (accuracy, completeness, consistency, timeliness) as tests. | S2, S1 |
| Specify | A shared data product has an ODPS descriptor: an owner and an ODCS contract per output port. | S3, S2 |
| Specify | Search: the database's own full-text search first (PostgreSQL `tsvector` with a GIN index), a search engine only for a measured need. The index follows the data by outbox or change data capture, per-tenant queries filter by tenant, and judged queries prove relevance. | S7 |
| Specify | Training data is a pinned, versioned snapshot, and each run and eval records its model and data snapshot in the model registry. A model and its datasets get a model card and a CycloneDX ML-BOM entry (`aidd-packs:ai-features`). | S8, S10 |
| Implement | Idempotent, restartable jobs, with the delivery guarantee stated against the transport's own; lineage emitted (OpenLineage); no personal data in fixtures or logs. | S6, S4 |
| Implement | Warehouse personal data is classified in the contract and masked by column; each reader role sees only the columns it needs. | S2 |
| Review | Lineage names each consumer of a changed field; a breaking contract change waits for them or keeps the old version to a stated date. | S4, S2 |
| Roll out | Row counts, freshness, and null rates are the observe signals; consumers are notified. For a model, input, prediction, and outcome drift too, wired to the rollback trigger. | S1, S9 |

## Test commands

| Command | Proves | How | Standard |
| --- | --- | --- | --- |
| Contract validation (emerging) | producers and consumers pass the contract | `uvx --from datacontract-cli==1.2.2 datacontract test datacontract.yaml` in CI | S2 |
| Data product descriptor check (emerging) | the ODPS descriptor validates | `uvx check-jsonschema==0.38.2` with the ODPS v1.1.0 schema | S3 |
| Data quality tests | the quality rules hold on the output | dbt tests (`dbt test`, dbt-core 1.12.6), Great Expectations (1.23.2), or Soda checks | S1, S5 |
| Lineage check | the changed job emits lineage | an OpenLineage test backend | S4 |

- From `aidd-team:schema-migration`: Migration up and down.
- From `aidd-packs:privacy-by-default`: Masked test data check.

## Owned elsewhere

- The migration plan, the backfill, and the rollback: `aidd-team:schema-migration`.
- Query plans for new queries and indexes: `aidd-team:performance`.
- Retention, deletion, and cross-border transfers of personal data: `aidd-packs:privacy-by-default`.

## Keep it current

Each entry is due after its `review_every_days` (else emerging 90 days, established 365) or by its `review_due`. An emerging one applies only when the change needs it, said in the Spec. One `aidd eval standards` lists as due is looked up in Specify, and a missing one is proposed to aidd's `schemas/industry-standards.json`.

Versions

The database has no other skills named data-and-analytics.

Database snapshot: 13.11.2026 · package version 50f465b04f7c (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 85data-analytics
    Data analysis, visualization, machine learning, business intelligence, and data engineering skills. SQL, dashboards, statistical analysis, predictive models, data pipelines. Triggers on: data analysis, dashboard, BI, SQL, machine learning, statistics, data pipeline, ETL, predictive analytics.
  • 90data-engineering
    Design and implement data pipelines (ETL/ELT), warehousing, and analytics engineering. Use when: data pipeline, etl, elt, spark, airflow, dbt, warehouse, big data, sql optimization, kafka.
  • 95senior-data-engineer
    World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when…
  • 90data-engineering-guide
    Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming. Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling, data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs,…
  • 100analytics-engineer
    Analytics engineering across data modeling, dbt, transformation, and semantic layers. Use when building dbt models, designing star schemas, writing staging or mart SQL, configuring data tests, or optimizing warehouse queries.
  • 95data-pipelines
    Use when building or debugging data pipelines with Airflow or Prefect, writing dbt models or tests, designing incremental loads, implementing idempotent ETL/ELT jobs, validating data quality, or orchestrating multi-step data workflows.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.