data-and-analytics
/data-and-analytics
In an aidd repository, use when code touches pipelines, ETL, dbt models, warehouses, SQL schemas, data contracts, PII columns, event tracking, or ML training data.
- Author
- thaithienvanid ↗
- Repository
- aidd ↗
- Licence
- Apache-2.0
- Last repository change
- 12.11.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 1
- URLs in SKILL.md
- 10
No signals detected. Repository changed on 12.11.2026, without references to retired models or APIs.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
- SKILL.md zawiera komentarze HTML (1). Nie widać ich w podglądzie Markdown, a model je czyta.
What it does and when to use it
In an aidd repository, use when code touches pipelines, ETL, dbt models, warehouses, SQL schemas, data contracts, PII columns, event tracking, or ML training data.
Original author description
In an aidd repository, use when code touches pipelines, ETL, dbt models, warehouses, SQL schemas, data contracts, PII columns, event tracking, or ML training data.
How it works
- Scope
- Standards
- Checklist by loop step
- Test commands
- Owned elsewhere
- Keep it current
SKILL.md preview
--- name: data-and-analytics description: "In an aidd repository, use when code touches pipelines, ETL, dbt models, warehouses, SQL schemas, data contracts, PII columns, event tracking, or ML training data." --- <!-- generated by `aidd gen-docs` from schemas/packs.json; edit the registry, not this file --> # Data and analytics ## Scope Applies when the change touches pipelines, warehouses, schemas, ML data. Not when the change touches one service's own tables with no schema, contract, or pipeline change. Neighbours: `aidd-team:schema-migration`, `aidd-packs:privacy-by-default`, `aidd-team:performance`. ## Standards | # | Standard | Edition | Checked | Source | | --- | --- | --- | --- | --- | | S1 | ISO/IEC 25012 data quality model | ISO/IEC 25012:2008 | 2026-10-16 | <https://webstore.iec.ch/en/publication/11246> | | S2 | Open Data Contract Standard (ODCS) (emerging) | v3.2.0 | 2026-10-16 | <https://github.com/bitol-io/open-data-contract-standard/releases> | | S3 | Open Data Product Standard (ODPS) (emerging) | v1.1.0 | 2026-10-16 | <https://github.com/bitol-io/open-data-product-standard/releases> | | S4 | OpenLineage | 1.53.0 | 2026-10-16 | <https://github.com/OpenLineage/OpenLineage/releases> | | S5 | Analytics engineering: tests on every model, contracts on published models, and one definition of each metric in a semantic layer | — | 2026-10-16 | <https://docs.getdbt.com/docs/collaborate/publish/model-contracts> | | S6 | Streaming semantics stated against the transport's own guarantee: idempotent producers and transactions where Kafka carries the data | — | 2026-10-16 | <https://kafka.apache.org/documentation/#semantics> | | S7 | PostgreSQL full text search | PostgreSQL 18 | 2026-10-17 | <https://www.postgresql.org/docs/current/textsearch-intro.html> | | S8 | Reproducible runs: experiment and model-registry tracking, datasets versioned, and every eval run pinned to a model and data snapshot | — | 2026-10-16 | <https://mlflow.org/docs/latest/> | | S9 | Data and model drift monitored in production, with eval-score and cost signals wired to the rollback trigger | — | 2026-10-16 | <https://research.google/pubs/pub46555/> | | S10 | Model cards and CycloneDX ML-BOM for models and datasets | CycloneDX 1.7.2 | 2026-10-16 | <https://cyclonedx.org/capabilities/mlbom/> | ### Open choices The team decides these. aidd's default holds until a `standards` entry fills one (`aidd standard list`). | Choice | Question | aidd's default | Options | | --- | --- | --- | --- | | `transformation` | How are models and transformations built and tested in the warehouse? | dbt Core: each model is a SELECT in the repository, with data tests on every model, contracts on published ones, and SQLFluff linting the SQL, all run by `dbt build` in CI. | dbt Core, SQLMesh, plain SQL files run by the orchestrator | | `orchestrator` | What schedules and runs the pipelines? | Apache Airflow for scheduled pipelines; Dagster when the team wants assets declared in Python with lineage and tests in the repository. One orchestrator per project. | Apache Airflow, Dagster, Prefect, the warehouse's or cloud's own scheduler | | `ingestion` | How does data from APIs, databases, and files reach the warehouse? | dlt, a Python library, so each pipeline lives and is tested in the repository; Airbyte when the team runs a connector platform with many ready-made sources, under a source-available license (its entry says which). | dlt, Airbyte, Fivetran, hand-written loaders | | `warehouse` | Where do the models run and the analysts query? | DuckDB, marked emerging, while the data fits one machine and CI runs the models with no server. Past that, the warehouse the team already runs, such as PostgreSQL, BigQuery, or Snowflake. | the team's warehouse, DuckDB, a query engine over the data lake, such as Trino | ## Checklist by loop step | Step | Item | Standard | | --- | --- | --- | | Before the first edit (`aidd risk --paths`) | A schema, contract, or pipeline change names its producers and consumers; a migration is major (the `migrations` area). | S2 | | Specify | The data contract change (ODCS) is written before Spec approval and listed as a `design-files` doc; quality rules (accuracy, completeness, consistency, timeliness) as tests. | S2, S1 | | Specify | A shared data product has an ODPS descriptor: an owner and an ODCS contract per output port. | S3, S2 | | Specify | Search: the database's own full-text search first (PostgreSQL `tsvector` with a GIN index), a search engine only for a measured need. The index follows the data by outbox or change data capture, per-tenant queries filter by tenant, and judged queries prove relevance. | S7 | | Specify | Training data is a pinned, versioned snapshot, and each run and eval records its model and data snapshot in the model registry. A model and its datasets get a model card and a CycloneDX ML-BOM entry (`aidd-packs:ai-features`). | S8, S10 | | Implement | Idempotent, restartable jobs, with the delivery guarantee stated against the transport's own; lineage emitted (OpenLineage); no personal data in fixtures or logs. | S6, S4 | | Implement | Warehouse personal data is classified in the contract and masked by column; each reader role sees only the columns it needs. | S2 | | Review | Lineage names each consumer of a changed field; a breaking contract change waits for them or keeps the old version to a stated date. | S4, S2 | | Roll out | Row counts, freshness, and null rates are the observe signals; consumers are notified. For a model, input, prediction, and outcome drift too, wired to the rollback trigger. | S1, S9 | ## Test commands | Command | Proves | How | Standard | | --- | --- | --- | --- | | Contract validation (emerging) | producers and consumers pass the contract | `uvx --from datacontract-cli==1.2.2 datacontract test datacontract.yaml` in CI | S2 | | Data product descriptor check (emerging) | the ODPS descriptor validates | `uvx check-jsonschema==0.38.2` with the ODPS v1.1.0 schema | S3 | | Data quality tests | the quality rules hold on the output | dbt tests (`dbt test`, dbt-core 1.12.6), Great Expectations (1.23.2), or Soda checks | S1, S5 | | Lineage check | the changed job emits lineage | an OpenLineage test backend | S4 | - From `aidd-team:schema-migration`: Migration up and down. - From `aidd-packs:privacy-by-default`: Masked test data check. ## Owned elsewhere - The migration plan, the backfill, and the rollback: `aidd-team:schema-migration`. - Query plans for new queries and indexes: `aidd-team:performance`. - Retention, deletion, and cross-border transfers of personal data: `aidd-packs:privacy-by-default`. ## Keep it current Each entry is due after its `review_every_days` (else emerging 90 days, established 365) or by its `review_due`. An emerging one applies only when the change needs it, said in the Spec. One `aidd eval standards` lists as due is looked up in Specify, and a missing one is proposed to aidd's `schemas/industry-standards.json`.
Versions
The database has no other skills named data-and-analytics.
Database snapshot: 13.11.2026 · package version 50f465b04f7c (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 85data-analytics
Data analysis, visualization, machine learning, business intelligence, and data engineering skills. SQL, dashboards, statistical analysis, predictive models, data pipelines. Triggers on: data analysis, dashboard, BI, SQL, machine learning, statistics, data pipeline, ETL, predictive analytics. - 90data-engineering
Design and implement data pipelines (ETL/ELT), warehousing, and analytics engineering. Use when: data pipeline, etl, elt, spark, airflow, dbt, warehouse, big data, sql optimization, kafka. - 95senior-data-engineer
World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when… - 90data-engineering-guide
Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming. Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling, data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs,… - 100analytics-engineer
Analytics engineering across data modeling, dbt, transformation, and semantic layers. Use when building dbt models, designing star schemas, writing staging or mart SQL, configuring data tests, or optimizing warehouse queries. - 95data-pipelines
Use when building or debugging data pipelines with Airflow or Prefect, writing dbt models or tests, designing incremental loads, implementing idempotent ETL/ELT jobs, validating data quality, or orchestrating multi-step data workflows.