Skip to content
Skills Library
Data & analytics · Learning & explanation

data-engineering-guide

/data-engineering-guide

Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming. Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling, data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs, streaming pipelines, lakehouse architecture, or data contracts.

Repository
agentspec ↗
Licence
MIT
Last repository change
08.10.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 08.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming. Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling, data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs, streaming pipelines, lakehouse architecture, or data contracts.

Original author description

Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming. Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling, data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs, streaming pipelines, lakehouse architecture, or data contracts.

How it works

  1. Quick Routing
  2. Knowledge Domains Available
  3. How Agents Use Knowledge
  4. When to Suggest Commands
Original SKILL.md headings · 46 lines, 318 words
SKILL.md preview
---
name: data-engineering-guide
description: |
  Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming.
  Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling,
  data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs, streaming pipelines,
  lakehouse architecture, or data contracts.
---

# Data Engineering Guide

You have access to 23 specialized knowledge base domains and 15+ data engineering agents. Route the user to the right tool based on their task.

## Quick Routing

| User Task | Command | Agent |
|-----------|---------|-------|
| Design a data pipeline / DAG | `/agentspec:pipeline` | pipeline-architect |
| Design a schema / star schema / data model | `/agentspec:schema` | schema-designer |
| Add data quality checks | `/agentspec:data-quality` | data-quality-analyst |
| Review SQL performance | `/agentspec:sql-review` | sql-optimizer |
| Choose table format (Iceberg/Delta) | `/agentspec:lakehouse` | lakehouse-architect |
| Build RAG / embedding pipeline | `/agentspec:ai-pipeline` | ai-data-engineer |
| Create a data contract | `/agentspec:data-contract` | data-contracts-engineer |
| Migrate legacy ETL | `/agentspec:migrate` | dbt-specialist + spark-engineer |

## Knowledge Domains Available

| Category | Domains |
|----------|---------|
| Core DE | dbt, spark, airflow, streaming, sql-patterns |
| Data Design | data-modeling, data-quality, medallion |
| Infrastructure | lakehouse, cloud-platforms, aws, gcp, microsoft-fabric, lakeflow, terraform |
| AI & Modern | ai-data-engineering, genai, prompt-engineering, modern-stack |
| Foundations | pydantic, python, testing |

## How Agents Use Knowledge

1. Agent reads KB index at `${CLAUDE_PLUGIN_ROOT}/kb/{domain}/index.md`
2. Loads specific pattern/concept file matching the task
3. Falls back to MCP if KB insufficient (max 3 MCP calls)
4. Calculates confidence from evidence matrix

## When to Suggest Commands

- User mentions "dbt model" or "staging model" → `/agentspec:schema` or delegate to dbt-specialist
- User mentions "pipeline" or "DAG" or "orchestration" → `/agentspec:pipeline`
- User mentions "data quality" or "expectations" or "tests" → `/agentspec:data-quality`
- User mentions "slow query" or "optimize SQL" → `/agentspec:sql-review`
- User mentions "Iceberg" or "Delta Lake" or "table format" → `/agentspec:lakehouse`
- User mentions "RAG" or "embeddings" or "vector" → `/agentspec:ai-pipeline`
- User mentions "contract" or "SLA" or "schema governance" → `/agentspec:data-contract`
- User mentions "migrate" or "legacy" or "SSIS" or "Informatica" → `/agentspec:migrate`

Versions

The database has no other skills named data-engineering-guide.

Identical content appears in 1 other GitHub folders. We show one copy with the highest score.

Database snapshot: 13.11.2026 · package version 47bb8e53b329 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90data-engineering
    Design and implement data pipelines (ETL/ELT), warehousing, and analytics engineering. Use when: data pipeline, etl, elt, spark, airflow, dbt, warehouse, big data, sql optimization, kafka.
  • 100data-engineering
    Design, build, validate, and operate reliable data pipelines, ingestion, transformation, orchestration, analytics models, and data contracts. Use for ETL/ELT, batch/stream processing, warehouse/lakehouse work, data quality, and governed data access.
  • 100senior-data-engineer
    Data engineering for batch and streaming pipelines with Airflow, dbt, Spark, and Kafka. Use when designing data architectures, building pipelines, adding data-quality checks, optimizing ETL/ELT, or troubleshooting pipeline failures.
  • 95senior-data-engineer
    World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when…
  • 100senior-data-engineer
    Data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data…
  • 100senior-data-engineer
    Data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.