data-engineering-guide
/data-engineering-guide
Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming. Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling, data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs, streaming pipelines, lakehouse architecture, or data contracts.
- Author
- luanmorenommaciel ↗
- Repository
- agentspec ↗
- Licence
- MIT
- Last repository change
- 08.10.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 08.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming. Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling, data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs, streaming pipelines, lakehouse architecture, or data contracts.
Original author description
Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming. Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling, data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs, streaming pipelines, lakehouse architecture, or data contracts.
How it works
- Quick Routing
- Knowledge Domains Available
- How Agents Use Knowledge
- When to Suggest Commands
SKILL.md preview
---
name: data-engineering-guide
description: |
Data engineering expertise for pipelines, schemas, data quality, SQL, lakehouse, and streaming.
Use PROACTIVELY when the user discusses data pipelines, ETL/ELT, schema design, dimensional modeling,
data quality checks, SQL optimization, dbt models, Spark jobs, Airflow DAGs, streaming pipelines,
lakehouse architecture, or data contracts.
---
# Data Engineering Guide
You have access to 23 specialized knowledge base domains and 15+ data engineering agents. Route the user to the right tool based on their task.
## Quick Routing
| User Task | Command | Agent |
|-----------|---------|-------|
| Design a data pipeline / DAG | `/agentspec:pipeline` | pipeline-architect |
| Design a schema / star schema / data model | `/agentspec:schema` | schema-designer |
| Add data quality checks | `/agentspec:data-quality` | data-quality-analyst |
| Review SQL performance | `/agentspec:sql-review` | sql-optimizer |
| Choose table format (Iceberg/Delta) | `/agentspec:lakehouse` | lakehouse-architect |
| Build RAG / embedding pipeline | `/agentspec:ai-pipeline` | ai-data-engineer |
| Create a data contract | `/agentspec:data-contract` | data-contracts-engineer |
| Migrate legacy ETL | `/agentspec:migrate` | dbt-specialist + spark-engineer |
## Knowledge Domains Available
| Category | Domains |
|----------|---------|
| Core DE | dbt, spark, airflow, streaming, sql-patterns |
| Data Design | data-modeling, data-quality, medallion |
| Infrastructure | lakehouse, cloud-platforms, aws, gcp, microsoft-fabric, lakeflow, terraform |
| AI & Modern | ai-data-engineering, genai, prompt-engineering, modern-stack |
| Foundations | pydantic, python, testing |
## How Agents Use Knowledge
1. Agent reads KB index at `${CLAUDE_PLUGIN_ROOT}/kb/{domain}/index.md`
2. Loads specific pattern/concept file matching the task
3. Falls back to MCP if KB insufficient (max 3 MCP calls)
4. Calculates confidence from evidence matrix
## When to Suggest Commands
- User mentions "dbt model" or "staging model" → `/agentspec:schema` or delegate to dbt-specialist
- User mentions "pipeline" or "DAG" or "orchestration" → `/agentspec:pipeline`
- User mentions "data quality" or "expectations" or "tests" → `/agentspec:data-quality`
- User mentions "slow query" or "optimize SQL" → `/agentspec:sql-review`
- User mentions "Iceberg" or "Delta Lake" or "table format" → `/agentspec:lakehouse`
- User mentions "RAG" or "embeddings" or "vector" → `/agentspec:ai-pipeline`
- User mentions "contract" or "SLA" or "schema governance" → `/agentspec:data-contract`
- User mentions "migrate" or "legacy" or "SSIS" or "Informatica" → `/agentspec:migrate`
Versions
The database has no other skills named data-engineering-guide.
Identical content appears in 1 other GitHub folders. We show one copy with the highest score.
Database snapshot: 13.11.2026 · package version 47bb8e53b329 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 90data-engineering
Design and implement data pipelines (ETL/ELT), warehousing, and analytics engineering. Use when: data pipeline, etl, elt, spark, airflow, dbt, warehouse, big data, sql optimization, kafka. - 100data-engineering
Design, build, validate, and operate reliable data pipelines, ingestion, transformation, orchestration, analytics models, and data contracts. Use for ETL/ELT, batch/stream processing, warehouse/lakehouse work, data quality, and governed data access. - 100senior-data-engineer
Data engineering for batch and streaming pipelines with Airflow, dbt, Spark, and Kafka. Use when designing data architectures, building pipelines, adding data-quality checks, optimizing ETL/ELT, or troubleshooting pipeline failures. - 95senior-data-engineer
World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when… - 100senior-data-engineer
Data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data… - 100senior-data-engineer
Data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data…