dataform-gcp-sql-transformations
/dataform-gcp-sql-transformations
Use this skill when managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.
- Author
- nextboxis ↗
- Repository
- library-skill-agents ↗
- Licence
- MIT
- Last repository change
- 28.10.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 28.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Use this skill when managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.
Original author description
Use this skill when managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.
How it works
- When to use
- Prerequisites
- Step-by-step workflow
- Commands / code examples
- Common mistakes & pitfalls
- Verification checklist
- Related skills
SKILL.md preview
---
name: dataform-gcp-sql-transformations
description: Use this skill when managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.
---
# Google Dataform SQL Transformations
## When to use
Use this skill when you need to managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.
## Prerequisites
- Deep understanding of modern AI/ML systems, data engineering pipelines, or distributed architectures.
- Testing environment with necessary libraries and credentials configured.
## Step-by-step workflow
1. **Analyze data & architectural requirements**: Evaluate throughput, latency, context windows, and schema constraints.
2. **Formulate pipeline / model contracts**: Define strict input/output specifications and validation schemas.
3. **Implement core execution logic**: Construct deterministic transformations, prompt chains, or streaming data flows.
4. **Validate performance & accuracy**: Run automated test assertions, evaluate accuracy metrics, and benchmark throughput.
5. **Enforce observability & guardrails**: Instrument workflows with tracing, logging, rate limits, and fallback strategies.
## Commands / code examples
```python
# Production pipeline and implementation pattern for dataform-gcp-sql-transformations
import os
def run_pipeline():
print("Executing operational pipeline: dataform-gcp-sql-transformations")
if __name__ == "__main__":
run_pipeline()
```
## Common mistakes & pitfalls
- Allowing untrusted inputs to bypass validation checks, resulting in data corruption or prompt injection.
- Accumulating unconstrained state in memory during stream processing, leading to Out-Of-Memory (OOM) crashes.
- Neglecting evaluation benchmarks, deploying models or prompts without quantitative quality scoring.
## Verification checklist
- [ ] Pipeline validates input schemas and filters malformed records.
- [ ] Concurrency and memory utilization remain within provisioned boundaries.
- [ ] Performance and accuracy metrics meet predefined SLA targets.
- [ ] Fallback and retry mechanisms handle external API or network interruptions gracefully.
## Related skills
- retrieval-augmented-generation-rag-pipeline
- prompt-injection-defense-guardrails
- apache-kafka-event-streaming-architecture
- distributed-tracing-opentelemetry-sdk
Versions
The database has no other skills named dataform-gcp-sql-transformations.
Database snapshot: 13.11.2026 · package version 071e0302105f (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100dataform-bigquery
Expertise in generating clean, correct, and efficient Dataform pipeline code for BigQuery ELT. Use this when creating or modifying Dataform pipelines, actions, or source declarations, when Dataform, SQLX, or BigQuery are mentioned in a transformation, when data needs to be… - 95dataform-bigquery
Expertise in generating clean, correct, and efficient Dataform pipeline code for BigQuery ELT. Use this when creating or modifying Dataform pipelines, actions, or source declarations, when Dataform, SQLX, or BigQuery are mentioned in a transformation, when data needs to be… - 90bigquery-and-dataform-platform-engineering
Guides agents through BigQuery- and Dataform-centered data engineering workflows. Use when designing BigQuery physical models, ingestion boundaries, Dataform transformation workflows, slot or cost controls, and platform decisions across BigQuery, Dataflow, Dataproc, and GCP orchestration services. - 87dataform-bigquery-warehouse
Design, build, extend and review a layered BigQuery warehouse in Google Dataform (SQLX) - declarations, L1 source models, L2 schemas per business unit of the company, L3P presentation tables for machine-to-machine targets (apps, APIs, CRM, reverse ETL) and L3R reporting views… - 100sqlanvil-engineering-fundamentals
Use whenever a task touches a sqlanvil data project (the Dataform fork for PostgreSQL, Supabase and MySQL/MariaDB) - writing or editing .sqlx models, workflow_settings.yaml, .df-credentials.json, definitions/*.js, actions.yaml, Supabase RLS policies, indexes, partitions,… - 100gcp-data-engineering
Guides architecture decisions for data pipelines on Google Cloud — lake/warehouse/lakehouse design, batch vs. streaming, orchestration tool selection (Dataform, Cloud Composer/Airflow, Dataflow/Beam), and BigQuery cost/performance tuning (partitioning, clustering, slots). Use…