Skip to content
Skills Library
Data & analytics · Building & coding

dataform-gcp-sql-transformations

/dataform-gcp-sql-transformations

Use this skill when managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.

Licence
MIT
Last repository change
28.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 28.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Use this skill when managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.

Original author description

Use this skill when managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.

How it works

  1. When to use
  2. Prerequisites
  3. Step-by-step workflow
  4. Commands / code examples
  5. Common mistakes & pitfalls
  6. Verification checklist
  7. Related skills
Original SKILL.md headings · 45 lines, 263 words
SKILL.md preview
---
name: dataform-gcp-sql-transformations
description: Use this skill when managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.
---
# Google Dataform SQL Transformations

## When to use
Use this skill when you need to managing data transformation workflows and dependency graphs in BigQuery natively using Dataform and SQLX.

## Prerequisites
- Deep understanding of modern AI/ML systems, data engineering pipelines, or distributed architectures.
- Testing environment with necessary libraries and credentials configured.

## Step-by-step workflow
1. **Analyze data & architectural requirements**: Evaluate throughput, latency, context windows, and schema constraints.
2. **Formulate pipeline / model contracts**: Define strict input/output specifications and validation schemas.
3. **Implement core execution logic**: Construct deterministic transformations, prompt chains, or streaming data flows.
4. **Validate performance & accuracy**: Run automated test assertions, evaluate accuracy metrics, and benchmark throughput.
5. **Enforce observability & guardrails**: Instrument workflows with tracing, logging, rate limits, and fallback strategies.

## Commands / code examples
```python
# Production pipeline and implementation pattern for dataform-gcp-sql-transformations
import os

def run_pipeline():
    print("Executing operational pipeline: dataform-gcp-sql-transformations")

if __name__ == "__main__":
    run_pipeline()
```

## Common mistakes & pitfalls
- Allowing untrusted inputs to bypass validation checks, resulting in data corruption or prompt injection.
- Accumulating unconstrained state in memory during stream processing, leading to Out-Of-Memory (OOM) crashes.
- Neglecting evaluation benchmarks, deploying models or prompts without quantitative quality scoring.

## Verification checklist
- [ ] Pipeline validates input schemas and filters malformed records.
- [ ] Concurrency and memory utilization remain within provisioned boundaries.
- [ ] Performance and accuracy metrics meet predefined SLA targets.
- [ ] Fallback and retry mechanisms handle external API or network interruptions gracefully.

## Related skills
- retrieval-augmented-generation-rag-pipeline
- prompt-injection-defense-guardrails
- apache-kafka-event-streaming-architecture
- distributed-tracing-opentelemetry-sdk

Versions

The database has no other skills named dataform-gcp-sql-transformations.

Database snapshot: 13.11.2026 · package version 071e0302105f (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100dataform-bigquery
    Expertise in generating clean, correct, and efficient Dataform pipeline code for BigQuery ELT. Use this when creating or modifying Dataform pipelines, actions, or source declarations, when Dataform, SQLX, or BigQuery are mentioned in a transformation, when data needs to be…
  • 95dataform-bigquery
    Expertise in generating clean, correct, and efficient Dataform pipeline code for BigQuery ELT. Use this when creating or modifying Dataform pipelines, actions, or source declarations, when Dataform, SQLX, or BigQuery are mentioned in a transformation, when data needs to be…
  • 90bigquery-and-dataform-platform-engineering
    Guides agents through BigQuery- and Dataform-centered data engineering workflows. Use when designing BigQuery physical models, ingestion boundaries, Dataform transformation workflows, slot or cost controls, and platform decisions across BigQuery, Dataflow, Dataproc, and GCP orchestration services.
  • 87dataform-bigquery-warehouse
    Design, build, extend and review a layered BigQuery warehouse in Google Dataform (SQLX) - declarations, L1 source models, L2 schemas per business unit of the company, L3P presentation tables for machine-to-machine targets (apps, APIs, CRM, reverse ETL) and L3R reporting views…
  • 100sqlanvil-engineering-fundamentals
    Use whenever a task touches a sqlanvil data project (the Dataform fork for PostgreSQL, Supabase and MySQL/MariaDB) - writing or editing .sqlx models, workflow_settings.yaml, .df-credentials.json, definitions/*.js, actions.yaml, Supabase RLS policies, indexes, partitions,…
  • 100gcp-data-engineering
    Guides architecture decisions for data pipelines on Google Cloud — lake/warehouse/lakehouse design, batch vs. streaming, orchestration tool selection (Dataform, Cloud Composer/Airflow, Dataflow/Beam), and BigQuery cost/performance tuning (partitioning, clustering, slots). Use…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.