Skip to content
Skills Library
Data & analytics · Analysis

data-map

/data-map

Produce a cross-table data health map for the active dataset: row counts, date alignment, join rates, relationships, opening hypothesis. Trigger on dataset-wide open questions: "tell me about this data", "what's in here", "give me an overview", "map out the data", "what do I have", "what does this data look like". NOT for table-scoped questions (data-quality-check), /explore follow-ups, schema listing (/data), or deep profiles (data-profiling).

Licence
MIT
Last repository change
26.08.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 26.08.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Produce a cross-table data health map for the active dataset: row counts, date alignment, join rates, relationships, opening hypothesis. Trigger on dataset-wide open questions: "tell me about this data", "what's in here", "give me an overview", "map out the data", "what do I have", "what does this data look like". NOT for table-scoped questions (data-quality-check), /explore follow-ups, schema listing (/data), or deep profiles (data-profiling).

Triggers from the description

tell me about this datawhatgive me an overviewmap out the datawhat do I havewhat does this data look like
Original author description

Produce a cross-table data health map for the active dataset: row counts, date alignment, join rates, relationships, opening hypothesis. Trigger on dataset-wide open questions: "tell me about this data", "what's in here", "give me an overview", "map out the data", "what do I have", "what does this data look like". NOT for table-scoped questions (data-quality-check), /explore follow-ups, schema listing (/data), or deep profiles (data-profiling).

How it works

  1. Purpose
  2. When to Fire
  3. Instructions
  4. Output Format
  5. Budget and Safeguards
  6. Anti-Patterns
  7. Interaction With Other Skills
Original SKILL.md headings · 228 lines, 1,668 words
SKILL.md preview
---
name: data-map
description: >-
  Produce a cross-table data health map for the active dataset: row counts, date alignment, join
  rates, relationships, opening hypothesis. Trigger on dataset-wide open questions: "tell me about
  this data", "what's in here", "give me an overview", "map out the data", "what do I have", "what
  does this data look like". NOT for table-scoped questions (data-quality-check), /explore
  follow-ups, schema listing (/data), or deep profiles (data-profiling).
---

# Skill: Data Map

## Purpose
Answer the broadest possible question — "tell me about this data" — with the broadest substantive answer: cross-table health, relationships, date alignment, and an opening analytical thread. This is first-contact dataset exploration, not a schema dump and not a steering question.

## When to Fire

**Fires on dataset-wide open questions:**
- "tell me about this data / the data / this dataset / the dataset"
- "what's in here / what's in this data / what's in the data"
- "give me an overview / give me the map / map out the data"
- "what do I have / what do we have here"
- "what does this data look like / show me what we've got"
- Any open question that references the dataset as a whole

**Does NOT fire when:**
- The question names a specific table → use `data-quality-check`
- User invoked `/explore` or is mid-exploration within an already-mapped dataset → use `explore`
- User invoked `/data` or `/data {table}` → use `data-inspect` (schema-only)
- User invoked `/run-pipeline` or a specific analysis → run the pipeline

**Fires regardless of:**
- Whether the dataset was recently profiled (data-profiling populates `.knowledge/`, this skill produces a live report)
- Whether the user explicitly asked for DQ (it's implicit in "tell me about")

## Instructions

### Step 0 — Resolve active dataset

1. Read `.knowledge/active.yaml` to get `active_dataset`. If missing, halt and tell the user to run `/connect-data` (knowledge-bootstrap can create the rest of the tree).
2. Read `.knowledge/datasets/{active}/manifest.yaml` for connection type and local paths.
3. Read `.knowledge/datasets/{active}/schema.md` for table list and column types.
4. Read `.knowledge/datasets/{active}/quirks.md` to surface known gotchas inline.
5. Verify connectivity through the session's active data connection (a connected warehouse, local DuckDB, or the CSV files, in that order of preference). Announce which source is active in one line.

### Step 1 — Table inventory and PK health

For every table in schema.md, run:

```sql
SELECT
  COUNT(*)                                       AS row_count,
  COUNT(DISTINCT {pk_col})                       AS distinct_pk,
  COUNT(*) - COUNT(DISTINCT {pk_col})            AS pk_dupes,
  SUM(CASE WHEN {pk_col} IS NULL THEN 1 ELSE 0 END) AS pk_nulls
FROM {schema}.{table};
```

PK inference order: (1) `{table_singular}_id` (e.g., `order_id` for `orders`), (2) column literally named `id`, (3) if neither, note "no single PK detected — composite candidate" and check the most plausible composite. For `order_items` specifically, use `order_item_id` if present, else `(order_id, product_id)`.

### Step 2 — Date range + primary timestamp

For each table, identify the primary timestamp column (prefer `*_timestamp`, then `*_date`, then `created_at`/`started_at`/`session_start`/`event_timestamp`). Run:

```sql
SELECT MIN({ts}) AS min_ts, MAX({ts}) AS max_ts, COUNT(DISTINCT DATE({ts})) AS distinct_days
FROM {schema}.{table};
```

Tables with no timestamp (e.g., `products`, `calendar`, `promotions` if static): note as "static/reference."

### Step 3 — Column completeness (null rates)

For every column in every non-reference table, compute null %. Use a single query per table:

```sql
SELECT
  SUM(CASE WHEN {col_1} IS NULL THEN 1 ELSE 0 END) * 100.0 / COUNT(*) AS {col_1}_null_pct,
  SUM(CASE WHEN {col_2} IS NULL THEN 1 ELSE 0 END) * 100.0 / COUNT(*) AS {col_2}_null_pct,
  ...
FROM {schema}.{table};
```

Flag any column with null% > 5%. For DOUBLE foreign-key columns (e.g., `orders.promo_id`), high nulls may be expected (not every order used a promo) — note but don't alarm.

### Step 4 — Foreign key inference and join-rate matrix

Infer FKs from naming: a column named `{entity}_id` in table T likely references the `{entity}s` (or `{entity}`) table's PK. For each inferred FK, run:

```sql
SELECT
  COUNT(*)                                          AS child_rows,
  COUNT(DISTINCT c.{fk_col})                        AS distinct_fks,
  SUM(CASE WHEN p.{pk_col} IS NULL THEN 1 ELSE 0 END) * 100.0 / COUNT(*) AS orphan_pct
FROM {schema}.{child_table} c
LEFT JOIN {schema}.{parent_table} p
  ON c.{fk_col} = p.{pk_col};
```

Expected FKs in a typical e-commerce schema (illustrative example):
- `orders.user_id` → `users.user_id`
- `order_items.order_id` → `orders.order_id`
- `order_items.product_id` → `products.product_id`
- `events.user_id` → `users.user_id`
- `events.session_id` → `sessions.session_id`
- `events.product_id` → `products.product_id` (nullable — only product-related events)
- `sessions.user_id` → `users.user_id`
- `memberships.user_id` → `users.user_id`
- `nps_responses.user_id` → `users.user_id`
- `support_tickets.user_id` → `users.user_id`
- `support_tickets.order_id` → `orders.order_id` (nullable)
- `experiment_assignments.user_id` → `users.user_id`
- `experiment_assignments.experiment_id` → `experiments.experiment_id`
- `orders.promo_id` → `promotions.promo_id` (nullable)

Generalize by scanning column names for `*_id` patterns — the FK list above is one example shape, not a required schema.

### Step 5 — Cross-table date alignment

Build a table: each row is a table, columns are (min_ts, max_ts, distinct_days). Identify:
- **Pivot range:** the intersection of all core fact tables' date ranges (the window where all data coexists).
- **Misalignments:** tables that start late or end early vs. the pivot. Flag if a fact table's range is <80% of the pivot.
- **Calendar overlap:** if a `calendar` table exists, confirm its range covers the fact tables.

### Step 6 — Relationship diagram

Render an ASCII DAG. Use this layout pattern:

```
              users ──┬── sessions ── events
                      ├── orders ── order_items ── products
                      ├── memberships
                      ├── nps_responses
                      ├── support_tickets ──(nullable)── orders
                      └── experiment_assignments ── experiments
                                     
              promotions ──(nullable)── orders
              calendar  (reference / date spine)
```

Only include tables that actually exist in the active schema.

### Step 7 — Surface quirks

From `quirks.md`, inline any noted gotchas in the relevant section of the output. If quirks.md is empty or placeholder-only, note: "No quirks documented yet — run `/data-profiling` for deep column-level analysis."

### Step 8 — Opening thread to pull

Close with one concrete analytical thread. Prefer a thread that:
1. Matches the dataset's apparent business theme (e.g. e-commerce → funnel, membership, support, experiments; SaaS → activation, usage, churn)
2. Uses the data you just confirmed is present and joinable
3. Points at a specific decision or question (not "segment by X")

Pick a thread grounded in the data's strongest signal — a visible spike, a clear cohort split, or a conspicuous gap. For example: if a `support_tickets` table shows a sharp weekly-volume spike, a strong opener is *"ticket volume jumped in week N — want to pull the thread on what drove it?"*

### Step 9 — Log every query

After running, append each query to `.knowledge/query-log.md` in the working folder: a dated line with the purpose, the SQL, and the row count.

## Output Format

```markdown
# Data Map: {display_name}

**Source:** {connection_type} @ {path}  •  **Active:** {min_date} → {max_date}  •  **Tables:** {n}

## Cross-Table Health

| Table | Rows | PK Unique | Primary Timestamp | Date Range | Days | Completeness |
|-------|-----:|:---------:|:------------------|:-----------|-----:|:-------------|
| users | 50,000 | ✅ | signup_timestamp | 2024-01-01 → 2024-12-31 | 366 | A |
| orders | 47,199 | ✅ | order_timestamp | 2024-01-02 → 2024-12-31 | 364 | A |
| ... | | | | | | |

**Legend:** ✅ PK unique, no nulls  •  ⚠ PK dupes or nulls  •  Completeness A (<1% null) / B (1-5%) / C (5-20%) / D (>20%)

## Relationship Map

[ASCII DAG]

## Join-Rate Matrix

| Child → Parent | Child Rows | Distinct FKs | Orphans |
|----------------|-----------:|-------------:|--------:|
| orders.user_id → users | 47,199 | 28,403 | 0.0% |
| ... | | | |

**Orphans > 1%** are flagged. Nullable FKs (e.g., `orders.promo_id`) are noted but not flagged.

## Date Alignment

- **Pivot range:** YYYY-MM-DD → YYYY-MM-DD ({N} days, all core facts present)
- **Misalignments:** [list any table with range <80% of pivot, or note "All core tables aligned."]
- **Calendar coverage:** [confirm calendar spans the pivot, or flag]

## Completeness Flags

[Any column with >5% nulls, excluding expected-nullable FKs. If none, write "No columns exceed 5% null threshold (excluding nullable FKs)."]

## Quirks

[From quirks.md, or "No quirks documented — run `/data-profiling` for deep profiling."]

## Thread to Pull

[One concrete analytical opener, 1-2 sentences, grounded in the data.]

---

_Generated by `data-map` skill. For a deeper statistical profile, run `/data-profiling`. For one-table DQ, ask about the table directly. For guided exploration, use `/explore`._
```

## Budget and Safeguards

- **Query budget:** ~2-4 queries per table + one per inferred FK. For a 13-table dataset this is typically 40-70 queries. All logged.
- **Large tables (>100M rows):** Before running null-rate or PK-dupe checks, ask the user: "Table X has {N} rows — run full probe or sample at {sample_pct}%?" Default to sampling at 1% for >100M-row tables, full scan otherwise.
- **Connection failures:** Fall back to the next available source (warehouse → local DuckDB → CSV) and note which source is active.
- **Schema drift:** If a column referenced in schema.md doesn't exist in the live data, catch the error, note it in the Completeness Flags section, and continue.
- **Reference tables (no timestamp):** `calendar`, `products`, `promotions`, `experiments` are static. Skip date range and note as "reference."

## Anti-Patterns

1. **Never answer with schema-only.** Schema.md is an input, not the output. The deliverable includes live probes.
2. **Never ask a steering question instead of running the map.** The whole point is that the broadest question gets the broadest substantive answer. Asking "what do you want to explore?" violates this skill.
3. **Never skip date alignment.** The "data starts 3 months apart" trap is the most common way analyses break — always check.
4. **Never skip the FK matrix.** Orphan rates determine whether joins will silently drop data.
5. **Never fabricate PKs or FKs.** If inference is ambiguous, say so and check the most plausible candidate.
6. **Never produce the thread-to-pull as a vague suggestion.** It should cite a concrete signal visible in the data you just probed.
7. **Never skip the quirks section.** Even "none documented yet" is a useful signal — it tells the student this dataset hasn't been explored in depth.

## Interaction With Other Skills

- **Runs alongside `data-quality-check`**: this skill is DQ at the dataset level; table-scoped DQ still fires on table-specific questions.
- **Supersedes `data-inspect`**: if the user's question is open-ended ("tell me about") rather than `/data`-commanded, run `data-map`, not `data-inspect`.
- **Does not replace `data-profiling`**: that skill is a deep, per-column statistical profile meant to run once after connection. `data-map` is the conversational first-contact answer.
- **Does not replace `/explore`**: `/explore` is for iterative, user-steered exploration within an already-mapped dataset.

Versions

RepositoryScoreLicence
This page · ai-analyst-lab/ai-analyst-plugin100MIT
ai-analyst-lab/ai-analyst87MIT

Database snapshot: 13.11.2026 · package version 61ad50af4a52 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 1 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100data-inspect
    Show the active dataset's schema: tables, columns, row counts, relationships. Trigger on "/data" or "/data {table}", "what tables do I have?", "show me the schema", "what columns are in the users table?", "list tables", "describe the data", "data dictionary". DISAMBIGUATION:…
  • 87data-inspect
    Show the active dataset's schema — tables, columns, row counts, and relationships. Optionally drill into a specific table. Use this skill whenever the user invokes `/data` or `/data {table}`, or asks questions like "what tables do I have?", "show me the schema", "what's in this…
  • 100explore
    Quick, interactive data exploration without the full pipeline. Trigger on "/explore", "let me explore the data", "browse the data", "show me a sample of the data", "what does the distribution look like?", "explore this table". DISAMBIGUATION: interactive poke-around within a…
  • 98eda-report
    Run exploratory data analysis on a dataset and produce a structured report. Use when the user says "explore this dataset", "EDA on X", "analyze this data", "what's in this dataset", "summarize this data", "first look at the data", "understand this dataset before modeling", "data…
  • 100profiling-tables
    Deep-dive data profiling for a specific table. Use when the user asks to profile a table, wants statistics about a dataset, asks about data quality, or needs to understand a table's structure and content. Requires a table name.
  • 100data-profiling
    Deep-profile the active dataset: distributions, temporal patterns, correlations, completeness gaps, anomalies. Use after connecting a new dataset. Trigger on "profile this data", "deep-profile the dataset", "run a data profile", "check distributions", "find anomalies in the…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.