Skip to content
Skills Library
Programming · Building & coding

data-time-machine

/data-time-machine

[WHAT IT DOES]: Orchestrates zero-downtime, zero-data-loss database schema migrations and data backfills using multi-phase dual-writing, adaptive replica throttling, and cryptographic Merkle-tree parity bisection. [WHEN TO USE]: Use when modifying live production database schemas, altering tables with >100k rows, splitting/merging high-traffic tables, migrating between SQL/NoSQL engines, or refactoring primary keys. [WHEN NOT TO USE]: Do NOT use for blank greenfield database initialization or local offline dev seed generation.

Licence
MIT
Last repository change
03.11.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
4
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 03.11.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

[WHAT IT DOES]: Orchestrates zero-downtime, zero-data-loss database schema migrations and data backfills using multi-phase dual-writing, adaptive replica throttling, and cryptographic Merkle-tree parity bisection. [WHEN TO USE]: Use when modifying live production database schemas, altering tables with >100k rows, splitting/merging high-traffic tables, migrating between SQL/NoSQL engines, or refactoring primary keys. [WHEN NOT TO USE]: Do NOT use for blank greenfield database initialization or local offline dev seed generation.

Original author description

[WHAT IT DOES]: Orchestrates zero-downtime, zero-data-loss database schema migrations and data backfills using multi-phase dual-writing, adaptive replica throttling, and cryptographic Merkle-tree parity bisection. [WHEN TO USE]: Use when modifying live production database schemas, altering tables with >100k rows, splitting/merging high-traffic tables, migrating between SQL/NoSQL engines, or refactoring primary keys. [WHEN NOT TO USE]: Do NOT use for blank greenfield database initialization or local offline dev seed generation.

How it works

  1. 1. When to Activate
  2. 2. The 4-Phase Migration Lifecycle
  3. 3. Reference Files
Original SKILL.md headings · 88 lines, 427 words
SKILL.md preview
---
name: data-time-machine
description: >-
  [WHAT IT DOES]: Orchestrates zero-downtime, zero-data-loss database schema migrations and data backfills
  using multi-phase dual-writing, adaptive replica throttling, and cryptographic Merkle-tree parity bisection.
  [WHEN TO USE]: Use when modifying live production database schemas, altering tables with >100k rows,
  splitting/merging high-traffic tables, migrating between SQL/NoSQL engines, or refactoring primary keys.
  [WHEN NOT TO USE]: Do NOT use for blank greenfield database initialization or local offline dev seed generation.
---

# Data Time-Machine: Zero-Downtime State & Schema Migration Engine

```
  +---------------------------------------------------------------------------------+
  |                               DATA TIME-MACHINE                                 |
  |             4-Phase Expand/Contract & Merkle Tree Parity Engine                 |
  +---------------------------------------------------------------------------------+

  [Live Traffic]
        |
        +-----------------------------------+
        |                                   |
        v                                   v
  +---------------+                 +--------------------+
  | Primary Writes|                 | Dual-Writes Shadow |
  | (Old Schema)  |                 | (New Schema)       |
  +---------------+                 +--------------------+
        |                                   |
        +-------------------+---------------+
                            |
                            v
             +------------------------------+
             | Adaptive Background Backfill |
             | (Replica Lag < 80ms Throttle)|
             +------------------------------+
                            |
                            v
             +------------------------------+
             | Merkle Tree Parity Bisection |
             | O(log N) Hash Tree Compare   |
             +------------------------------+
                            |
            +---------------+---------------+
            |                               |
       [Parity 100%]                  [Hash Divergence]
            |                               |
            v                               v
    [Atomic Cutover]              [Pinpoint Row Delta]
    - Reads -> New Schema         - Auto-Repair Corrupted Rows
    - Drop Old Columns
```

---

## 1. When to Activate

Trigger `data-time-machine` when:
- **Altering High-Volume Tables:** Adding columns, changing data types, or adding unique indexes to production tables where table locking causes downtime.
- **Architectural Normalization:** Splitting a monolithic table (e.g., `orders` $\to$ `orders` + `order_fulfillments`).
- **Engine / Cloud Migrations:** Moving data from self-hosted PostgreSQL to DynamoDB, MongoDB to Postgres, or Cloud SQL to Spanner.
- **Data Parity Auditing:** Proving that an active replica or backfilled cache is 100% synchronized with the primary database.

---

## 2. The 4-Phase Migration Lifecycle

### Phase 1: Expand (Dual-Writing)
1. Add target columns or shadow tables as backward-compatible structures.
2. Deploy application code that executes **Dual Writes** (writes to both old and new schema atomically via transaction or transactional outbox).
3. Confirm write error rate stays at $0.00\%$ over a multi-hour window.

### Phase 2: Idempotent Chunked Backfill
1. Run background batch jobs copying records in primary-key sorted chunks (e.g. 500 rows per batch).
2. Measure replication lag on read replicas continuously. If lag exceeds 80ms, pause backfill workers adaptively.
3. Use upsert semantics (`INSERT ... ON CONFLICT DO UPDATE`) to prevent overwriting newer dual-written updates.

### Phase 3: Merkle Tree Parity Bisection
1. Run `verify_merkle_sync.py`:
   ```bash
   python scripts/verify_merkle_sync.py --demo
   ```
2. The auditor constructs binary Merkle trees over both old and new datasets.
3. If roots match: Mathematical proof of $100.00\%$ data parity.
4. If roots differ: Recursively descends the tree in $O(\log N)$ steps to output the exact primary keys and data diffs.

### Phase 4: Contract (Cutover & Cleanup)
1. Switch application reads to the new schema.
2. Keep dual-writes active for a 24-hour observation buffer.
3. Disable writes to the legacy schema.
4. Safely drop obsolete legacy columns and tables.

---

## 3. Reference Files
- [verify_merkle_sync.py](./scripts/verify_merkle_sync.py): O(log N) cryptographic tree auditor.
- [migration_plan.md](./templates/migration_plan.md): Step-by-step production rollback and cutover checklist.
Files in this package (4)
  • .gitignore 1 KB
  • LICENSE 1 KB
  • README.md 4 KB
  • SKILL.md 5 KB

Versions

The database has no other skills named data-time-machine.

Database snapshot: 13.11.2026 · package version 84b5dca809cf (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100database-migrations
    Plans and writes zero-downtime database schema migrations (expand/contract, safe indexes, batched backfills, rollback). Use when adding, renaming or dropping columns or tables, changing types or constraints, adding indexes, or backfilling data on a live database.
  • 100database-migration-patterns
    Manage database schema changes safely with migration tools, zero-downtime strategies, and rollback procedures. Covers Alembic, SQL migrations, data migrations, and testing strategies. Triggers on database migration, schema changes, or Alembic configuration requests.
  • 100database-migration
    Database migrations with Alembic, rollbacks, and zero-downtime deployment. Use for schema changes.
  • 100database-migration
    Execute database migrations across ORMs and platforms with zero-downtime strategies, data transformation, and rollback procedures. Use when migrating databases, changing schemas, performing data transformations, or implementing zero-downtime deployment strategies.
  • 100database-migration
    Execute database migrations across ORMs and platforms with zero-downtime strategies, data transformation, and rollback procedures. Use when migrating databases, changing schemas, performing data transformations, or implementing zero-downtime deployment strategies.
  • 100database-migration
    Execute database migrations across ORMs and platforms with zero-downtime strategies, data transformation, and rollback procedures. Use when migrating databases, changing schemas, performing data transformations, or implementing zero-downtime deployment strategies.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.