Skip to content
Skills Library
Data & analytics · Automation

ability

/ability

Entity resolution and record matching — find which records across one or more datasets refer to the same real person, company, place, or thing, then merge them into one trusted "golden record" with full source history. Use this skill whenever the user wants to deduplicate a list, merge customer/patient/contact/vendor files, match records without a shared ID, link addresses, clean a CRM, build a master data (MDM) pipeline, score fuzzy name matches, choose blocking keys, set match thresholds, or apply Fellegi-Sunter / probabilistic record linkage — even if they never say "entity resolution".

Licence
MIT
Last repository change
04.11.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
3
Scripts
1
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 04.11.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What to check before installing
  • 1 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: scripts/fs_score.py

What it does and when to use it

Entity resolution and record matching — find which records across one or more datasets refer to the same real person, company, place, or thing, then merge them into one trusted "golden record" with full source history. Use this skill whenever the user wants to deduplicate a list, merge customer/patient/contact/vendor files, match records without a shared ID, link addresses, clean a CRM, build a master data (MDM) pipeline, score fuzzy name matches, choose blocking keys, set match thresholds, or apply Fellegi-Sunter / probabilistic record linkage — even if they never say "entity resolution".

Triggers from the description

golden recordentity resolution
Original author description

Entity resolution and record matching — find which records across one or more datasets refer to the same real person, company, place, or thing, then merge them into one trusted "golden record" with full source history. Use this skill whenever the user wants to deduplicate a list, merge customer/patient/contact/vendor files, match records without a shared ID, link addresses, clean a CRM, build a master data (MDM) pipeline, score fuzzy name matches, choose blocking keys, set match thresholds, or apply Fellegi-Sunter / probabilistic record linkage — even if they never say "entity resolution".

How it works

  1. Before starting: the alignment check
  2. The 5-stage pipeline
  3. Validating results
  4. Evidence tags
  5. Checkpoint review
  6. Output for a matching job
  7. Privacy
Original SKILL.md headings · 165 lines, 1,258 words
SKILL.md preview
---
name: ability
description: Entity resolution and record matching — find which records across one or more datasets refer to the same real person, company, place, or thing, then merge them into one trusted "golden record" with full source history. Use this skill whenever the user wants to deduplicate a list, merge customer/patient/contact/vendor files, match records without a shared ID, link addresses, clean a CRM, build a master data (MDM) pipeline, score fuzzy name matches, choose blocking keys, set match thresholds, or apply Fellegi-Sunter / probabilistic record linkage — even if they never say "entity resolution".
metadata:
  author: Garrett C. Gauthier
  version: "1.0"
  layer: session (active only while loaded; removing the skill removes the behavior)
---

# Ability — Entity Resolution

Ability turns messy, overlapping records into one clean, traceable set of entities.
It works in five stages and never destroys source data: every merged record keeps
links back to the originals it came from.

Record linkage is the matching math (does record A match record B?).
Entity resolution is the whole job: matching, clustering matches into entities,
scoring confidence, and keeping provenance. Do the whole job.

## Before starting: the alignment check

Messy requests are normal. Read typos and shorthand for intent and never comment
on spelling. Before real work, state three short lines so the user can correct
course cheaply:

```
SAID:     <what the user literally asked>
INFERRED: <the actual problem to solve>
ASSUMED:  <the main assumption you are making — ask at most one question if truly blocked>
```

Skip this for quick questions; use it when you are about to process data or design a pipeline.

## The 5-stage pipeline

### 1. Profile
Look at every field before matching anything. Report per field: null rate,
distinct count, top values, format patterns, and encoding problems (UTF-8 vs Latin-1).
Profiling tells you which fields are trustworthy enough to match on and which
values are so common (e.g. "Smith", "123 Main St") that agreement means little.

### 2. Standardize
Normalize before comparing, or the scores will be wrong: lowercase, strip
punctuation, expand abbreviations (St → Street), parse names into parts, normalize
phone numbers to digits, normalize company suffixes (Inc., Ltd, LLC), and
standardize addresses (postal verification where available).

Messy real data almost always also needs:
- **Nickname table** (Bob/Robert, Peggy/Margaret, Sasha/Alexander, Trish/Patricia) and
  initials ("J.") treated as compatible with any name starting with that letter.
- **Swapped name fields** — also compare first↔last reversed.
- **Ambiguous dates** — "04/05/1990" could be April 5 or May 4. Keep both readings and
  treat a day/month swap as a partial match, not a conflict.

### 3. Block
Comparing every record to every other is N² — 1M records means ~500 billion pairs.
Blocking only compares records that share a key. Use several passes with
different keys so a typo in one field does not hide a true match:

- Phonetic surname (Soundex / NYSIIS) + birth year
- Postcode prefix + house number
- Last 4 of an ID + ZIP
- Phone or email
- For text-heavy data: MinHash LSH on token sets, or embeddings + nearest-neighbour search (FAISS/HNSW)

Measure it:
- **Reduction ratio** = 1 − (pairs compared ÷ all possible pairs) — aim for > 99%
- **Pair completeness** = true matches that land in some block ÷ all true matches — aim for ≥ 95%

### 4. Score — the 3-step cascade
Run the cheap, certain checks first and the expensive ones only where needed.

1. **Deterministic**: a reliable unique ID agrees (national ID, tax ID, account number, verified email) → match.
2. **Fuzzy**: no ID match → compare fields with the right algorithm per field
   (see table) and combine them with Fellegi-Sunter weights.
3. **Machine learning / human review**: still ambiguous → a classifier
   (e.g. logistic regression on the field-similarity features) if labeled data exists,
   otherwise route to a human review queue.

| Field type | Use |
|---|---|
| Person names | Jaro-Winkler (rewards matching prefixes), plus nickname lists |
| Company names | Token sort ratio (word order doesn't matter) after suffix cleanup |
| Addresses | Levenshtein after standardization; house and unit numbers must agree exactly |
| IDs, phone, email | Exact after normalization |
| Long text | Jaccard / cosine on tokens |

**Fellegi-Sunter in brief** — for each field k:
- m = P(field agrees | records truly match) — how reliable the field is
- u = P(field agrees | records don't match) — how often it agrees by chance
- agree weight = log2(m/u); disagree weight = log2((1−m)/(1−u))
- total score = sum of the weights for each field's outcome

Rare values carry more evidence: agreeing on "Lebowski" (tiny u) is far stronger
than agreeing on "Smith". Use value-specific u where you can. In practice: a rare
full name plus an exact birth date is strong evidence even when address, phone and
email all changed (the person moved); a common name plus birth date is not.

To score pairs, run `scripts/fs_score.py` instead of computing by hand — it applies
the weights, converts the score to a match probability given prior odds, and assigns
the decision band. Run `python scripts/fs_score.py --help` for usage.
Full formulas, the worked example, and m/u estimation (including EM) are in
`references/methods.md`; read it when the user wants the math or needs to tune weights.

**Two traps the score alone won't catch:**
- **Households.** Twins, spouses, and roommates share surname, address, and sometimes
  birth date, so their score can look like a near-certain match. If first names differ
  (and aren't nicknames of each other) and personal contacts (email, phone) conflict,
  keep them apart or send them to review — whatever the score says.
- **Placeholder values.** Dates like 2000-01-01 or 1900-01-01, phones like 555-0000, and
  emails like none@none.com are filler, not evidence. Profiling (stage 1) should flag
  values that repeat suspiciously; give them no weight.

**Decision bands** (tune on labeled data; these are starting points):

| Match probability | Action |
|---|---|
| ≥ 0.95 | Auto-match |
| 0.80 – 0.94 | Human review queue |
| < 0.80 | Non-match |

### 5. Merge — survivorship and golden records
Group matched pairs into entities (connected components / graph clustering — watch
for chains where A~B and B~C but A≁C; split those for review). For each entity build
one golden record by choosing each field with a stated rule:

- **Most recent** (latest timestamp) — for addresses, phone
- **Most complete** (non-null, longest) — for names, descriptions
- **Trusted source** (system ranking) — for IDs, legal names

Never overwrite or delete originals. Store: golden record ID → list of source record
IDs, the rule that picked each field, and the match score that joined them.

## Validating results

Ask for, or help build, a small labeled set (a few hundred pairs marked match/non-match).
Report precision, recall, and F1 on it, plus reduction ratio and pair completeness
for blocking. Thresholds without validation are guesses — say so when there is no
labeled set.

## Evidence tags

When claiming a result, say how you know it:
- **[DOC]** — from a spec or documentation, not run
- **[RAN]** — executed in this session
- **[PASS]** — checked against labeled/test data

## Checkpoint review

On long jobs (roughly every five working turns, when the user reports a mistake, or
when they type "checkpoint" or "dream"): pause the project work and briefly review
your own recent turns. List each mistake or drift from the user's stated constraints,
its cause, and the fix you will apply going forward. Then resume. Keep the review out
of deliverables — it is for the conversation, not the user's data or documents.

## Output for a matching job

End a completed job with:

```
ENTITIES: <input records> → <resolved entities>  (<duplicates found>)
BANDS: auto <n> | review <n> | non-match <n>
QUALITY: precision <x> | recall <x> | F1 <x>   (or "not validated — no labeled set")
FILES: <where the golden records, crosswalk, and review queue were saved>
```

## Privacy

Matching often involves personal data. Process it locally, don't paste identifiers
into URLs or external services, and when linking across organizations suggest
privacy-preserving linkage (Bloom-filter encoding of names, compared with Dice
similarity) — details in `references/methods.md`.
Files in this package (3)
  • SKILL.md 8 KB
  • references/methods.md 6 KB
  • scripts/fs_score.py 4 KB

Versions

The database has no other skills named ability.

Database snapshot: 13.11.2026 · package version b5da4370f764 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90master-data-and-entity-resolution
    Guides agents through master data and entity resolution workflows. Use when matching identities across systems, defining canonical entities, resolving duplicates, or building golden records for shared downstream use.
  • 85Intelligent Lookup Wizard
    Master fuzzy matching and intelligent lookup configuration for data harmonization and record matching. Use this when you need to match company names across sources, reconcile customer addresses, match product descriptions, deduplicate data, or harmonize master data from multiple…
  • 100entity-resolution
    Entity resolution (ER) for multi-source RAG and knowledge graphs. Dedup entities across documents ("Acme Corp" = "Acme, Inc." = "Acme Corporation"). Covers blocking, probabilistic linking (Splink), Dedupe.io, embedding-based matching, LLM-assisted resolution with rules,…
  • 95cross-system-identity-resolution
    Use when stitching the same real-world entity — a customer account — across multiple source systems into one conformed spine; e.g. Salesforce + Planhat + Intercom + Slack. Operationalizes the deterministic-keys-before-fuzzy best practice: inventory candidate keys, build the…
  • 100crm-data-cleanup
    Cleans up CRM data from HubSpot, Salesforce, Pipedrive, Zoho, Close, Attio, or any CRM CSV export. Finds duplicate contacts and companies with fuzzy matching (not just exact email), normalizes emails, phones to E.164, names, domains, states, and countries, flags stale, invalid,…
  • 100company-identity-resolution
    Resolve a messy company name to the right domain and canonical record, and say how confident the match is. Use when a list has company names but no domains, when names are ambiguous, or when enrichment is matching the wrong company.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.