Skip to content
Skills Library
Data & analytics · Conversion & formats

convert-knowledge-base

/convert-knowledge-base

Turn an existing knowledge base (a spreadsheet, database, set of YAML or JSON files, wiki, or a collection of repositories) into a Mech: survey it, map it to records, shape the schema, write a conversion script on the Mech's `just convert` helper, carry its facts over with checked quotes, and curate what it never recorded. Use when someone already has a knowledge base and wants it as a Mech, or asks to import, migrate or convert one. For a Mech started from nothing, use make-mech.

Repository
mechmaker ↗
Licence
BSD-3-Clause
Last repository change
23.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
1
Deprecation

No signals detected. Repository changed on 23.10.2026, without references to retired models or APIs.

Licence

BSD-3-Clause, otwarta.

What it does and when to use it

Turn an existing knowledge base (a spreadsheet, database, set of YAML or JSON files, wiki, or a collection of repositories) into a Mech: survey it, map it to records, shape the schema, write a conversion script on the Mech's `just convert` helper, carry its facts over with checked quotes, and curate what it never recorded. Use when someone already has a knowledge base and wants it as a Mech, or asks to import, migrate or convert one. For a Mech started from nothing, use make-mech.

Original author description

Turn an existing knowledge base (a spreadsheet, database, set of YAML or JSON files, wiki, or a collection of repositories) into a Mech: survey it, map it to records, shape the schema, write a conversion script on the Mech's `just convert` helper, carry its facts over with checked quotes, and curate what it never recorded. Use when someone already has a knowledge base and wants it as a Mech, or asks to import, migrate or convert one. For a Mech started from nothing, use make-mech.

How it works

  1. 1. Survey the knowledge base
  2. 2. Map it to a Mech
  3. 3. Generate and design
  4. 4. Write the conversion script
  5. 5. Trial
  6. 6. Curate what the source lacks
  7. 7. Batch
  8. 8. Record the source
  9. Report
Original SKILL.md headings · 207 lines, 1,576 words
SKILL.md preview
---
name: convert-knowledge-base
description: >-
  Turn an existing knowledge base (a spreadsheet, database, set of YAML or
  JSON files, wiki, or a collection of repositories) into a Mech: survey it,
  map it to records, shape the schema, write a conversion script on the
  Mech's `just convert` helper, carry its facts over with checked quotes,
  and curate what it never recorded. Use when someone already has a
  knowledge base and wants it as a Mech, or asks to import, migrate or
  convert one. For a Mech started from nothing, use make-mech.
---

# Convert a knowledge base into a Mech

A conversion keeps the old knowledge base as it is. The old one becomes a
source. The new Mech holds records that quote it, and the things it never
said are curated from elsewhere or marked as gaps.

| Step | Who | Output |
|---|---|---|
| 1. Survey the knowledge base | you | the brief's first section |
| 2. Map it to a Mech | you, with `survey-domain` | the rest of the brief, `answers.yml` |
| 3. Generate and design | Copier, then you with `design-mech-schema` | the repository, `docs/DOMAIN.md`, the schema |
| 4. Write the conversion script | you | `scripts/convert_<source>.py` |
| 5. Trial | you | three to five DRAFT records, `just qc-full` green |
| 6. Curate what the source lacks | you, with the Mech's `curate-record` | PROPOSED records |
| 7. Batch | you and the person | reviewable pull requests |
| 8. Record the source | you | `curation/source_queue.tsv`, `registry/<slug>.md` |

Steps 3 and later follow `make-mech` where this skill says nothing.

## 1. Survey the knowledge base

Measure it. Do not describe it from its README.

- **Where and what format.** One file, a database, an export, many
  repositories. How to read it without a person's help.
- **Entries.** Count them. Count how many follow the main layout and how
  many do not. The ones that do not are out of scope for the first
  conversion. Say so in the brief.
- **Fields.** For each field: how many entries fill it, what its values
  look like, and whether they are identifiers, free text or numbers.
- **Identifiers.** Which field names an entry. Check a sample of the
  identifiers it uses for other things against their registry or ontology.
  Report how many fail. The `search-bioregistry` skill names an unfamiliar
  prefix, checks a sample of CURIEs against their patterns in one call, and
  turns IRIs into CURIEs. In the Monarch ingests, 13 of 40 infores ids were
  not in the Biolink catalog.
- **Citations.** What form they take: PMIDs, DOIs, free text, none.
- **License.** Of the knowledge base, and of what it describes, separately.
- **Listings.** Search KG-Registry for the knowledge base by name, and for
  its sources (`search-kg-registry`). A listing gives its products, what
  it is built from, and which graphs already ingest it: neighbors the
  brief names. Search MechRegistry too (`search-mechregistry`): a Mech
  that already converts one of its sources has a script to port.
- **Another Mech.** The knowledge base may itself be a Mech. Read it at a
  pinned commit or release, keep the evidence its records carry (the
  reference and the quote), and read its data license.
  `search-mechregistry` has the details.

Pick a trial set of three to five entries that span it: one typical, one
with the most structure, one odd.

### License is a hard stop

Read the license of what you will copy. If it does not allow
redistribution of the text you would put in records, stop. Tell the person
what the license says, and convert nothing until they decide. A Mech is
published, and its records carry quotes.

## 2. Map it to a Mech

Run `survey-domain` with the knowledge base as the main source. Decide:

- **One record is one what.** Usually one entry. Sometimes an entry is a
  field of a larger record, or one entry is several records. Write the rule.
- **The id.** An ontology CURIE if one keys the entity; else the source's
  own identifier in the Mech's namespace, when it is stable and unique; else
  leave `id` out and `convert` mints `<slug>:<uuid>`. A source identifier
  that is not the id goes in a slot the schema adds for it, or, when it
  names the same entity in another database (an OMIM column, a Wikidata
  column), in the record's `mappings`: turn on `sssom`, and see
  `search-mappings` for the predicate each column means. `convert` skips
  an entry whose id an earlier entry or a record on disk already holds.
  Never as a synonym. Synonyms are names. A record keyed by an ontology
  CURIE names it again in `record_term`, with the ontology's label; the
  Mech refuses the record without it.
- **Field by field.** Each source field goes to a section, a slot, or
  nowhere, and the brief says which. A field with no
  home is a design question now, and costly later.
- **What the source never records.** This is often what the Mech is for.
  List it. It becomes curation, not conversion.
- **Already ingested?** A knowledge base that is a public database may
  have a Monarch ingest that releases it as KGX rows every month
  (`ingest-structured-data`, `ingest_catalog.py search <name>`). Reading a
  locked release is cheaper, and reproducible, where the rows carry what
  the records need. A source that needs a repeatable ingest of its own
  can get one with Koza (`koza_ingests`) instead of a one-off script.

## 3. Generate and design

`make-mech` steps 2 and 3, then `design-mech-schema`, with the source's
fields as the inventory. If the knowledge base has a schema of its own
(LinkML, JSON Schema, OWL, XSD, a frictionless package), import it with
`just import-schema` instead of rebuilding it field by field. Put `repository` or `source` and a version (a
commit, a release, an export date) on the record when the source changes
over time. Evidence will be pinned to it.

## 4. Write the conversion script

Every Mech has `src/<slug>/convert.py`. The script reads the source and
yields one `Entry` (the source key and the record) or one `Skip` (the key
and why) per entry. The helper does the rest the same way every time: DRAFT
status, a CREATE event naming the entry, validation through
`write_validated_record`, never overwriting, a report of every skip, a dry
run unless `--apply`, and one history record per applied batch.

```python
from <slug>.convert import Entry, Skip, run

def entries():
    for row in read_the_source():
        if not row["name"]:
            yield Skip(row["key"], "no name")
            continue
        yield Entry(row["key"], {"id": ..., "name": row["name"], ...})

if __name__ == "__main__":
    raise SystemExit(run(entries(), source="Old KB", script=__file__))
```

```bash
just convert scripts/convert_old_kb.py --limit 5          # dry run
just convert scripts/convert_old_kb.py --only KEY         # one entry
just convert scripts/convert_old_kb.py --model <id> --apply
```

Rules for the script:

- **Quote what you copy.** Each fact copied from the source gets an
  evidence item whose snippet is the source's own text: the line, the cell,
  the sentence. Point the reference at a fixed version:
  `url:https://raw.githubusercontent.com/<org>/<repo>/<commit>/<path>` for
  a repository, a DOI or release URL for an export. `just qc-full` then
  checks every copied fact. A local file only you have cannot be checked by
  anyone else.
- **Look up, never guess.** Map values to CURIEs with lookups (OLS, the
  source's registry, the Bioregistry for prefixes). A value that does not map keeps its text as
  `preferred_term` and gets a `CURATION_TODO` discussion. An identifier the
  registry does not know is left out, with a `CURATION_TODO`.
- **Write the gaps down.** Each thing the source never records becomes a
  `CURATION_TODO` on the record, attached to the section it belongs in.
- **Skip loudly.** An entry the script cannot convert is a `Skip` with a
  reason. A network failure is a skip too, and a rerun picks it up.
- **Cache what you fetch** under `build/`, so reruns are cheap and do not
  hit rate limits.

## 5. Trial

Convert the trial set with `--apply`. Then:

```bash
just qc-full
```

It must pass. Read each record in full. Then run the Mech's `review-record`
on one. Fix the schema now, while there are five records, with
`extend-schema`, and reconvert: delete the trial records and run again.

## 6. Curate what the source lacks

Use the Mech's `curate-record`. Set the curated field, then add its quote:

```bash
just add-evidence data/.../x.yaml --at 'section[0].field' --ref REF \
  --snippet "..." --source SOURCE_KIND --apply
```

`add-evidence` fetches the source and refuses a quote it cannot find.

- A quote that crosses a link or markup in the source fails. Quote the
  parts on each side, joined with ` ... `.
- A page that refuses scripted requests is not quoted from memory. The
  field stays empty and its discussion becomes a `KNOWLEDGE_GAP` that says
  what was tried.
- A value no source states stays unset. Do not fill a field because its
  likely value is obvious.

Close each discussion the curation answered: `status: RESOLVED`, with
`resolved_date` and `resolution_note`. A record whose sections are filled
or explained is `PROPOSED`. Add a history record per record.

## 7. Batch

Convert the rest in batches of a few dozen per pull request, with
`--only` or by ranges of the source. Each batch is one `--apply` and one
history record. Say in each pull request what was skipped and why.

## 8. Record the source

- `curation/source_queue.tsv`: the knowledge base, its license, and its
  status (`EVALUATING` during the trial, then `ADOPTED`, the values
  `curation/README.md` defines).
- `registry/<slug>.md`: the knowledge base in `data_sources` with
  `relation_type: prov:wasDerivedFrom`. Registries and ontologies used to
  check it are `prov:used`.

## Report

End with: the counts (entries, converted, skipped and why), what `just
qc-full` said, the gaps left open, and the schema decisions the person
should look at. Then run `audit-mech`, with the person's asks in
`requests.yml` (the conversion itself is one; give it a `records` check
with the count you expect), and give its checklist, gaps and next steps.

The IngestMech example in mechmaker's `example/` did all of this for
three Monarch ingests.

Versions

The database has no other skills named convert-knowledge-base.

Database snapshot: 13.11.2026 · package version b520fc61ed07 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100ingest-news
    Set a Mech up to curate from news reports: turn on mechmaker's news feature (`news_leads`, and the weekly `news-watch` workflow), write the Mech's conf/news.yaml so its records' own fields drive the searches, and prove it on a few records. Use when a person wants a Mech that…
  • 95ci-doctor
    Set up or repair a repo's mech-review CI. Use when adopting mech-review, or when its check reports `no result`, `n/a`, or a CI entry.
  • 100sync-mech
    Bring an existing Mech up to date with a newer mechmaker: find what changed in the template since the Mech was made or last synced, show the person each change (by mechmaker pull request) and how it meets the Mech's own edits, take the ones they choose, merge them into the…
  • 100knowledge-setup
    Bootstrap a cogni-knowledge knowledge base — a wiki base + a binding manifest that records every research project deposited into it. Scaffolds the wiki skeleton natively (the wiki engine is vendored; no separate plugin required) if it does not exist, then writes…
  • 100feishu-wiki
    Feishu knowledge-base navigation workflows. Activate when the user mentions a knowledge base, wiki, or wiki link.
  • 100wiki-ingest
    Compile articles, documents, or notes into a structured wiki knowledge base. Use when user says 'ingest to wiki', 'compile to knowledge base', 'update wiki', 'wiki ingest', 'add this to wiki', or invokes /wiki-ingest. Supports single or batch ingest. Triggers: wiki, ingest,…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.