source-trace
/source-trace
Audit an AI-generated knowledge tree (knowledge graph, concept map, mind map; 知识树、知识图谱、思维导图) against the source material it was built from, such as lecture slides or courseware exported as PNG/JPG images (课件、讲义截图). Finds wrong definitions, wrong relationships, content not in the source, overgeneralizations, vague points and important omissions, and backs every finding with evidence IDs, page numbers and quotes. Produces report.md and audit.json. Use this skill whenever the user wants to check, verify, review, fact-check or proofread a knowledge tree or study notes generated by AI against their
- Author
- ChenXuanyu-1216 ↗
- Repository
- SourceTrace ↗
- Licence
- MIT
- Last repository change
- 11.11.2026
Assessment and security
- Files in this package
- 10
- Scripts
- 3
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 11.11.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
- 3 scripts in this package. Review them before installing, as Claude may execute them.
Scripts: scripts/build_report.py, scripts/validate_evidence.py, scripts/validate_input.py
What it does and when to use it
Audit an AI-generated knowledge tree (knowledge graph, concept map, mind map; 知识树、知识图谱、思维导图) against the source material it was built from, such as lecture slides or courseware exported as PNG/JPG images (课件、讲义截图). Finds wrong definitions, wrong relationships, content not in the source, overgeneralizations, vague points and important omissions, and backs every finding with evidence IDs, page numbers and quotes. Produces report.md and audit.json. Use this skill whenever the user wants to check, verify, review, fact-check or proofread a knowledge tree or study notes generated by AI against their
Original author description
Audit an AI-generated knowledge tree (knowledge graph, concept map, mind map; 知识树、知识图谱、思维导图) against the source material it was built from, such as lecture slides or courseware exported as PNG/JPG images (课件、讲义截图). Finds wrong definitions, wrong relationships, content not in the source, overgeneralizations, vague points and important omissions, and backs every finding with evidence IDs, page numbers and quotes. Produces report.md and audit.json. Use this skill whenever the user wants to check, verify, review, fact-check or proofread a knowledge tree or study notes generated by AI against their
How it works
- Inputs
- Outputs
- Workflow
- When the input is unusual
SKILL.md preview
---
name: source-trace
description: Audit an AI-generated knowledge tree (knowledge graph, concept map, mind map; 知识树、知识图谱、思维导图) against the source material it was built from, such as lecture slides or courseware exported as PNG/JPG images (课件、讲义截图). Finds wrong definitions, wrong relationships, content not in the source, overgeneralizations, vague points and important omissions, and backs every finding with evidence IDs, page numbers and quotes. Produces report.md and audit.json. Use this skill whenever the user wants to check, verify, review, fact-check or proofread a knowledge tree or study notes generated by AI against their slides, asks whether an AI summary of a course is accurate or complete, or asks "这个知识树对不对" / "对照课件检查" — even if they don't say "audit".
compatibility: Python 3.9+ with jsonschema (pip install jsonschema). The agent must be able to read images.
---
# SourceTrace
SourceTrace checks an AI-generated knowledge tree against the source images it came from, and explains every problem with traceable evidence. The user should never have to take a finding on trust: each one says **why** it is a problem, **where** the source material is (page and quote), and **what** it rests on (evidence IDs).
A plain model can usually spot the errors. What SourceTrace adds is that the result is *checkable*: every node and edge gets a verdict, every verdict cites evidence, categories and risk levels are fixed, and the output has the same shape every time.
## Inputs
- **Source images:** PNG/JPG, one image per page (e.g. slides exported as images). They are the single source of truth. If the user has a PPT or PDF, ask them to export it to images first.
- **Knowledge tree:** JSON following `schemas/knowledge-tree.schema.json` (nodes with `id`, `label`, `definition`, `kind`; edges with `id`, `source`, `target`, `relation`). If the tree uses another format or other relation names, convert it first and tell the user what you mapped.
Paths below such as `scripts/…`, `schemas/…` and `references/…` are relative to this skill's folder.
## Outputs
Write all files to one output folder, by default `sourcetrace-output/` next to the knowledge tree:
| File | Stage | Content |
|------|-------|---------|
| `evidence-index.json` | 2 | Every piece of evidence extracted from the images (you write it) |
| `audit.json` | 4–6 | A verdict for every node and edge, plus omissions (you write the judgments; the script completes it) |
| `report.md` | 6 | The human-readable report (generated by `scripts/build_report.py`; do not write it yourself) |
Never modify the user's knowledge tree. SourceTrace reports and recommends; the user decides.
Put your own working files (drafts, helper scripts, enlarged crops of an image) in a `work/` folder inside the output folder, never next to the user's files or anywhere else. They are not part of the result. Mention the folder in your final message, so the user can delete it.
## Workflow
Run the six stages in order. Do not skip ahead to judging the tree before the evidence index exists: auditing against an index, instead of against your impression of the slides, is what makes the findings traceable.
The scripts write UTF-8 themselves. If you run your own Python commands that print Chinese on Windows, run them as `python -X utf8 …`, or the console may fail with `UnicodeEncodeError`. Setting `PYTHONIOENCODING=utf-8` also works, but not together with `python -I` or `-E`, which ignore it.
### 1. Source ingestion
```bash
python scripts/validate_input.py --tree <knowledge-tree.json> --images <image folder or files> --json
```
- **Errors** (`"valid": false`): stop, show the user the errors in plain words, and ask them to fix the input. Do not audit an invalid tree.
- **Warnings** (e.g. `DISCONNECTED_TREE`, `ISOLATED_NODE`, `DUPLICATE_EDGE`): continue. `build_report.py` puts them in the report's Structure section.
- `summary.images` gives each image its `source_id`, `page` and `path`. Use exactly these in stage 2. The paths are relative to the default output folder; if you write the outputs somewhere else, pass that folder with `--out`.
Then look at every image and note its title, text, tables and diagrams.
### 2. Evidence extraction
Build `evidence-index.json` following `references/evidence-extraction.md`. Read that file before extracting; in short:
- One item per slide title, bullet, table row and diagram relationship.
- Quote text verbatim in its original language. Small wording changes are what the audit must catch.
- Extract everything, including homework or reading notes.
- Set `certainty` honestly; `low` evidence cannot reject a tree item on its own.
Then validate it, and fix every error before continuing:
```bash
python scripts/validate_evidence.py --index <output>/evidence-index.json
```
### 3. Knowledge mapping
Treat the tree as a set of claims to test, never as fact. Split it into:
- **Nodes:** is the label and definition correct?
- **Edges:** is the relationship between the two nodes correct? Read edges as `source relation target`.
Audit them separately. A node can be correct while an edge using it is wrong (and the other way round), and each gets its own verdict.
For each node and edge, list the evidence items that bear on it. Search the whole index, not just the page you expect: some claims are only confirmed or contradicted by combining two pages.
### 4. Bidirectional audit
- **Tree → source:** for every node and edge, decide whether the evidence supports it. Every node and edge gets a verdict, including the correct ones; confirming what is right is part of the result.
- **Source → tree:** go through the evidence index and find significant content that no node or edge covers.
Follow `references/audit-rules.md` for verdicts, what counts as significant, and the edge cases.
### 5. Conflict detection
Classify every problem into exactly one category and give it a risk level. Use the decision order in `references/audit-rules.md`: first gather all the item's evidence (text, tables and figures on every page), then go through its questions in order and stop at the first "yes". Do not pick a category by how a finding feels; the order is what keeps two runs consistent.
| Category | Meaning |
|----------|---------|
| `conceptual-error` | A node's definition contradicts the source in meaning: both cannot be true at once. Opposite-looking words alone are not enough. |
| `incorrect-relationship` | An edge contradicts the source, even if both nodes are correct. |
| `unsupported-expansion` | Content not found in the source, even if true in general. Scope `whole` (medium) when the term or relationship is absent; scope `detail` (low) when only one added detail of an otherwise supported definition is. |
| `overgeneralization` | Supported content stretched beyond what the source says. |
| `ambiguous` | Cannot verify: the source only names the term and never explains it, or the evidence is unreadable (`low` certainty). |
| `omission` | Significant source content missing from the tree. |
### 6. Audit report
Write your judgments to `audit.json`, then let the script check them and produce the report. You decide; the script counts, numbers, sorts and formats. That split is what makes two runs produce the same report.
`audit.json` as you write it (v0.1, schema `schemas/audit.schema.json`):
```json
{
"schema_version": "0.1",
"items": [
{ "target": "N001", "verdict": "supported", "evidence": ["EV-002"] },
{ "target": "N002", "verdict": "finding", "evidence": ["EV-008"],
"category": "conceptual-error", "risk": "high",
"reason": "...", "recommendation": "..." },
{ "target": "N028", "verdict": "finding", "evidence": ["EV-167"],
"category": "unsupported-expansion", "scope": "detail", "risk": "low",
"reason": "...", "recommendation": "..." }
],
"omissions": [
{ "category": "omission", "tier": "core", "risk": "medium", "evidence": ["EV-016"],
"reason": "...", "recommendation": "..." },
{ "category": "omission", "tier": "core", "risk": "high", "risk_reason": "central concept of the lecture",
"evidence": ["EV-003"], "reason": "...", "recommendation": "..." }
]
}
```
- **`items`:** exactly one entry per node and edge of the tree.
- **`reason`:** why it is a problem, in one or two sentences, quoting the exact words that differ. A `supported` item has no `reason`, except a short note when the verdict had to resolve a problem in the source itself (see "Problems in the source itself" in `references/audit-rules.md`); the report shows that note in the confirmed list.
- **`recommendation`:** a concrete revision that stays within the tree schema (allowed `kind` and `relation` values only). For a wrong edge: if the source shows the correct relationship, recommend replacing it and give the new `source`, `relation` and `target`; if not, or if another edge already says it, recommend deleting it. Do not invent a replacement the source does not state. If no allowed relation expresses the correct relationship (e.g. "ReLU 可缓解梯度消失": there is no "mitigates"), recommend deleting the edge and stating the relationship in a node's definition. Do not fall back on `related-to`, which loses the direction. This applies only when no relation value fits: when the edge's relation is right and only its `source` or `target` is wrong, keep the relation and correct the node (e.g. 过拟合 related-to 高偏差 → 过拟合 related-to 高方差).
- **Keep the definition useful for learning.** For an `ambiguous` or a `detail` finding, do not shrink the definition to something like "资料列出的一种方法" just because that is easier to trace. Recommend keeping it, and marking the unverified part as an explanation from outside the source (资料外释义), or rewording only the detail.
- **Add no claims of your own.** A suggested wording may use the tree's words and the source's words, nothing else. For example, do not turn "区分 A 与 B 来分析" into "把误差分为 A、B 两部分", which says the split is complete.
- **Check the recommendation against the evidence, as strictly as the finding.** Keep the source's conditions and scope ("通过适当的预处理", "2006 年的一篇比较研究"), so a result from one study does not read as a general rule. Do not copy a source sentence that other evidence in the source contradicts: for example, "概率比可以表达为似然比" next to a formula that keeps the prior ratio p(S)/p(¬S). Use the consistent evidence (the formula) instead. Some contradictions take a short derivation to see: before quoting a general claim of the source (正数, 反对称, a condition for a result), test it against the source's own formulas at the boundary cases, as "Check a source claim" in `references/audit-rules.md` describes.
- **`scope`** (`unsupported-expansion` only): `detail` when one added detail is unsupported; leave it out otherwise.
- **`tier`** (omissions only): `core` or `supporting`, as defined in `references/audit-rules.md`.
- **`risk_reason`:** only when the risk differs from the default.
- Leave out `summary`, `structure_warnings` and `finding_id`; the script computes them.
Then run:
```bash
python scripts/build_report.py --tree <knowledge-tree.json> --index <output>/evidence-index.json --audit <output>/audit.json
```
- **Errors:** nothing is written. Fix `audit.json` and run again until it succeeds. Do not work around the checks.
- **Success:** the script rewrites `audit.json` with `summary`, `structure_warnings` and `finding_id`s (F-001, F-002, … in report order), and writes `report.md` next to it. Findings with scope `detail` are listed in a folded "来源提示" / "Source notes" section after the other findings, because they mark a source gap, not an error. The report language follows the slides; pass `--lang zh` or `--lang en` if the user asks for another.
Finish by telling the user where the report is, the counts from the summary, and the top problems in a few lines. If you noticed problems in the source itself (it contradicts itself, has an evident slip, or a claim fails a check against its own formulas), list them briefly too, and say they are not findings about the tree. A wording that only outside knowledge suggests is wrong goes in as a possible slip, without asserting the correct version. Do not paste the whole report.
## When the input is unusual
- **Web page or document screenshots** instead of slides: follow the web-page section of `references/evidence-extraction.md` (overlapping screenshots, site menus and navigation boxes).
- **The source contradicts itself or has a slip:** quote it as printed and follow "Problems in the source itself" in `references/audit-rules.md`. It is not a finding about the tree.
- **Unreadable image:** extract what you can with `low` certainty, mention the page in the affected findings' `reason`, and tell the user; do not invent content.
- **The tree covers material not in the images:** report those items as `unsupported-expansion`. Tell the user that missing pages may be the cause only if there is a sign of it: gaps in the slide numbers, an agenda topic with no slide of its own, or a whole topic in the tree that the deck never touches. Otherwise do not suggest it.
- **Very large decks or trees:** work page by page in stage 2 and node by node in stage 4, writing results to the files as you go rather than holding everything in mind.
Files in this package (10)
- SKILL.md 13 KB
- references/audit-rules.md 23 KB
- references/evidence-extraction.md 7 KB
- schemas/audit.schema.json 3 KB
- schemas/evidence-index.schema.json 1 KB
- schemas/evidence.schema.json 2 KB
- schemas/knowledge-tree.schema.json 2 KB
- scripts/build_report.py 18 KB
- scripts/validate_evidence.py 6 KB
- scripts/validate_input.py 10 KB
Versions
The database has no other skills named source-trace.
Database snapshot: 13.11.2026 · package version 59f691cf440e (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 95latex-mindmap
Turn LaTeX lecture notes, a textbook, or a thesis into a single self-contained HTML mind map — every definition, theorem, lemma, proposition, corollary, remark, example and exercise as a bubble in a dependency tree, with statements, proofs and a searchable cheat sheet built in.… - 100citation-check
Use when the user wants to verify citations in AI-generated or human-written text against the local knowledge base and catch hallucinated, wrong, or missing references. - 100citation-check
Verify citations in AI-generated or human-written text against the local knowledge base. Catches hallucinated references, wrong metadata, and missing papers. Use when the user wants to check if citations are real and accurate.