wiki-ingest

---
name: wiki-ingest
description: "Use when ingesting canonical PARA-ZK sources into the LLM-Wiki — per-import, delta, uncited, or targeted re-ingest."
argument-hint: "mode=<uncited|delta|per-import|re-ingest>"
---

# Wiki Ingest

Orchestrate a bounded LLM-Wiki ingest run as **Plan → Fill → Synthesize**: YOU (the orchestrator)
read the candidates' structure and the existing wiki and decide the whole page structure first, then
spawn one `wiki-weaver` per planned page to fill it in parallel, then a hub pass builds the navigable
spine. The weaver is `para-zk:wiki-weaver` in Claude Code, `wiki-weaver` in Codex (after running the
bundled `codex-setup` skill).

## Context

LLM-Wiki pages are LLM-owned derived synthesis under `LLM-Wiki/`; canonical knowledge stays in
Resources, PARA, and ZK notes. The wiki is a **compounding, interlinked web of concept pages** drawn
from the WHOLE canonical base (resources, digests, permanents, subnotes) — pages cross-link, a rich
source spans several pages, and each ingest extends existing pages rather than mirroring sources 1:1.

Cohesion (the right page set, the right granularity, consistent domains, a navigable spine) is a
GLOBAL property, so **you decide it yourself, up front, before any page is written** — and you do it
HERE in the orchestrator (not in a sub-agent) precisely because you hold THIS conversation: the plan
must reflect what the user actually wants (what to include/skip, domain preferences). You discover
the source set, read each candidate's HEAD (a bounded structural peek — not the full body) plus the
existing wiki, and produce one global plan: the page set, domains, granularity, source→page
assignments, cross-links, and the spine. Then you spawn one `wiki-weaver` per planned page. Weavers
do the heavy work — full-body reads and the actual writing (`create-llm-wiki`/`update-llm-wiki` with
`by=<model-id>`) — but never decide which pages exist, so they cannot diverge on domains, over-split
a source into micro-pages, or duplicate concepts. You never write wiki prose or read full bodies; you
never edit source notes. Wiki prose mirrors the user's OWN language register, which the weavers
derive from the material.

## Argument Routing

`mode` (required) selects what to discover. `uncited`/`delta` take no source args; `per-import`/`re-ingest`
require one of `source_path` or `source_paths`, passed through to `para-zk:wiki-ingest-candidates`.
`type` is optional and filters ingestable source kind (`resource`, `digest`, `permanent`, `subnote`);
use `mode=uncited type=resource` to inspect existing Resources that have never been cited by the wiki.

| `mode` | Targets |
|--------|---------|
| `uncited` | all ingestable sources missing an incoming LLM-Wiki citation |
| `delta` | new/uncited sources + sources whose `updated` is newer than the LLM-Wiki pages citing them (best-effort, via `updated` timestamps; force-refresh a known change with `re-ingest`) |
| `per-import` | exactly the given `source_path`/`source_paths` |
| `re-ingest` | exactly the given `source_path`/`source_paths`, even if already cited/fresh |

On invalid args (source args with `uncited`/`delta`, or none with `per-import`/`re-ingest`), stop with
the concrete routing error — do not ask the user.

## Execution

1. **Orient**: Run `optsidian para-zk:conventions` ONCE first for this task, then call
   `optsidian para-zk:describe` / `optsidian para-zk:describe type=<t>` as reference. Use the
   returned invocation style for subsequent `para-zk:*` calls. Examples below use `optsidian`;
   substitute the native `obsidian` CLI (`obsidian para-zk:…`) when `optsidian` isn't installed —
   the two are interchangeable, so use whichever is on `PATH`.
   Confirm the surface exposes `para-zk:wiki-ingest-candidates`, `para-zk:list`,
   `para-zk:read-llm-wiki`, `para-zk:create-llm-wiki`, `para-zk:update-llm-wiki`, and
   `para-zk:refile-llm-wiki`. If the vault
   is unavailable, stop with the CLI error; do not fall back to direct file writes.

2. **Resolve mode**: Normalize exactly one mode from `{per-import, delta, uncited, re-ingest}`. Preserve
   `limit`/`offset` and optional `type` when supplied. For targeted modes, preserve exactly one of `source_path` or
   `source_paths`; for untargeted modes, reject either source selector before doing any reads.

3. **Discover candidates**:

   ```bash
   optsidian para-zk:wiki-ingest-candidates mode=<mode> [type=<resource|digest|permanent|subnote>] [source_path=<path>|source_paths='<json-or-comma-list>'] [limit=<n|all>] [offset=<n>]
   ```

   Read the text output. It starts with a candidate count and then one line per candidate:
   `<path> [<reason>]`, optionally followed by `(stale: <wiki pages>)`. Candidate reasons are
   `missing_wiki_citation` or `source_newer_than_wiki`; targeted modes may also report
   `per_import` or `reingest_requested`. If the command prints `error:`, stop. If it reports
   `0 candidates`, report that none were returned and stop. If it prints a trailing pagination hint,
   plan/ingest only the returned page and report that another bounded page remains; do not auto-page
   into a full scan. Collect candidate paths and any stale wiki labels; for subnote candidates,
   collect their parent project/area note paths as framing `context`.

4. **Plan** (you, the orchestrator — inline): read enough to decide the global page structure, then
   decide it.
   - **Read the head of each candidate** — `optsidian read path="<path>" lines=1:200` (frontmatter +
     abstract + lead/early sections). If the head does not make the source clear enough to classify,
     read FURTHER (more line windows, or `grep '^#{1,6} '` for its headings) until you understand it
     well enough to place it — reading more of a source when you need it is fine. You just don't
     routinely page whole bodies for every source (that's the weaver's job and would overflow you).
   - **Read the existing wiki**: `optsidian para-zk:list type=llm-wiki limit=all` for the complete
     roster (every `<domain>/<concept>` path → the domains already in use). Then, for the domains the
     candidates touch, **read that domain's `<domain>/index` hub FIRST** (`read-llm-wiki title="<domain>/index"`)
     — it is the area map and the cheapest way to learn the domain's structure; only read individual
     concept pages when the index is insufficient.
   - **Decide the plan**, applying these rules:
     - **Granularity:** a PAGE is a concept that recurs across ≥2 sources OR is a canonical,
       standalone concept/method/system. A one-off mechanism from a single source is a **section
       within** a broader page, listed in that page's `sections` — NOT its own page. A per-source
       SUMMARY page (`type:"source-summary"`) only when a work is distinct enough that concept pages
       do not cover it. Project-operational / ephemeral material (status memos, test logs, trip
       checklists) is NOT a concept — skip it (record why) or fold under a project hub. Prefer FEWER,
       denser pages.
     - **Domains:** one level, `<domain>/<concept>`. Treat a domain as a stable research lineage,
       problem area, or system family that can sustain a useful hub — not as a universal umbrella.
       Avoid vague catch-all domains such as `ai`, `ml`, `research`, or `papers` unless the existing
       wiki already uses one coherently and the candidate truly cannot fit a narrower lineage. For
       broad corpora like "LLM papers", infer the corpus axes from the sources and prefer the
       narrowest stable domain that can hold multiple pages (for example `scaling-laws`,
       `chain-of-thought`, `mixture-of-experts`, `retrieval-augmented-generation`, `alignment`, or
       `language-models`). A method name becomes a domain only when it has enough internal structure
       for multiple child pages; otherwise keep it as a concept page under a broader domain. A
       one-page domain is acceptable only when it names a durable area expected to grow or when the
       existing wiki already uses it as a necessary bridge; otherwise prefer a concept page under the
       nearest coherent domain. REUSE a roster domain when a concept fits; mint a new domain only for
       a genuinely new area; never near-synonyms (`RL` vs `Reinforcement Learning`). Before finalizing
       each new domain, write a one-sentence domain rationale in the plan guidance: what stable
       lineage/problem area it represents, expected child concepts, and why it is not better placed
       under an existing broader/narrower domain. One concept = one page across the whole wiki.
     - **Assignment & cross-links:** assign every candidate source to ≥1 page (a source spanning N
       concepts goes into N pages' `sources`); set each page's `links` to the related pages.
     - **Domain moves:** when an existing concept page belongs under a different domain, add it to a
       separate `domain_moves` list (`title`, `from_domain`, `to_domain`, `reason`). Do not hide the
       move inside a page write. Apply it only when the user explicitly requested/approved domain
       cleanup, using `para-zk:refile-llm-wiki title="<current>" domain="<target>"`; use
       `rename-llm-wiki` only for same-domain concept-name changes.
     - **Spine:** every domain's hub is `<domain>/index` (create-llm-wiki auto-mints it as an empty
       scaffold when the domain's first page is created, so it always exists — you don't create it).
       Plan it as `type:"hub"` with `title="<domain>/index"` and `children` = that domain's leaf
       titles, for the Synthesize pass to FILL. Add an entity hub (descriptive title) for a large
       sub-cluster when it helps.
     - **Honor THIS conversation:** include/exclude and domain choices the user expressed here win.
   - Produce the page list — each page `{title, type, existing, sources, sections, links, children,
     guidance}` (`type` ∈ `concept`|`source-summary`|`hub`) — plus a `skipped` list with reasons.

5. **Fill** (leaf pages): for EACH planned page with `type` `concept` or `source-summary`, spawn one
   `wiki-weaver`, IN PARALLEL, with a `WeavePacket`:

   ```json
   { "mode": "<mode>", "by": "<model-id>", "page": <the planned page object>,
     "plan_pages": ["<every planned page title>", "..."], "context": ["<parent note path>", "..."] }
   ```

   Inject `by` from your current model id; `plan_pages` is every planned title so a weaver's
   cross-links resolve to real (current or about-to-exist) pages. One weaver owns one page → leaf
   pages do not contend. If a weaver reports `remaining_sources` (a context boundary), spawn a FRESH
   weaver for the SAME page with the remaining paths (its compare-and-swap writes make continuation
   safe). Do not post-process a weaver's writes.

   Claude Code spawn shape:

   ```text
   Agent({ subagent_type: "para-zk:wiki-weaver", run_in_background: true,
           prompt: "Fill this WeavePacket's one page and report per your Output Format.\n\n<WEAVE_PACKET_JSON>" })
   ```

   Codex (after `codex-setup` has installed the custom agents into `~/.codex/agents/`; if `wiki-weaver`
   is unavailable, stop and tell the user to run `codex-setup`, then restart Codex):

   ```text
   Spawn a `wiki-weaver` subagent: "Fill this WeavePacket's one page and report per your Output Format.\n\n<WEAVE_PACKET_JSON>"
   ```

6. **Synthesize** (spine): AFTER the leaf weavers complete (hubs link pages that must already exist),
   spawn one `wiki-weaver` in hub mode for EACH planned `type:"hub"` page (the per-domain
   `<domain>/index`, plus any entity hub) — same packet shape; a hub `page` carries `page.children`
   (planned page titles) and no `sources`. The weaver fills the hub as a RELATIONAL MAP of the area
   (grouped children, a one-line gist each, and how they relate) — the LLM's substitute for the graph
   view it cannot see. For targeted modes, this refreshes only the hubs your plan lists.

7. **Report launch**: Return the mode, the plan summary (counts of concept / source-summary / hub
   pages, sources assigned vs `skipped`), whether `has_more` was true, and the weaver launch
   handle(s). The observable write contract is the weavers' direct page writes succeeding.

8. **Verify on completion (clean-context pass)**: When the weavers finish, read the touched pages
   (`read-llm-wiki key=body` each). You never generated this prose, so this catches generation slips.
   Fix high-confidence orthographic/generation slips in place — malformed Korean syllables where a
   wrong 받침/jamo yields a well-formed but wrong word (궤적→궁적, 댄스→댓스, 앉기→앙기) — via
   `update-llm-wiki key=body op=replace by=<model-id>`. Surface (do NOT auto-apply): ambiguous
   corrections, any `unplanned_concepts` weavers reported (so the user can decide whether to plan a
   page for them next run), and any semantic findings (contradictions, a leaf missing from its hub,
   missing cross-references). If the touched-page volume would bloat your context, isolate this pass
   in a freshly spawned verifier sub-agent — but still report rather than auto-apply. (A whole-wiki
   health sweep is the separate, human-requested `wiki-lint`, not part of ingest.)

9. **Fold in conversation-derived insight (optional, human-judged)**: The background weavers only saw
   their packets. If THIS conversation surfaced a genuine, wiki-worthy insight the source bodies alone
   do not capture (a synthesis, a key tension, a correction the user raised), fold it into the relevant
   touched page(s) via `update-llm-wiki key=body op=... by=<model-id>` after verification, citing any
   canonical source as a backtick `` `PZ[id]` ``, keeping links single-direction, and writing any formula
   as Obsidian MathJax (`$…$`/`$$…$$`, single backslashes, never backtick-wrapped or `\(...\)`/`\[...\]`). Guardrails: ONLY
   genuine user-surfaced insight — never invent, embellish, or inject tangential conversation; if
   unsure whether it belongs, surface it to the user instead of writing it; if nothing genuine
   emerged, skip. Report what was folded in.