Przejdź do treści
Biblioteka skilli
Dokumenty i biuro · Przegląd i audyt

docs-grounding-verifier

/docs-grounding-verifier

Weryfikuje na poziomie twierdzeń, czy strony dokumentacji zgadzają się z kodem źródłowym, sprawdza fakty w sekcjach i wskazuje twierdzenia, które odeszły od źródła.

Repozytorium
apm ↗
Licencja
Apache-2.0
Ostatnia zmiana repozytorium
13.11.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
101
Skrypty
4
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 13.11.2026, bez odwołań do wycofanych modeli i API.

Licencja

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

Co sprawdzić przed instalacją
  • 4 skrypty w paczce. Przejrzyj je przed instalacją, bo Claude może je uruchomić.

Skrypty: evals/run-evals.sh, scripts/extract-claims.py, scripts/retrieve-evidence.sh, scripts/verify-page.sh

Co robi i kiedy

Weryfikuje na poziomie twierdzeń, czy strony dokumentacji zgadzają się z kodem źródłowym, sprawdza fakty w sekcjach i wskazuje twierdzenia, które odeszły od źródła.

Wyzwalacze z opisu

is this doc accurateverify the page against the codefact-check this sectionany claims that drifted from sourcefact-checkinggrounding auditdrift huntclaim verification
Oryginalny opis (EN)

Use this skill to verify CLAIM-LEVEL grounding of a documentation page (or set of pages) against the source code. Activate when you have specific pages to check for factual accuracy -- not when sweeping a whole corpus (use docs-corpus-audit for that) and not when triaging a PR diff (use docs-sync for that). Trigger nouns: "is this doc accurate", "verify the page against the code", "fact-check this section", "any claims that drifted from source", "fact-checking", "grounding audit", "drift hunt", "claim verification". Returns per-claim verdicts (GROUNDED | PARTIAL | CONTRADICTED | UNSUPPORTED) w

Jak działa

  1. Sibling contract
  2. When to activate
  3. When NOT to activate
  4. Architecture (PIPELINE-of-PANELS)
  5. Phase 1: SCOPE
  6. Phase 2: EXTRACT (parallel)
  7. Phase 3: RETRIEVE (deterministic, batched)
  8. Phase 4: JUDGE (parallel)
  9. Phase 5: SYNTHESIZE
  10. Phase 6: ALIGNMENT LOOP (A8)
Sekcje z nagłówków SKILL.md, w oryginale · 167 linii, 862 słów
Podgląd SKILL.md
---
name: docs-grounding-verifier
description: 'Use this skill to verify CLAIM-LEVEL grounding of a documentation page (or set of pages) against the source code. Activate when you have specific pages to check for factual accuracy -- not when sweeping a whole corpus (use docs-corpus-audit for that) and not when triaging a PR diff (use docs-sync for that). Trigger nouns: "is this doc accurate", "verify the page against the code", "fact-check this section", "any claims that drifted from source", "fact-checking", "grounding audit", "drift hunt", "claim verification". Returns per-claim verdicts (GROUNDED | PARTIAL | CONTRADICTED | UNSUPPORTED) with file:line evidence citations. Catches paragraph-level inaccuracies that page-level audit averages over -- e.g. a paragraph with 5 claims where 4 are grounded and 1 is fabricated. Does NOT modify files (returns advisory only); does NOT re-architect the docs; does NOT triage PRs.'
---

# docs-grounding-verifier

CLAIM-LEVEL grounding verification. Adapts the RAGAS faithfulness-eval
pattern (proven in RAG literature) to docs/code instead of generated-
answers/retrieved-context. Source code is the ground truth; docs
paragraphs are the candidate text under audit.

[python-architect persona](../../personas/python-architect.persona.md)
[doc-writer persona](../../personas/doc-writer.persona.md)

## Sibling contract

This skill is a SIBLING of `docs-corpus-audit` and `docs-sync`. The
boundary is load-bearing:

| Skill                  | Trigger                                | Scope            | Granularity     |
| ---------------------- | -------------------------------------- | ---------------- | --------------- |
| docs-sync              | PR opened/synchronized                 | PR diff only     | Page-level      |
| docs-corpus-audit      | Maintainer asks for whole-corpus pass  | Entire corpus    | Page-level      |
| **docs-grounding-verifier** | **Verify specific pages factually**    | **1..N pages**    | **CLAIM-level** |

`docs-corpus-audit` invokes this skill in its VERIFY phase on the
highest-risk pages of each wave. `docs-sync` can invoke it on the
specific pages in a PR diff. The skill is also runnable standalone.

## When to activate

- Maintainer says "verify <page> against the code".
- An audit wave wants per-claim grounding scores for its highest-risk pages.
- A PR review wants to confirm that prose changes are not just plausible
  but actually consistent with the implementation.
- A "fact-check" or "grounding" or "drift hunt" request.

## When NOT to activate

- Whole-corpus sweep with no specific page list -> use `docs-corpus-audit`.
- PR review with mixed code+docs diff -> use `docs-sync`.
- Editorial / tone review -> use `editorial-owner` persona directly.

## Architecture (PIPELINE-of-PANELS)

```
PARENT
  -> [Stage 1: EXTRACT claims, fan-out PANEL]
       per page -> LLM extracts atomic factual claims as JSON
       script: scripts/extract-claims.py
  -> [Stage 2: RETRIEVE evidence, deterministic S7]
       per claim -> grep over src/ via keywords + hints
       script: scripts/retrieve-evidence.sh   (NO LLM)
  -> [Stage 3: JUDGE grounding, adversarial A7]
       per (claim, evidence) -> LLM rules GROUNDED|PARTIAL|CONTRADICTED|UNSUPPORTED
       asset: assets/judge-prompt.md
  -> [Stage 4: SYNTHESIZE]
       aggregate ungrounded -> doc-writer for fix
       re-verify after fix (A8 ALIGNMENT LOOP)
```

Stage 2 is the load-bearing design choice: evidence retrieval is
DETERMINISTIC (grep + AST hints), not LLM. The judge in Stage 3 can
only rule on evidence it actually receives -- it cannot hallucinate
support that the retriever did not find. This is the structural
guard against the failure mode "the LLM convinces itself the docs
match the code."

## Phase 1: SCOPE

Input: list of page paths to verify (1..N). If a `risk_class` is
attached (e.g. "high-stakes"), prefer it; otherwise treat all as equal.

Out-of-scope:
- Pages outside `docs/src/content/docs/` or
  `packages/apm-guide/.apm/skills/apm-usage/`.
- Pages with no factual claims (pure editorial / landing). Skip
  rather than force-extract.

## Phase 2: EXTRACT (parallel)

For each page, dispatch ONE claim-extractor agent:
- Prompt template: `scripts/extract-claims.py <page>` produces the
  prompt and embeds the page content.
- Returns: JSON `{"page", "claims":[{"id","text","section","keywords",
  "expected_source_areas"}]}` capped at 15 claims per page.

Parallel safe; no shared state between extractors.

## Phase 3: RETRIEVE (deterministic, batched)

For each claim, pipe to `scripts/retrieve-evidence.sh`:
- Uses keywords + expected_source_areas to grep src/.
- Returns one-line JSON: `{"claim_id","claim_text","evidence":[...],
  "evidence_count"}`.

Sequential is fine (grep is fast). No LLM. Diagnostics on stderr,
data on stdout.

## Phase 4: JUDGE (parallel)

For each (claim, evidence) tuple, dispatch ONE grounding-judge agent:
- Load `assets/judge-prompt.md`.
- Send the prompt + the tuple.
- Returns: JSON verdict per the schema in `judge-prompt.md`.

Batching across claims-of-one-page into a single judge call is fine
(prompt with all tuples at once). Across pages, fan out.

## Phase 5: SYNTHESIZE

Aggregate verdicts. Materialize the report:

```
{
  "summary": {
    "pages_verified": N,
    "claims_total": N,
    "grounded": N, "partial": N, "contradicted": N, "unsupported": N,
    "grounding_rate": N/total
  },
  "actionable": [
    {"page", "claim", "verdict", "evidence_cited", "fix_suggestion"}
  ]
}
```

CONTRADICTED and PARTIAL are doc-writer work items. UNSUPPORTED is
split: if `retrieval_fix_suggestion` is plausible, retry retrieval
with the suggested keywords; if still empty, treat as CONTRADICTED.

## Phase 6: ALIGNMENT LOOP (A8)

Hand actionable items to doc-writer (one subagent per page). After
edits, RE-RUN the pipeline on the same pages. The grounding_rate
must MONOTONICALLY INCREASE between iterations or the loop has
diverged -- stop and escalate to the operator.

## Ship gate

- grounding_rate >= 0.9 on each verified page after the alignment loop.
- Every CONTRADICTED claim cited a specific code file:line that
  disproves it -- not vague "the code doesn't say that".
- The eval-runner (see `evals/`) passes on the trigger evals and
  the content evals before the skill is treated as production-ready.

## Bundled assets

- `scripts/extract-claims.py` -- Stage 1 prompt builder. `--help`, `--schema`.
- `scripts/retrieve-evidence.sh` -- Stage 2 retriever. Deterministic. `--help`.
- `scripts/verify-page.sh` -- end-to-end orchestrator. `--help`.
- `assets/judge-prompt.md` -- Stage 3 adversarial judge prompt.
- `evals/trigger-evals.json` -- 20 dispatch queries (10 should, 10 shouldn't).
- `evals/content-evals.json` -- seeded-drift recall scenarios.
- `evals/run-evals.sh` -- the eval-runner that turns JSON into metrics.

## Failure modes guarded against

- **Hallucinated grounding**: Stage 2 is deterministic; judge sees only
  real evidence.
- **Adversarial weakness**: Stage 3 prompt defaults to SKEPTICAL.
- **Page-level averaging**: claim-level granularity surfaces partials.
- **Bundle leakage**: design notes / one-time scripts stay in session
  state, never in `references/`.
- **Phantom dependency**: SKILL.md links its persona deps via relative
  paths; A9 PROBE before invoking docs-corpus-audit's substrate.
- **Dispatch collision** with sibling skills: trigger-eval validation
  split is the ship gate (must distinguish from docs-sync /
  docs-corpus-audit triggers).
Pliki w paczce (101)
  • SKILL.md 7 KB
  • assets/judge-prompt.md 3 KB
  • evals/content-evals.json 2 KB
  • evals/run-evals.sh 5 KB
  • evals/trigger-evals.json 2 KB
  • evals/runs/20260527-194228/trigger-prompts.txt 2 KB
  • evals/runs/20260527-194228/seeded-corpus/drift-install-flag/page.md 14 KB
  • evals/runs/20260527-194228/seeded-corpus/drift-install-flag/scenario.json 1 KB
  • evals/runs/20260527-194228/seeded-corpus/drift-policy-reject/page.md 7 KB
  • evals/runs/20260527-194228/seeded-corpus/drift-policy-reject/scenario.json 1 KB
  • evals/runs/20260527-194228/seeded-corpus/drift-registry-resolver/page.md 21 KB
  • evals/runs/20260527-194228/seeded-corpus/drift-registry-resolver/scenario.json 1 KB
  • evals/runs/proof/trigger-responses.json 1 KB
  • evals/runs/proof/claims/copilot.json 4 KB
  • evals/runs/proof/claims/install.json 5 KB
  • evals/runs/proof/claims/pkgtypes.json 4 KB
  • evals/runs/proof/claims/policy.json 4 KB
  • evals/runs/proof/claims/registries.json 5 KB
  • evals/runs/proof/copilot/judge-batch-docs_src_content_docs_integrations_copilot-app_md.txt 45 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c1.json 3 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c10.json 2 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c11.json 2 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c12.json 3 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c13.json 2 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c14.json 2 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c15.json 2 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c2.json 3 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c3.json 3 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c4.json 3 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c5.json 3 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c6.json 2 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c7.json 2 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c8.json 3 KB
  • evals/runs/proof/copilot/evidence/docs_src_content_docs_integrations_copilot-app_md_c9.json 3 KB
  • evals/runs/proof/install/judge-batch-docs_src_content_docs_reference_cli_install_md.txt 38 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c1.json 3 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c10.json 2 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c11.json 1 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c12.json 2 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c13.json 2 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c14.json 3 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c15.json 3 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c2.json 1 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c3.json 2 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c4.json 1 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c5.json 2 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c6.json 2 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c7.json 5 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c8.json 1 KB
  • evals/runs/proof/install/evidence/docs_src_content_docs_reference_cli_install_md_c9.json 1 KB
  • evals/runs/proof/pkgtypes/judge-batch-docs_src_content_docs_reference_package-types_md.txt 51 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c1.json 3 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c10.json 2 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c11.json 2 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c12.json 2 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c13.json 3 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c14.json 2 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c15.json 4 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c2.json 4 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c3.json 3 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c4.json 4 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c5.json 3 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c6.json 3 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c7.json 3 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c8.json 2 KB
  • evals/runs/proof/pkgtypes/evidence/docs_src_content_docs_reference_package-types_md_c9.json 2 KB
  • evals/runs/proof/policy/judge-batch-docs_src_content_docs_enterprise_apm-policy_md.txt 51 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c1.json 4 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c10.json 2 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c11.json 2 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c12.json 2 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c13.json 3 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c14.json 3 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c15.json 4 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c2.json 3 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c3.json 3 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c4.json 3 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c5.json 2 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c6.json 3 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c7.json 2 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c8.json 2 KB
  • evals/runs/proof/policy/evidence/docs_src_content_docs_enterprise_apm-policy_md_c9.json 1 KB
  • evals/runs/proof/registries/judge-batch-docs_src_content_docs_guides_registries_md.txt 49 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c1.json 3 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c10.json 2 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c11.json 3 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c12.json 3 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c13.json 2 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c14.json 3 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c15.json 3 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c2.json 3 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c3.json 2 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c4.json 3 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c5.json 4 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c6.json 3 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c7.json 1 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c8.json 3 KB
  • evals/runs/proof/registries/evidence/docs_src_content_docs_guides_registries_md_c9.json 2 KB
  • scripts/extract-claims.py 3 KB
  • scripts/retrieve-evidence.sh 3 KB
  • scripts/verify-page.sh 2 KB

Wersje

W bazie nie ma innych skilli o nazwie docs-grounding-verifier.

Migawka bazy z 13.11.2026 · wersja paczki ffec794d5653 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100verify-openspec-docs
    Sprawdza fakty w dokumentacji użytkownika OpenSpec przez subagenta ze świeżym kontekstem, który ponownie uruchamia polecenia i porównuje twierdzenia ze źródłem; uruchamiany ręcznie.
  • 100docs-corpus-audit
    Przeprowadza całościowy przegląd dokumentacji microsoft/apm względem bieżącego kodu, strona po stronie, i wskazuje punktowe poprawki nieaktualnych twierdzeń, np. przed wydaniem.
  • 100docs-corpus-audit
    Przeprowadza całościowy przegląd dokumentacji microsoft/apm względem bieżącego kodu, strona po stronie, i wskazuje punktowe poprawki nieaktualnych twierdzeń, np. przed wydaniem.
  • 90epistemic-grounding
    Sprawdza, czy twierdzenia mają pokrycie w dowodach: przypisuje poziomy pewności, weryfikuje fakty i odróżnia wskazówkę od ustalenia.
  • 100verify-both-ways
    Sprawdza twierdzenie w obie strony (fact-check) lub poprawia na miejscu twierdzenia w artefakcie albo dokumencie HTML, w trybach claim i artifact.
  • 100fact-checking
    Weryfikuje twierdzenia techniczne w kodzie, dokumentacji i komentarzach przed scaleniem zmian, sprawdzając, czy zapisane stwierdzenia są prawdziwe.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.