verify
/verify
Fact-check LLM-generated content against source-of-truth — confirm imports import, CLI flags are real, links resolve, counts match. Triggers on: verify docs, fact-check, did the model hallucinate, check generated content, are these claims real.
- Author
- smart-ai-memory ↗
- Repository
- attune-ai ↗
- Licence
- Apache-2.0
- Last repository change
- 24.10.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 24.10.2026, without references to retired models or APIs.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
What it does and when to use it
Fact-check LLM-generated content against source-of-truth — confirm imports import, CLI flags are real, links resolve, counts match. Triggers on: verify docs, fact-check, did the model hallucinate, check generated content, are these claims real.
Original author description
Fact-check LLM-generated content against source-of-truth — confirm imports import, CLI flags are real, links resolve, counts match. Triggers on: verify docs, fact-check, did the model hallucinate, check generated content, are these claims real.
How it works
- What It Does
- How To Run It
- Declaring Tighter Boundaries
- Reading The Report
- When It Belongs In A Pipeline
SKILL.md preview
---
name: verify
description: "Fact-check LLM-generated content against source-of-truth — confirm imports import, CLI flags are real, links resolve, counts match. Triggers on: verify docs, fact-check, did the model hallucinate, check generated content, are these claims real."
---
# Verify
**IMPORTANT: Start your response by telling the user:**
> **Verify** — Fact-checking the named entities in this content
> against the project as source-of-truth.
## What It Does
`/verify` is the output-side fact-checker for LLM-generated content
(docs, READMEs, tutorials). It catches the hallucination classes that
unit tests never see — an invented CLI flag, an import of a package
that doesn't exist, a dead cross-reference, a wrong count — by
confirming the *named entities* in the content actually exist.
It runs in two layers:
1. **Deterministic checkers** (`attune_verify`, authoritative): the
*top-level package* of each import resolves via `find_spec`, CLI
flags appear in `--help`, markdown links resolve under the project
root, numeric claims match a declared count source. These are ground
truth — no LLM, no guessing.
2. **Ambient semantic cross-check** (you, the agent): after the
deterministic pass, read the content against its source and flag
claims the checkers can't see (a security caveat dropped, a route
path that's plausible-but-wrong, a *private submodule* of an
installed package that doesn't actually exist — the import checker
only validates the top-level package, so `from pkg.fake_sub import X`
passes deterministically when `pkg` is installed). This layer is a
*cross-check only* — it over-flags on truncated context, so never let
it override a deterministic pass. If the deterministic layer says a
top-level import resolves, it resolves.
## How To Run It
### Shared command workspace (preferred)
Open adapter `verify` with the validated generated-content path and optional
`hard_gate=true` for pipeline callers. The invocation authorizes this
read-only run, so there is no confirmation action. Execute
`attune_verify.verify` exactly as below and publish its authoritative outcome
as `deterministic_result`, retaining every finding's kind, severity, detail,
evidence, and location.
Perform the ambient semantic pass only after the deterministic result and
publish it as `cross_check_result`. Ambient findings remain explicitly labeled
warnings and cannot override deterministic entity-existence results. In hard
gate mode, any deterministic error keeps `hard_gate_passed=false`. Checker or
cross-check failure must render “did not complete,” never a clean report.
Present the terminal widget or Markdown and preserve the full evidence chain
in text fallback.
The deterministic layer is `attune_verify.verify(content, context)`.
The caller declares the truth boundaries; verify performs the lookups —
it never auto-discovers them. Run this via Bash from the project root,
passing the file under test as `F`:
```bash
F="<path/to/generated.md>" python -c "
import json, os
from pathlib import Path
from attune_verify import verify, VerifyContext
content = Path(os.environ['F']).read_text(encoding='utf-8')
ctx = VerifyContext(project_root=Path.cwd())
result = verify(content, ctx)
print(json.dumps({
'ok': result.ok,
'checked': result.checked,
'findings': [
{'kind': f.kind.value, 'severity': f.severity,
'detail': f.detail, 'evidence': f.evidence, 'location': f.location}
for f in result.findings
],
}, ensure_ascii=False, indent=2))
"
```
Then present the findings as a readable report (errors first, then
warnings), grouped by kind. Each finding carries `kind`
(`unresolved_import` / `unknown_flag` / `dead_link` / `count_mismatch`),
`severity` (`error` | `warning`), `detail`, `evidence`, and `location`.
## Declaring Tighter Boundaries
The default context only checks imports and links (project root). To
catch flag and count hallucinations, declare the sources the content
claims against — `VerifyContext` is the single place to do it:
- **CLI flags** — pre-capture `--help` and pass it so the checker
knows the real flags:
```python
ctx = VerifyContext(
project_root=Path.cwd(),
help_commands={"attune": <captured --help text>},
allowed_help_cmds=frozenset({"attune"}), # safe to shell out to
)
```
- **Counts** — declare what a number should be, by label. Values may be
plain ints or zero-arg callables:
```python
ctx = VerifyContext(
project_root=Path.cwd(),
count_sources={"templates": 259, "skills": lambda: len(skill_dirs)},
)
```
A flag whose command is neither pre-captured nor in `allowed_help_cmds`
yields a **warning, not a silent pass** — surface it so the user knows
it went unchecked.
## Reading The Report
- **`result.ok is False`** → at least one error-severity finding. These
are confirmed hallucinations (the import does not resolve, the flag is
not in `--help`). Quote the `evidence` line and point at where to fix.
- **Warnings** → something couldn't be checked (no count source
declared, a command not on the allow-list). Not a failure — a gap in
the declared boundaries. Tell the user what to declare to close it.
- **Empty findings** → the deterministic layer found nothing wrong.
Then do the ambient semantic cross-check and report any claims worth a
human's eye, clearly labeled as cross-check (not deterministic).
Do **not** invent findings, and do **not** soften a deterministic error
into a "maybe" — the checkers are authoritative for entity existence.
## When It Belongs In A Pipeline
For a hard gate (fail the run on any error finding), the library
exposes `raise_if_failed(result)`. The `/verify` skill itself is the
interactive, report-only surface — it never hard-gates. The hard gate
is for callers like attune-author's post-generation step.
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · smart-ai-memory/attune-ai | 100 | Apache-2.0 |
| yonatangross/orchestkit | 100 | MIT |
| yonatangross/orchestkit | 100 | MIT |
| NolwennD/gabier | 100 | Apache-2.0 |
| ericrisco/rsc-harness | 100 | MIT |
| sentimony/agentic-workflow | 100 | MIT |
| JoDi-27/JoDi-SDD | 100 | MIT |
| marcel-bich/marcel-bich-claude-marketplace | 100 | MIT |
| kdlbs/kandev | 100 | AGPL-3.0 |
Identical content appears in 1 other GitHub folders. We show one copy with the highest score.
Database snapshot: 13.11.2026 · package version 355ea58248a3 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100dehallucination
Use when verifying that AI-generated claims, references, or assertions are grounded in reality. Triggers: 'does this actually exist', 'is this real', 'did you hallucinate', 'verify these references', 'check if this is fabricated', 'reality check', 'ground truth'. Invoked as… - 92fact-checker
This skill should be used when an agent's claims are suspect — either content claims (finding X at line Y) that may be hallucinated, or state claims (claimed to edit file Z) where git diff should confirm. Use this skill — not validator — when the concern is whether the agent… - 100hallucination-detection-llm-output-fact-checking
Use this skill when audit AI-generated claims rigorously against verified documentation and runtime tests. - 100verify-claims
Run Chain-of-Verification (CoVe) on a draft or a block of text with factual claims. Spawns the `claim-verifier` agent in a fresh context (never a conversation fork) so it never sees the draft — then reports which claims are supported, contradicted, or unverifiable. Use when user… - 90verify-claims
Fact-checks a draft, answer, report or AI-generated text before it is used or published. Use when the user asks to check, verify or fact-check something; before finalizing anything with facts, numbers, quotes or citations; or for high-stakes content (legal, medical, financial,… - 100verify-claims
对草稿或包含事实论断的文本运行验证链(Chain-of-Verification,CoVe)。在独立上下文中启动 `claim-verifier` 代理,绝不使用对话分叉,并确保它始终看不到草稿,然后报告哪些论断得到支持、被反驳或无法核验。用户说 "verify these citations"、"check the claims in X"、"did I hallucinate anything"、"fact-check this draft"、"run CoVe on this"、"核验这些引文"、"检查这篇草稿的事实" 或 "看看有没有编造内容"…