Skip to content
Skills Library
AI & agents · Review & audit

verify

/verify

Fact-check LLM-generated content against source-of-truth — confirm imports import, CLI flags are real, links resolve, counts match. Triggers on: verify docs, fact-check, did the model hallucinate, check generated content, are these claims real.

Repository
attune-ai ↗
Licence
Apache-2.0
Last repository change
24.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 24.10.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Fact-check LLM-generated content against source-of-truth — confirm imports import, CLI flags are real, links resolve, counts match. Triggers on: verify docs, fact-check, did the model hallucinate, check generated content, are these claims real.

Original author description

Fact-check LLM-generated content against source-of-truth — confirm imports import, CLI flags are real, links resolve, counts match. Triggers on: verify docs, fact-check, did the model hallucinate, check generated content, are these claims real.

How it works

  1. What It Does
  2. How To Run It
  3. Declaring Tighter Boundaries
  4. Reading The Report
  5. When It Belongs In A Pipeline
Original SKILL.md headings · 135 lines, 768 words
SKILL.md preview
---
name: verify
description: "Fact-check LLM-generated content against source-of-truth — confirm imports import, CLI flags are real, links resolve, counts match. Triggers on: verify docs, fact-check, did the model hallucinate, check generated content, are these claims real."
---
# Verify

**IMPORTANT: Start your response by telling the user:**

> **Verify** — Fact-checking the named entities in this content
> against the project as source-of-truth.

## What It Does

`/verify` is the output-side fact-checker for LLM-generated content
(docs, READMEs, tutorials). It catches the hallucination classes that
unit tests never see — an invented CLI flag, an import of a package
that doesn't exist, a dead cross-reference, a wrong count — by
confirming the *named entities* in the content actually exist.

It runs in two layers:

1. **Deterministic checkers** (`attune_verify`, authoritative): the
   *top-level package* of each import resolves via `find_spec`, CLI
   flags appear in `--help`, markdown links resolve under the project
   root, numeric claims match a declared count source. These are ground
   truth — no LLM, no guessing.
2. **Ambient semantic cross-check** (you, the agent): after the
   deterministic pass, read the content against its source and flag
   claims the checkers can't see (a security caveat dropped, a route
   path that's plausible-but-wrong, a *private submodule* of an
   installed package that doesn't actually exist — the import checker
   only validates the top-level package, so `from pkg.fake_sub import X`
   passes deterministically when `pkg` is installed). This layer is a
   *cross-check only* — it over-flags on truncated context, so never let
   it override a deterministic pass. If the deterministic layer says a
   top-level import resolves, it resolves.

## How To Run It

### Shared command workspace (preferred)

Open adapter `verify` with the validated generated-content path and optional
`hard_gate=true` for pipeline callers. The invocation authorizes this
read-only run, so there is no confirmation action. Execute
`attune_verify.verify` exactly as below and publish its authoritative outcome
as `deterministic_result`, retaining every finding's kind, severity, detail,
evidence, and location.

Perform the ambient semantic pass only after the deterministic result and
publish it as `cross_check_result`. Ambient findings remain explicitly labeled
warnings and cannot override deterministic entity-existence results. In hard
gate mode, any deterministic error keeps `hard_gate_passed=false`. Checker or
cross-check failure must render “did not complete,” never a clean report.
Present the terminal widget or Markdown and preserve the full evidence chain
in text fallback.

The deterministic layer is `attune_verify.verify(content, context)`.
The caller declares the truth boundaries; verify performs the lookups —
it never auto-discovers them. Run this via Bash from the project root,
passing the file under test as `F`:

```bash
F="<path/to/generated.md>" python -c "
import json, os
from pathlib import Path
from attune_verify import verify, VerifyContext
content = Path(os.environ['F']).read_text(encoding='utf-8')
ctx = VerifyContext(project_root=Path.cwd())
result = verify(content, ctx)
print(json.dumps({
    'ok': result.ok,
    'checked': result.checked,
    'findings': [
        {'kind': f.kind.value, 'severity': f.severity,
         'detail': f.detail, 'evidence': f.evidence, 'location': f.location}
        for f in result.findings
    ],
}, ensure_ascii=False, indent=2))
"
```

Then present the findings as a readable report (errors first, then
warnings), grouped by kind. Each finding carries `kind`
(`unresolved_import` / `unknown_flag` / `dead_link` / `count_mismatch`),
`severity` (`error` | `warning`), `detail`, `evidence`, and `location`.

## Declaring Tighter Boundaries

The default context only checks imports and links (project root). To
catch flag and count hallucinations, declare the sources the content
claims against — `VerifyContext` is the single place to do it:

- **CLI flags** — pre-capture `--help` and pass it so the checker
  knows the real flags:

  ```python
  ctx = VerifyContext(
      project_root=Path.cwd(),
      help_commands={"attune": <captured --help text>},
      allowed_help_cmds=frozenset({"attune"}),  # safe to shell out to
  )
  ```

- **Counts** — declare what a number should be, by label. Values may be
  plain ints or zero-arg callables:

  ```python
  ctx = VerifyContext(
      project_root=Path.cwd(),
      count_sources={"templates": 259, "skills": lambda: len(skill_dirs)},
  )
  ```

A flag whose command is neither pre-captured nor in `allowed_help_cmds`
yields a **warning, not a silent pass** — surface it so the user knows
it went unchecked.

## Reading The Report

- **`result.ok is False`** → at least one error-severity finding. These
  are confirmed hallucinations (the import does not resolve, the flag is
  not in `--help`). Quote the `evidence` line and point at where to fix.
- **Warnings** → something couldn't be checked (no count source
  declared, a command not on the allow-list). Not a failure — a gap in
  the declared boundaries. Tell the user what to declare to close it.
- **Empty findings** → the deterministic layer found nothing wrong.
  Then do the ambient semantic cross-check and report any claims worth a
  human's eye, clearly labeled as cross-check (not deterministic).

Do **not** invent findings, and do **not** soften a deterministic error
into a "maybe" — the checkers are authoritative for entity existence.

## When It Belongs In A Pipeline

For a hard gate (fail the run on any error finding), the library
exposes `raise_if_failed(result)`. The `/verify` skill itself is the
interactive, report-only surface — it never hard-gates. The hard gate
is for callers like attune-author's post-generation step.

Versions

RepositoryScoreLicence
This page · smart-ai-memory/attune-ai100Apache-2.0
yonatangross/orchestkit100MIT
yonatangross/orchestkit100MIT
NolwennD/gabier100Apache-2.0
ericrisco/rsc-harness100MIT
sentimony/agentic-workflow100MIT
JoDi-27/JoDi-SDD100MIT
marcel-bich/marcel-bich-claude-marketplace100MIT
kdlbs/kandev100AGPL-3.0

Identical content appears in 1 other GitHub folders. We show one copy with the highest score.

Database snapshot: 13.11.2026 · package version 355ea58248a3 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100dehallucination
    Use when verifying that AI-generated claims, references, or assertions are grounded in reality. Triggers: 'does this actually exist', 'is this real', 'did you hallucinate', 'verify these references', 'check if this is fabricated', 'reality check', 'ground truth'. Invoked as…
  • 92fact-checker
    This skill should be used when an agent's claims are suspect — either content claims (finding X at line Y) that may be hallucinated, or state claims (claimed to edit file Z) where git diff should confirm. Use this skill — not validator — when the concern is whether the agent…
  • 100hallucination-detection-llm-output-fact-checking
    Use this skill when audit AI-generated claims rigorously against verified documentation and runtime tests.
  • 100verify-claims
    Run Chain-of-Verification (CoVe) on a draft or a block of text with factual claims. Spawns the `claim-verifier` agent in a fresh context (never a conversation fork) so it never sees the draft — then reports which claims are supported, contradicted, or unverifiable. Use when user…
  • 90verify-claims
    Fact-checks a draft, answer, report or AI-generated text before it is used or published. Use when the user asks to check, verify or fact-check something; before finalizing anything with facts, numbers, quotes or citations; or for high-stakes content (legal, medical, financial,…
  • 100verify-claims
    对草稿或包含事实论断的文本运行验证链(Chain-of-Verification,CoVe)。在独立上下文中启动 `claim-verifier` 代理,绝不使用对话分叉,并确保它始终看不到草稿,然后报告哪些论断得到支持、被反驳或无法核验。用户说 "verify these citations"、"check the claims in X"、"did I hallucinate anything"、"fact-check this draft"、"run CoVe on this"、"核验这些引文"、"检查这篇草稿的事实" 或 "看看有没有编造内容"…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.