santa-method
/santa-method
Przepuszcza wynik przez dwóch niezależnych recenzentów, zanim trafi dalej, gdy kod, treść lub inny dokument nie może oprzeć się na jednym przeglądzie.
- Autor
- peterbamuhigire ↗
- Repozytorium
- chwezi-dev-engine ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 24.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 2
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 24.10.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
- SKILL.md zawiera komentarze HTML (2). Nie widać ich w podglądzie Markdown, a model je czyta.
Co robi i kiedy
Przepuszcza wynik przez dwóch niezależnych recenzentów, zanim trafi dalej, gdy kod, treść lub inny dokument nie może oprzeć się na jednym przeglądzie.
Oryginalny opis (EN)
Use when output must clear two independent adversarial reviewers before it ships — production code, published content, or any deliverable where a single self-reviewing agent shares the blind spots that produced the output. Not for internal drafts or deterministic checks (use build/test/lint pipelines for those).
Jak działa
- The Problem This Solves
- Use When
- Do Not Use When
- Required Inputs
- Workflow
- Phase 1: Generate
- Phase 2: Dual Independent Review
- Quality Standards
- Outputs
- Anti-Patterns
Podgląd SKILL.md
---
name: santa-method
description: Use when output must clear two independent adversarial reviewers before it ships — production code, published content, or any deliverable where a single self-reviewing agent shares the blind spots that produced the output. Not for internal drafts or deterministic checks (use build/test/lint pipelines for those).
metadata:
portable: true
compatible_with:
- claude-code
- codex
origin: "Adapted from affaan-m/ECC skills/santa-method/SKILL.md (Ronald Skelton, RapportScore.ai)"
---
# Santa Method
Multi-agent adversarial verification with a convergence loop. Two independent reviewers, no shared
context, must both PASS before output ships.
## The Problem This Solves
A single agent reviewing its own output shares the same biases, knowledge gaps, and systematic
errors that produced the output. This engine already has `sdlc-meta/skill-engine-audit` (audits a
whole skill engine) and `digital-research-engine/skills/peer-review-loop` (reviewer independence
for research claims, with a red-team memo and dissent disposition). Santa Method is narrower and
more mechanical than either: it is the specific **dual-independent-PASS gate** for a single
deliverable — code, a generated document, a batch of generated content — not a whole-engine audit
or a research claim review. Use it as the gate; use `skill-engine-audit` or `peer-review-loop` for
their own broader jobs.
<!-- dual-compat-start -->
## Use When
- Output will be published, deployed, or consumed by end users
- Compliance, regulatory, or brand constraints must be enforced
- Code ships to production without human review
- Content accuracy matters (technical docs, educational material, customer-facing copy)
- Batch generation at scale where spot-checking misses systemic patterns
- Hallucination risk is elevated (claims, statistics, API references, legal language)
## Do Not Use When
Do not use for internal drafts, exploratory research, or tasks with deterministic verification —
run this engine's build/type/lint/test pipeline for those instead (see
`rules/common/verification.md`).
## Required Inputs
- The task specification the output was generated against.
- The output under review, in full (or the sample, under Batch Sampling).
- A rubric whose every criterion has an objective pass/fail condition (see Rubric Design).
- Authority to launch two fresh, non-fork reviewer agents per round; without it, see Degraded Mode
under Quality Standards.
## Workflow
```
GENERATE -> DUAL INDEPENDENT REVIEW (B, C; no shared context, same rubric)
-> VERDICT GATE: B=PASS AND C=PASS -> SHIP
otherwise -> FIX CYCLE (max 3 rounds, fresh reviewers each
round) -> re-review -> escalate to human if
still NAUGHTY after 3 rounds
```
## Phase 1: Generate
Run the primary task normally. Santa Method is a post-generation verification layer, not a
generation strategy.
## Phase 2: Dual Independent Review
Spawn two review agents (use the `Agent` tool with `subagent_type` other than `fork` — a fork
inherits your context, which defeats the independence requirement here). Critical invariants:
1. **Context isolation** — neither reviewer sees the other's assessment, and neither is a fork of
the generating session.
2. **Identical rubric** — both receive the same evaluation criteria.
3. **Same inputs** — both receive the original spec AND the generated output.
4. **Structured output** — each returns a typed verdict, not free prose.
Reviewer prompt shape:
```
You are an independent quality reviewer. You have NOT seen any other review of this output.
## Task Specification
{task_spec}
## Output Under Review
{output}
## Evaluation Rubric
{rubric}
## Instructions
Evaluate the output against EACH rubric criterion:
- PASS: criterion fully met, no issues
- FAIL: specific issue found (cite the exact problem, file/line where applicable)
Return:
{
"verdict": "PASS" | "FAIL",
"checks": [{"criterion": "...", "result": "PASS|FAIL", "detail": "..."}],
"critical_issues": ["..."],
"suggestions": ["..."]
}
Be rigorous. Your job is to find problems, not to approve.
```
### Rubric Design
Every criterion must have an objective pass/fail condition — vague rubrics produce vague reviews.
| Criterion | Pass Condition | Failure Signal |
|---|---|---|
| Factual accuracy | All claims verifiable against source material | Invented statistics, wrong version numbers, nonexistent APIs |
| Hallucination-free | No fabricated entities, quotes, URLs, references | Links or citations that don't resolve |
| Completeness | Every requirement in the spec is addressed | Missing sections, skipped edge cases |
| Compliance | Passes project-specific constraints | Banned terms, tone violations, regulatory gaps |
| Internal consistency | No contradictions within the output | Section A says X, section B says not-X |
| Technical correctness | Code compiles/runs, logic is sound | Syntax errors, logic bugs |
Domain extensions for this engine's typical deliverables:
- **Code (PHP/TypeScript)**: type safety, error handling coverage, no hardcoded secrets, input
validation, test coverage for new paths — this overlaps with `agents/code-reviewer.md`,
`agents/php-reviewer.md`, `agents/typescript-reviewer.md`; use one of those agents as a Santa
reviewer directly rather than re-deriving the rubric.
- **Content/documentation**: banned-slop terms (see `rules/common/coding-style.md` and the design
engine's anti-slop doctrine where the output is presentation-layer), claims traceable to a real
source per `digital-research-engine`.
### Phase 3: Verdict Gate
Both reviewers must PASS. No partial credit. If only one reviewer catches an issue, that issue is
real — the other reviewer's blind spot is exactly the failure mode this skill exists to eliminate.
### Phase 4: Fix Until Nice (Convergence Loop)
- Max 3 iterations. Fix ONLY the flagged critical issues — no unrequested refactors.
- Each round spawns **fresh reviewer agents** with no memory of the previous round; prior context
creates anchoring bias.
- After 3 rounds without a double-PASS, stop and escalate to a human instead of looping further.
### Pattern: Batch Sampling
For large batches (100+ items), full dual review on every item is cost-prohibitive:
1. Run Santa Method on a random sample (10-15% of the batch, minimum 5 items).
2. Classify failures by type (hallucination, compliance, completeness, etc.).
3. If a systematic pattern emerges, apply a targeted fix to the whole batch, not just the sample.
4. Re-sample and re-verify the fixed batch. Repeat until a clean sample passes.
## Quality Standards
- Both reviewers PASS in the same round; there is no partial credit and no majority vote.
- Reviewers never see each other's assessment, and each round uses fresh reviewers.
- Fixes address only the flagged critical issues; three rounds without a double PASS escalate to a human.
- Degraded mode: when independent agents cannot be launched, the gate cannot be met. Record the
verdict as NOT_ASSESSED for the dual-PASS gate and hand the deliverable to a human reviewer,
rather than reporting a self-review as a Santa PASS.
## Outputs
- One structured verdict per reviewer per round.
- A reconciliation record ending in SHIP or escalation to a human.
## Anti-Patterns
| Failure mode | Symptom | Mitigation |
|---|---|---|
| Infinite loop | Reviewers keep finding new issues after fixes | Max iteration cap (3), then escalate |
| Rubber stamping | Both reviewers pass everything | Adversarial prompt: "Your job is to find problems, not approve" |
| Subjective drift | Reviewers flag style, not errors | Tight rubric, objective pass/fail only |
| Fix regression | Fixing A introduces B | Fresh reviewers each round catch it |
| Agreement bias | Both miss the same thing | Mitigated by independence, not eliminated — add a third reviewer for critical output |
| Cost explosion | Too many iterations on large outputs | Batch sampling pattern; budget caps per cycle |
## Evidence Produced
| Category | Artifact | Format | Example |
|----------|----------|--------|---------|
| Correctness | Two independent reviewer verdicts per round, each against the same rubric | JSON in the reviewer-prompt shape (`verdict`, `checks`, `critical_issues`, `suggestions`) | `reviews/santa/round-1-reviewer-b.json`, `reviews/santa/round-1-reviewer-c.json` |
| Release evidence | Reconciliation record: rounds run, both verdicts per round, issues flagged by one or both reviewers, fixes applied, final outcome (SHIP or escalated) | Markdown | `reviews/santa/reconciliation.md` — "Round 1: B=FAIL (invented API), C=PASS; fixed; round 2: B=PASS, C=PASS; SHIP" |
| Correctness | Batch sample record (Batch Sampling only): sample size and selection, failure taxonomy, batch-wide fix, re-sample result | Markdown table | "120 items, 15 sampled; 3 hallucinated citations; citation check applied to all 120; clean re-sample of 15" |
## References
Companion rules, agents and skills:
| Skill/agent | Relationship |
|---|---|
| `rules/common/verification.md` | Run deterministic checks (build, type, lint, test) first; Santa Method handles the semantic checks those pipelines cannot. |
| `agents/code-reviewer.md`, `php-reviewer.md`, `typescript-reviewer.md` | Use as the reviewer agents themselves for code deliverables instead of writing a fresh rubric each time. |
| `sdlc-meta/skill-engine-audit` | Use for whole-engine audits; Santa Method is for a single deliverable. |
| `digital-research-engine/skills/peer-review-loop` | Use for research-claim review; Santa Method is the mechanical dual-PASS gate, narrower in scope. |
## Metrics
- **First-pass rate**: % of outputs that pass on round 1 (target >70%)
- **Mean iterations to convergence**: average rounds to a double PASS (target <1.5)
- **Issue taxonomy**: distribution of failure types
- **Reviewer agreement**: % of issues flagged by both reviewers vs. only one — low agreement means the rubric needs tightening
- **Escape rate**: issues found post-ship that this gate should have caught (target 0)
<!-- dual-compat-end -->
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · peterbamuhigire/chwezi-dev-engine | 100 | MIT |
| SHRIJANAGAIN/Nemo-Cli | 98 | Apache-2.0 |
| ginaservice/ginaservice-skills | 98 | MIT |
Migawka bazy z 13.11.2026 · wersja paczki 38e8f0078501 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 90interrogate
Kilku recenzentów LLM kwestionuje zmiany z niezależnych punktów widzenia, zanim trafią do scalenia. - 90interrogate
Kilku recenzentów LLM przegląda zmianę z niezależnych stron i zgłasza to, co wymaga poprawy. - 90interrogate
Uruchamia adwersaryjny przegląd kodu lub różnic przez niezależnych recenzentów i zbiera wspólne ustalenia. - 90interrogate
Kilku recenzentów modeli językowych kwestionuje zmiany z różnych stron i zgłasza wątpliwości. - 100adversarial-reviewer
Przeprowadza krytyczny przegląd zmian przez wrogie persony recenzentów, by wyłapać ślepe punkty, których autor nie widzi sam. - 96test-driven-development
Sprawdza, czy test powstał przed kodem przy nowym zachowaniu lub poprawce błędu, zanim kod trafi dalej.