Skip to content
Skills Library
AI & agents · Testing

e2e-validation

/e2e-validation

Periodically exercise ALL nexus-agents feature families end-to-end against a real task — research → synthesize → vote → plan → dev-pipeline → graph workflow → memory → audit — capturing ACTUAL outputs to validate that fixes, docs, and claims hold in real usage (the things unit tests miss: live voter panels, adapter routing, pipeline stage wiring, audit chains). Use after a release, weekly, when several fixes have landed since the last run, or on demand. Triggers on "e2e validation", "end-to-end validation", "dogfood the loops", "validate end to end", "real usage test", "periodic validation run

Licence
MIT
Last repository change
27.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Periodically exercise ALL nexus-agents feature families end-to-end against a real task — research → synthesize → vote → plan → dev-pipeline → graph workflow → memory → audit — capturing ACTUAL outputs to validate that fixes, docs, and claims hold in real usage (the things unit tests miss: live voter panels, adapter routing, pipeline stage wiring, audit chains). Use after a release, weekly, when several fixes have landed since the last run, or on demand. Triggers on "e2e validation", "end-to-end validation", "dogfood the loops", "validate end to end", "real usage test", "periodic validation run

Triggers from the description

e2e validationend-to-end validationdogfood the loopsvalidate end to endreal usage test
Original author description

Periodically exercise ALL nexus-agents feature families end-to-end against a real task — research → synthesize → vote → plan → dev-pipeline → graph workflow → memory → audit — capturing ACTUAL outputs to validate that fixes, docs, and claims hold in real usage (the things unit tests miss: live voter panels, adapter routing, pipeline stage wiring, audit chains). Use after a release, weekly, when several fixes have landed since the last run, or on demand. Triggers on "e2e validation", "end-to-end validation", "dogfood the loops", "validate end to end", "real usage test", "periodic validation run

How it works

  1. When to run
  2. Preconditions (fail closed)
  3. The loop — exercise every feature family
  4. Capture protocol
  5. On mismatch → file a tracked issue
  6. Output — a short validation report
Original SKILL.md headings · 95 lines, 815 words
SKILL.md preview
---
name: e2e-validation
description: |
  Periodically exercise ALL nexus-agents feature families end-to-end against a
  real task — research → synthesize → vote → plan → dev-pipeline → graph workflow
  → memory → audit — capturing ACTUAL outputs to validate that fixes, docs, and
  claims hold in real usage (the things unit tests miss: live voter panels,
  adapter routing, pipeline stage wiring, audit chains). Use after a release,
  weekly, when several fixes have landed since the last run, or on demand.
  Triggers on "e2e validation", "end-to-end validation", "dogfood the loops",
  "validate end to end", "real usage test", "periodic validation run",
  "full-loop validation".
allowed-tools: Read, Edit, Write, Bash, Grep, Glob, Task, WebSearch, WebFetch
---

# End-to-End Validation Skill

Unit tests prove a function does what its test says. They do **not** prove the
real loops work: that the live voter panel actually returns 7 votes, that
`run_dev_pipeline` wires every stage, that routing picks a reachable adapter,
that the audit chain verifies. This skill runs the **real** loops with live
adapters and compares observed behavior against what the code, docs, and recent
"fixed" claims assert. Where reality diverges, it files a tracked issue.

> Why this exists: this is dogfooding for **validation**, not implementation.
> `dogfooding-issues` implements open issues; `dev-pipeline` runs one pipeline;
> `system-review` is a static health check. This skill is the periodic _real-usage
> smoke test of the whole substrate_.

## When to run

- **After every release** (a published version is a claim that the loops work).
- **Weekly**, or once **≥3 behavior-affecting fixes** have landed since the last run.
- **On demand** when a claim is in doubt (e.g., "the voter race is fixed").

Record the trigger in the report. One run per trigger is enough — don't loop it.

## Preconditions (fail closed)

1. **Live adapter required.** Confirm at least one real CLI/model adapter is
   authenticated (`nexus-agents doctor`). This skill validates _real_ behavior —
   **never** use `consensus_vote { simulateVotes: true }` (random output, #2319)
   or any simulate/mock path. If no live adapter is available, record every step
   as `BLOCKED` and stop — do not fabricate a pass.
2. **Cost awareness.** Real loops spend tokens. If work is cost-gated, note it and
   get authorization before the full run; a reduced run (e.g. `--quick` votes,
   `dryRun` pipelines) is an acceptable partial — label it as such.
3. **Note the versions.** Capture the installed `nexus-agents` version and git SHA
   so the report is reproducible.

## The loop — exercise every feature family

Pick one **real, non-trivial seed task** (a genuine backlog item works best, so the
run produces real value). Drive it through the families below, capturing the
**actual** tool output each step. Don't paraphrase success — paste/observe the real
result and judge it.

| #   | Family        | Tools to exercise                                                                                  | What "real" validates                                                                 |
| --- | ------------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| 1   | Research      | `research_discover`, `research_synthesize`, `research_query`                                       | discovery returns results; synthesis preserves provenance                             |
| 2   | Consensus     | `consensus_vote` (run BOTH `--quick` 3-voter AND full 7-voter, `higher_order` + `simple_majority`) | **all** voters return — catches dead/404 voters (e.g. #3497), OAuth races, escalation |
| 3   | Planning/exec | `orchestrate`, `execute_expert`, `delegate_to_model`                                               | adapter routing picks a reachable model; expert personas resolve                      |
| 4   | Pipelines     | `run_dev_pipeline` (dryRun → real), `run_pipeline`, `run_graph_workflow`                           | every stage wired; no missing-stage / template-resolution gaps                        |
| 5   | Memory        | `memory_write`, `memory_query`, `memory_stats`                                                     | writes persist and read back; counts move                                             |
| 6   | Audit/health  | `verify_audit_chain`, `ci_health_check`, `query_trace`                                             | the run left a verifiable audit trail; traces resolve                                 |
| 7   | Repo/analysis | `repo_analyze`, `search_codebase`, `extract_symbols`                                               | analysis tools run against this repo                                                  |

Adapt the set to what changed since the last run — but a full periodic run should
touch all seven families at least once.

## Capture protocol

For each step record: **what you ran**, the **actual output** (or error), the
**claimed/expected** behavior (from code/docs/changelog), and a verdict —
`PASS` / `FAIL` / `BLOCKED` / `PARTIAL`. A vague "seemed to work" is a FAIL of the
capture, not a PASS of the step.

On a real failure, follow the orchestrator-fallback rule (one retry max, then treat
as a blocker — never silently retry past it) and apply the **Q Protocol** to triage.

## On mismatch → file a tracked issue

When observed behavior contradicts a claim (a "fixed" bug that reproduces, a doc that
lies, a dead voter, a missing stage), file a GitHub issue per the Discovered-Issues
4-point gate — this is canonical tracking, not a memory note.

- **OPSEC:** scrub government / organization / provider references from titles,
  bodies, and pasted output; generalize to "the configured provider" / "an upstream
  provider". Keep the technical content. The auto-file path applies the
  operator-configured `NEXUS_SENSITIVE_REFS` denylist automatically — set it locally
  to your org's terms (they are intentionally not hardcoded in this repo).
- Cite the run (version + SHA), paste the real evidence, and link the claim it
  contradicts (changelog line, doc, or issue that declared it fixed).

## Output — a short validation report

Close with a report (post to the tracking issue and/or `docs/ops/`):

```
E2E Validation — <date> — nexus-agents <version> (<sha>)
Trigger: <release | weekly | N fixes | on-demand>
Coverage: 7/7 families  | Adapters live: <which>
Result: <PASS n> / <FAIL n> / <BLOCKED n> / <PARTIAL n>
Issues filed: #.... , #....
Notes: <one or two lines on anything surprising>
```

Keep it to ~10 lines. The value is the issues filed and the honest verdict, not prose.

Versions

The database has no other skills named e2e-validation.

Database snapshot: 13.11.2026 · package version 02bef9b89f42 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100i4h-workflow-e2e
    Run the maintained workflow data-to-policy pipeline from recording through checkpoint validation. Use for full end-to-end requests; do not use for one individual stage.
  • 100dev-pipeline
    Multi-agent development pipeline (Orchestrator + workers + consensus vote). Use when the user asks to "build a feature", "fix a bug with the pipeline", or "run the dev pipeline". Triggers on "dev pipeline", "multi-agent pipeline", "run pipeline".
  • 93relay-80-100-workflow
    Use when writing agent-relay workflows that must fully validate features end-to-end before merging. Covers the 80-to-100 pattern - going beyond "code compiles" to "feature works, tested E2E locally." Includes repair-before-failure validation gates, review-depth fresh-eyes…
  • 90build-loop-claude-code
    Use when building features with **Claude Code** in any codebase and the work should go through a disciplined build → review → test → fix loop. Triggers on "run the build loop", "build the next task", "continue the plan", "build this feature properly", or any request to implement…
  • 100plugin-test
    Use when writing or reviewing tests for DeepSeek Harness plugins, external DSH plugin packages, or package changes in the deepseek-harness repository. Select the minimum sufficient levels from unit tests, coverage, real-API end-to-end tests, snapshots, Web tests, real…
  • 90build-loop-cursor
    Use when building features with **Cursor** in any codebase and the work should go through a disciplined build → review → test → fix loop. Triggers on "run the build loop", "build the next task", "continue the plan", "build this feature properly", or any request to implement work…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.