Skip to content
Skills Library
Programming · Testing

validate-pr

/validate-pr

Generates and self-executes a diff-derived test plan for a PR. Use when validating PR changes before merge. Do not use for code review; use sanctum:pr-review.

Licence
MIT
Last repository change
01.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 01.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Generates and self-executes a diff-derived test plan for a PR. Use when validating PR changes before merge. Do not use for code review; use sanctum:pr-review.

Original author description

Generates and self-executes a diff-derived test plan for a PR. Use when validating PR changes before merge. Do not use for code review; use sanctum:pr-review.

How it works

  1. When To Use
  2. When NOT To Use
  3. Algorithm
  4. Step 1: Fetch Diff and Detect Areas
  5. Step 2: Generate and Execute Steps per Area
  6. Step 3: Revert-Test Quality Check
  7. Step 4: Final Full-Suite Run
  8. Step 5: Produce Summary Table
  9. Step 6: Posting (--post flag only)
  10. Failure Behaviour
Original SKILL.md headings · 254 lines, 1,319 words
SKILL.md preview
---
name: validate-pr
description: Generates and self-executes a diff-derived test plan for a PR.
  Use when validating PR changes before merge. Do not use for code review;
  use sanctum:pr-review.
alwaysApply: false
category: validation
tags:
- pr
- validation
- test-plan
- diff
- revert-test
- evidence
tools: []
usage_patterns:
- diff-derived-test-plan
- revert-test-quality-check
- evidence-capture
complexity: intermediate
model_hint: standard
estimated_tokens: 650
progressive_loading: false
dependencies:
- leyline:git-platform
- imbue:proof-of-work
role: entrypoint
---

# validate-pr: Diff-Derived Test Plan

Generate and self-execute a validation plan matched to what actually changed
in a PR. Replaces generic "tests pass" with area-targeted evidence and
revert-test quality checks that prove tests catch regressions.

## When To Use

- End of `/fix-pr` Step 5 (Validate), before Step 6 (Complete)
- Standalone after any PR fix, to generate targeted validation evidence
- When you need proof that revert-tests are genuine guards

## When NOT To Use

- `--scope minor` with only formatting or doc changes (no logic changed)
- No diff available (clean branch, nothing changed)
- `--skip-validate` passed to `/fix-pr`

## Algorithm

```
fetch diff -> group by area -> generate steps -> execute -> revert-test -> table
```

## Step 1: Fetch Diff and Detect Areas

```bash
# Get changed file list from the PR
PR_NUMBER=<number from invocation or current branch>
CHANGED=$(gh pr diff "$PR_NUMBER" --name-only)
# Fallback when no PR number:
# CHANGED=$(git diff "origin/$(git rev-parse --abbrev-ref HEAD@{upstream})...HEAD" \
#   --name-only 2>/dev/null)
```

Group changed files into areas using ripgrep (grep if rg unavailable):

```bash
RUST_FILES=$(echo "$CHANGED"  | rg '\.rs$|Cargo\.(toml|lock)$' || true)
PY_FILES=$(echo "$CHANGED"    | rg '\.py$|pyproject\.toml$|requirements.*\.txt$' || true)
SH_FILES=$(echo "$CHANGED"    | rg '\.sh$|\.githooks' || true)
GRAMMAR_FILES=$(echo "$CHANGED" | rg '\.(lark|peg|g4)$' || true)
```

Area routing table:

| Area | File patterns | Verification type |
|------|---------------|-------------------|
| Rust | `*.rs`, `Cargo.toml`, `Cargo.lock` | cargo build + per-crate test |
| Python | `*.py`, `pyproject.toml` | pytest per changed module |
| Shell | `*.sh`, `.githooks/*` | shellcheck |
| Grammar | `*.lark`, `*.peg`, `*.g4` | language-specific lint |
| Build/config | `*.yaml`, `*.json`, `*.toml` | parse check |

## Step 2: Generate and Execute Steps per Area

For each non-empty area, generate and run at least one verification step.
Assign `[E1]`, `[E2]`, ... labels to each captured output.

### Rust

```bash
# Build with default features
cargo build --workspace 2>&1
# Evidence: [En] → "0 errors, 0 warnings"

# Build with --all-features
cargo build --workspace --all-features 2>&1
# Evidence: [En+1]

# Per-crate test for each changed crate
# Extract crate directory from changed path, e.g. crates/token-types/src/lib.rs
CHANGED_CRATES=$(echo "$RUST_FILES" \
  | rg -o '(?:crates|src)/[^/]+' \
  | sort -u \
  | xargs -I{} basename {})
for CRATE in $CHANGED_CRATES; do
  cargo test -p "$CRATE" 2>&1
done
```

### Python

```bash
# Targeted test per changed module
for PY_FILE in $PY_FILES; do
  MODULE=$(basename "${PY_FILE%.py}")
  TEST_FILE="tests/test_${MODULE}.py"
  if [[ -f "$TEST_FILE" ]]; then
    uv run pytest "$TEST_FILE" -v 2>&1
  fi
done

# Or project-specific runner if Makefile target exists
make test 2>&1 || uv run pytest tests/ -v 2>&1
```

### Shell

```bash
for SH_FILE in $SH_FILES; do
  [[ -f "$SH_FILE" ]] && shellcheck "$SH_FILE" 2>&1
done
```

### Build/config parse check

```bash
# YAML files
for YML in $(echo "$CHANGED" | rg '\.ya?ml$' || true); do
  [[ -f "$YML" ]] && python3 -c "import yaml; yaml.safe_load(open('$YML'))" \
    && echo "PASS: $YML" || echo "FAIL: $YML"
done

# JSON files
for JSON_F in $(echo "$CHANGED" | rg '\.json$' || true); do
  [[ -f "$JSON_F" ]] && python3 -m json.tool "$JSON_F" > /dev/null \
    && echo "PASS: $JSON_F" || echo "FAIL: $JSON_F"
done
```

## Step 3: Revert-Test Quality Check

Prove at least one test is a genuine guard, not a dead assertion.

**Safety: abort if the working tree has uncommitted changes.**

```bash
if ! git diff --exit-code > /dev/null 2>&1; then
  echo "[RT] SKIP: working tree dirty: revert-test unsafe"
  # Mark INCONCLUSIVE and continue
fi
```

**Algorithm (one representative fix):**

1. From the changed source files, find one that has a corresponding test.
   - Rust: a `#[test]` in the same crate that exercises a changed function.
   - Python: `tests/test_<module>.py` for a changed `<module>.py`.
   - Shell: a test harness that invokes the changed script.
2. Identify the specific changed line or block from the diff.
3. Edit that line to revert the fix to its broken state.
4. Run the targeted test: confirm it **FAILS** with exit code `1`
   specifically. A pytest usage error (`4`) or an empty collection (`5`)
   is also non-zero, so a harness that only checks "not zero" reports a
   dead assertion as a genuine guard.
5. Restore: `git checkout -- <file>` (git-based restore, safe on interrupt).
6. Run the targeted test again: confirm it **PASSES**.
7. If any step cannot complete, mark INCONCLUSIVE with the reason.

**Revert-test output format:**

```
[RT-1] Target: <file>:<line>: <description of fix>
[RT-2] Broke fix: <edit description>
[RT-3] Ran: <test command> → <test name> FAILED (expected)
[RT-4] Restored: git checkout -- <file>
[RT-5] Ran: <test command> → <test name> PASSED
Result: PASS: test is a genuine guard
```

**Reverting a test that guards document content:**

Content tests assert on prose, and this repo wraps prose at 80 columns,
so any anchor phrase long enough to be meaningful eventually straddles a
line break. Collapse whitespace before matching. Otherwise a pure reflow
turns the test red and tempts an author to "fix" it by unwrapping the
line.

Normalizing reintroduces the hazard the revert test exists to catch: a
rejoined anchor can also appear elsewhere in the file, so deleting the
paragraph the test guards leaves it green. Anchor on a full clause that
is unique to that paragraph, then delete the paragraph and confirm the
test goes red. A DDD paradigm test passed its revert check this way in
PR #612 while guarding nothing.

**When no covering test exists:**

```
Revert-test: INCONCLUSIVE: no covering test for <changed area>
Recommendation: add a test for <changed function or behaviour>
```

## Step 4: Final Full-Suite Run

After all area checks and the revert-test:

```bash
# Rust workspace
cargo test --workspace 2>&1

# Python project
uv run pytest tests/ -v 2>&1

# Mixed project: run both
cargo test --workspace 2>&1 && uv run pytest tests/ -v 2>&1
```

Capture full output as final evidence `[En]`.

## Step 5: Produce Summary Table

```markdown
### validate-pr: <PR title or number>

| Area | Step | Evidence | Result |
|------|------|----------|--------|
| Rust: token-types | cargo build --workspace | [E1] 0 errors | PASS |
| Rust: token-types | cargo test -p token-types | [E2] 12 passed | PASS |
| Rust: token-types | cargo build --all-features | [E3] 0 errors | PASS |
| Shell: hooks/pre-commit | shellcheck | [E4] 0 issues | PASS |
| Revert-test: lib.rs:45 | break/fail/restore | [RT-1..5] genuine guard | PASS |
| Final: cargo test --workspace | full suite | [E5] 694 passed, 0 failed | PASS |

**Totals**: 6 steps: 6 PASS, 0 FAIL, 0 INCONCLUSIVE
```

## Step 6: Posting (--post flag only)

When `--post` is given, post the summary table as a PR comment:

```bash
gh pr comment "$PR_NUMBER" --body "$(cat /tmp/validate-pr-summary.md)"
```

Skip posting when invoked from `/fix-pr`: results feed into the Gate 3
summary comment instead.

## Failure Behaviour

When any step produces **FAIL**:

- Surface the failures in the summary table with the evidence reference.
- When called from `/fix-pr`: halt before Step 6 (Complete). The user must
  fix the failures or pass `--skip-validate` to `/fix-pr` to bypass.
- When called standalone: report failures and exit with non-zero status.

INCONCLUSIVE results are reported but do not halt the workflow.

## Exit Criteria

- [ ] `gh pr diff --name-only` returned a non-empty file list (diff fetched)
- [ ] Every detected area has at least one row in the summary table
- [ ] Every row shows an Evidence reference (`[E1]`, `[E2]`, etc.) with the
  actual command output, not fabricated
- [ ] Revert-test attempted for at least one area with a covering test, or
  documented as INCONCLUSIVE with reason
- [ ] Final full-suite run appears in the summary table
- [ ] Summary table is present with columns: Area, Step, Evidence, Result
- [ ] Any FAIL result halts `/fix-pr` before Step 6 when called from fix-pr
- [ ] Working tree is clean after skill completes (git checkout restore
  confirmed successful for any revert-test mutation)

Versions

RepositoryScoreLicence
This page · athola/claude-night-market100MIT
arraydude/agent-skills100MIT

Database snapshot: 13.11.2026 · package version 5327d2285169 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100code-review
    Review code changes for bugs, security holes and regressions before they merge. Use when the user says "review my changes", "review this PR", "check this diff", "is this safe to merge", or names a branch, commit or files to look over.
  • 100expect
    Diff-aware AI browser testing — reads the git diff, maps changes to affected pages via the route map, generates a targeted test plan, and executes it via agent-browser (Rust daemon + CDP, ARIA-tree-first) with pass/fail reporting. Use when testing UI changes, verifying PRs…
  • 90code-review
    Review code changes for correctness, security, performance, accessibility, maintainability, tests, dependencies, design-system adherence, and localization. Use when the user asks for a code review, PR review, review of local changes, risk assessment, code quality feedback, or…
  • 100go-code-review
    WHEN: User wants an automated first-pass code review on a PR, diff, or set of changes. Also when asked "review this PR", "review my changes", or "what do you think of this code?" WHEN NOT: Full project audits (use go-code-audit), or when user just wants linting (use go-lint-audit).
  • 100review
    Use when code changes need review for security, cost, architecture, or acceptance criteria verification. Triggers on /review, after implementation, pre-merge.
  • 90review
    Review a branch, diff, or pull request in this repository before merge. Use when asked to review changes or a PR, self-review work before opening or marking a PR ready, or check a change for correctness, backward compatibility, security, tests, docs, and changesets.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.