verify

---
name: verify
description: Test a behaviour claim as an investigator and report whether the evidence confirms or refutes it — CONFIRMED / REFUTED / INCONCLUSIVE with bash-captured evidence. Use when asked to verify a claim, prove a fix works, or check that a PR does what it says.
version: 1.0.0
tags: [github, verify, testing, evidence]
---

# Verify

Test a specific claim about behaviour and report whether the evidence supports
or refutes it. The deliverable is a **verdict with evidence**, not an opinion.

This skill uses the **building** skill for installing dependencies and running
the repo's build/test commands in the sandbox.

> **Scope (text-evidence path).** In the default text phase the sandbox agent
> has bash, file read, and the github tools — no browser, no screenshots, no
> video. Prove claims with what bash can capture: test output, command
> stdout/stderr, `curl` against a dev-server you start, log excerpts, file
> contents, exit codes. Driving a real browser and attaching screenshots is the
> separate `browser-qa` skill, which is staged only in the gated browser phase
> running on the docker QA image (Playwright + Chromium). If that skill isn't in
> your catalogue and a claim genuinely can only be shown in a rendered UI, say
> so under **INCONCLUSIVE** rather than guessing — the browser phase will pick
> it up when available.

## Ground rule — investigator, not advocate

Your job is to find out whether the claim is **true**, not to make it look true.
A conclusive "this is broken" with clear evidence is exactly as valuable as a
"this works". Specifically:

- **Never fabricate, hardcode, mock, or stage evidence** to match an expected
  outcome. Do not edit the code under test to make a result appear.
- If the behaviour you observe **contradicts** the claim, that is the result —
  report it as REFUTED with the evidence inline. Don't bury it.
- **Don't retry a failing test hoping for a different answer.** Retry only after
  *changing* the environment or procedure (wrong branch, missing build step),
  and only when you have reason to believe the test setup — not the code — was
  wrong. One unexpected result that reproduces is a finding.

## Parse the target

The Context block gives you the claim and (usually) a PR or issue. The claim may
arrive as:

- A **direct claim** — "ESC cancels streaming in bash mode", "the `--fork` flag
  creates a new session".
- A **PR reference + claim** — verify the claim against that PR's head.
- A **PR reference only** — read the PR description + diff and identify the most
  important, most testable claim it makes; state which claim you chose.

When a PR/issue is given, read its description and diff for context (from the
local checkout — see the **building** skill's workspace notes; don't pull the
diff through the API).

## Decide what would convince a skeptic

Name the single behaviour to observe and the evidence that settles it:

- **Functional claim** (a feature works, a flow completes) → run it and capture
  the output / exit code / resulting state.
- **Regression / fix claim** ("no longer clears the screen", "stops erroring") →
  show the **before and after**: reproduce on the base ref, then show it gone on
  the head ref. Both states must appear in the evidence — an off-camera "it's
  fixed" doesn't count.
- **Output/encoding claim** → capture the exact bytes/text (`xxd`, `od -c`, raw
  stdout) rather than describing them.

## Run the test

Follow the **building** skill to install and build. Then exercise the claim with
the minimal sequence that demonstrates it, capturing evidence at the decisive
step. Start a dev-server in the background and `curl` it when the claim is about
a running service; run the CLI directly when it's a CLI claim; run the targeted
test when the repo already encodes the behaviour as a test.

If the environment blocks a clean test (missing dependency the sandbox can't
provide, build failure you can't resolve, infra that won't run), capture what
blocked it — that drives an **INCONCLUSIVE** result, not a guess.

## Report

Produce your report as your **final message** in this shape — the workflow posts
it for you (don't `github_add_issue_comment` yourself; that would double-post).
**Keep it concise**: a one-line environment summary, the decisive evidence
(commands + key output, not every line), and a one-paragraph conclusion — not an
exhaustive walkthrough.

```
## Verify: <claim>

**Environment:** <branch / commit, package manager, how the app was run>

### Evidence
<commands run + their captured output — fenced. Include both states for a
before/after claim.>

### Conclusion
**CONFIRMED** | **REFUTED** | **INCONCLUSIVE**

<one paragraph: what the evidence shows and why it settles the claim.>
```

- **REFUTED** — state the expected behaviour (from the claim), the observed
  behaviour (from the evidence), include the evidence inline, and note any
  environmental factors (branch, commit, OS). This is a valuable finding, not a
  failed run.
- **INCONCLUSIVE** — report exactly what blocked the test and what would be
  needed to resolve it. Do not guess at the outcome.

## Do NOT

- Retry a failing test more than once without changing the environment/procedure.
- Hardcode expected output, mock responses, or edit the code under test to force
  a result.
- Assume unexpected behaviour means *you* made a mistake — it may be a real bug.
- Omit evidence that contradicts the claim.