run-e2e-tests
/run-e2e-tests
Uruchamia skonfigurowane zestawy testów produktu, zapisuje wyniki do artefaktu i przy błędzie prowadzi pętlę zgłoszeń regresji.
- Autor
- globalmindsolution ↗
- Repozytorium
- gms-marketplace ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 26.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 3
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 26.10.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Uruchamia skonfigurowane zestawy testów produktu, zapisuje wyniki do artefaktu i przy błędzie prowadzi pętlę zgłoszeń regresji.
Wyzwalacze z opisu
Argumenty
/run-e2e-tests [ticket-id] [documents…] [prompt] [--suite <name>]
Oryginalny opis (EN)
Run this product's configured test suites (all of them, a --suite-selected subset, or the suites one run's test cases name), capture pass/fail results to an auditable run artifact, and on a failure triage and drive a closed regression-ticket loop. Use when asked to run the test suites, run a named suite (e.g. "run the e2e suite"), or check whether anything broke. Use for any request to run, execute or kick off the repo's test suites — all of them or named ones (smoke, api, integration, e2e, nightly) — including after a rebase or when CI is down; it reads the configured suites from acs settings
Jak działa
- Start
- Step 1 — Resolve context and arguments
- Step 2 — Per-suite execution: setup → command → teardown
- Step 3 — Write the results artifact
- Step 4 — All-green short-circuit (deterministic guarantee)
- Which suites this run runs
- Step 4a — Triage (model step, failure path only)
- Step 4b — Dedup / recurrence lookup
- Step 5 — Report
- Scheduling surface
Podgląd SKILL.md
---
name: run-e2e-tests
description: Run this product's configured test suites (all of them, a --suite-selected subset, or the suites one run's test cases name), capture pass/fail results to an auditable run artifact, and on a failure triage and drive a closed regression-ticket loop. Use when asked to run the test suites, run a named suite (e.g. "run the e2e suite"), or check whether anything broke. Use for any request to run, execute or kick off the repo's test suites — all of them or named ones (smoke, api, integration, e2e, nightly) — including after a rebase or when CI is down; it reads the configured suites from acs settings itself, so do not look for package.json, CI files or a test runner. Call it as your first action on such a request — do not Glob, Grep or Read for the ticket, plan, run or repo files, and do not look for a shell: it locates all of them itself.
argument-hint: "[ticket-id] [documents…] [prompt] [--suite <name>]"
disallowed-tools: Edit, NotebookEdit
---
You are the coordinator of `/acs:run-e2e-tests`, the acs suite runner. It runs
the e2e suites `/acs:create-e2e-tests` wrote — and any other suite the project
configures — and records what happened.
It is a **step of `ship.yaml`** and it is also a standing command you can run
at any time against a repo. Those are the same skill and the same protocol:
the step records its start and finish like every other, and a standing run
records them against whatever run the checkout is on. There is no second mode
and no second set of rules (§3.11).
`/acs:create-e2e-tests` WRITES a run's e2e suites; this skill RUNS the
configured suites. It spawns no subagents: the work is running commands and
reading their output, which is what the coordinator is already for.
Scope honesty up front: this skill is **not read-only**. Every run **writes** a
results artifact (see Step 3), and on a failure path it **mutates** ticket
state (regression tickets minted, commented, or linked).
## Start
MANDATORY first action — run exactly:
```bash
python3 "${CLAUDE_PLUGIN_ROOT}/hooks/scripts/acs.py" step start --step run-e2e-tests
```
If it exits non-zero: STOP and surface its stderr verbatim. **When the plan
says nothing is owed, the pre-hook does not start you at all**: it reads ONE
thing — the `owes.e2e` flag in the `## Contract` block of the run's
`steps/create-impl-plan/plan.md` (`acs_lib.stepgate.noop_decision`) — and when
that flag is `false` it records this step completed with
`outcome: nothing_to_run` and the plan's own reason, and this skill is never
invoked (§2.2). Zero tokens. (`/acs:create-e2e-tests` no-ops on the same flag
with `no_e2e_owed`; the pre-hook reads the flag, not that step's record.)
That is the ONLY thing the pre-hook settles. It never reads the settings, so a
project that configures no suite is not caught there, and neither is a run
with no plan, or a plan that is silent on `e2e` (silence is not permission to
skip). Those reach you, and Step 1 and "Which suites this run runs" decide
them — `no_harness` or `nothing_to_run` — so do not assume, once you are
running, that there is something to run.
**Requirements: `context.requirements` / `acs.py requirements show` — a ticket
id, documents and a prompt are only where they came from; never read
ticket.json for acceptance criteria.** No ticket is required: a run started on
a prompt or documents runs its suites the same way.
`${CLAUDE_PLUGIN_ROOT}/docs/INTERNALS.md` carries resume-and-reconcile,
context pressure and the completion report — the parts every acs skill shares.
**Clarification ledger first.** Before asking the user anything, run
`python3 "${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clarify.py" list` and reuse any
recorded answer. When ≥2 clarifications are open, present them in ONE grouped
interaction. Record each answer as its own `clarify.py add --skill
run-e2e-tests` entry — one `C-<n>` per question — BEFORE acting on it. Never
skip a question, never merge two into one entry, and never auto-answer one
outside the `--source assumption --rationale "..."` rule; when the user is
unavailable, record the decision as an assumption under that rule.
## Step 1 — Resolve context and arguments
Call `acs_lib.build_context(cwd)` to resolve `settings`, `workspace`, and
`repo_id` exactly as other unhooked skills do. Read
`ctx["settings"].get("tests", {})` — the suites this run can run are its
keys other than `coverage` (the coverage target is a number, not a suite);
call that map `suites` below.
Parse `$ARGUMENTS` for zero or more `--suite <name>` flags:
- **No `--suite` flag:** run every entry in this run's **run set** — see
"Which suites this run runs" below.
- **One or more `--suite <name>` flags:** run only the named subset, in the
order given, narrowing whichever run set the mode already resolved. If a
named suite is not a key in `suites`, fail fast with a clear error
identifying the unknown name(s) — do not silently skip it or fall back to
running all suites.
- If `suites` itself is `{}` — the repo configures no suite under
`settings.tests` — report that plainly ("no
suites configured: no harness to run") and finish `completed` with
`outcome: no_harness`.
- If `suites` is non-empty but this run's run set resolves to `{}` (see
"Which suites this run runs"), report that plainly ("suites are
configured, none of them is in this run's run set") and finish `completed`
with `outcome: nothing_to_run`.
- In both cases there is nothing to execute, but you still emit a valid,
empty-arrays artifact (`suites: [], regressions: []`) per Step 3.
## Step 2 — Per-suite execution: setup → command → teardown
For each suite to run, in order:
- Record a start timestamp.
- If the suite entry has a non-empty `setup` string, run it via the shell
exactly as configured (same trust boundary as the already-shipped `e2e`
runner — no new arbitrary-input path).
- Run the suite's `command` string, capturing its exit code and
stdout/stderr.
- **Always** run `teardown` (if present) — teardown runs even after a
failing `command`, pass or fail, and its own exit code never overwrites
the suite's verdict (the verdict is `command`'s exit code only).
- Record an end timestamp; `duration_s` is the elapsed wall-clock time across
`setup`→`command`→`teardown` combined (the full cost the caller pays per
suite).
- Suite `status` is `"pass"` when `command`'s exit code is `0`, `"fail"`
otherwise. `failure_output` is populated only when `status` is `"fail"`
(captured stdout/stderr from `command`, truncated to a reasonable bound so
the artifact stays readable and auditable, never multi-megabyte).
**R1 safety — no interpolation.** Never build `setup`, `command`, or
`teardown` by string-interpolating captured failure output, another suite's
exit code, or any other runtime-captured text into a shell command. Each
suite's `setup`/`command`/`teardown` strings come verbatim from
`settings.tests.<name>` as configured — never rewritten, wrapped, or
concatenated with captured data before execution. Captured output is stored
as artifact data only; it is never re-injected as executable input.
## Step 3 — Write the results artifact
After all selected suites have run (or immediately, in the zero-suites case),
write JSON to `<workspace>/<repo_id>/test-runs/<run-id>/results.json` through
`acs.py write` (as in Finish; the absolute path, parents created for you), where
`<workspace>` and `<repo_id>` are exactly what `build_context()` resolved
(the same repo-level directory `acs_lib.repo_dir(workspace, repo_id)`
returns — sibling to `tickets-index.json`, NOT inside any
ticket partition, since `/acs:run-e2e-tests` is unticketed). `<run-id>` is
`run-<ISO8601>` (an ISO-8601 UTC timestamp with filesystem-path-safe
characters — colons replaced or omitted).
Artifact shape:
```json
{
"run_id": "run-<ISO8601>",
"started_at": "<ISO8601>",
"ended_at": "<ISO8601>",
"suites": [
{
"name": "<suite-name>",
"command": "<configured command string>",
"exit_code": 0,
"duration_s": 0.0,
"status": "pass",
"failure_output": "<only present when status is \"fail\">"
}
],
"regressions": []
}
```
`regressions` is **always** emitted as an array — this spec's runs always emit
the empty list `[]` (only the failure-path closed loop, layered onto this
same file, ever populates entries into it).
## Step 4 — All-green short-circuit (deterministic guarantee)
After the results artifact is written, check whether every suite that ran has
`status: "pass"`. **If so, stop here.** This is a hard requirement, not a
default that happens to be true today: on an all-green run, **no triage step
runs and no model call of any kind is made** — `regressions` stays the empty
list `[]`. A future edit must not silently introduce a model call on this
branch.
Only when at least one suite's `status` is `"fail"` does control pass to the
failure-path steps below (triage, regression-key derivation, dedup/recurrence,
ticket mint/comment/link).
## Which suites this run runs
The run set is **narrowed by the run's own subject**, always. There is no
second mode: a standing invocation and a `ship.yaml` step resolve the same way
(§3.11), which is what makes the two the same skill rather than two skills
sharing a file.
The run's DOCUMENTS narrow it; the `e2e` key never does on its own. Which
rule applies depends only on which documents the run has:
1. The reserved `e2e` key, if `suites` carries one, is
ADDED to whatever rule 2 or 3 selects — it is the suite
`/acs:create-e2e-tests` wrote into, so a run that has documents always
runs it. It is not a selector by itself: it never decides between rules
2-4, and on its own it narrows nothing.
2. When the run wrote a `test-cases.md`: the suites named in it — the `Suite`
column of its `## Cases` table, at `steps/create-test-docs/test-cases.md`
— plus rule 1. `/acs:create-test-docs` is what assigns each case to a
suite, so the run's own cases are what scopes it.
3. When the run wrote no `test-cases.md` but has a plan: the suites named in
the plan's Test-strategy section, at `steps/create-impl-plan/plan.md` —
plus rule 1.
4. When the run has NEITHER document — a run started from a prompt against a
repo, with nothing planned — every entry in `suites`,
whether or not an `e2e` key is among them. A ticket run with no
`test-cases.md` and no plan lands here too: the `e2e` key alone does not
narrow it to `{e2e}`. That is the honest answer there: nothing has said
which suites this change bears on, so the run cannot narrow.
This is re-resolved on every invocation, never cached from an earlier call, so
a later case-document or plan write is picked up automatically. A suite named
in either document is included only when it is also a key in
`suites`.
**When nothing resolves** — suites are configured, and rule 2 or 3 applied
but named none of them and there is no `e2e` key — record
`outcome: nothing_to_run` and finish. A run with no suite to run is a
completed step with a reason, not a failure: the absence of a suite is not
evidence that anything is broken. (A repo that configures no suite at all is
`no_harness`, Step 1 — rule 4 over an empty map.)
## Step 4a — Triage (model step, failure path only)
For each suite whose `status` is `"fail"`, read that suite's captured
`failure_output`, `exit_code`, and `command` (from the in-memory run state or
the just-written `results.json` — either is acceptable) and produce:
1. A **regression summary**: one paragraph covering what broke, the likely
cause, and the first-glance affected component.
2. A **stable regression key** of the form:
```
<suite-name>:<normalized-failing-test-id>
```
where `<suite-name>` is the exact suite name (the `settings.tests` key) the failure came from, and `<normalized-failing-test-id>` is the
lowercase, whitespace-collapsed identifier of the single failing test
parsed out of the suite's failure output.
**C-1 fallback (binding, not optional):** when the suite's failure output
yields **no parseable individual failing-test id** — a bare non-zero exit
code, a compile/build error before any test runs, or any other
suite-level failure with no per-test identifier to extract — the
regression key falls back to the **coarse suite-level key**:
```
<suite>:__suite__
```
This is a **literal fixed marker, not a content fingerprint or hash**: a
suite known to be flaky is a suite-authoring concern, not something this
closed loop should amplify by minting a new ticket per distinct
(unparseable) failure blob. The fallback intentionally collapses every
unparseable failure from the same suite onto **one ticket per broken
suite**.
Multiple failing suites in the same run each get their own regression key
(and therefore their own dedup/mint/bump decision) — process the
`regressions` candidates one key at a time.
## Step 4b — Dedup / recurrence lookup
For each regression key, search `tickets-index.json`
(`acs_lib.index_path(workspace, repo_id)`, read via `acs_lib.read_json`) for
an existing ticket whose description carries that exact regression-key
marker (see "Regression key marker convention" below) — scanning each
candidate's own `description` field loaded via `acs_lib.load_ticket` at the
partition resolved by `acs_lib.find_ticket_partition(workspace, repo_id,
ticket_id)` (active partition first, then `archive/`).
Given a matching ticket (if any) and its `status`
(`acs_lib.TICKET_STATUSES = ["open", "in_progress", "in_review", "done"]`),
apply this **three-way policy** exactly:
1. **Not found → mint a new ticket.** Invoke `new-ticket.py` (unchanged —
this skill reuses it as-is, no edits to the file) with `--title "<suite>:
<test> regression"` (or, under the suite-level fallback, `--title
"<suite>: suite regression"`), `--type task`, and `--description "..."`
embedding, in order: the regression-key marker, the triage summary, and a
link to the results artifact
(`<workspace>/<repo>/test-runs/<run-id>/results.json`). Record the
printed `{"ticket_id", "partition"}` and mark this regression's action as
`"minted"`. If `new-ticket.py` exits non-zero, STOP and surface its stderr
verbatim — never invent a `ticket_id` for the `regressions[]` entry. In
particular, on a workspace partition that has never allocated an id it
refuses with exit 2 and a local-evidence reconciliation proposal
(`allocate_ticket_id`'s fail-closed gate, MAR-402) instead of minting the
regression ticket: relay that stderr, obtain the confirmed start number
from the user, and re-run `new-ticket.py` with `--seed-next <n>` added
(only `new-ticket.py` mints ids — never hand-pick one).
2. **Found, ticket status is `open`, `in_progress`, or `in_review` →
comment-bump the existing ticket — never mint a duplicate, never silently skip.**
Append fresh evidence (the new run id, the current timestamp, and
the latest failure output) to the existing ticket's `description` (loaded
via `acs_lib.load_ticket`, appended to, saved via `acs_lib.save_ticket`,
re-indexed via `acs_lib.update_index`). Mark this regression's action as
`"commented"` with `ticket_id` set to the existing ticket's id.
3. **Found, ticket status is `done` → the regression recurred after being
marked fixed: mint a NEW ticket linked to the old one — never silently reopen
the closed ticket.** Call `new-ticket.py` exactly as in case 1, including
its non-zero-exit / reconciliation-refusal handling above, but
the `--description` additionally states explicitly that this is a
recurrence and names the old (closed) ticket id it links back to. Mark
this regression's action as `"minted_linked"` with `linked_ticket_id` set
to the old ticket's id. The closed ticket itself is left untouched
(status stays `done`) — reopening it would hide that a shipped fix
regressed.
**Regression key marker convention.** Every minted/bumped ticket's
`description` embeds the literal marker line:
```
acs-regression-key: <suite>:<normalized-failing-test-id-or-__suite__>
```
on its own line (mirrors the existing `acs-ticket: <ID>` convention already
used elsewhere in ticket descriptions). The dedup lookup searches for this
exact line, not a fuzzy match on summary prose.
**R1 safety — no interpolation (restated for the failure path).** No failure
content (suite output, parsed test ids, triage summary text) is **ever
interpolated into a shell command**. The only commands this step invokes are
in-process reads (`acs_lib.read_json`, `acs_lib.load_ticket`,
`acs_lib.find_ticket_partition`) and one write path — invoking `new-ticket.py`
as a subprocess with `--title`, `--type`, and `--description` passed as
**argument values**, never shell-interpolated into a command string. Failure
output/triage text becomes ticket **description content only**, never a
command or command fragment.
After processing every failing suite's regression key, populate the results
artifact's `regressions[]` array (Step 3's shape) with one entry per
processed key:
```json
{
"key": "<suite>:<normalized-failing-test-id-or-__suite__>",
"ticket_id": "<TICKET-ID>",
"action": "minted" | "commented" | "minted_linked",
"linked_ticket_id": "<OLD-TICKET-ID>"
}
```
`linked_ticket_id` is present only when `action` is `"minted_linked"`.
## Step 5 — Report
Print a run summary: suites run (count and names), pass/fail counts, and
tickets minted/bumped/linked (on an all-green run, this line is simply "0
tickets minted/bumped/linked" since `regressions` is always `[]` on that
path). State explicitly that the results artifact is left in place on disk
after the run — it is not cleaned up — so it can be read later.
**Under `/acs:ship`.** Replace the tickets-minted/bumped/linked line with the
verdict (`pass` / `fail`) and name the run, so the caller can read the outcome
out of a transcript without opening the artifact.
## Scheduling surface
`/acs:run-e2e-tests` carries no notion of "I am scheduled" versus "I was
invoked by a person" — it is a pure function of "which suites, right now,"
and the caller (cron, a CI scheduled workflow, or a Claude Code routine)
decides when to invoke it. See
`${CLAUDE_PLUGIN_ROOT}/skills/run-e2e-tests/references/test-scheduling.md` for
the concrete cron/CI/routine recipe; this skill itself has no built-in
scheduler (ADR 0011 G8).
## Finish
MANDATORY final step — never skipped, also on failure. Write `steps/run-e2e-tests/result.json`
through `acs.py write` (never the Write tool), below. A run that completed carries
`"status": "completed"` and an `outcome` of `passed`, `no_harness` or
`nothing_to_run`. A run where any suite failed carries `"status": "failed"`
and no `outcome` — the failing suites and the regressions this run minted,
bumped or linked are in its `summary` and `findings`, and the results artifact
is the evidence. Then:
```bash
python3 "${CLAUDE_PLUGIN_ROOT}/hooks/scripts/acs.py" write steps/run-e2e-tests/result.json <<'ACS_EOF'
<the result document>
ACS_EOF
python3 "${CLAUDE_PLUGIN_ROOT}/hooks/scripts/post-run-e2e-tests.py" --result-file "<the result.json you just wrote>"
```
The status and outcome are read from that document — a step's transition is
read from its result, not asserted on the command line. The two empty
outcomes mean one thing each, the same as in the fragment
(`skills/run-e2e-tests/state.schema.json`): `no_harness` — the repo
configures no suite at all (no suite under `settings.tests`); `nothing_to_run` — suites are configured but this run
has none to run (the plan's `owes.e2e: false`, settled by the pre-hook, or a
run set that resolved empty). Both are honest completions, not failures: a
step with nothing to do says which kind of nothing, rather than failing a run
over the absence of its own evidence.
## Completion report (normative)
End your final message with the standard completion block; replace the Ticket
line with **Run** (this skill is run-scoped, not tied to one ticket). Under
`/acs:ship` the **Findings** line states the verdict (`pass` / `fail`) instead
of a tickets-minted count:
```markdown
## /acs:run-e2e-tests · <status>
- **Run**: <run-id>, <N> suites run
- **Status**: <status> — <one line>
- **Results**: <pass count>/<N> suites passed
- **Findings**: <regressions minted/bumped/linked, or "none — all suites passed">
- **Artifacts**: results artifact at test-runs/<run-id>/results.json (left in place)
- **Metrics**: n/a
- **Next**: <e.g. re-run after a fix lands, or schedule a standing run>
```
Pliki w paczce (3)
- SKILL.md 20 KB
- state.schema.json 1 KB
- references/test-scheduling.md 2 KB
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · globalmindsolution/gms-marketplace | 100 | MIT |
| codescene-oss/codescene-mcp-server | 100 | Apache-2.0 |
Migawka bazy z 13.11.2026 · wersja paczki dde47f9b3d18 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100run-regression
Uruchamia zestaw testów regresji projektu w całości, dla jednego repozytorium lub zakresu przypadków, i raportuje wyniki pass i fail. - 100api-test-suite-builder
Generuje testy API, zestawy testów integracyjnych, testy endpointów REST i testy kontraktowe. - 100test-all
Uruchamia pełny zestaw testów Momentum CMS, wszystkie suity bez pomijania, na każdą prośbę o przetestowanie całości. - 85proof
Działa jako inżynier QA i testów: strategia testów, zestawy E2E, testy API, triage niestabilnych testów i pokrycie. - 100run-tests
Buduje aplikację z ostatniego commita, uruchamia ją w laboratorium VM, przechodzi zestawy testów i poprawia błędy, aż testy przejdą. - 100api-test-suite-builder
Generuje zestawy testów API z definicji tras w różnych frameworkach: uwierzytelnianie, walidacja, kontrakty, testy obciążeniowe k6, mocki i generowanie z OpenAPI.