Przejdź do treści
Biblioteka skilli
Programowanie · Testowanie

run-e2e-tests

/run-e2e-tests

Uruchamia skonfigurowane zestawy testów produktu, zapisuje wyniki do artefaktu i przy błędzie prowadzi pętlę zgłoszeń regresji.

Licencja
MIT
Ostatnia zmiana repozytorium
26.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
3
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 26.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Uruchamia skonfigurowane zestawy testów produktu, zapisuje wyniki do artefaktu i przy błędzie prowadzi pętlę zgłoszeń regresji.

Wyzwalacze z opisu

run the e2e suite

Argumenty

/run-e2e-tests [ticket-id] [documents…] [prompt] [--suite <name>]
Oryginalny opis (EN)

Run this product's configured test suites (all of them, a --suite-selected subset, or the suites one run's test cases name), capture pass/fail results to an auditable run artifact, and on a failure triage and drive a closed regression-ticket loop. Use when asked to run the test suites, run a named suite (e.g. "run the e2e suite"), or check whether anything broke. Use for any request to run, execute or kick off the repo's test suites — all of them or named ones (smoke, api, integration, e2e, nightly) — including after a rebase or when CI is down; it reads the configured suites from acs settings

Jak działa

  1. Start
  2. Step 1 — Resolve context and arguments
  3. Step 2 — Per-suite execution: setup → command → teardown
  4. Step 3 — Write the results artifact
  5. Step 4 — All-green short-circuit (deterministic guarantee)
  6. Which suites this run runs
  7. Step 4a — Triage (model step, failure path only)
  8. Step 4b — Dedup / recurrence lookup
  9. Step 5 — Report
  10. Scheduling surface
Sekcje z nagłówków SKILL.md, w oryginale · 401 linii, 2 866 słów
Podgląd SKILL.md
---
name: run-e2e-tests
description: Run this product's configured test suites (all of them, a --suite-selected subset, or the suites one run's test cases name), capture pass/fail results to an auditable run artifact, and on a failure triage and drive a closed regression-ticket loop. Use when asked to run the test suites, run a named suite (e.g. "run the e2e suite"), or check whether anything broke. Use for any request to run, execute or kick off the repo's test suites — all of them or named ones (smoke, api, integration, e2e, nightly) — including after a rebase or when CI is down; it reads the configured suites from acs settings itself, so do not look for package.json, CI files or a test runner. Call it as your first action on such a request — do not Glob, Grep or Read for the ticket, plan, run or repo files, and do not look for a shell: it locates all of them itself.
argument-hint: "[ticket-id] [documents…] [prompt] [--suite <name>]"
disallowed-tools: Edit, NotebookEdit
---

You are the coordinator of `/acs:run-e2e-tests`, the acs suite runner. It runs
the e2e suites `/acs:create-e2e-tests` wrote — and any other suite the project
configures — and records what happened.

It is a **step of `ship.yaml`** and it is also a standing command you can run
at any time against a repo. Those are the same skill and the same protocol:
the step records its start and finish like every other, and a standing run
records them against whatever run the checkout is on. There is no second mode
and no second set of rules (§3.11).

`/acs:create-e2e-tests` WRITES a run's e2e suites; this skill RUNS the
configured suites. It spawns no subagents: the work is running commands and
reading their output, which is what the coordinator is already for.

Scope honesty up front: this skill is **not read-only**. Every run **writes** a
results artifact (see Step 3), and on a failure path it **mutates** ticket
state (regression tickets minted, commented, or linked).

## Start

MANDATORY first action — run exactly:

```bash
python3 "${CLAUDE_PLUGIN_ROOT}/hooks/scripts/acs.py" step start --step run-e2e-tests
```

If it exits non-zero: STOP and surface its stderr verbatim. **When the plan
says nothing is owed, the pre-hook does not start you at all**: it reads ONE
thing — the `owes.e2e` flag in the `## Contract` block of the run's
`steps/create-impl-plan/plan.md` (`acs_lib.stepgate.noop_decision`) — and when
that flag is `false` it records this step completed with
`outcome: nothing_to_run` and the plan's own reason, and this skill is never
invoked (§2.2). Zero tokens. (`/acs:create-e2e-tests` no-ops on the same flag
with `no_e2e_owed`; the pre-hook reads the flag, not that step's record.)

That is the ONLY thing the pre-hook settles. It never reads the settings, so a
project that configures no suite is not caught there, and neither is a run
with no plan, or a plan that is silent on `e2e` (silence is not permission to
skip). Those reach you, and Step 1 and "Which suites this run runs" decide
them — `no_harness` or `nothing_to_run` — so do not assume, once you are
running, that there is something to run.

**Requirements: `context.requirements` / `acs.py requirements show` — a ticket
id, documents and a prompt are only where they came from; never read
ticket.json for acceptance criteria.** No ticket is required: a run started on
a prompt or documents runs its suites the same way.

`${CLAUDE_PLUGIN_ROOT}/docs/INTERNALS.md` carries resume-and-reconcile,
context pressure and the completion report — the parts every acs skill shares.

**Clarification ledger first.** Before asking the user anything, run
`python3 "${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clarify.py" list` and reuse any
recorded answer. When ≥2 clarifications are open, present them in ONE grouped
interaction. Record each answer as its own `clarify.py add --skill
run-e2e-tests` entry — one `C-<n>` per question — BEFORE acting on it. Never
skip a question, never merge two into one entry, and never auto-answer one
outside the `--source assumption --rationale "..."` rule; when the user is
unavailable, record the decision as an assumption under that rule.

## Step 1 — Resolve context and arguments

Call `acs_lib.build_context(cwd)` to resolve `settings`, `workspace`, and
`repo_id` exactly as other unhooked skills do. Read
`ctx["settings"].get("tests", {})` — the suites this run can run are its
keys other than `coverage` (the coverage target is a number, not a suite);
call that map `suites` below.

Parse `$ARGUMENTS` for zero or more `--suite <name>` flags:

- **No `--suite` flag:** run every entry in this run's **run set** — see
  "Which suites this run runs" below.
- **One or more `--suite <name>` flags:** run only the named subset, in the
  order given, narrowing whichever run set the mode already resolved. If a
  named suite is not a key in `suites`, fail fast with a clear error
  identifying the unknown name(s) — do not silently skip it or fall back to
  running all suites.
- If `suites` itself is `{}` — the repo configures no suite under
  `settings.tests` — report that plainly ("no
  suites configured: no harness to run") and finish `completed` with
  `outcome: no_harness`.
- If `suites` is non-empty but this run's run set resolves to `{}` (see
  "Which suites this run runs"), report that plainly ("suites are
  configured, none of them is in this run's run set") and finish `completed`
  with `outcome: nothing_to_run`.
- In both cases there is nothing to execute, but you still emit a valid,
  empty-arrays artifact (`suites: [], regressions: []`) per Step 3.

## Step 2 — Per-suite execution: setup → command → teardown

For each suite to run, in order:

- Record a start timestamp.
- If the suite entry has a non-empty `setup` string, run it via the shell
  exactly as configured (same trust boundary as the already-shipped `e2e`
  runner — no new arbitrary-input path).
- Run the suite's `command` string, capturing its exit code and
  stdout/stderr.
- **Always** run `teardown` (if present) — teardown runs even after a
  failing `command`, pass or fail, and its own exit code never overwrites
  the suite's verdict (the verdict is `command`'s exit code only).
- Record an end timestamp; `duration_s` is the elapsed wall-clock time across
  `setup`→`command`→`teardown` combined (the full cost the caller pays per
  suite).
- Suite `status` is `"pass"` when `command`'s exit code is `0`, `"fail"`
  otherwise. `failure_output` is populated only when `status` is `"fail"`
  (captured stdout/stderr from `command`, truncated to a reasonable bound so
  the artifact stays readable and auditable, never multi-megabyte).

**R1 safety — no interpolation.** Never build `setup`, `command`, or
`teardown` by string-interpolating captured failure output, another suite's
exit code, or any other runtime-captured text into a shell command. Each
suite's `setup`/`command`/`teardown` strings come verbatim from
`settings.tests.<name>` as configured — never rewritten, wrapped, or
concatenated with captured data before execution. Captured output is stored
as artifact data only; it is never re-injected as executable input.

## Step 3 — Write the results artifact

After all selected suites have run (or immediately, in the zero-suites case),
write JSON to `<workspace>/<repo_id>/test-runs/<run-id>/results.json` through
`acs.py write` (as in Finish; the absolute path, parents created for you), where
`<workspace>` and `<repo_id>` are exactly what `build_context()` resolved
(the same repo-level directory `acs_lib.repo_dir(workspace, repo_id)`
returns — sibling to `tickets-index.json`, NOT inside any
ticket partition, since `/acs:run-e2e-tests` is unticketed). `<run-id>` is
`run-<ISO8601>` (an ISO-8601 UTC timestamp with filesystem-path-safe
characters — colons replaced or omitted).

Artifact shape:

```json
{
  "run_id": "run-<ISO8601>",
  "started_at": "<ISO8601>",
  "ended_at": "<ISO8601>",
  "suites": [
    {
      "name": "<suite-name>",
      "command": "<configured command string>",
      "exit_code": 0,
      "duration_s": 0.0,
      "status": "pass",
      "failure_output": "<only present when status is \"fail\">"
    }
  ],
  "regressions": []
}
```

`regressions` is **always** emitted as an array — this spec's runs always emit
the empty list `[]` (only the failure-path closed loop, layered onto this
same file, ever populates entries into it).

## Step 4 — All-green short-circuit (deterministic guarantee)

After the results artifact is written, check whether every suite that ran has
`status: "pass"`. **If so, stop here.** This is a hard requirement, not a
default that happens to be true today: on an all-green run, **no triage step
runs and no model call of any kind is made** — `regressions` stays the empty
list `[]`. A future edit must not silently introduce a model call on this
branch.

Only when at least one suite's `status` is `"fail"` does control pass to the
failure-path steps below (triage, regression-key derivation, dedup/recurrence,
ticket mint/comment/link).

## Which suites this run runs

The run set is **narrowed by the run's own subject**, always. There is no
second mode: a standing invocation and a `ship.yaml` step resolve the same way
(§3.11), which is what makes the two the same skill rather than two skills
sharing a file.

The run's DOCUMENTS narrow it; the `e2e` key never does on its own. Which
rule applies depends only on which documents the run has:

1. The reserved `e2e` key, if `suites` carries one, is
   ADDED to whatever rule 2 or 3 selects — it is the suite
   `/acs:create-e2e-tests` wrote into, so a run that has documents always
   runs it. It is not a selector by itself: it never decides between rules
   2-4, and on its own it narrows nothing.
2. When the run wrote a `test-cases.md`: the suites named in it — the `Suite`
   column of its `## Cases` table, at `steps/create-test-docs/test-cases.md`
   — plus rule 1. `/acs:create-test-docs` is what assigns each case to a
   suite, so the run's own cases are what scopes it.
3. When the run wrote no `test-cases.md` but has a plan: the suites named in
   the plan's Test-strategy section, at `steps/create-impl-plan/plan.md` —
   plus rule 1.
4. When the run has NEITHER document — a run started from a prompt against a
   repo, with nothing planned — every entry in `suites`,
   whether or not an `e2e` key is among them. A ticket run with no
   `test-cases.md` and no plan lands here too: the `e2e` key alone does not
   narrow it to `{e2e}`. That is the honest answer there: nothing has said
   which suites this change bears on, so the run cannot narrow.

This is re-resolved on every invocation, never cached from an earlier call, so
a later case-document or plan write is picked up automatically. A suite named
in either document is included only when it is also a key in
`suites`.

**When nothing resolves** — suites are configured, and rule 2 or 3 applied
but named none of them and there is no `e2e` key — record
`outcome: nothing_to_run` and finish. A run with no suite to run is a
completed step with a reason, not a failure: the absence of a suite is not
evidence that anything is broken. (A repo that configures no suite at all is
`no_harness`, Step 1 — rule 4 over an empty map.)

## Step 4a — Triage (model step, failure path only)

For each suite whose `status` is `"fail"`, read that suite's captured
`failure_output`, `exit_code`, and `command` (from the in-memory run state or
the just-written `results.json` — either is acceptable) and produce:

1. A **regression summary**: one paragraph covering what broke, the likely
   cause, and the first-glance affected component.
2. A **stable regression key** of the form:

   ```
   <suite-name>:<normalized-failing-test-id>
   ```

   where `<suite-name>` is the exact suite name (the `settings.tests` key) the failure came from, and `<normalized-failing-test-id>` is the
   lowercase, whitespace-collapsed identifier of the single failing test
   parsed out of the suite's failure output.

   **C-1 fallback (binding, not optional):** when the suite's failure output
   yields **no parseable individual failing-test id** — a bare non-zero exit
   code, a compile/build error before any test runs, or any other
   suite-level failure with no per-test identifier to extract — the
   regression key falls back to the **coarse suite-level key**:

   ```
   <suite>:__suite__
   ```

   This is a **literal fixed marker, not a content fingerprint or hash**: a
   suite known to be flaky is a suite-authoring concern, not something this
   closed loop should amplify by minting a new ticket per distinct
   (unparseable) failure blob. The fallback intentionally collapses every
   unparseable failure from the same suite onto **one ticket per broken
   suite**.

Multiple failing suites in the same run each get their own regression key
(and therefore their own dedup/mint/bump decision) — process the
`regressions` candidates one key at a time.

## Step 4b — Dedup / recurrence lookup

For each regression key, search `tickets-index.json`
(`acs_lib.index_path(workspace, repo_id)`, read via `acs_lib.read_json`) for
an existing ticket whose description carries that exact regression-key
marker (see "Regression key marker convention" below) — scanning each
candidate's own `description` field loaded via `acs_lib.load_ticket` at the
partition resolved by `acs_lib.find_ticket_partition(workspace, repo_id,
ticket_id)` (active partition first, then `archive/`).

Given a matching ticket (if any) and its `status`
(`acs_lib.TICKET_STATUSES = ["open", "in_progress", "in_review", "done"]`),
apply this **three-way policy** exactly:

1. **Not found → mint a new ticket.** Invoke `new-ticket.py` (unchanged —
   this skill reuses it as-is, no edits to the file) with `--title "<suite>:
   <test> regression"` (or, under the suite-level fallback, `--title
   "<suite>: suite regression"`), `--type task`, and `--description "..."`
   embedding, in order: the regression-key marker, the triage summary, and a
   link to the results artifact
   (`<workspace>/<repo>/test-runs/<run-id>/results.json`). Record the
   printed `{"ticket_id", "partition"}` and mark this regression's action as
   `"minted"`. If `new-ticket.py` exits non-zero, STOP and surface its stderr
   verbatim — never invent a `ticket_id` for the `regressions[]` entry. In
   particular, on a workspace partition that has never allocated an id it
   refuses with exit 2 and a local-evidence reconciliation proposal
   (`allocate_ticket_id`'s fail-closed gate, MAR-402) instead of minting the
   regression ticket: relay that stderr, obtain the confirmed start number
   from the user, and re-run `new-ticket.py` with `--seed-next <n>` added
   (only `new-ticket.py` mints ids — never hand-pick one).

2. **Found, ticket status is `open`, `in_progress`, or `in_review` →
   comment-bump the existing ticket — never mint a duplicate, never silently skip.**
   Append fresh evidence (the new run id, the current timestamp, and
   the latest failure output) to the existing ticket's `description` (loaded
   via `acs_lib.load_ticket`, appended to, saved via `acs_lib.save_ticket`,
   re-indexed via `acs_lib.update_index`). Mark this regression's action as
   `"commented"` with `ticket_id` set to the existing ticket's id.

3. **Found, ticket status is `done` → the regression recurred after being
   marked fixed: mint a NEW ticket linked to the old one — never silently reopen
   the closed ticket.** Call `new-ticket.py` exactly as in case 1, including
   its non-zero-exit / reconciliation-refusal handling above, but
   the `--description` additionally states explicitly that this is a
   recurrence and names the old (closed) ticket id it links back to. Mark
   this regression's action as `"minted_linked"` with `linked_ticket_id` set
   to the old ticket's id. The closed ticket itself is left untouched
   (status stays `done`) — reopening it would hide that a shipped fix
   regressed.

**Regression key marker convention.** Every minted/bumped ticket's
`description` embeds the literal marker line:

```
acs-regression-key: <suite>:<normalized-failing-test-id-or-__suite__>
```

on its own line (mirrors the existing `acs-ticket: <ID>` convention already
used elsewhere in ticket descriptions). The dedup lookup searches for this
exact line, not a fuzzy match on summary prose.

**R1 safety — no interpolation (restated for the failure path).** No failure
content (suite output, parsed test ids, triage summary text) is **ever
interpolated into a shell command**. The only commands this step invokes are
in-process reads (`acs_lib.read_json`, `acs_lib.load_ticket`,
`acs_lib.find_ticket_partition`) and one write path — invoking `new-ticket.py`
as a subprocess with `--title`, `--type`, and `--description` passed as
**argument values**, never shell-interpolated into a command string. Failure
output/triage text becomes ticket **description content only**, never a
command or command fragment.

After processing every failing suite's regression key, populate the results
artifact's `regressions[]` array (Step 3's shape) with one entry per
processed key:

```json
{
  "key": "<suite>:<normalized-failing-test-id-or-__suite__>",
  "ticket_id": "<TICKET-ID>",
  "action": "minted" | "commented" | "minted_linked",
  "linked_ticket_id": "<OLD-TICKET-ID>"
}
```

`linked_ticket_id` is present only when `action` is `"minted_linked"`.

## Step 5 — Report

Print a run summary: suites run (count and names), pass/fail counts, and
tickets minted/bumped/linked (on an all-green run, this line is simply "0
tickets minted/bumped/linked" since `regressions` is always `[]` on that
path). State explicitly that the results artifact is left in place on disk
after the run — it is not cleaned up — so it can be read later.

**Under `/acs:ship`.** Replace the tickets-minted/bumped/linked line with the
verdict (`pass` / `fail`) and name the run, so the caller can read the outcome
out of a transcript without opening the artifact.

## Scheduling surface

`/acs:run-e2e-tests` carries no notion of "I am scheduled" versus "I was
invoked by a person" — it is a pure function of "which suites, right now,"
and the caller (cron, a CI scheduled workflow, or a Claude Code routine)
decides when to invoke it. See
`${CLAUDE_PLUGIN_ROOT}/skills/run-e2e-tests/references/test-scheduling.md` for
the concrete cron/CI/routine recipe; this skill itself has no built-in
scheduler (ADR 0011 G8).

## Finish

MANDATORY final step — never skipped, also on failure. Write `steps/run-e2e-tests/result.json`
through `acs.py write` (never the Write tool), below. A run that completed carries
`"status": "completed"` and an `outcome` of `passed`, `no_harness` or
`nothing_to_run`. A run where any suite failed carries `"status": "failed"`
and no `outcome` — the failing suites and the regressions this run minted,
bumped or linked are in its `summary` and `findings`, and the results artifact
is the evidence. Then:

```bash
python3 "${CLAUDE_PLUGIN_ROOT}/hooks/scripts/acs.py" write steps/run-e2e-tests/result.json <<'ACS_EOF'
<the result document>
ACS_EOF
python3 "${CLAUDE_PLUGIN_ROOT}/hooks/scripts/post-run-e2e-tests.py" --result-file "<the result.json you just wrote>"
```

The status and outcome are read from that document — a step's transition is
read from its result, not asserted on the command line. The two empty
outcomes mean one thing each, the same as in the fragment
(`skills/run-e2e-tests/state.schema.json`): `no_harness` — the repo
configures no suite at all (no suite under `settings.tests`); `nothing_to_run` — suites are configured but this run
has none to run (the plan's `owes.e2e: false`, settled by the pre-hook, or a
run set that resolved empty). Both are honest completions, not failures: a
step with nothing to do says which kind of nothing, rather than failing a run
over the absence of its own evidence.

## Completion report (normative)

End your final message with the standard completion block; replace the Ticket
line with **Run** (this skill is run-scoped, not tied to one ticket). Under
`/acs:ship` the **Findings** line states the verdict (`pass` / `fail`) instead
of a tickets-minted count:

```markdown
## /acs:run-e2e-tests · <status>

- **Run**: <run-id>, <N> suites run
- **Status**: <status> — <one line>
- **Results**: <pass count>/<N> suites passed
- **Findings**: <regressions minted/bumped/linked, or "none — all suites passed">
- **Artifacts**: results artifact at test-runs/<run-id>/results.json (left in place)
- **Metrics**: n/a
- **Next**: <e.g. re-run after a fix lands, or schedule a standing run>
```
Pliki w paczce (3)
  • SKILL.md 20 KB
  • state.schema.json 1 KB
  • references/test-scheduling.md 2 KB

Wersje

RepozytoriumWynikLicencja
Ta strona · globalmindsolution/gms-marketplace100MIT
codescene-oss/codescene-mcp-server100Apache-2.0

Migawka bazy z 13.11.2026 · wersja paczki dde47f9b3d18 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100run-regression
    Uruchamia zestaw testów regresji projektu w całości, dla jednego repozytorium lub zakresu przypadków, i raportuje wyniki pass i fail.
  • 100api-test-suite-builder
    Generuje testy API, zestawy testów integracyjnych, testy endpointów REST i testy kontraktowe.
  • 100test-all
    Uruchamia pełny zestaw testów Momentum CMS, wszystkie suity bez pomijania, na każdą prośbę o przetestowanie całości.
  • 85proof
    Działa jako inżynier QA i testów: strategia testów, zestawy E2E, testy API, triage niestabilnych testów i pokrycie.
  • 100run-tests
    Buduje aplikację z ostatniego commita, uruchamia ją w laboratorium VM, przechodzi zestawy testów i poprawia błędy, aż testy przejdą.
  • 100api-test-suite-builder
    Generuje zestawy testów API z definicji tras w różnych frameworkach: uwierzytelnianie, walidacja, kontrakty, testy obciążeniowe k6, mocki i generowanie z OpenAPI.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.