Przejdź do treści
Biblioteka skilli
Programowanie · Testowanie

test-mutation

/test-mutation

Uruchamia testy mutacyjne w StrykerJS i mutmut, by sprawdzić, czy testy faktycznie weryfikują zachowanie, a nie tylko wykonują linie kodu.

Repozytorium
cursor-kenji ↗
Licencja
MIT
Ostatnia zmiana repozytorium
19.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
1
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 19.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Uruchamia testy mutacyjne w StrykerJS i mutmut, by sprawdzić, czy testy faktycznie weryfikują zachowanie, a nie tylko wykonują linie kodu.

Wyzwalacze z opisu

add mutation testingare our tests realcan our test suite be gamed
Oryginalny opis (EN)

Run mutation testing (StrykerJS, mutmut) to measure whether tests assert behavior, not just execute lines. Use when "add mutation testing", "are our tests real", "can our test suite be gamed", or after an agent bulk-generated tests.

Jak działa

  1. This skill vs neighbors
  2. How to reason (every survivor)
  3. Worked example
  4. Phase 0 — Detect stack and scope deliberately [HIGH freedom]
  5. Phase 1 — Configure for signal, not noise [HIGH freedom]
  6. Phase 2 — First run and triage the survivors [LOW freedom — classify every survivor]
  7. Phase 3 — Wire into CI (sustainably) [HIGH freedom; floor from measured score = LOW]
  8. Definition of Done
  9. Self-critique before reporting [LOW freedom — do not skip]
  10. Output format
Sekcje z nagłówków SKILL.md, w oryginale · 152 linii, 1 088 słów
Podgląd SKILL.md
---
name: test-mutation
description: >
  Run mutation testing (StrykerJS, mutmut) to measure whether tests assert
  behavior, not just execute lines. Use when "add mutation testing", "are our
  tests real", "can our test suite be gamed", or after an agent bulk-generated
  tests.
license: MIT
---

# test-mutation — Coverage proves the test ran; mutants prove it would notice

**Degree of freedom: MIXED** — scope and triage `[HIGH freedom]`; first run
and CI floor `[LOW freedom — measure, then ratchet]`. Never start with the
whole repo.

Install and run the gate coverage cannot close. **Coverage proves the test
*ran* the code; mutation testing proves the test would *notice* if the
code were wrong.** The tool makes hundreds of small deliberate bugs
(mutants) — flip `>` to `>=`, delete a statement, replace a return — and
reruns the tests. Survivors mark code the tests execute but verify
nothing.

This is the specific counter to agent-generated suites: an agent can
trivially produce 90% coverage with no meaningful assertions, and every
coverage ratchet will wave them through. Mutants do not care about
coverage theater.

## This skill vs neighbors

| Skill | Owns |
|---|---|
| **test-mutation** (this) | Install / run / triage mutation testing; score ratchet |
| `plan-test-coverage` | What *should* be tested — plan only, no harness |
| `test-unit` | Write the tests |
| `audit-gate-logic` / `housekeep-gates` | Ratchet policy this score plugs into |
| `enhance-agent-guardrails` | Wording of the agent-rule hook |

## How to reason (every survivor)

1. **Observe** — mutant (operator/line) and which tests still passed
2. **Interpret** — missing assertion, untested branch, dead code, or equivalent?
3. **Classify** — the four triage classes in Phase 2
4. **Severity** — blast radius: entitlements/auth/money outrank formatters

## Worked example

> **Observe:** Stryker flipped `>` to `>=` in `entitlements.ts` expiry check;
> suite green; mutation score 62% on a 94% coverage file.
> **Interpret:** tests execute the line but never assert the boundary.
> **Classify:** missing assertion.
> **Severity:** high — paid access could extend a day.
> **Fix:** assert expired-at-exact-boundary is denied; then re-run incremental.

---

## Phase 0 — Detect stack and scope deliberately  [HIGH freedom]

Identify the runner and pick the tool: **StrykerJS** (Jest / Vitest /
Mocha, TS-aware), **mutmut** / cosmic-ray (Python), or the ecosystem
built-in. Then scope — this step decides whether mutation testing
*survives* in the repo:

**Never start with the whole repo.** Each mutant is a test run. A 6-hour
first run gets the tool deleted. Scope the first pass to where a silent
bug costs most: money movement, auth, entitlements, data mutation, core
business logic. Explicitly exclude: generated code, config, type-only
files, UI glue better covered by `test-visual-regression`, and third-party
wrappers.

Record the suite's baseline runtime. Mutation runtime ≈ (mutants ×
affected-test runtime). Turn on per-test coverage analysis from day one
so only tests covering the mutated line rerun.

---

## Phase 1 — Configure for signal, not noise  [HIGH freedom]

- **Mutate patterns:** scoped critical paths only, via `mutate:` globs.
- **Incremental mode on** (Stryker `--incremental`): later runs only
  re-test mutants in changed code — this is what makes per-PR runs
  feasible.
- **Thresholds:** `high` / `low` for reporting, and `break` (CI-failing
  floor) at or slightly below the *measured* first score. Never invent an
  aspirational floor; measure, then ratchet. Same
  auto-tighten-with-reviewed-resets policy as `housekeep-gates`.
- **Timeouts:** mutants that loop are killed by timeout — keep the factor
  sane or runs balloon.
- **Reporters:** HTML for humans (each survivor in context), JSON for the
  ratchet.

---

## Phase 2 — First run and triage the survivors  [LOW freedom — classify every survivor]

Run, then classify every surviving mutant — this triage **is** the
deliverable:

1. **Missing assertion** — the test executes the line but asserts nothing
   about its effect. Fix: strengthen the test. Each of these is a hole a
   bug could walk through today.
2. **Untested branch** — no test reaches the mutated logic despite
   file-level coverage. Fix: add the case (hand to `plan-test-coverage`).
3. **Dead / vestigial code** — the mutant survives because the code has
   no observable effect. Fix: delete the code, not add a test
   (`workflow-housekeep`).
4. **Equivalent mutant** — the mutation does not change behavior (e.g.
   `<` vs `<=` on a boundary that cannot occur). No test can kill it;
   mark ignored with a comment. Expect a small percentage; a large one
   means the mutator is mutating the wrong things.

Prioritize by blast radius: a survivor in `entitlements.ts` outranks
fifty in a formatting helper.

---

## Phase 3 — Wire into CI (sustainably)  [HIGH freedom; floor from measured score = LOW]

- **Per-PR:** incremental mutation on changed files only, as a job wired
  into the aggregator gate (`housekeep-gates`).
- **Scheduled full run** (nightly / weekly) over the mutated scope;
  publishes the score, updates the ratchet floor upward per policy, and
  files the new-survivor list.
- **Score ratchet:** `break` rises as the score does (flake tolerance);
  lowering it is a separate reviewed PR, never bundled.
- **Agent-rule hook:** bulk-generated tests must pass a mutation run over
  the touched files before "done" counts. `enhance-agent-guardrails`
  installs the wording.

---

## Definition of Done

- [ ] Tool chosen for the stack; per-test coverage analysis enabled
- [ ] Scope limited to critical paths with exclusions recorded; whole-repo explicitly deferred
- [ ] Incremental mode on; timeouts and reporters configured
- [ ] First run complete; baseline mutation score recorded
- [ ] Every survivor triaged into missing-assertion / untested-branch / dead-code / equivalent, with fixes filed by class
- [ ] `break` threshold set from the measured score and wired to auto-tighten; resets require a separate reviewed PR
- [ ] Per-PR incremental job in the aggregator gate; scheduled full run publishing the score
- [ ] Agent-rule hook added: generated tests validated by mutation before done
- [ ] Total added CI time measured and reported

## Self-critique before reporting  [LOW freedom — do not skip]

1. **Scope was narrow** — whole-repo first run is a failure mode, not a flex
2. **Every survivor classified** — no "look at the HTML later"
3. **Floor measured** — set the `break` floor from the first measured score, not an aspiration
4. **Right owner** — write the missing test → `test-unit`; delete dead code → `plan-dead-code` → `housekeep-dead-code`
5. **Strengthen tests only on approval**

## Output format

1. **Scope & config** — mutated paths | exclusions | tool + modes
2. **Score report** — baseline mutation score | per-module breakdown | coverage-vs-mutation gap (high coverage + low mutation score = assertion theater)
3. **Survivor triage** — mutant | file:line | class | fix filed
4. **CI wiring** — PR job, scheduled run, ratchet policy

Implement the harness and triage. Strengthen tests only on approval —
each fix is a reviewable change.

Wersje

W bazie nie ma innych skilli o nazwie test-mutation.

Migawka bazy z 13.11.2026 · wersja paczki f703436a8894 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100running-mutation-tests
    Uruchamia testowanie mutacyjne, które sprawdza, czy testy faktycznie wychwytują błędy w kodzie.
  • 100mutation-testing
    Prowadzi testy mutacyjne ze Stryker (TS i JS) oraz mutmut (Python), aby znaleźć słabe testy przechodzące na zmutowanym kodzie i poprawić jakość testów.
  • 100running-mutation-tests
    Wykonuje testy mutacyjne, aby ocenić skuteczność zestawu testów.
  • 100mutation-test
    Uruchamia StrykerJS na logice biznesowej dostarczonej dla use case'u i raportuje mutation score oraz każdego ocalałego mutanta wraz z asercją, która by go zabiła.
  • 100mutation-test
    Uruchamia testy mutacyjne, aby ocenić jakość testów ponad samo pokrycie kodu, znaleźć słabe testy i sprawdzić skuteczność zestawu testów.
  • 100mutation-test
    Uruchamia testy mutacyjne, aby ocenić jakość testów ponad samo pokrycie kodu, znaleźć słabe testy i sprawdzić skuteczność zestawu testów.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.