flaky-test-fix
/flaky-test-fix
Bada i naprawia niestabilny test wskazany adresem nieudanego przebiegu CI lub nazwą, usuwając przyczynę zamiast pomijać test lub ponawiać jego uruchomienie.
- Autor
- cardstack ↗
- Repozytorium
- boxel ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 27.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 27.10.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Bada i naprawia niestabilny test wskazany adresem nieudanego przebiegu CI lub nazwą, usuwając przyczynę zamiast pomijać test lub ponawiać jego uruchomienie.
Wyzwalacze z opisu
Oryginalny opis (EN)
Investigate and fix a flaky test reported by a CI failure URL (e.g. a GitHub Actions job link) or a known-flaky test name. Use whenever someone hands over a flaky/intermittent test and wants it fixed — not skipped or retried. Triggers on "this test is flaky: <URL>", "intermittent failure in <test>", or a CI job link for a test that passes on re-run.
Jak działa
- Why this protocol
- Procedure
- When the failure isn't fixable yet
- Related repo conventions
Podgląd SKILL.md
--- name: flaky-test-fix description: Investigate and fix a flaky test reported by a CI failure URL (e.g. a GitHub Actions job link) or a known-flaky test name. Use whenever someone hands over a flaky/intermittent test and wants it fixed — not skipped or retried. Triggers on "this test is flaky: <URL>", "intermittent failure in <test>", or a CI job link for a test that passes on re-run. --- # Flaky Test Fix A flaky test reported by a CI URL is a request for five things, done in order: **prior-art search → Linear issue → worktree → root-cause fix → diagnostic logging**. Do not paper over the failure with a retry, an increased timeout, or a `skip`/`todo` unless the failure genuinely cannot be diagnosed without more data — and if so, lead with the diagnostics and say so explicitly. ## Why this protocol Flaky tests are uncertain by nature. A fix that looks correct in local repro often does not address the real race / ordering / resource issue that triggers it in CI. Two safeguards counter that: - **Prior-art search** catches the common case where the same test has been "fixed" before. Without that context you risk re-doing a fix that already proved insufficient, or undoing an intentional accommodation. - **Diagnostic logging bundled with the fix** means that if the fix misses the real root cause, the _next_ CI failure produces enough signal to diagnose it — instead of burning another CI cycle staring at the same opaque output. ## Procedure ### 1. Search for prior fix attempts first Before touching anything, find out whether this test has been worked on before. Search by test name, file path, and failure signature: ```bash gh search prs "<test name or file>" # searches open + closed by default gh pr list --state=all --search "<test name or failure signature>" git log --oneline -- <path/to/test/file> ``` If you find a prior PR, read its description, diff, and review thread before designing your fix. You may be looking at a regression, a partial fix, or a class of flakiness someone has already characterized. Surface any prior attempts early so the reporter can weigh in if the new fix should look materially different from the last. ### 2. Create a Linear issue for the flaky test Unless the reporter already names a ticket, recommend a Linear issue that tracks the flake, and create it once they agree. Search Linear for the test name first: an existing issue for the same flake is updated, not duplicated. Give a new issue these settings, and set the state of an existing issue too: - **Team:** Cardstack Engineering (`CS`) - **Project:** Bugs to Fix for Launch - **Priority:** none - **Cycle:** the team's current cycle - **Assignee:** the current user (`me`) - **State:** In Progress Put the CI failure URL, the failure signature, and any prior attempts from step 1 in the description. Name the branch with the issue's `cs-<id>-` prefix, so that Linear links the PR to the issue. ### 3. Work in a worktree off `origin/main` Spawn a fresh git worktree based on `origin/main` (not the current branch) and do all the work there — never in the main checkout — unless an active ticket-specific override says otherwise. Prefer launching an Agent with `isolation: "worktree"` for this. ### 4. Attempt a real fix for the root cause Diagnose the actual race, ordering, or resource issue and fix that. If a prior attempt exists, explicitly account for why this fix will succeed where the previous one didn't — or why it is complementary rather than a repeat. ### 5. Add diagnostic logging alongside the fix Add logging that would make the _next_ failure diagnosable if this fix turns out not to catch the root cause. The logging is insurance: it stays in even when the fix looks obviously right. Aim it at the uncertain part — the timing, the ordering, the shared resource state — so a future CI failure prints the signal you wished you'd had this time. ## When the failure isn't fixable yet If there genuinely isn't enough information to identify a root cause, don't guess at a fix. Land the diagnostic logging on its own, say clearly that it's diagnostics-only, and explain what signal the next failure will produce. ## Related repo conventions - Reproduce locally with a tight loop (stop on first failure) rather than waiting out the full suite. - For host integration tests, watch for shared global state leaking across tests (a frequent flakiness source) — register/unregister symmetry, store/loader resets, lingering DOM or timers.
Wersje
W bazie nie ma innych skilli o nazwie flaky-test-fix.
Migawka bazy z 13.11.2026 · wersja paczki 4322518adeff (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100fix
Naprawia padające i niestabilne testy Playwright, w tym testy przechodzące tylko czasami i błędy sporadyczne. - 100fix
Naprawia padające i niestabilne testy Playwright, w tym testy przechodzące tylko czasami i błędy sporadyczne. - 100fix
Naprawia padające i niestabilne testy Playwright, w tym testy przechodzące tylko czasami i błędy sporadyczne. - 100flaky-repro
Odtwarza niestabilny test na żądanie z zapisanym ziarnem i poleceniem powtórki, a poprawkę potwierdza deterministycznym testem obciążeniowym. - 100fix-flaky-test
Znajduje i usuwa przyczynę testu, który czasem nie przechodzi, zamiast dokładać ponowienia i dłuższe czekanie. - 100fix-flaky-tests
Bada i naprawia niestabilne testy w dowolnym frameworku i systemie CI: wykrywa framework oraz dostawcę CI i stosuje pasujące wzorce klasyfikacji oraz poprawek.