msa-test-reliability
/msa-test-reliability
Use for flaky tests, conflicting AI reviews, weak oracles or suspected false passes; measure the inspection system before trusting its verdict.
- Author
- gukhoe ↗
- Repository
- production-inspired-coding-skills ↗
- Licence
- MIT
- Last repository change
- 12.11.2026
Assessment and security
- Files in this package
- 3
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 12.11.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Use for flaky tests, conflicting AI reviews, weak oracles or suspected false passes; measure the inspection system before trusting its verdict.
Original author description
Use for flaky tests, conflicting AI reviews, weak oracles or suspected false passes; measure the inspection system before trusting its verdict.
How it works
- Output and evidence
SKILL.md preview
--- name: msa-test-reliability description: Use for flaky tests, conflicting AI reviews, weak oracles or suspected false passes; measure the inspection system before trusting its verdict. metadata: version: "2.0" --- # Msa Test Reliability 1. Start with labeled good/bad cases and a passing control. Perturb one condition; require the corresponding assertion to fail for the intended reason. 2. Use mutation tests and known-bad sentinels to measure detection bias. Separate product failure, environment failure and equivalent/surviving mutants; never majority-vote away a real failure. 3. For AI reviewers, record model/version/prompt and blinded labeled decisions. Report confusion matrix, false-pass/false-fail rates and Cohen kappa where defined; keep one-class/degenerate kappa unavailable. 4. Complete when results identify what the oracle can discriminate. Unrun paired model trials remain UNVERIFIED; test counts and instruction quality do not prove improved coding outcomes. ## Output and evidence Use [the blank output form](assets/output-template.md) for this workflow and [the portable record](assets/review-record.json) when transferring it to the auditor. Fill only observed facts; link raw evidence rather than copying logs into the conversation. A completed form is a document, not execution evidence or an automatic supported score.
Files in this package (3)
- SKILL.md 1 KB
- assets/output-template.md 1 KB
- assets/review-record.json 1 KB
Versions
The database has no other skills named msa-test-reliability.
Database snapshot: 13.11.2026 · package version 344409ab3d0d (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100test-quality-analysis
Detect test smells, overmocking, flaky tests, and coverage issues. Analyze test effectiveness, maintainability, and reliability. Use when reviewing tests or improving test quality. - 100test-audit
Use when asked to audit unit tests, check test quality/health, find flaky or weak tests, or review the test suite for smells — repo-wide or on changed tests - 100flaky-detect
Identify flaky tests from CI history and test execution patterns. Use when debugging intermittent test failures, auditing test reliability, or improving CI stability. - 100flaky-detect
Identify flaky tests from CI history and test execution patterns. Use when debugging intermittent test failures, auditing test reliability, or improving CI stability. - 90testing-reliability
Diagnose flaky tests, misleading mocks or fakes, sleep-based waits, brittle fixtures, test-only production hooks, weak assertions, and mutation-testing gaps. Use when CI behavior differs, tests hide defects, or coverage creates false confidence. - 100testing-audit
Audit the test suite for coverage gaps, test quality, flaky tests, and strategy alignment. Use when assessing test health before a release, or when tests feel weak, slow, or flaky.