Skip to content
Skills Library
Programming · Testing

msa-test-reliability

/msa-test-reliability

Use for flaky tests, conflicting AI reviews, weak oracles or suspected false passes; measure the inspection system before trusting its verdict.

Licence
MIT
Last repository change
12.11.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
3
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 12.11.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Use for flaky tests, conflicting AI reviews, weak oracles or suspected false passes; measure the inspection system before trusting its verdict.

Original author description

Use for flaky tests, conflicting AI reviews, weak oracles or suspected false passes; measure the inspection system before trusting its verdict.

How it works

  1. Output and evidence
Original SKILL.md headings · 11 lines, 153 words
SKILL.md preview
---
name: msa-test-reliability
description: Use for flaky tests, conflicting AI reviews, weak oracles or suspected false passes; measure the inspection system before trusting its verdict.
metadata:
  version: "2.0"
---
# Msa Test Reliability

1. Start with labeled good/bad cases and a passing control. Perturb one condition; require the corresponding assertion to fail for the intended reason.
2. Use mutation tests and known-bad sentinels to measure detection bias. Separate product failure, environment failure and equivalent/surviving mutants; never majority-vote away a real failure.
3. For AI reviewers, record model/version/prompt and blinded labeled decisions. Report confusion matrix, false-pass/false-fail rates and Cohen kappa where defined; keep one-class/degenerate kappa unavailable.
4. Complete when results identify what the oracle can discriminate. Unrun paired model trials remain UNVERIFIED; test counts and instruction quality do not prove improved coding outcomes.

## Output and evidence

Use [the blank output form](assets/output-template.md) for this workflow and [the portable record](assets/review-record.json) when transferring it to the auditor. Fill only observed facts; link raw evidence rather than copying logs into the conversation. A completed form is a document, not execution evidence or an automatic supported score.
Files in this package (3)
  • SKILL.md 1 KB
  • assets/output-template.md 1 KB
  • assets/review-record.json 1 KB

Versions

The database has no other skills named msa-test-reliability.

Database snapshot: 13.11.2026 · package version 344409ab3d0d (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100test-quality-analysis
    Detect test smells, overmocking, flaky tests, and coverage issues. Analyze test effectiveness, maintainability, and reliability. Use when reviewing tests or improving test quality.
  • 100test-audit
    Use when asked to audit unit tests, check test quality/health, find flaky or weak tests, or review the test suite for smells — repo-wide or on changed tests
  • 100flaky-detect
    Identify flaky tests from CI history and test execution patterns. Use when debugging intermittent test failures, auditing test reliability, or improving CI stability.
  • 100flaky-detect
    Identify flaky tests from CI history and test execution patterns. Use when debugging intermittent test failures, auditing test reliability, or improving CI stability.
  • 90testing-reliability
    Diagnose flaky tests, misleading mocks or fakes, sleep-based waits, brittle fixtures, test-only production hooks, weak assertions, and mutation-testing gaps. Use when CI behavior differs, tests hide defects, or coverage creates false confidence.
  • 100testing-audit
    Audit the test suite for coverage gaps, test quality, flaky tests, and strategy alignment. Use when assessing test health before a release, or when tests feel weak, slow, or flaky.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.