model-evaluation-reporting
/model-evaluation-reporting
Use when writing or reviewing a model evaluation report -- accuracy is being quoted alone, or stakeholders need failure modes, segment performance, and business trade-offs made explicit.
- Author
- yeaight7 ↗
- Repository
- agent-powerups ↗
- Licence
- Apache-2.0
- Last repository change
- 27.09.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 27.09.2026, without references to retired models or APIs.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
What it does and when to use it
Use when writing or reviewing a model evaluation report -- accuracy is being quoted alone, or stakeholders need failure modes, segment performance, and business trade-offs made explicit.
Original author description
Use when writing or reviewing a model evaluation report -- accuracy is being quoted alone, or stakeholders need failure modes, segment performance, and business trade-offs made explicit.
How it works
- Purpose
- When to Use
- Inputs
- Workflow
- Output
- Verification
- Failure Modes
SKILL.md preview
--- name: model-evaluation-reporting description: Use when writing or reviewing a model evaluation report -- accuracy is being quoted alone, or stakeholders need failure modes, segment performance, and business trade-offs made explicit. --- ## Purpose Raw accuracy metrics are not enough. Evaluation must reflect the actual business impact and failure modes of the model; this standard turns metrics into a report a stakeholder can act on. ## When to Use - Producing an evaluation report for a trained model - Reviewing a report that quotes a single headline metric - A go/no-go decision depends on understanding error costs ## Inputs - Model predictions on a held-out set - Business context: cost of errors, key user/customer segments ## Workflow 1. **Beyond accuracy**: include the confusion matrix; report Precision, Recall, and F1; explain the cost of a False Positive vs a False Negative in the business context. 2. **Slice analysis**: report performance on key segments. A model can be 95% accurate overall but only 40% accurate on new users. 3. **Calibration**: if the model outputs probabilities, verify they are calibrated — a prediction of 0.8 should come true about 80% of the time. 4. **Format as a Markdown report** that a non-technical stakeholder can read, highlighting trade-offs and worst-case scenarios. ## Output - A Markdown evaluation report: headline metrics, confusion matrix, per-segment table, calibration assessment, and explicit trade-offs/worst cases ## Verification - [ ] Confusion matrix plus Precision/Recall/F1 present (not accuracy alone) - [ ] False Positive vs False Negative costs stated in business terms - [ ] Key segments sliced and reported - [ ] Calibration assessed whenever probabilities are output - [ ] Report readable by a non-technical stakeholder ## Failure Modes - **Headline-metric reporting** — one aggregate number hides segment failures; always include slices. - **Cost-free framing** — treating FP and FN as equivalent when their business impact differs by orders of magnitude. - **Uncalibrated probabilities read as confidence** — stakeholders will read 0.8 as "80% sure"; verify calibration before they do.
Versions
The database has no other skills named model-evaluation-reporting.
Identical content appears in 1 other GitHub folders. We show one copy with the highest score.
Database snapshot: 13.11.2026 · package version 8876f66a8146 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 90score-eval
Design an evaluation framework for a ML model — metrics, splits, and reporting. Use when asked "how should we evaluate this model", "design evaluation metrics", or "plan our train test split". - 100model-evaluation-report
Produces a rigorous, honest evaluation report for an ML model - real baselines, business-matched metrics with confidence intervals, slice-level error analysis, calibration checks, and a go/no-go recommendation. Use when someone asks "is this model good enough to ship", "write up… - 90model-evaluation
Evaluate models with cost-aligned metrics, slice analysis, calibration, and uncertainty, not a single accuracy number. Use when choosing evaluation metrics or judging whether a model is fit to ship. - 90score-recon
Audit existing model evaluation code — find metric misuse, missing CIs, and evaluation leakage. Use when asked to "audit our model evaluation", "find metric misuse", or "check for evaluation leakage". - 100model-evaluation-metrics
Build model evaluation metrics operations. Auto-activating skill for ML Training. Triggers on: model evaluation metrics, model evaluation metrics Part of the ML Training skill category. Use when working with model evaluation metrics functionality. Trigger with phrases like…