Skip to content
Skills Library
Data & analytics · Writing

model-evaluation-reporting

/model-evaluation-reporting

Use when writing or reviewing a model evaluation report -- accuracy is being quoted alone, or stakeholders need failure modes, segment performance, and business trade-offs made explicit.

Licence
Apache-2.0
Last repository change
27.09.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 27.09.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Use when writing or reviewing a model evaluation report -- accuracy is being quoted alone, or stakeholders need failure modes, segment performance, and business trade-offs made explicit.

Original author description

Use when writing or reviewing a model evaluation report -- accuracy is being quoted alone, or stakeholders need failure modes, segment performance, and business trade-offs made explicit.

How it works

  1. Purpose
  2. When to Use
  3. Inputs
  4. Workflow
  5. Output
  6. Verification
  7. Failure Modes
Original SKILL.md headings · 41 lines, 300 words
SKILL.md preview
---
name: model-evaluation-reporting
description: Use when writing or reviewing a model evaluation report -- accuracy is being quoted alone, or stakeholders need failure modes, segment performance, and business trade-offs made explicit.
---

## Purpose

Raw accuracy metrics are not enough. Evaluation must reflect the actual business impact and failure modes of the model; this standard turns metrics into a report a stakeholder can act on.

## When to Use

- Producing an evaluation report for a trained model
- Reviewing a report that quotes a single headline metric
- A go/no-go decision depends on understanding error costs

## Inputs

- Model predictions on a held-out set
- Business context: cost of errors, key user/customer segments

## Workflow

1. **Beyond accuracy**: include the confusion matrix; report Precision, Recall, and F1; explain the cost of a False Positive vs a False Negative in the business context.
2. **Slice analysis**: report performance on key segments. A model can be 95% accurate overall but only 40% accurate on new users.
3. **Calibration**: if the model outputs probabilities, verify they are calibrated — a prediction of 0.8 should come true about 80% of the time.
4. **Format as a Markdown report** that a non-technical stakeholder can read, highlighting trade-offs and worst-case scenarios.

## Output

- A Markdown evaluation report: headline metrics, confusion matrix, per-segment table, calibration assessment, and explicit trade-offs/worst cases

## Verification

- [ ] Confusion matrix plus Precision/Recall/F1 present (not accuracy alone)
- [ ] False Positive vs False Negative costs stated in business terms
- [ ] Key segments sliced and reported
- [ ] Calibration assessed whenever probabilities are output
- [ ] Report readable by a non-technical stakeholder

## Failure Modes

- **Headline-metric reporting** — one aggregate number hides segment failures; always include slices.
- **Cost-free framing** — treating FP and FN as equivalent when their business impact differs by orders of magnitude.
- **Uncalibrated probabilities read as confidence** — stakeholders will read 0.8 as "80% sure"; verify calibration before they do.

Versions

The database has no other skills named model-evaluation-reporting.

Identical content appears in 1 other GitHub folders. We show one copy with the highest score.

Database snapshot: 13.11.2026 · package version 8876f66a8146 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90score-eval
    Design an evaluation framework for a ML model — metrics, splits, and reporting. Use when asked "how should we evaluate this model", "design evaluation metrics", or "plan our train test split".
  • 100model-evaluation-report
    Produces a rigorous, honest evaluation report for an ML model - real baselines, business-matched metrics with confidence intervals, slice-level error analysis, calibration checks, and a go/no-go recommendation. Use when someone asks "is this model good enough to ship", "write up…
  • 90model-evaluation
    Evaluate models with cost-aligned metrics, slice analysis, calibration, and uncertainty, not a single accuracy number. Use when choosing evaluation metrics or judging whether a model is fit to ship.
  • 90score-recon
    Audit existing model evaluation code — find metric misuse, missing CIs, and evaluation leakage. Use when asked to "audit our model evaluation", "find metric misuse", or "check for evaluation leakage".
  • 100model-evaluation-metrics
    Build model evaluation metrics operations. Auto-activating skill for ML Training. Triggers on: model evaluation metrics, model evaluation metrics Part of the ML Training skill category. Use when working with model evaluation metrics functionality. Trigger with phrases like…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.