Skip to content
Skills Library
Medicine & health · Testing

model-validation

/model-validation

Designs and executes validation studies for radiology AI models to ensure clinical reliability and regulatory compliance. Use when user mentions "validate model performance", "external validation", "statistical analysis", "clinical validation", or needs model evaluation.

Licence
MIT
Last repository change
21.06.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
2
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 21.06.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Designs and executes validation studies for radiology AI models to ensure clinical reliability and regulatory compliance. Use when user mentions "validate model performance", "external validation", "statistical analysis", "clinical validation", or needs model evaluation.

Triggers from the description

validate model performanceexternal validationstatistical analysisclinical validation
Original author description

Designs and executes validation studies for radiology AI models to ensure clinical reliability and regulatory compliance. Use when user mentions "validate model performance", "external validation", "statistical analysis", "clinical validation", or needs model evaluation.

How it works

  1. Triggers
  2. Parameters
  3. Validation Framework
  4. Output Format
  5. Usage Examples
Original SKILL.md headings · 77 lines, 285 words
SKILL.md preview
---
name: model-validation
description: Designs and executes validation studies for radiology AI models to ensure clinical reliability and regulatory compliance. Use when user mentions "validate model performance", "external validation", "statistical analysis", "clinical validation", or needs model evaluation.
---

# Model Validation Skill

## Triggers

- "validate model performance"
- "external validation"
- "statistical analysis"
- "clinical validation"
- "model comparison"
- "regulatory submission"
- "performance benchmarking"
- "fairness audit"

## Parameters

- `validation_type` (required): Type of validation needed
  - `internal` - Retrospective internal dataset
  - `external` - Prospective/out-of-distribution testing
  - `prospective` - Clinical deployment study
  - `regulatory` - FDA/EMA submission prep
  - `fairness` - Subgroup disparity analysis
  - `comparison` - Head-to-head model comparison
- `model_task` (required): Model's intended use
  - `detection` - Sensitivity, specificity, PPV, NPV
  - `segmentation` - Dice, IoU, Hausdorff distance
  - `classification` - Accuracy, AUC, F1 score
  - `regression` - MAE, RMSE, correlation
- `modality` (optional): Imaging modality
- `regulatory_path` (optional): Target clearance pathway

## Validation Framework

### Performance Metrics
| Task | Primary Metrics | Secondary |
|------|-----------------|-----------|
| Detection | Sensitivity, Specificity, AUC | PPV, NPV, FROC |
| Segmentation | Dice, IoU | Hausdorff, ASD |
| Classification | Accuracy, AUC, F1 | Sensitivity, Specificity |
| Regression | MAE, RMSE | Correlation, Bland-Altman |

### Statistical Methods
- Confidence intervals (bootstrap, binominal)
- Significance testing (McNemar, DeLong for AUC)
- Power analysis for sample sizing
- Multiple comparison correction
- Subgroup interaction testing

### Regulatory Standards
- FDA 510(k) predicate comparison
- FDA De Novo requirements
- EU MDR clinical evaluation
- IMDRF clinical evidence framework
- ACR-SIIM AI performance standards

## Output Format

Returns structured JSON with:
- Validation protocol and methodology
- Required sample size with power analysis
- Statistical test selection and rationale
- Results template with standard metrics
- Interpretation guidelines
- Regulatory compliance checklist

## Usage Examples

```
validation_type: external
model_task: detection
modality: CT

validation_type: regulatory
model_task: classification
regulatory_path: 510k
```
Files in this package (2)
  • SKILL.md 2 KB
  • evals/evals.json 3 KB

Versions

The database has no other skills named model-validation.

Identical content appears in 8 other GitHub folders. We show one copy with the highest score.

Database snapshot: 13.11.2026 · package version bcd845a5d3a3 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100llm-radiology-use
    Use LLM APIs for radiology tasks. Also use when integrating medical LLMs (MedPaLM, MedLM, Google Health, Amazon HealthLake) for report analysis, clinical reasoning, or radiology AI workflows.
  • 100radiology-context
    Establish and manage user's clinical radiology environment configuration - PACS, EHR, modality types, AI tools, and workflow settings. Also use when setting up or updating radiology workflow context, configuring PACS connections, or managing clinical environment settings.
  • 100prediction-model-design
    ML/prediction-model module. Plans development or validation of a clinical prediction or machine-learning model: intended use, predictors and leakage, baseline regression model, sample size for development and external validation, validation design, discrimination, calibration,…
  • 90clinical-trial
    Designs and analyzes clinical trials including sample size calculation, randomization schemes, endpoint selection, CONSORT reporting, and interim analysis planning; trigger when users ask about RCTs, Phase I-IV trials, or clinical study design.
  • 90alterlab-clinical-reports
    Writes comprehensive clinical reports — case reports (CARE guidelines), diagnostic reports (radiology, pathology, lab), clinical trial reports (ICH-E3, SAE, CSR), and patient documentation (SOAP notes, H&P, discharge summaries) — with templates, regulatory compliance (HIPAA,…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.