validate
/validate
Evaluate a frozen model and decision system against intended-use evidence and guardrails. Use for independent quality, uncertainty, slice, interpretation, fairness, robustness, or readiness checks; model selection belongs to train.
- Author
- Kiran-980 ↗
- Repository
- 1ml ↗
- Licence
- MIT
- Last repository change
- 04.11.2026
Assessment and security
- Files in this package
- 2
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 04.11.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Evaluate a frozen model and decision system against intended-use evidence and guardrails. Use for independent quality, uncertainty, slice, interpretation, fairness, robustness, or readiness checks; model selection belongs to train.
Original author description
Evaluate a frozen model and decision system against intended-use evidence and guardrails. Use for independent quality, uncertainty, slice, interpretation, fairness, robustness, or readiness checks; model selection belongs to train.
How it works
- Decide and route
SKILL.md preview
--- name: validate description: Evaluate a frozen model and decision system against intended-use evidence and guardrails. Use for independent quality, uncertainty, slice, interpretation, fairness, robustness, or readiness checks; model selection belongs to train. --- # Validate the exact decision system For a design-only review, load system evidence but not project-state or notebook files. Resolve links relative to this skill directory, preserving `../`; shared files live under the plugin root's `references/`. Identify the frozen preprocessing, estimator, calibration, policy, fallback, and human workflow. Confirm data lineage, independent evaluation role, supported population, metrics, costs, and thresholds. Read [system evidence and release decisions](references/system-evidence.md) and relevant [state](../../references/project-state.md). Use the selected [task profile](../../references/tasks/index.md) when evaluation semantics differ. Measure the whole system under its declared use, not only successful predictions. Report sample/event support, uncertainty, slices, exclusions, failures, review/fallback workload, and operational limits. Distinguish calibrated probabilities, metric confidence intervals, prediction intervals, and distribution-shift uncertainty. Conformal guarantees require their stated calibration/exchangeability conditions; average coverage does not guarantee every subgroup. For a fixed set of scores and evaluation population, changing the action threshold changes workload, precision/recall and costs, not the score-ranking AUROC. Re-evaluate the affected policy outcomes. Excluding failed/abstained cases can bias quality in either direction; report selective score metrics with coverage and full-workflow outcomes rather than asserting every omitted-case score must be inflated. Choose explanation and fairness analyses for the actual claim and harm context. Permutation importance and partial dependence describe model behavior under assumptions, not causal effects. Correlated inputs can distort perturbations. Passing one parity metric does not establish fairness of the system. Check schema, units, unknown/missing handling, row alignment, packaged parity, dependencies, latency/memory, fallback behavior, and rollback readiness to the extent relevant. Use domain-valid perturbations; impossible feature combinations do not demonstrate useful robustness. Keep final-test inspection separate from adaptive development: changes based on its result require renewed evaluation and invalidation. Required artifact checks must assert or raise on failure; printing `FAILED` or catching an exception without failing does not pass a gate. Reuse saved evaluation predictions within their declared role, and make summaries name the actual population, candidate and metric. Read [diagnostic implementation checks](references/system-evidence.md#diagnostic-implementation-checks) only when implementing those diagnostics. Separate observed results, untested plans, and assumptions; neither a sealed split nor a model family guarantees independence or calibration. When a notebook is a requested deliverable, apply the [notebook completion review](../../references/notebook-protocol.md#review-before-calling-the-notebook-complete) to the actual saved analysis and figures. Check that its decisions and transitions are explained and agree with measured evidence, separately from execution and model-parity checks. Repair presentation gaps without changing a frozen candidate in response to final-test outcomes; disclose any remaining verification gap. For an end-to-end request, reconcile the lifecycle coverage record with executed evidence, including preparation checks and development error diagnostics. A high aggregate score, a figure count, or a fresh-process reload pass cannot stand in for these analyses. Return missing applicable computations to the owning skill before final-test inspection; after final-test inspection, retain the frozen bundle and distinguish added diagnostics from model adaptation requiring renewed evaluation. ## Decide and route - Contract/pipeline failure: return the failing evidence to [data](../data/SKILL.md) or [prepare](../prepare/SKILL.md). - Quality/support failure: return a development hypothesis to [train](../train/SKILL.md), or a population/collection limitation to [frame](../frame/SKILL.md); preserve test history. - Inconclusive precision: state what evidence is missing; keep the baseline or define a bounded learning pilot. - Critical slice or workload failure: repair the policy/system or narrow supported use with the owner. Average quality does not override it. - Predictive success with an unresolved intervention claim: use [experiment](../experiment/SKILL.md). - Supported requested operational use: pass the exact bundle and release record to [release](../release/SKILL.md). Save actual results, intended/excluded uses, limits, and the decision in the [notebook](../../references/notebook-protocol.md). An analysis report can be the endpoint. Equivalent quality with an accepted cost benefit can justify replacement; superiority on every metric is not mandatory. A score alone does not authorize deployment.
Files in this package (2)
- SKILL.md 5 KB
- references/system-evidence.md 10 KB
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · Kiran-980/1ml | 100 | MIT |
| DiogoHSM/driver | 100 | MIT |
| boshu2/agentops | 100 | Apache-2.0 |
| RemiPelloux/kill-my-idea | 100 | MIT |
| lalitmaganti/syntaqlite | 100 | Apache-2.0 |
| zanderadam/engy | 100 | MIT |
| hirogakatageri/hirokata-claude-marketplace | 100 | MIT |
| fabricio21x/fbs | 100 | MIT |
| whawkinsiv/solo-founder-skills | 100 | MIT |
Database snapshot: 13.11.2026 · package version b2e6dbf8087b (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100validating-ai-ethics-and-fairness
Validate AI/ML models and datasets for bias, fairness, and ethical concerns. Use when auditing AI systems for ethical compliance, fairness assessment, or bias detection. Trigger with phrases like "evaluate model fairness", "check for bias", or "validate AI ethics". - 95release
Package a supported model or decision system and prepare or perform an authorized rollout. Use for serving contracts, artifact parity, resource checks, fallback, rollback, and monitoring ownership after intended-use evidence is available. - 90model-selection
Choose a model for an AI use case on your own eval suite (cost per success, latency, hosting) and write the decision record. Use for "which model?", a new release or a cost review. - 100ml-evaluation
This skill should be used when the user asks to evaluate, validate, compare, explain, or debug model performance. PROACTIVELY activate for: (1) metrics selection for classification, regression, ranking, NLP, CV, recommender, forecasting, and generative AI, (2)… - 86zils-train
Train a custom Zils decision model on the user's own labeled data and call it. Use when the user wants a model that answers one repeated question (yes/no, one of a list, or a score) from their data, or asks to train, fine-tune or replace an LLM classifier with Zils. - 9012-ai-model-card
Use when a production AI feature needs a version-specific model card covering purpose, data, evaluation, limitations, bias, intended use, prohibited use and operational pins; use AI architecture for system design and evaluation artefacts as evidence inputs.