Skip to content
Skills Library
Data & analytics · Analysis

hyperparameter-tuning

/hyperparameter-tuning

Search hyperparameters with budgets, early stopping, and validation hygiene so gains are real, not overfit to the dev set. Use when tuning models or reviewing tuning claims.

Repository
AI-SKILLS ↗
Licence
MIT
Last repository change
05.09.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 05.09.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Search hyperparameters with budgets, early stopping, and validation hygiene so gains are real, not overfit to the dev set. Use when tuning models or reviewing tuning claims.

Original author description

Search hyperparameters with budgets, early stopping, and validation hygiene so gains are real, not overfit to the dev set. Use when tuning models or reviewing tuning claims.

How it works

  1. Method
  2. Boundaries
Original SKILL.md headings · 57 lines, 416 words
SKILL.md preview
---
name: hyperparameter-tuning
description: Search hyperparameters with budgets, early stopping, and validation hygiene so gains are real, not overfit to the dev set. Use when tuning models or reviewing tuning claims.
---

# Hyperparameter tuning

Tuning is compute spent buying metric points. Spend it with a budget, a
search strategy better than grid, and the humility that hundreds of
trials against one validation set will overfit to that set.

## Method

1. **Tune only after the pipeline is trustworthy.** Splits disciplined
   (see train-test-discipline), features leak-free (see
   feature-engineering), baseline recorded (see ml-baselines). Tuning
   a leaky pipeline polishes a lie; and data/feature improvements
   usually dwarf hyperparameter deltas, so exhaust them first.
2. **Search the parameters that matter, on the right scales.**
   Learning rate (log scale), regularization strength (log),
   tree depth/leaves, subsampling rates; leave third-order knobs at
   defaults. Random or quasi-random search beats grid at equal budget
   (grid wastes trials on unimportant axes); Bayesian/model-based
   search (Optuna-class) pays off when single trials are expensive.
3. **Use early stopping as both tuner and guard.** For iterative
   learners (boosting, neural nets), fix a generous iteration cap and
   early-stop on validation: it tunes the most sensitive parameter
   for free. At the search level, successive-halving/Hyperband prunes
   bad trials at low fidelity so the budget concentrates on
   contenders.
4. **Set the budget before the first trial.** N trials or X GPU-hours,
   decided from the metric's value (see cloud-cost-optimization
   thinking); log every trial's config and result (see
   experiment-tracking). Open-ended tuning runs until someone gets
   bored, which is a stopping rule that favors noise.
5. **Protect the validation set from the search.** Hundreds of trials
   maximize validation-set luck along with skill: use K-fold CV
   inside the search when data is small (see cross-validation),
   re-fit the winner on train+validation only per your protocol, and
   report the final number from the untouched test set once. Expect
   test to come in slightly under the tuned validation score; if it
   craters, the search overfit.
6. **Prefer flat optima and reproducibility.** Between configs within
   noise of each other, take the one in a wide flat region (stable to
   drift and retrains) over a sharp peak; verify the winner across
   2-3 seeds before believing it (see model-evaluation uncertainty).
   Record final config in code, not in a notebook's memory.

## Boundaries

- Tuning cannot rescue the wrong model class or missing signal; a
  plateaued search is evidence to change representation, not to add
  trials.
- Percent-level gains that vanish across seeds are noise laundered
  through effort; the ship decision weighs stability and cost, not
  leaderboard decimals.
- AutoML earns its keep for breadth on tabular problems; it still
  obeys every rule here (budget, validation hygiene, test-once), and
  its winner still needs the deployment review (see
  model-deployment).

Versions

RepositoryScoreLicence
This page · Amey-Thakur/AI-SKILLS90MIT
seb1n/awesome-ai-agent-skills100MIT
param087/agent-ml-skills100MIT
Harrypatria/data-mining-ml90MIT

Database snapshot: 13.11.2026 · package version 44273896db2e (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90ml-hyperparameter-tuning
    Use this skill when performing hyperparameter tuning, optimizing model performance via search strategies, or configuring tuning frameworks (Optuna, Ray Tune, Hyperopt). This skill enforces: search space definition, strategy selection (grid/random/Bayesian), framework…
  • 100hyperparameter-tuning-without-target-leakage
    Use when a search selects hyperparameters for a model whose labels are forward-looking and overlapping, such as h-bar returns or triple-barrier outcomes; purged and embargoed nested cross-validation so the selection folds stay clean.
  • 100hyperparameter-optimization
    Optimizes forecaster hyperparameters using grid search, random search, or Bayesian search (Optuna). Covers single-series and multi-series search, cross-validation configuration, and search space definition. Use when the user wants to find the best model configuration.
  • 100hyperparameter-optimization
    Optimizes forecaster hyperparameters using grid search, random search, or Bayesian search (Optuna). Covers single-series and multi-series search, cross-validation configuration, and search space definition. Use when the user wants to find the best model configuration.
  • 100tuning-hyperparameters
    Optimize machine learning model hyperparameters using grid search, random search, or Bayesian optimization. Finds best parameter configurations to maximize performance. Use when asked to "tune hyperparameters" or "optimize model". Trigger with relevant phrases based on skill purpose.
  • 100tuning-hyperparameters
    Optimize machine learning model hyperparameters using grid search, random search, or Bayesian optimization. Finds best parameter configurations to maximize performance. Use when asked to "tune hyperparameters" or "optimize model". Trigger with relevant phrases based on skill purpose.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.