Skip to content
Skills Library
AI & agents · Analysis

optimizer-selection

/optimizer-selection

Choose an optimiser and its hyperparameters based on the problem rather than habit, and know what each actually does. Use when training is unstable, slow to converge, or generalising poorly.

Repository
AI-SKILLS ↗
Licence
MIT
Last repository change
05.09.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 05.09.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Choose an optimiser and its hyperparameters based on the problem rather than habit, and know what each actually does. Use when training is unstable, slow to converge, or generalising poorly.

Original author description

Choose an optimiser and its hyperparameters based on the problem rather than habit, and know what each actually does. Use when training is unstable, slow to converge, or generalising poorly.

How it works

  1. Method
  2. Boundaries
Original SKILL.md headings · 39 lines, 231 words
SKILL.md preview
---
name: optimizer-selection
description: Choose an optimiser and its hyperparameters based on the problem rather than habit, and know what each actually does. Use when training is unstable, slow to converge, or generalising poorly.
---

# Optimizer selection

Optimisers differ in how they use gradient history to set the step. The
adaptive ones are robust defaults, plain momentum sometimes generalises
better, and the choice interacts strongly with learning rate and
regularisation.

## Method

1. **Start with a well-tested adaptive default.** It works across a wide
   range of problems with little tuning, which is why it is the default
   in nearly every codebase.
2. **Understand momentum as smoothing.** Accumulating past gradients
   damps oscillation across narrow valleys and accelerates along
   consistent directions.
3. **Know what adaptivity does.** Per-parameter step scaling helps when
   gradient magnitudes vary widely across parameters, which is typical
   in deep networks.
4. **Decouple weight decay from the gradient.** Applying decay as part
   of the gradient interacts badly with adaptive scaling, and the
   decoupled variant is the correct default (see
   regularization-techniques).
5. **Tune the learning rate before anything else.** It matters more than
   the optimiser choice by a wide margin (see learning-rate-schedules).
6. **Watch the optimiser's memory cost.** Adaptive methods store
   per-parameter state, adding substantially to memory for large models
   (see gpu-memory-hierarchy).
7. **Change one hyperparameter at a time.** Optimiser settings interact,
   and simultaneous changes make results uninterpretable (see
   experiment-tracking).

## Boundaries

Optimiser choice matters less than data quality and architecture for
most outcomes. Published settings are tuned to their paper's setup and
transfer imperfectly. Second-order methods are rarely practical at
scale despite their theoretical appeal.

Versions

The database has no other skills named optimizer-selection.

Database snapshot: 13.11.2026 · package version 1bc844a24b09 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100optimize-plugin
    This skill should be used when the user asks to "validate a plugin", "optimize plugin", "check plugin quality", "review plugin structure", or "run plugin optimizer".

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.