Skip to content
Skills Library
Data & analytics · Learning & explanation

training-protocol

/training-protocol

Reference for trustworthy neural network experiments - seeds and repeated runs, compute budget, tuning protocol, early stopping, experiment tracking and pretrained models. Use when planning, running or comparing training runs.

Licence
MIT
Last repository change
12.11.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 12.11.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Reference for trustworthy neural network experiments - seeds and repeated runs, compute budget, tuning protocol, early stopping, experiment tracking and pretrained models. Use when planning, running or comparing training runs.

Original author description

Reference for trustworthy neural network experiments - seeds and repeated runs, compute budget, tuning protocol, early stopping, experiment tracking and pretrained models. Use when planning, running or comparing training runs.

How it works

  1. Seeds and repeated runs
  2. Compute budget (human decision)
  3. Tuning protocol (fixed in advance)
  4. Tracking
  5. Pretrained models
Original SKILL.md headings · 33 lines, 316 words
SKILL.md preview
---
name: training-protocol
description: Reference for trustworthy neural network experiments - seeds and repeated runs, compute budget, tuning protocol, early stopping, experiment tracking and pretrained models. Use when planning, running or comparing training runs.
---

# Training protocol

Training runs are expensive and noisy. These rules make results comparable and claims defensible.

## Seeds and repeated runs

- Set and log the seed of every source of randomness (framework, NumPy, Python, data loader).
- Before claiming that configuration A beats B, run each with **at least three seeds** and report mean and spread. An improvement smaller than the spread is not an improvement.
- Report the spread in findings, not only the best run.

## Compute budget (human decision)

Before tuning starts, the user sets the budget: number of runs, maximum time or GPU hours. Record it on the experiment card. Stop when the budget is spent, and report the best result within budget.

## Tuning protocol (fixed in advance)

1. Search space: which hyperparameters, which ranges, on which scale (learning rate on a log scale).
2. Method: grid, random or Bayesian search, with the budget above.
3. Selection: by the validation metric from the analysis plan, averaged over seeds for the finalists.
4. Stopping rule: early stopping on validation loss with a fixed patience.
5. Final estimate: retrain the selected configuration as planned and evaluate on the test set **once**.

Record every run, not only the best: tuning many configurations and reporting the best one overstates performance.

## Tracking

Each run logs: date, card, commit, data version, full configuration, seed, metrics per epoch (training and validation), final metrics per segment, runtime. Use a tracking tool (e.g. MLflow, Weights & Biases) or a CSV next to `experiments/LOG.md`. Summarise in the log only runs that change a decision.

## Pretrained models

Record the model name and version, its source, its license, and what is known about its training data. Ask whether that data could overlap with the evaluation data (e.g. public benchmark images); if so, evaluate on data the model cannot have seen. Freezing and fine-tuning choices are data decisions for `DECISIONS.md`.

Versions

The database has no other skills named training-protocol.

Database snapshot: 13.11.2026 · package version 4e7ef7c8b9f2 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100reproducible-runs
    Use when writing, changing, or running a machine learning training, fine-tuning, or experiment script or notebook, and whenever a result cannot be reproduced or changes between runs. Sets seeds, records config, environment, git commit and data hashes in one directory per run,…
  • 95domino-experiment-tracking
    Track traditional ML experiments in Domino using the MLflow-based Experiment Manager. Covers experiment setup, auto-logging for sklearn/TensorFlow/PyTorch, manual logging, artifact storage, run comparison, and model registration. Use when training ML models, logging metrics and…
  • 100hyperparameter-tuning
    Use when optimizing model hyperparameters. Covers search strategy (random vs Bayesian/Optuna), leakage-safe tuning inside CV, search-space design, early stopping, and budget management.
  • 90experiment
    Discipline for ML, RL, GAN, LLM and benchmark experiments: baseline first, fixed seeds, one change per run, config and metrics logged to files, and every reported number explained and traceable. Use when training or evaluating a model, tuning hyperparameters, comparing methods,…
  • 86reproducible-ml-experiments
    Implement, run, and compare ML experiments with traceable configurations and artifacts. Use when training, fine-tuning, evaluating, or reproducing a model.
  • 100domino-experiment-tracking
    Track traditional ML experiments in Domino's MLflow-based Experiment Manager. Covers unique experiment naming, auto-logging for scikit-learn, TensorFlow, PyTorch, XGBoost and LightGBM, manual logging of params, metrics and artifacts, Domino context tags, large-artifact multipart…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.