training-protocol
/training-protocol
Reference for trustworthy neural network experiments - seeds and repeated runs, compute budget, tuning protocol, early stopping, experiment tracking and pretrained models. Use when planning, running or comparing training runs.
- Author
- steffen74 ↗
- Repository
- agentic-data-science ↗
- Licence
- MIT
- Last repository change
- 12.11.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 12.11.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Reference for trustworthy neural network experiments - seeds and repeated runs, compute budget, tuning protocol, early stopping, experiment tracking and pretrained models. Use when planning, running or comparing training runs.
Original author description
Reference for trustworthy neural network experiments - seeds and repeated runs, compute budget, tuning protocol, early stopping, experiment tracking and pretrained models. Use when planning, running or comparing training runs.
How it works
- Seeds and repeated runs
- Compute budget (human decision)
- Tuning protocol (fixed in advance)
- Tracking
- Pretrained models
SKILL.md preview
--- name: training-protocol description: Reference for trustworthy neural network experiments - seeds and repeated runs, compute budget, tuning protocol, early stopping, experiment tracking and pretrained models. Use when planning, running or comparing training runs. --- # Training protocol Training runs are expensive and noisy. These rules make results comparable and claims defensible. ## Seeds and repeated runs - Set and log the seed of every source of randomness (framework, NumPy, Python, data loader). - Before claiming that configuration A beats B, run each with **at least three seeds** and report mean and spread. An improvement smaller than the spread is not an improvement. - Report the spread in findings, not only the best run. ## Compute budget (human decision) Before tuning starts, the user sets the budget: number of runs, maximum time or GPU hours. Record it on the experiment card. Stop when the budget is spent, and report the best result within budget. ## Tuning protocol (fixed in advance) 1. Search space: which hyperparameters, which ranges, on which scale (learning rate on a log scale). 2. Method: grid, random or Bayesian search, with the budget above. 3. Selection: by the validation metric from the analysis plan, averaged over seeds for the finalists. 4. Stopping rule: early stopping on validation loss with a fixed patience. 5. Final estimate: retrain the selected configuration as planned and evaluate on the test set **once**. Record every run, not only the best: tuning many configurations and reporting the best one overstates performance. ## Tracking Each run logs: date, card, commit, data version, full configuration, seed, metrics per epoch (training and validation), final metrics per segment, runtime. Use a tracking tool (e.g. MLflow, Weights & Biases) or a CSV next to `experiments/LOG.md`. Summarise in the log only runs that change a decision. ## Pretrained models Record the model name and version, its source, its license, and what is known about its training data. Ask whether that data could overlap with the evaluation data (e.g. public benchmark images); if so, evaluate on data the model cannot have seen. Freezing and fine-tuning choices are data decisions for `DECISIONS.md`.
Versions
The database has no other skills named training-protocol.
Database snapshot: 13.11.2026 · package version 4e7ef7c8b9f2 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100reproducible-runs
Use when writing, changing, or running a machine learning training, fine-tuning, or experiment script or notebook, and whenever a result cannot be reproduced or changes between runs. Sets seeds, records config, environment, git commit and data hashes in one directory per run,… - 95domino-experiment-tracking
Track traditional ML experiments in Domino using the MLflow-based Experiment Manager. Covers experiment setup, auto-logging for sklearn/TensorFlow/PyTorch, manual logging, artifact storage, run comparison, and model registration. Use when training ML models, logging metrics and… - 100hyperparameter-tuning
Use when optimizing model hyperparameters. Covers search strategy (random vs Bayesian/Optuna), leakage-safe tuning inside CV, search-space design, early stopping, and budget management. - 90experiment
Discipline for ML, RL, GAN, LLM and benchmark experiments: baseline first, fixed seeds, one change per run, config and metrics logged to files, and every reported number explained and traceable. Use when training or evaluating a model, tuning hyperparameters, comparing methods,… - 86reproducible-ml-experiments
Implement, run, and compare ML experiments with traceable configurations and artifacts. Use when training, fine-tuning, evaluating, or reproducing a model. - 100domino-experiment-tracking
Track traditional ML experiments in Domino's MLflow-based Experiment Manager. Covers unique experiment naming, auto-logging for scikit-learn, TensorFlow, PyTorch, XGBoost and LightGBM, manual logging of params, metrics and artifacts, Domino context tags, large-artifact multipart…