Skip to content
Skills Library
AI & agents · Building & coding

ml-training

/ml-training

Fine-tunes and trains model weights with LoRA, QLoRA, SFT, DPO, distributed training and GPUs. Use when a training run hits CUDA out of memory, a transformers or TRL script breaks after an upgrade, choosing between SFT, DPO and GRPO, picking FSDP or DeepSpeed parallelism, or reviewing a training script before spending GPU hours.

Repository
hyperskills ↗
Licence
MIT
Last repository change
04.11.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
25
Scripts
7
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 04.11.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What to check before installing
  • 7 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: evals/files/align/train_grpo.py, evals/files/api/classify.py, evals/files/data/build_dataset.py, evals/files/longctx/train_sft_long.py, evals/files/sft/train_sft.py, scripts/dataset_overlap.py, scripts/vram_ledger.py

What it does and when to use it

Fine-tunes and trains model weights with LoRA, QLoRA, SFT, DPO, distributed training and GPUs. Use when a training run hits CUDA out of memory, a transformers or TRL script breaks after an upgrade, choosing between SFT, DPO and GRPO, picking FSDP or DeepSpeed parallelism, or reviewing a training script before spending GPU hours.

Original author description

Fine-tunes and trains model weights with LoRA, QLoRA, SFT, DPO, distributed training and GPUs. Use when a training run hits CUDA out of memory, a transformers or TRL script breaks after an upgrade, choosing between SFT, DPO and GRPO, picking FSDP or DeepSpeed parallelism, or reviewing a training script before spending GPU hours.

How it works

  1. Scope
  2. Read first
  3. Core rules
  4. Workflows
  5. Topic router
  6. Output format
  7. Environment
Original SKILL.md headings · 327 lines, 2,837 words
SKILL.md preview
---
name: ml-training
description: "Fine-tunes and trains model weights with LoRA, QLoRA, SFT, DPO, distributed training and GPUs. Use when a training run hits CUDA out of memory, a transformers or TRL script breaks after an upgrade, choosing between SFT, DPO and GRPO, picking FSDP or DeepSpeed parallelism, or reviewing a training script before spending GPU hours."
license: MIT (upstream attributions in NOTICE.md)
metadata:
  author: HyperSkills
  version: "2026.11.04"
  category: task
---

# ml-training

Paths below are relative to this skill's directory.

## Scope

Everything between "we have a model and some data" and "the weights we
produced are serving traffic": method choice, the memory arithmetic that
decides whether a run is possible, distributed strategy, dataset hygiene,
evaluation that can be believed, checkpointing, and the handoff to an
inference server.

Calling a model someone else trained is the `ai-engineering` skill's job; changing the weights is the `ml-training` skill's.

Not covered — do not answer from this skill:

- Prompting, RAG, agent loops, and anything whose remedy is a better request
  to a hosted model: the `ai-engineering` skill.
- Running someone else's diffusion or speech weights to produce images,
  video or audio, and the reproducibility of those renders: the
  `generative-media` skill. Training or adapting those weights belongs here.
- Python packaging, virtualenvs, type checking, the language itself: the
  `python` skill. Training code is Python, but its problems here are
  numerical and memory-shaped.
- Dockerfiles, Kubernetes, job schedulers: the `containers` skill.
- Inference-serving performance: benchmarking a deployment, KV cache and
  `max-model-len` sizing, batching and chunked prefill, prefix caching,
  inference-side parallelism, and replica scaling: the `model-serving` skill.
  The handoff of a checkpoint to a server belongs here; everything the server
  does with it afterwards does not.
- GPU instance types, quota requests, spot interruption, storage classes:
  the `aws`, `azure` and `gcp` skills.
- Vector stores and index tuning: the `redis`, `mongodb`, `postgres` and
  `elasticsearch` skills.
- Writing a SKILL.md: the `skill-authoring` skill.
- Classical tabular ML, forecasting, computer-vision pipelines that are not
  transformer fine-tunes, and scientific or bioinformatics workflows. Nothing
  here is tuned for them.

## Read first

`references/api-versions.md` before touching any training script. Most
scripts in circulation were written against transformers 4.x and trl 0.x and
fail at config construction on transformers 5 / trl 1, one `TypeError` per
launch. Fix them in one pass.

## Core rules

Each rule is an invariant; the clause after the dash is what violating it costs.

1. Before any run, write the memory ledger — weights, gradients, optimizer
   state, activations, separately. A total without a breakdown cannot tell
   you which knob to turn, and the default guess (batch size) is wrong
   whenever model state dominates.
2. Full fine-tuning costs about 16 bytes per parameter of model state before
   a single activation: 2 for the bf16 weight, 2 for the gradient, 8 for
   Adam's two fp32 moments, 4 for the fp32 master copy. An 8B model is
   therefore ~119 GiB and does not fit on one 80 GB card at any batch size —
   proposing a smaller batch there wastes a launch and a queue slot.
3. `bf16=True` does not halve training memory. It halves weights and
   gradients and leaves the optimizer term alone, so expect 25-30% off model
   state, not 50% — a plan built on "half" is short by a card.
4. Gradient accumulation changes the effective batch, not the activation
   peak; the micro-batch is what activations scale with. Reaching for
   `gradient_accumulation_steps` to fix an OOM without lowering the
   per-device batch changes nothing and costs a run.
5. Sharding (ZeRO, FSDP) divides model state across the data-parallel group
   and leaves activations untouched. A run whose dominant term is activations
   gets no relief from more GPUs, and the cluster bill says otherwise.
6. LoRA freezes the base model; it does not shrink it. Gradients and
   optimizer state collapse to the adapter's size, the weights stay resident
   in full — so LoRA does not rescue a model whose *weights* alone exceed the
   card. QLoRA is the tool for that term.
7. Set `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` before changing any
   parallelism degree. Fragmentation-induced OOM is common and this costs
   nothing, where doubling tensor parallelism to fix the same OOM has been
   measured at -28% throughput for 3 GB. (Incompatible with `--use-nccl-ub`.)
8. Keep tensor parallelism inside one NVLink domain and cross node boundaries
   with pipeline or data parallelism — TP over the network turns an
   all-reduce per transformer block into the step's critical path.
9. Minimum GPUs for a combined config is `PP * max(TP * CP, EP * ETP)`, not
   the product of every dimension: the dense and expert meshes share the GPUs
   inside a pipeline stage. The product form over-allocates hardware and is
   in many sizing tables.
10. Deduplicate and decontaminate the pooled corpus **before** splitting.
    Deduplicating each split separately leaves every cross-split pair intact,
    which is exactly the pair that inflates the eval number you will report.
11. Split by the unit of independence — thread, document, customer, time —
    never by row. A random row split over re-opened tickets or chunked
    documents measures memorisation.
12. Any public benchmark you report on is checked against the training corpus
    by n-gram overlap (n=13 is the standard), and the contaminated fraction
    is reported with the score. An unchecked benchmark number is not a
    measurement, and mixing the benchmark into training for "domain coverage"
    destroys it outright.
13. Measure the base model on the same eval set before training. Without that
    baseline the fine-tune gets credit for everything the base model already
    did, and a pointless training cycle becomes a quarterly ritual.
14. Every fine-tune runs a regression suite against general capability, not
    only its own task metric — a model that is better at one task and worse
    at everything else passes task-only evaluation and fails in the product.
15. Preference methods run at roughly 1e-6, SFT at roughly 2e-5 (the trl 1.13
    defaults). Carrying an SFT learning rate into DPO or GRPO moves the
    policy off the manifold: it wins the preference eval and cannot hold a
    conversation.
16. DPO's `beta` is the KL anchor to the reference model, and the reference
    is the SFT checkpoint the run started from. Anchoring to the base model,
    or dropping `beta` to learn faster, buys preference-set wins and pays in
    everything else.
17. GRPO requires a reward that is verifiable — a test passes, the arithmetic
    checks, the schema validates. A reward that can be satisfied by emitting
    a fixed string will be, especially at the trl 1.13 default `beta=0.0`,
    which switches the KL penalty off.
18. GRPO's `generation_batch_size` (per-device batch x world size x
    steps-per-generation) must be divisible by `num_generations`, or the
    config raises before the first step. On one GPU this is what forces
    gradient accumulation: a group of two is a coin flip, not an advantage
    estimate.
19. A resumable checkpoint carries optimizer state, scheduler state, step
    count, RNG state and the dataloader position. Weights alone resume as a
    warm start with zero momentum and a restarted warmup, and the loss curve
    shows it.
20. Under FSDP or ZeRO-3, save with Distributed Checkpoint or the distributed
    state-dict helpers. `torch.save(model.state_dict())` over DTensors writes
    a file that loads nowhere else, and that is discovered at serving time.
21. Verify the loss mask by decoding the positions where `labels != -100` on
    one real batch. If that text is not exactly what the model should
    produce, every subsequent number is measuring the wrong objective.
22. Render one training example and compare it byte-for-byte with what the
    serving stack sends. A template or BOS mismatch degrades everything and
    fails no test.
23. Two runs differing by a fraction of a point are the same run. Establish
    the noise floor with two seeds before attributing a delta to a change —
    reduction order, kernel versions and world size all move weights even
    when fully seeded.
24. Change one thing per run and record the resolved config, package
    versions, dataset hash and base-model revision with the checkpoint. A run
    you cannot reconstruct is an anecdote, and a config where three knobs
    moved cannot be bisected.

## Workflows

### plan-a-run

Before requesting GPUs.

- [ ] State the goal as an acceptance criterion that can come out false, and
      name the eval set it will be judged on. Freeze that set now.
- [ ] Check whether weights need to change at all: write ten examples of the
      wanted behaviour and try them as a prompt. If that works, stop — this
      is the `ai-engineering` skill's problem.
- [ ] Pick the method from the data you have, not the one you want:
      `references/method-selection.md`.
- [ ] Compute the token-length distribution with the real tokenizer and chat
      template. Set `max_length` from the p99, and note the truncated
      fraction.
- [ ] Run `uv run scripts/vram_ledger.py` for the candidate configuration.
      Identify the dominant term and choose the remedy that moves *that*
      term: `references/memory-ledger.md`.
- [ ] If it needs more than one GPU, choose the parallelism from the model
      size and the interconnect, and compute the minimum GPU count:
      `references/parallelism.md`.
- [ ] Run `uv run scripts/dataset_overlap.py --train ... --eval ...` and fix
      what it reports before training: `references/dataset-hygiene.md`.
- [ ] Decide checkpoint cadence, retention and disk budget at roughly 16
      bytes per parameter per checkpoint.
- [ ] Smoke-run: 10 steps at the real sequence length and micro-batch, with
      eval and checkpointing both triggered once. Record peak memory.
- [ ] **Gate — THE LEDGER MATCHES THE SMOKE RUN:** measured peak is within
      about 20% of the estimate, the overlap check is clean, and the decoded
      loss mask is the text you want generated. A mismatch means the
      configuration is not the one you costed.

### debug-oom

- [ ] Read the allocator message: allocated, reserved-but-unallocated, and
      the size of the failed allocation. Reserved-but-unallocated in the
      gigabytes means fragmentation.
- [ ] Set `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` and retry before
      anything else.
- [ ] Establish *when* it OOMs — step 1, the first eval boundary, the first
      save, or gradually. Each has a different cause:
      `references/oom-and-throughput.md`.
- [ ] Run the ledger, then walk the fixed order in that file. Re-run the
      ledger after each change; the dominant term moves.
- [ ] **Gate — THE FIX MATCHES THE DOMINANT TERM:** you can name the term
      that dominated, the remedy that moves it, and the throughput it cost.
      "It stopped OOMing after I changed four things" is not a diagnosis.

### review-training-code

Reviewing a script or config before GPU hours are spent.

- [ ] Check it against the installed versions first: the renames in
      `references/api-versions.md` fail at construction and hide everything
      behind them.
- [ ] Recompute the memory ledger from the config's own numbers and say
      whether it fits the stated hardware.
- [ ] Recompute the effective batch and, for GRPO, the generation-batch
      divisibility.
- [ ] Check learning rate against the method (rule 15), and the epoch count
      against the dataset size.
- [ ] Trace the data path: where the split happens relative to dedup and
      decontamination, what the loss is masked to, what gets truncated.
- [ ] Check what is saved and whether it can be resumed and served.
- [ ] **Gate — EVERY FINDING NAMES ITS FAILURE:** each finding says whether
      it stops the script, wastes the run, or silently degrades the result.
      A finding you cannot classify is a style preference; drop it.

### diagnose-quality

The metric and reality disagree, or the fine-tune did not help.

- [ ] Check contamination first — benchmark against corpus, and both sides of
      the split against each other. It is the most likely cause and the
      cheapest to rule out.
- [ ] Check independence: is the reported subset carved out of the eval
      split, is the eval set derived from anything used in filtering.
- [ ] Compare against the base model on the same set with the same decoding
      parameters. Then compare against the previous release.
- [ ] Check the metric against the goal: exact match rewards reproducing a
      reference string, which memorisation does perfectly.
- [ ] Check training-time defects last: loss mask, template, truncation,
      learning rate, epochs. `references/evaluation.md`.
- [ ] **Gate — THE DISAGREEMENT IS EXPLAINED, NOT NARROWED:** you can say
      which measurement was wrong and why, and which numbers must be
      re-measured. Adjusting a threshold until the numbers agree is not an
      explanation.

### Rationalizations

| Excuse | Reality |
|---|---|
| "Just lower the batch size" | Only if activations dominate. Say which term does first. |
| "We'll use LoRA, so memory is fine" | The frozen base is still resident. LoRA moves two of four terms. |
| "More GPUs will fix the OOM" | Not if activations dominate; sharding does not touch them. |
| "The benchmark is 86%, ship it" | Check whether the benchmark is in the training set before believing it. |
| "We deduplicated the data" | Per split, after splitting? Then the leakage is still there. |
| "It converged, so the data is fine" | Convergence on leaked data is memorisation with a nice curve. |
| "bf16 halves the memory" | It halves two terms of four. The optimizer is unchanged. |
| "Same hyperparameters as the SFT run" | For DPO or GRPO that is a twentyfold learning rate. |
| "We'll evaluate after we see the loss" | Then the criterion will describe whatever you built. |

### Red flags

Stop when you catch any of these: an OOM "fixed" by changing several knobs at
once; a reported benchmark that was never checked against the corpus; a
train/eval split created before deduplication; a preference run at 2e-5; a
GRPO reward that a fixed string satisfies; a checkpoint directory containing
only weights; a run comparison across a transformers major version; a
`max_length` chosen without looking at the length distribution; "we'll merge
the adapter later" with no record of the base revision; a quantized model
shipped without a bf16 number to compare against.

## Topic router

| Topic | Read when | File |
|---|---|---|
| Bytes per parameter, the four terms, what mixed precision really costs, what each remedy moves, worked ledgers | Sizing a run, or an OOM whose cause is unclear | `references/memory-ledger.md` |
| SFT vs DPO vs GRPO preconditions and data shapes, learning-rate scales, LoRA/QLoRA/full choice, continued pretraining | Choosing how to train, or reviewing someone's choice | `references/method-selection.md` |
| DDP/ZeRO/FSDP2/TP/PP/CP ladder, communication costs, minimum GPU count, effective batch across world sizes | The run needs more than one GPU | `references/parallelism.md` |
| transformers 5 / trl 1 removed arguments, changed defaults, gradient-checkpointing signature, reading a version from its error | Before editing any training script | `references/api-versions.md` |
| Dedup/decontaminate/split order, n-gram contamination, token-length inspection, loss masking, how much data is enough | Building or auditing a dataset pipeline | `references/dataset-hygiene.md` |
| The four measurements, reading loss curves, benchmark-versus-human disagreement, regression suites, LLM-as-judge discipline | Deciding whether a model is better | `references/evaluation.md` |
| The fixed OOM order, reading the allocator message, OOM-at-step-200 causes, throughput thieves | Something is out of memory or slower than it should be | `references/oom-and-throughput.md` |
| What a resumable checkpoint contains, distributed checkpoints, adapter provenance, seeding, irreducible nondeterminism | Setting up saving/resume, or a result will not reproduce | `references/checkpointing-and-reproducibility.md` |
| PTQ vs QAT vs QLoRA, format choice, what quantization actually buys, measuring the damage | Shrinking a model for training or serving | `references/quantization.md` |
| The three silent mismatches at the handoff (chat template, tokenizer, stop strings), establishing the bf16 baseline, adapter provenance | Handing a trained checkpoint to an inference server | `references/serving-handoff.md` |

## Output format

**Training-code review** — the `review-training-code` workflow. Group by
severity, because the classes have different urgency: a construction error
costs a launch, a hyperparameter error costs the run, a data defect costs
every number measured afterwards.

```
## Blocks the launch
path:line - <finding> - <the error it raises> - <fix>

## Wastes the run
path:line - <finding> - <what the run will produce instead> - <fix>

## Silently degrades the result
path:line - <finding> - <what it corrupts and how you would notice> - <fix>

## Memory ledger for this config
weights <X> | gradients <X> | optimizer <X> | activations <X> | total <X>
dominant term: <term> -> <remedy that moves that term>
fits <hardware>: yes / no
```

**Run plan** — the `plan-a-run` workflow. Short, and every line a number
someone can check.

```
Method: <SFT|DPO|GRPO|continued pretraining> because <the data you have>
Base: <model id @ revision>
Data: <rows> rows, <tokens> tokens, p99 <N> tokens, truncated at max_length=<N>: <X>%
Overlap check: <clean | N eval rows share a 13-gram with train>
Ledger: <per-GPU total> on <hardware>, dominant term <term>
Parallelism: <config>, min GPUs <PP * max(TP*CP, EP*ETP)>
Effective batch: <per-device> x <accum> x <world> = <N>
Acceptance: <criterion that can come out false>, measured on <frozen eval set>
Regression: <suite>, tolerance <N points>
```

## Environment

The scripts in `scripts/` are dependency-free and run anywhere:

```bash
uv run scripts/vram_ledger.py --params 8e9 --seq 4096 --micro-batch 4
uv run scripts/dataset_overlap.py --train train.jsonl --eval eval.jsonl
```

Both are **Run** scripts. `dataset_overlap.py` exits non-zero when it finds
leakage, so it can gate a pipeline.

For the training stack itself, read the versions rather than assuming them:

```bash
python -c "import torch, transformers, trl, peft; print(torch.__version__, transformers.__version__, trl.__version__, peft.__version__)"
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.device_count())"
```
Files in this package (25)
  • NOTICE.md 5 KB
  • SKILL.md 19 KB
  • SOURCES.yaml 10 KB
  • evals/evals.json 9 KB
  • evals/files/align/train_grpo.py 2 KB
  • evals/files/api/classify.py 1 KB
  • evals/files/data/build_dataset.py 2 KB
  • evals/files/data/eval-report.md 1 KB
  • evals/files/longctx/run-notes.md 1 KB
  • evals/files/longctx/train_sft_long.py 1 KB
  • evals/files/serve/vllm-deploy.md 1 KB
  • evals/files/sft/run.md 1 KB
  • evals/files/sft/train_sft.py 1 KB
  • references/api-versions.md 7 KB
  • references/checkpointing-and-reproducibility.md 7 KB
  • references/dataset-hygiene.md 7 KB
  • references/evaluation.md 6 KB
  • references/memory-ledger.md 7 KB
  • references/method-selection.md 9 KB
  • references/oom-and-throughput.md 6 KB
  • references/parallelism.md 8 KB
  • references/quantization.md 5 KB
  • references/serving-handoff.md 2 KB
  • scripts/dataset_overlap.py 6 KB
  • scripts/vram_ledger.py 7 KB

Versions

RepositoryScoreLicence
This page · lynricsy/hyperskills100MIT
JosiahSiegel/claude-plugin-marketplace100MIT

Database snapshot: 13.11.2026 · package version 62ab8ce54b94 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 95trl-training
    Post-train LLMs with TRL (Transformers Reinforcement Learning) — SFT, DPO, GRPO, KTO, and reward-model training. Use when writing or debugging training code with the TRL Python API or the trl CLI.
  • 95trl-training
    Post-train LLMs with TRL (Transformers Reinforcement Learning) — SFT, DPO, GRPO, KTO, and reward-model training. Use when writing or debugging training code with the TRL Python API or the trl CLI.
  • 100fine-tuning-with-trl
    Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.
  • 90post-training-strategy
    为 Agent 模型做后训练或选路线时使用——判断该先补 Mid-training、用 SFT 立协议还是直接上 RL;构造 Mid-training 语料与 SFT 数据、搭建 RL 环境、在 GRPO/PPO/DPO 与蒸馏之间取舍;排查 pass@k 近零、格式不稳就训 RL、组内无奖励差异、训练-推理数值失配、过度训练泛化下降。触发词:后训练、Mid-training、继续预训练、SFT、RL、GRPO、PPO、DPO、拒绝采样、RFT、蒸馏、on-policy、RLVR、LoRA。
  • 95soup
    Drive Soup (`soup-cli`), a CLI-first tool for fine-tuning and post-training LLMs with one YAML config and one command — SFT, DPO/GRPO/ORPO/SimPO/KTO, QLoRA/DoRA/LoRA+, layer streaming for 4-8 GB GPUs, eval-gated training, and serving. Use when the user wants to `soup init`/`soup…
  • 95fine-tuning-with-trl
    Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.