finetuning-method-selection
/finetuning-method-selection
Rozstrzyga, czy w ogóle dostrajać model, i dobiera metodę (SFT, DPO, ORPO, KTO, GRPO, RLVR, dalszy pretraining) oraz model bazowy, także gdy wystarczyłby RAG lub prompt.
- Autor
- wshobson ↗
- Repozytorium
- agents ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 17.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 3
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 17.10.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Rozstrzyga, czy w ogóle dostrajać model, i dobiera metodę (SFT, DPO, ORPO, KTO, GRPO, RLVR, dalszy pretraining) oraz model bazowy, także gdy wystarczyłby RAG lub prompt.
Oryginalny opis (EN)
Decide whether to fine-tune at all, and route to the right method (SFT, DPO/ORPO/KTO, GRPO/RLVR, continued pretraining) and base model. Use when starting any fine-tuning effort, when unsure whether RAG or prompting would suffice, or when choosing between preference-optimization and reinforcement methods.
Jak działa
- When to Use This Skill
- Quick Reference
- Off-Ramps First
- Method Router
- Key Routing Facts
- Model Selection
- Memory Feasibility
- Related Skills
Podgląd SKILL.md
--- name: finetuning-method-selection description: Decide whether to fine-tune at all, and route to the right method (SFT, DPO/ORPO/KTO, GRPO/RLVR, continued pretraining) and base model. Use when starting any fine-tuning effort, when unsure whether RAG or prompting would suffice, or when choosing between preference-optimization and reinforcement methods. --- # Fine-Tuning Method Selection This is the router skill for the fine-tuning lifecycle: it decides whether fine-tuning is the right tool at all, and if so, which method and which base-model size class. Every other skill in this plugin assumes this routing already happened — start here before opening `lora-qlora-recipes`, `preference-optimization`, or `grpo-rlvr-training`. ## When to Use This Skill - Starting any fine-tuning effort, before a framework or base model has been chosen. - Unsure whether RAG or prompt engineering would solve the problem more cheaply than training. - Choosing between preference optimization (DPO family) and a reinforcement method (GRPO/RLVR) for the same underlying task. - Sizing a candidate model/method combination before committing to a run. ## Quick Reference | Situation | Route | |---|---| | Facts change often (prices, docs, news) | RAG, not fine-tuning | | Desired behavior still being figured out | Prompt engineering | | Stable domain knowledge, ≥500MB text | CPT then SFT — see Off-Ramps First | | Have input/output demonstrations | SFT — see `lora-qlora-recipes` | | Have preference pairs or thumbs-up/down | DPO/ORPO/KTO — see `preference-optimization` | | Have a verifiable pass/fail signal | GRPO+RLVR — see `grpo-rlvr-training` | | No eval harness yet | Stop — see `eval-harness-first` | ## Off-Ramps First Most requests that sound like "fine-tune this" are served better and cheaper elsewhere. Check these off-ramps before opening a training run: - **Knowledge-bound and volatile** (the gap is facts that change — prices, docs, current events): route to RAG, not fine-tuning. A fine-tuned model bakes in a snapshot; volatile facts go stale immediately. - **Behavior-bound and shifting** (the desired behavior is still being figured out, or changes per request): route to prompt engineering. Fine-tuning locks in a behavior; don't lock in one that hasn't stabilized yet. - **Stable, dense domain knowledge**: this is where continued pretraining (CPT) enters, sized by how much domain text exists: | Domain text volume | Route | |---|---| | <10MB | RAG only | | 10MB–500MB | RAG + fine-tune | | 500MB–10GB | CPT, then SFT | | >10GB | CPT required | CPT learning rate ≈ **10% of the pretraining LR**. CPT is guidance-only in this plugin — sizing and LR guidance live here, but this plugin does not execute a CPT run. ## Method Router Once the off-ramps are ruled out, this is the full decision tree (verbatim from the research this plugin is built on): ``` New FACTS? volatile → RAG | stable+dense → CPT (LR ~10% of pretrain) → SFT New BEHAVIOR? shifting → prompt-engineering | stable: demos → SFT (LoRA/QLoRA, all-linear, α=2r) preference pairs → DPO (SimPO if length-bias, ORPO if memory-bound) unpaired 👍/👎 → KTO verifiable success → RLVR + GRPO (DAPO/GSPO/Dr.GRPO per failure mode) Deploy: FP8 (Hopper+) | NVFP4 (Blackwell scale) | AWQ (older) | GGUF+imatrix (edge) BEFORE ANY OF THIS: the eval harness must exist first. ``` Read the tree top-down: answer "new facts or new behavior," then follow the branch that matches the data shape in hand (demos, preference pairs, thumbs up/down, or verifiable success/failure). The data shape picks the method — not the other way around. ### Worked Routing Examples - *"Users want the assistant to follow our support macros exactly."* Behavior is stable and demonstrable from transcripts → demos → **SFT**. - *"We have pairs of good/bad responses from reviewer thumbs-up/down, unpaired."* → unpaired signal → **KTO**, not DPO (DPO needs paired preferences). - *"The model can already solve some of these math problems and we can grade correctness automatically."* → verifiable success signal → **GRPO+RLVR**, and only after confirming the model succeeds at least sometimes (see Key Routing Facts below). - *"We want the model to know this week's pricing page."* → volatile facts → **RAG**, no training run at all. ## Key Routing Facts - **Loss-function choice is low-leverage.** A 240-H100-run study found method choice worth ~1 percentage point versus ~50 points for model scale, and zero of 20 DPO variants beat vanilla DPO. Don't spend a routing decision agonizing over DPO-variant selection — spend it on getting the data shape and scale right. - **DPO is for taste, GRPO+RLVR is for reasoning.** Preference pairs that encode a subjective judgment (tone, style, "which answer is better") route to DPO. Tasks with a verifiable pass/fail signal (math, code, tool calls) route to GRPO+RLVR instead. - **RL is not the fix for a model that never succeeds.** GRPO and other RL methods sharpen an existing capability — they don't teach one from zero. If the model doesn't yet understand the task or output format, run SFT first; only bring in RL once the model succeeds at least sometimes. ### Common Routing Mistakes - Reaching for fine-tuning to fix facts that change weekly — that's a RAG problem, and fine-tuning will just go stale faster than the source data does. - Picking a DPO variant before checking whether the actual bottleneck is data quality or model scale — variant choice is the ~1pp lever, not the ~50pp one. - Starting an RL run on a model that fails every rollout — route to SFT first so RL has something to sharpen. - Treating CPT as the default for "the model doesn't know our domain" — check the data volume thresholds first; under 500MB, RAG or RAG+fine-tune iterates faster than a CPT run. ## Model Selection Base-model choice is size-class first, family second, and it goes stale fast — so it lives in exactly one place: `references/model-catalog.md`. That file is the only place in this plugin (and in the DGX Spark ops plugin) that names a base model family. Neither this skill nor `references/memory-math.md` names one; both describe models by size class only (for example, "8B-class LoRA," not a model name). The catalog is dated on purpose — model rankings turn over quarterly. It carries a "last verified" date and a refresh checklist. Before trusting a row, check that date; if stale, work the refresh checklist in the catalog before recommending a model from it. **Precedence when the catalog and a method skill disagree:** the catalog's per-row Notes column states hardware/size-class *feasibility*, not a method recommendation — `lora-qlora-recipes`'s LoRA vs QLoRA vs Full FT table (routed by task shape) governs the actual method choice. ## Memory Feasibility Before committing to a method, size it: total memory ≈ **params × dtype bytes + optimizer state + gradients + activations**. Work each term for the chosen dtype and method (full fine-tune, LoRA, or QLoRA) — worked worksheets and size-class examples live in `references/memory-math.md`. On DGX Spark specifically, unified-memory behavior breaks the naive estimate (transient load peaks, `nvidia-smi` underreporting, thermal throttling on long runs). Once the `dgx-spark-ops` plugin is installed, defer Spark-specific feasibility calls to its `spark-memory-thermal-ops` skill rather than re-deriving them here. ## Related Skills Once this skill has picked a method, hand off to the skill that executes it: - `lora-qlora-recipes` — SFT via LoRA/QLoRA - `preference-optimization` — DPO, ORPO, KTO - `grpo-rlvr-training` — GRPO with verifiable rewards No method is selected before the eval harness exists — see `eval-harness-first`.
Pliki w paczce (3)
- SKILL.md 8 KB
- references/memory-math.md 5 KB
- references/model-catalog.md 3 KB
Wersje
W bazie nie ma innych skilli o nazwie finetuning-method-selection.
Migawka bazy z 13.11.2026 · wersja paczki a8d7df6a39ea (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100finetuning
Dostosowuje model open-weight do tonu, formatu lub wzorca rozumowania przez LoRA i QLoRA lub pełny fine-tuning z TRL SFTTrainer, a potem optymalizację preferencji (DPO, ORPO, KTO, GRPO). - 95finetuning-setup
Dobiera model bazowy i technikę dostrajania SFT, DPO lub RLVR, sprawdzając dostępność modeli w SageMaker Hub. - 100finetuning
Generuje kod dostrajający model bazowy w serwerowych zadaniach treningowych SageMaker, z trenerami SFT, DPO, RLVR i RLAIF oraz funkcją nagrody Lambda. - 95finetuning
Generuje notebook Jupyter, który dostraja model w bezserwerowych zadaniach SageMaker: SFT, DPO lub RLVR z funkcją nagrody Lambda. - 100finetuning-technique
Dobiera technikę dostrajania (SFT, DPO, RLVR lub RLAIF) do przypadku użycia i waliduje ją względem dostępnych przepisów wybranego modelu bazowego. - 100preference-optimization
Dopasowuje dostrojony model do danych preferencji metodami DPO, ORPO, KTO lub SimPO i pomaga dobrać metodę oraz hiperparametry albo debugować przebieg DPO.