speculative-decoding
/speculative-decoding
Trenuje, debuguje i waliduje model roboczy do dekodowania spekulatywnego (EAGLE3, DFlash, DSpark, Domino) w potoku ModelOpt, w tym analizę nieudanych przebiegów i progów akceptacji.
- Autor
- nvidia ↗
- Repozytorium
- model-optimizer ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 27.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 12
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 27.10.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Trenuje, debuguje i waliduje model roboczy do dekodowania spekulatywnego (EAGLE3, DFlash, DSpark, Domino) w potoku ModelOpt, w tym analizę nieudanych przebiegów i progów akceptacji.
Wyzwalacze z opisu
Oryginalny opis (EN)
Train, debug, and validate a speculative decoding draft model (EAGLE3, DFlash, DSpark, Domino) through the ModelOpt launcher pipeline. Use when the user wants to add a new model to a draft-training pipeline, asks why a pipeline run failed, wants experiment logs reviewed, or wants to check whether a run's acceptance rate meets threshold. Triggers on "EAGLE3", "DFlash", "DSpark", "Domino", "draft model", "acceptance rate", "speculative decoding pipeline". Do NOT use for quantizing a model (use ptq) or serving a checkpoint (use deployment).
Jak działa
- Two axes: stage and algorithm
- Where the code lives
- Stages
- Algorithms
- End-to-end: a new model
Podgląd SKILL.md
---
name: speculative-decoding
description: >-
Train, debug, and validate a speculative decoding draft model (EAGLE3, DFlash,
DSpark, Domino) through the ModelOpt launcher pipeline. Use when the user wants
to add a new model to a draft-training pipeline, asks why a pipeline run failed,
wants experiment logs reviewed, or wants to check whether a run's acceptance rate
meets threshold. Triggers on "EAGLE3", "DFlash", "DSpark", "Domino", "draft model",
"acceptance rate", "speculative decoding pipeline". Do NOT use for quantizing a
model (use ptq) or serving a checkpoint (use deployment).
user_invocable: true
---
# Speculative Decoding Draft-Model Training
Everything needed to take a target model from "no draft head" to "validated
acceptance rate" lives in this directory. Work through the stages below in order
for a new model; jump straight to a stage when you already know which one you need.
## Two axes: stage and algorithm
The pipeline is the same shape for every draft-model algorithm — synthesize data,
dump base-model hidden states, train the draft, benchmark acceptance rate. What
changes between algorithms is which training script and recipe run, which knobs
matter, and which failures are typical.
So this skill is split along those two axes, and **you almost always read one file
from each**:
| Axis | Directory | What it holds |
| --- | --- | --- |
| Stage | `references/stages/` | The procedure — algorithm-independent |
| Algorithm | `references/algorithms/` | The data sheet — scripts, recipe, knobs, thresholds, known failures |
Read the stage file for *what to do*, and the algorithm sheet for *the values to
plug in*. When a stage file says "see the algorithm sheet", it means the section of
`references/algorithms/<algorithm>.md` with the matching heading.
## Where the code lives
The launcher scripts in `tools/launcher/common/{eagle3,specdec}/` are thin wrappers around
`examples/speculative_decoding/` — `main.py` (training), `collect_hidden_states/` (dumps),
`scripts/{export_hf_checkpoint,ar_validate,quantize_drafter}.py`. Read a wrapper for the
flags it passes, that directory for what they do. Its `README.md` is also the non-launcher
path: training invoked directly, without Slurm.
## Stages
| Stage | Reference | Use when |
| --- | --- | --- |
| 1. Configure | `references/stages/configure.md` | Adding a model that has no pipeline YAML yet |
| 2. Review logs | `references/stages/review-logs.md` | A run finished (or died) and you want a pass/fail summary |
| 3. Triage | `references/stages/triage.md` | A task failed and you need root cause plus a fix |
| 4. Validate | `references/stages/validate.md` | All tasks passed and you need to confirm the acceptance rate gate |
Review-logs and triage overlap by design: review-logs is the fast sweep across all
tasks, triage is the deep dive into one failing task. Start with review-logs unless
the user already knows which task broke.
## Algorithms
All recipes live in `modelopt_recipes/general/speculative_decoding/<algorithm>.yaml`.
| Algorithm | Sheet | Family |
| --- | --- | --- |
| EAGLE3 | `references/algorithms/eagle3.md` | Autoregressive draft head |
| DFlash | `references/algorithms/dflash.md` | Block diffusion |
| DSpark | `references/algorithms/dspark.md` | DFlash backbone + Markov head + optional confidence head |
| Domino | `references/algorithms/domino.md` | DFlash backbone + GRU causal correction head |
| DFlash2 | `references/algorithms/dflash2.md` | DFlash backbone + sublayer convolutions + candidate selector |
| LiLiCorr | `references/algorithms/lilicorr.md` | DFlash backbone + candidate-lattice reranker (covers `lilicorr_conv.yaml`) |
Everything below EAGLE3 is a **DFlash variant**, not a separate pipeline: same
`recipe_type: speculative_dflash`, same training script, same `dflash.*` config
namespace, selected by `dflash_architecture_config.projector_type`. Read
`references/algorithms/dflash.md` first, then the variant's sheet for the delta.
LiLiCorr ships two recipes — `lilicorr.yaml` and `lilicorr_conv.yaml`, the latter
adding DFlash2's sublayer convolutions — but one sheet covers both.
If the user's algorithm has no sheet yet, the stage procedures still apply — derive
the missing values from an existing launcher example for that algorithm
(`tools/launcher/examples/*/*/hf_*_<algorithm>.yaml`) and its recipe, then write the
sheet as you go. `references/algorithms/README.md` defines what a sheet must contain.
## End-to-end: a new model
1. Confirm the algorithm and find the closest existing launcher example.
2. **Configure** — write the pipeline YAML (`references/stages/configure.md`).
3. Preview with `--dryrun`, then submit:
```bash
cd tools/launcher
uv run launch.py --yaml examples/<Org>/<Model>/<config>.yaml --yes
```
4. Register the job and set up monitoring per the **monitor skill**.
5. **Review logs** when it finishes (`references/stages/review-logs.md`).
6. **Triage** anything that failed (`references/stages/triage.md`), fix, re-run only
the failed tasks onward via `pipeline.task_N.skip=true`.
7. **Validate** once all tasks pass (`references/stages/validate.md`).
Model-support gaps that need code changes land in `modelopt/torch/speculative/` and
require a separate ModelOpt PR — the pipeline YAML alone cannot fix an unrecognized
architecture.
Pliki w paczce (12)
- SKILL.md 5 KB
- references/algorithms/README.md 3 KB
- references/algorithms/dflash.md 11 KB
- references/algorithms/dflash2.md 10 KB
- references/algorithms/domino.md 7 KB
- references/algorithms/dspark.md 11 KB
- references/algorithms/eagle3.md 6 KB
- references/algorithms/lilicorr.md 13 KB
- references/stages/configure.md 2 KB
- references/stages/review-logs.md 3 KB
- references/stages/triage.md 6 KB
- references/stages/validate.md 5 KB
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · nvidia/model-optimizer | 100 | MIT |
| synthetic-sciences/openscience | 100 | Apache-2.0 |
| davila7/claude-code-templates | 95 | MIT |
Migawka bazy z 13.11.2026 · wersja paczki 1e1e51aae395 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100modelopt-speculative-decoding
Trenuje, debuguje i waliduje model szkicowy do spekulatywnego dekodowania (EAGLE3, DFlash, DSpark, Domino) w pipeline ModelOpt. - 100train-dflash
Trenuje lub adaptuje roboczy model spekulatywny DFlash albo DFlash2 względem zamrożonego modelu SALM speech-LLM w NeMo speechlm2, z ewaluacją akceptacji, wznawianiem i eksportem checkpointów. - 100modelopt-day0-release
Prowadzi kwantyzację, ewaluację i porównanie modelu w dniu zero, z bramkami między etapami i decyzją o publikacji. - 100ptq
Kwantyzuje model po treningu (NVFP4, FP8, INT8, INT4 AWQ) w ModelOpt i tworzy checkpoint dla Hugging Face lub TensorRT-LLM. - 85domino-modeling-assistant
Pomaga pisać kod modeli i uruchamiać zadania w Domino z asystentami AI, łącząc serwery MCP, zachowując bezpieczeństwo i powtarzalność. - 100domino-model-serving
Wdraża, wywołuje i wycofuje API modeli oraz modele zarejestrowane w Domino przez REST: cykl modelServing, ścieżki registered-models v1 i v2, MLflow, adresy inferencji i endpointy GenAI.