Skip to content
Skills Library
AI & agents · Building & coding

speculative-decoding

/speculative-decoding

Train, debug, and validate a speculative decoding draft model (EAGLE3, DFlash, DSpark, Domino) through the ModelOpt launcher pipeline. Use when the user wants to add a new model to a draft-training pipeline, asks why a pipeline run failed, wants experiment logs reviewed, or wants to check whether a run's acceptance rate meets threshold. Triggers on "EAGLE3", "DFlash", "DSpark", "Domino", "draft model", "acceptance rate", "speculative decoding pipeline". Do NOT use for quantizing a model (use ptq) or serving a checkpoint (use deployment).

Licence
MIT
Last repository change
27.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
12
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Train, debug, and validate a speculative decoding draft model (EAGLE3, DFlash, DSpark, Domino) through the ModelOpt launcher pipeline. Use when the user wants to add a new model to a draft-training pipeline, asks why a pipeline run failed, wants experiment logs reviewed, or wants to check whether a run's acceptance rate meets threshold. Triggers on "EAGLE3", "DFlash", "DSpark", "Domino", "draft model", "acceptance rate", "speculative decoding pipeline". Do NOT use for quantizing a model (use ptq) or serving a checkpoint (use deployment).

Triggers from the description

s acceptance rate meets threshold. Triggers onDFlashDSparkDominodraft modelacceptance ratespeculative decoding pipeline
Original author description

Train, debug, and validate a speculative decoding draft model (EAGLE3, DFlash, DSpark, Domino) through the ModelOpt launcher pipeline. Use when the user wants to add a new model to a draft-training pipeline, asks why a pipeline run failed, wants experiment logs reviewed, or wants to check whether a run's acceptance rate meets threshold. Triggers on "EAGLE3", "DFlash", "DSpark", "Domino", "draft model", "acceptance rate", "speculative decoding pipeline". Do NOT use for quantizing a model (use ptq) or serving a checkpoint (use deployment).

How it works

  1. Two axes: stage and algorithm
  2. Where the code lives
  3. Stages
  4. Algorithms
  5. End-to-end: a new model
Original SKILL.md headings · 94 lines, 660 words
SKILL.md preview
---
name: speculative-decoding
description: >-
  Train, debug, and validate a speculative decoding draft model (EAGLE3, DFlash,
  DSpark, Domino) through the ModelOpt launcher pipeline. Use when the user wants
  to add a new model to a draft-training pipeline, asks why a pipeline run failed,
  wants experiment logs reviewed, or wants to check whether a run's acceptance rate
  meets threshold. Triggers on "EAGLE3", "DFlash", "DSpark", "Domino", "draft model",
  "acceptance rate", "speculative decoding pipeline". Do NOT use for quantizing a
  model (use ptq) or serving a checkpoint (use deployment).
user_invocable: true
---

# Speculative Decoding Draft-Model Training

Everything needed to take a target model from "no draft head" to "validated
acceptance rate" lives in this directory. Work through the stages below in order
for a new model; jump straight to a stage when you already know which one you need.

## Two axes: stage and algorithm

The pipeline is the same shape for every draft-model algorithm — synthesize data,
dump base-model hidden states, train the draft, benchmark acceptance rate. What
changes between algorithms is which training script and recipe run, which knobs
matter, and which failures are typical.

So this skill is split along those two axes, and **you almost always read one file
from each**:

| Axis | Directory | What it holds |
| --- | --- | --- |
| Stage | `references/stages/` | The procedure — algorithm-independent |
| Algorithm | `references/algorithms/` | The data sheet — scripts, recipe, knobs, thresholds, known failures |

Read the stage file for *what to do*, and the algorithm sheet for *the values to
plug in*. When a stage file says "see the algorithm sheet", it means the section of
`references/algorithms/<algorithm>.md` with the matching heading.

## Where the code lives

The launcher scripts in `tools/launcher/common/{eagle3,specdec}/` are thin wrappers around
`examples/speculative_decoding/` — `main.py` (training), `collect_hidden_states/` (dumps),
`scripts/{export_hf_checkpoint,ar_validate,quantize_drafter}.py`. Read a wrapper for the
flags it passes, that directory for what they do. Its `README.md` is also the non-launcher
path: training invoked directly, without Slurm.

## Stages

| Stage | Reference | Use when |
| --- | --- | --- |
| 1. Configure | `references/stages/configure.md` | Adding a model that has no pipeline YAML yet |
| 2. Review logs | `references/stages/review-logs.md` | A run finished (or died) and you want a pass/fail summary |
| 3. Triage | `references/stages/triage.md` | A task failed and you need root cause plus a fix |
| 4. Validate | `references/stages/validate.md` | All tasks passed and you need to confirm the acceptance rate gate |

Review-logs and triage overlap by design: review-logs is the fast sweep across all
tasks, triage is the deep dive into one failing task. Start with review-logs unless
the user already knows which task broke.

## Algorithms

All recipes live in `modelopt_recipes/general/speculative_decoding/<algorithm>.yaml`.

| Algorithm | Sheet | Family |
| --- | --- | --- |
| EAGLE3 | `references/algorithms/eagle3.md` | Autoregressive draft head |
| DFlash | `references/algorithms/dflash.md` | Block diffusion |
| DSpark | `references/algorithms/dspark.md` | DFlash backbone + Markov head + optional confidence head |
| Domino | `references/algorithms/domino.md` | DFlash backbone + GRU causal correction head |
| DFlash2 | `references/algorithms/dflash2.md` | DFlash backbone + sublayer convolutions + candidate selector |
| LiLiCorr | `references/algorithms/lilicorr.md` | DFlash backbone + candidate-lattice reranker (covers `lilicorr_conv.yaml`) |

Everything below EAGLE3 is a **DFlash variant**, not a separate pipeline: same
`recipe_type: speculative_dflash`, same training script, same `dflash.*` config
namespace, selected by `dflash_architecture_config.projector_type`. Read
`references/algorithms/dflash.md` first, then the variant's sheet for the delta.

LiLiCorr ships two recipes — `lilicorr.yaml` and `lilicorr_conv.yaml`, the latter
adding DFlash2's sublayer convolutions — but one sheet covers both.

If the user's algorithm has no sheet yet, the stage procedures still apply — derive
the missing values from an existing launcher example for that algorithm
(`tools/launcher/examples/*/*/hf_*_<algorithm>.yaml`) and its recipe, then write the
sheet as you go. `references/algorithms/README.md` defines what a sheet must contain.

## End-to-end: a new model

1. Confirm the algorithm and find the closest existing launcher example.
2. **Configure** — write the pipeline YAML (`references/stages/configure.md`).
3. Preview with `--dryrun`, then submit:

   ```bash
   cd tools/launcher
   uv run launch.py --yaml examples/<Org>/<Model>/<config>.yaml --yes
   ```

4. Register the job and set up monitoring per the **monitor skill**.
5. **Review logs** when it finishes (`references/stages/review-logs.md`).
6. **Triage** anything that failed (`references/stages/triage.md`), fix, re-run only
   the failed tasks onward via `pipeline.task_N.skip=true`.
7. **Validate** once all tasks pass (`references/stages/validate.md`).

Model-support gaps that need code changes land in `modelopt/torch/speculative/` and
require a separate ModelOpt PR — the pipeline YAML alone cannot fix an unrecognized
architecture.
Files in this package (12)
  • SKILL.md 5 KB
  • references/algorithms/README.md 3 KB
  • references/algorithms/dflash.md 11 KB
  • references/algorithms/dflash2.md 10 KB
  • references/algorithms/domino.md 7 KB
  • references/algorithms/dspark.md 11 KB
  • references/algorithms/eagle3.md 6 KB
  • references/algorithms/lilicorr.md 13 KB
  • references/stages/configure.md 2 KB
  • references/stages/review-logs.md 3 KB
  • references/stages/triage.md 6 KB
  • references/stages/validate.md 5 KB

Versions

RepositoryScoreLicence
This page · nvidia/model-optimizer100MIT
synthetic-sciences/openscience100Apache-2.0
davila7/claude-code-templates95MIT

Database snapshot: 13.11.2026 · package version 1e1e51aae395 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100modelopt-speculative-decoding
    Train, debug, and validate a speculative decoding draft model (EAGLE3, DFlash, DSpark, Domino) through the ModelOpt launcher pipeline. Use when the user wants to add a new model to a draft-training pipeline, asks why a pipeline run failed, wants experiment logs reviewed, or…
  • 100train-dflash
    Train or adapt a DFlash or DFlash2 speculative draft against a frozen SALM speech-LLM target with NeMo speechlm2. Use for online argmax or offline model-generated labels, pretrained draft adaptation, projection-only ablations, checkpoint resume/export, and acceptance evaluation.…
  • 100modelopt-day0-release
    Deterministic end-to-end driver for day-0 quantized-checkpoint releases — chains PTQ → evaluation → comparison with enforced gates between stages (the evaluation stage deploys the checkpoint itself), and returns a publish decision (ACCEPT / REGRESSION / INDETERMINATE / ANOMALOUS…
  • 100ptq
    This skill should be used when the user asks to "quantize a model", "run PTQ", "post-training quantization", "NVFP4 quantization", "FP8 quantization", "INT8 quantization", "INT4 AWQ", "quantize LLM", "quantize MoE", "quantize VLM", or needs to produce a quantized HuggingFace or…
  • 85domino-modeling-assistant
    Enable AI-assisted model development within Domino by writing needed model and training code and using MCP (Model Context Protocol) servers to execute domino jobs. AI coding assistants like Cursor and GitHub Copilot can execute commands as Domino jobs, maintaining security,…
  • 100domino-model-serving
    Deploy, invoke, and retire Domino model APIs and registered models via REST. Covers modelServing lifecycle, registered-models v1 vs v2 paths, MLflow tracking vs registry API, inference URLs vs management API, and GenAI endpoint vanity URLs. Use when automating model deployment,…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.