Skip to content
Skills Library
AI & agents · Automation

quantization-deployment

/quantization-deployment

Quantize a trained model to INT8 or INT4 for inference, calibrate the ranges, and gate the release on a measured quality regression. Use when serving needs lower latency and memory and you will spend effort keeping accuracy inside a defined budget.

Repository
AI-SKILLS ↗
Licence
MIT
Last repository change
05.09.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 05.09.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Quantize a trained model to INT8 or INT4 for inference, calibrate the ranges, and gate the release on a measured quality regression. Use when serving needs lower latency and memory and you will spend effort keeping accuracy inside a defined budget.

Original author description

Quantize a trained model to INT8 or INT4 for inference, calibrate the ranges, and gate the release on a measured quality regression. Use when serving needs lower latency and memory and you will spend effort keeping accuracy inside a defined budget.

How it works

  1. Method
  2. Checks
  3. Boundaries
Original SKILL.md headings · 67 lines, 557 words
SKILL.md preview
---
name: quantization-deployment
description: Quantize a trained model to INT8 or INT4 for inference, calibrate the ranges, and gate the release on a measured quality regression. Use when serving needs lower latency and memory and you will spend effort keeping accuracy inside a defined budget.
---

# Quantization deployment

Quantization maps floating-point weights and activations to low-bit integers,
cutting memory 2x to 4x and letting integer tensor cores do the math faster. The
catch is that it is lossy: pick the wrong ranges or scheme and accuracy falls
off a cliff, often on exactly the inputs you never tested. The discipline is to
calibrate ranges on real data, choose the scheme per tensor, and refuse to ship
until a quality gate passes.

## Method

1. **Start with post-training quantization, escalate to QAT only on failure.**
   Post-training quantization (PTQ) needs no retraining: convert an existing FP
   model and calibrate. If PTQ misses the budget, quantization-aware training
   (QAT) simulates the rounding during fine-tuning and usually recovers most of
   the gap, at the cost of a training run. Do the cheap thing first.
2. **Calibrate activation ranges on representative data.** Weights have fixed
   values, but activation ranges depend on inputs. Push a few hundred
   representative samples through the model to record per-tensor histograms,
   then set clipping thresholds. Entropy or percentile calibration (clipping
   the top 0.01 percent of outliers) beats plain min/max, which a single
   outlier ruins.
3. **Tame activation outliers before they wreck the scale.** In transformers a
   few channels carry huge activation magnitudes that stretch the range and
   crush everything else. SmoothQuant shifts that difficulty from activations
   into weights so a per-tensor activation scale still fits. Apply it before
   choosing thresholds, not after.
4. **Match the scheme to the tensor.** Use per-channel (per-output-channel)
   scales for weights and per-tensor scales for activations; per-channel weight
   quantization alone recovers much of the loss. Prefer symmetric quantization
   for weights and asymmetric with a nonzero zero-point for one-sided
   activations like post-ReLU outputs.
5. **Choose bit width against the model's tolerance.** INT8 is the safe default
   and often loses under 1 percent accuracy. INT4 doubles the saving again but
   needs group-wise scales (for example groups of 128 weights) and a
   salient-weight scheme like GPTQ or AWQ to stay usable. Do not reach for INT4
   before INT8 is validated.
6. **Keep sensitive layers in higher precision.** The first and last layers, and
   any layer with wide activation ranges, often dominate the error. Run a
   sensitivity sweep that quantizes one layer at a time and measures the drop,
   then leave the worst offenders in INT8 or FP16. Mixed precision trades a
   little speed for a lot of accuracy.
7. **Gate the release on a measured regression.** Fix the budget before you
   quantize (top-1 within 1 percent, or perplexity increase under 2 percent) and
   evaluate on a held-out set separate from calibration data. Block the deploy
   if the gate fails, and record before/after numbers and the scheme in the
   release notes.

## Checks

- Does the quantized model meet the pre-stated budget on a held-out set separate
  from calibration data?
- Were activation ranges calibrated on representative inputs with outlier
  clipping, not raw min/max?
- Are weights per-channel and activations per-tensor, with symmetric or
  asymmetric chosen to fit each?
- Did a per-layer sensitivity sweep justify which layers stay higher precision?

## Boundaries

This targets integer quantization for inference serving. Floating-point half
formats (FP16, BF16, FP8) and their loss scaling are mixed-precision-deployment.
Making the resulting INT8 matmuls actually hit the integer tensor cores is
tensor-core-utilization. The accuracy budget itself is a product decision:
quantization measures the cost, the owner decides if it is acceptable.

Versions

The database has no other skills named quantization-deployment.

Database snapshot: 13.11.2026 · package version d36fadf82563 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100model-quantization-tool
    Build model quantization tool operations. Auto-activating skill for ML Deployment. Triggers on: model quantization tool, model quantization tool Part of the ML Deployment skill category. Use when working with model quantization tool functionality. Trigger with phrases like…
  • 100ptq
    This skill should be used when the user asks to "quantize a model", "run PTQ", "post-training quantization", "NVFP4 quantization", "FP8 quantization", "INT8 quantization", "INT4 AWQ", "quantize LLM", "quantize MoE", "quantize VLM", or needs to produce a quantized HuggingFace or…
  • 100modelopt-ptq
    Use when the user asks to "quantize a model", "run PTQ", "post-training quantization", "NVFP4 quantization", "FP8 quantization", "INT8 quantization", "INT4 AWQ", "quantize LLM", "quantize MoE", "quantize VLM", or needs to produce a quantized HuggingFace checkpoint from a…
  • 100ptq
    Use when the user asks to "quantize a model", "run PTQ", "post-training quantization", "NVFP4 quantization", "FP8 quantization", "INT8 quantization", "INT4 AWQ", "quantize LLM", "quantize MoE", "quantize VLM", or needs to produce a quantized HuggingFace checkpoint from a…
  • 95awq-quantization
    Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and…
  • 100cortex-model
    Build an ML pipeline — from data to trained model to serving endpoint. Use when asked to "build ML model", "train a model", "prediction pipeline", "classification", or "regression".

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.