Przejdź do treści
Biblioteka skilli
AI i agenci · Automatyzacja

quantization-deployment

/quantization-deployment

Kwantyzuje wytrenowany model do INT8 lub INT4 dla wnioskowania, kalibruje zakresy i warunkuje wydanie zmierzonym spadkiem jakości.

Repozytorium
AI-SKILLS ↗
Licencja
MIT
Ostatnia zmiana repozytorium
05.09.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
1
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 05.09.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Kwantyzuje wytrenowany model do INT8 lub INT4 dla wnioskowania, kalibruje zakresy i warunkuje wydanie zmierzonym spadkiem jakości.

Oryginalny opis (EN)

Quantize a trained model to INT8 or INT4 for inference, calibrate the ranges, and gate the release on a measured quality regression. Use when serving needs lower latency and memory and you will spend effort keeping accuracy inside a defined budget.

Jak działa

  1. Method
  2. Checks
  3. Boundaries
Sekcje z nagłówków SKILL.md, w oryginale · 67 linii, 557 słów
Podgląd SKILL.md
---
name: quantization-deployment
description: Quantize a trained model to INT8 or INT4 for inference, calibrate the ranges, and gate the release on a measured quality regression. Use when serving needs lower latency and memory and you will spend effort keeping accuracy inside a defined budget.
---

# Quantization deployment

Quantization maps floating-point weights and activations to low-bit integers,
cutting memory 2x to 4x and letting integer tensor cores do the math faster. The
catch is that it is lossy: pick the wrong ranges or scheme and accuracy falls
off a cliff, often on exactly the inputs you never tested. The discipline is to
calibrate ranges on real data, choose the scheme per tensor, and refuse to ship
until a quality gate passes.

## Method

1. **Start with post-training quantization, escalate to QAT only on failure.**
   Post-training quantization (PTQ) needs no retraining: convert an existing FP
   model and calibrate. If PTQ misses the budget, quantization-aware training
   (QAT) simulates the rounding during fine-tuning and usually recovers most of
   the gap, at the cost of a training run. Do the cheap thing first.
2. **Calibrate activation ranges on representative data.** Weights have fixed
   values, but activation ranges depend on inputs. Push a few hundred
   representative samples through the model to record per-tensor histograms,
   then set clipping thresholds. Entropy or percentile calibration (clipping
   the top 0.01 percent of outliers) beats plain min/max, which a single
   outlier ruins.
3. **Tame activation outliers before they wreck the scale.** In transformers a
   few channels carry huge activation magnitudes that stretch the range and
   crush everything else. SmoothQuant shifts that difficulty from activations
   into weights so a per-tensor activation scale still fits. Apply it before
   choosing thresholds, not after.
4. **Match the scheme to the tensor.** Use per-channel (per-output-channel)
   scales for weights and per-tensor scales for activations; per-channel weight
   quantization alone recovers much of the loss. Prefer symmetric quantization
   for weights and asymmetric with a nonzero zero-point for one-sided
   activations like post-ReLU outputs.
5. **Choose bit width against the model's tolerance.** INT8 is the safe default
   and often loses under 1 percent accuracy. INT4 doubles the saving again but
   needs group-wise scales (for example groups of 128 weights) and a
   salient-weight scheme like GPTQ or AWQ to stay usable. Do not reach for INT4
   before INT8 is validated.
6. **Keep sensitive layers in higher precision.** The first and last layers, and
   any layer with wide activation ranges, often dominate the error. Run a
   sensitivity sweep that quantizes one layer at a time and measures the drop,
   then leave the worst offenders in INT8 or FP16. Mixed precision trades a
   little speed for a lot of accuracy.
7. **Gate the release on a measured regression.** Fix the budget before you
   quantize (top-1 within 1 percent, or perplexity increase under 2 percent) and
   evaluate on a held-out set separate from calibration data. Block the deploy
   if the gate fails, and record before/after numbers and the scheme in the
   release notes.

## Checks

- Does the quantized model meet the pre-stated budget on a held-out set separate
  from calibration data?
- Were activation ranges calibrated on representative inputs with outlier
  clipping, not raw min/max?
- Are weights per-channel and activations per-tensor, with symmetric or
  asymmetric chosen to fit each?
- Did a per-layer sensitivity sweep justify which layers stay higher precision?

## Boundaries

This targets integer quantization for inference serving. Floating-point half
formats (FP16, BF16, FP8) and their loss scaling are mixed-precision-deployment.
Making the resulting INT8 matmuls actually hit the integer tensor cores is
tensor-core-utilization. The accuracy budget itself is a product decision:
quantization measures the cost, the owner decides if it is acceptable.

Wersje

W bazie nie ma innych skilli o nazwie quantization-deployment.

Migawka bazy z 13.11.2026 · wersja paczki d36fadf82563 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100model-quantization-tool
    Prawdopodobnie pomaga w kwantyzacji modeli uczenia maszynowego przed wdrożeniem, w ramach kategorii ML Deployment.
  • 100ptq
    Kwantyzuje model po treningu (NVFP4, FP8, INT8, INT4 AWQ) w ModelOpt i tworzy checkpoint dla Hugging Face lub TensorRT-LLM.
  • 100modelopt-ptq
    Kwantyzuje model po treningu w ModelOpt do checkpointu HuggingFace: NVFP4, FP8, INT8 lub INT4 AWQ, także MoE i VLM.
  • 100ptq
    Kwantyzuje model po treningu (PTQ) przez ModelOpt do formatów NVFP4, FP8, INT8 i INT4 AWQ, tworząc skwantyzowany checkpoint HuggingFace z wytrenowanego modelu, także MoE i VLM.
  • 95awq-quantization
    Kwantyzuje duże modele językowe do 4 bitów metodą AWQ, z mniejszym spadkiem dokładności i szybszym wnioskowaniem niż GPTQ.
  • 100cortex-model
    Buduje potok ML od danych przez wytrenowany model po endpoint serwujący, na przykład dla klasyfikacji, regresji i potoków predykcyjnych.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.