Skip to content
Skills Library
AI & agents · Conversion & formats

convert-quantize

/convert-quantize

Use this skill when the user wants to convert a Hugging Face model to MLX or quantize/dequantize one with mlx_vlm.convert, including bits and group size, quant modes (affine, mxfp4, nvfp4, mxfp8), RTN vs AWQ, mixed-bit recipes, dtype casts, calibration (text or multimodal), local vs Hub paths, revisions, uploading to the Hub, and convert/quant errors.

Repository
mlx-vlm ↗
Licence
MIT
Last repository change
27.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Use this skill when the user wants to convert a Hugging Face model to MLX or quantize/dequantize one with mlx_vlm.convert, including bits and group size, quant modes (affine, mxfp4, nvfp4, mxfp8), RTN vs AWQ, mixed-bit recipes, dtype casts, calibration (text or multimodal), local vs Hub paths, revisions, uploading to the Hub, and convert/quant errors.

Original author description

Use this skill when the user wants to convert a Hugging Face model to MLX or quantize/dequantize one with mlx_vlm.convert, including bits and group size, quant modes (affine, mxfp4, nvfp4, mxfp8), RTN vs AWQ, mixed-bit recipes, dtype casts, calibration (text or multimodal), local vs Hub paths, revisions, uploading to the Hub, and convert/quant errors.

How it works

  1. First Checks
  2. Command Patterns
  3. Key Facts
  4. Validation
Original SKILL.md headings · 76 lines, 492 words
SKILL.md preview
---
name: convert-quantize
description: Use this skill when the user wants to convert a Hugging Face model to MLX or quantize/dequantize one with mlx_vlm.convert, including bits and group size, quant modes (affine, mxfp4, nvfp4, mxfp8), RTN vs AWQ, mixed-bit recipes, dtype casts, calibration (text or multimodal), local vs Hub paths, revisions, uploading to the Hub, and convert/quant errors.
---

# Convert & Quantize

Use this workflow for `mlx_vlm.convert` — turning a Hugging Face checkpoint into MLX format, optionally quantizing it.

## First Checks

1. Confirm the source: a Hugging Face repo id or a local path (`--hf-path`, alias `--model`).
2. Confirm the model family is supported in `mlx_vlm/models/` (a folder named after the `config.json` `model_type`). If not, this is a porting task — switch to `Skill("mlx-vlm-skills:add-new-model")`.
3. Verify current flags before finalizing: `uv run mlx_vlm.convert --help`.
4. Entry point is `mlx_vlm.convert`. `python -m mlx_vlm.convert` is deprecated; use `uv run mlx_vlm.convert ...` or `python -m mlx_vlm convert ...`.

## Command Patterns

Plain convert (no quantization), saves to `./mlx_model` by default:

```bash
uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-mlx
```

4-bit affine quantization (RTN, the default method):

```bash
uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-4bit -q --q-bits 4 --q-group-size 64
```

Other quant modes (`--q-mode` sets its own bit/group defaults):

```bash
# mxfp4 (group 32, 4 bit), nvfp4 (group 16, 4 bit), mxfp8 (group 32, 8 bit)
uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-mxfp4 -q --q-mode mxfp4
```

Mixed-bit recipe (per-layer bit allocation, llama.cpp-style):

```bash
# recipes: mixed_2_6 mixed_3_4 mixed_3_5 mixed_3_6 mixed_3_8 mixed_4_6 mixed_4_8
uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-mixed -q --quant-predicate mixed_3_6
```

AWQ (activation-aware, needs a calibration pass):

```bash
uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-awq -q --quant-method awq \
  --calibration multimodal --calibration-data /path/to/media   # or --calibration text (default)
```

dtype cast only / dequantize:

```bash
uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-bf16 --dtype bfloat16
uv run mlx_vlm.convert --hf-path <quantized-repo> --mlx-path ./out-fp -d   # dequantize
```

Upload the result to the Hub:

```bash
uv run mlx_vlm.convert --hf-path <repo> --mlx-path ./out -q --upload-repo <user>/<name>-mlx
```

## Key Facts

- **Multimodal modules are skipped from quantization by default** (`skip_multimodal_module`) — vision/audio towers stay full precision; only the language model is quantized. This is expected, not a bug.
- `--q-mode` choices: `affine` (default, group 64 / 4 bit), `mxfp4` (32/4), `nvfp4` (16/4), `mxfp8` (32/8). `--q-bits`/`--q-group-size` override the mode defaults.
- `--quant-method`: `rtn` (default, round-to-nearest) or `awq` (needs calibration; `--calibration text|multimodal`, optional `--calibration-data`).
- `-q`/`--quantize` and `-d`/`--dequantize` are mutually exclusive.
- `--dtype` (from `config.json` `torch_dtype` if unset) casts float weights; useful for shrinking fp32 → bf16 without quantizing.
- The converted folder gets the weights, copied `*.py`/`*.json`, the processor, a regenerated `config.json`, and a model card — it is ready for `mlx_vlm.generate` and the server.

## Validation

- After converting, load and run a tiny generation to prove the checkpoint works — see `Skill("mlx-vlm-skills:cli-inference")`. Run on Apple Silicon with enough RAM for the model; do not attempt a large model on an 8 GB machine.
- For quality, compare a few greedy outputs between the source and the quantized model; large drops usually mean too-low bits or a wrong `--q-mode` for the model.
- For code changes touching conversion, run `uv run --with pytest python -m pytest mlx_vlm/tests/test_utils.py -q` and any model-specific test.
- If the outcome is a bug report, switch to `Skill("mlx-vlm-skills:reproducible-github-issues")`.

Versions

The database has no other skills named convert-quantize.

Database snapshot: 13.11.2026 · package version 91a021f4edac (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100ptq
    This skill should be used when the user asks to "quantize a model", "run PTQ", "post-training quantization", "NVFP4 quantization", "FP8 quantization", "INT8 quantization", "INT4 AWQ", "quantize LLM", "quantize MoE", "quantize VLM", or needs to produce a quantized HuggingFace or…
  • 100fskills-compression
    Model compression and quantization of a HF checkpoint (v1 method: RTN weight-only quantize-dequantize, int4 or int8, group-wise). Use it to preflight a host for compression (vendored preflight.py; classifies FAIL, UNKN and PROBE_ERROR results), to plan a probe-first run that…
  • 100modelopt-ptq
    Use when the user asks to "quantize a model", "run PTQ", "post-training quantization", "NVFP4 quantization", "FP8 quantization", "INT8 quantization", "INT4 AWQ", "quantize LLM", "quantize MoE", "quantize VLM", or needs to produce a quantized HuggingFace checkpoint from a…
  • 100ptq
    Use when the user asks to "quantize a model", "run PTQ", "post-training quantization", "NVFP4 quantization", "FP8 quantization", "INT8 quantization", "INT4 AWQ", "quantize LLM", "quantize MoE", "quantize VLM", or needs to produce a quantized HuggingFace checkpoint from a…
  • 100resolve
    Always resolve Hugging Face models via `model-shelf` before any download. Supports GGUF, MLX, and safetensors. Triggers whenever the user asks to load, run, or use a local LLM model (llama.cpp / Ollama / MLX / vLLM / transformers flows).
  • 100cli-inference
    Use this skill when the user wants to run or debug MLX-VLM inference from the command line, including uv run mlx_vlm.generate, image/audio/video inputs, local model paths, Hugging Face model IDs, deterministic repro commands, and CLI errors around processors, prompts, model…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.