server-inference
/server-inference
Use this skill when the user wants to run or debug MLX-VLM server inference, including uv run mlx_vlm.server, /v1/models, /v1/chat/completions, /v1/responses, streaming, OpenAI-compatible clients, health checks, metrics, model unload/reload, adapters, trust-remote-code, and server request/response failures.
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 5
No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Use this skill when the user wants to run or debug MLX-VLM server inference, including uv run mlx_vlm.server, /v1/models, /v1/chat/completions, /v1/responses, streaming, OpenAI-compatible clients, health checks, metrics, model unload/reload, adapters, trust-remote-code, and server request/response failures.
Original author description
Use this skill when the user wants to run or debug MLX-VLM server inference, including uv run mlx_vlm.server, /v1/models, /v1/chat/completions, /v1/responses, streaming, OpenAI-compatible clients, health checks, metrics, model unload/reload, adapters, trust-remote-code, and server request/response failures.
How it works
- First Checks
- Startup
- Minimal Checks
- Endpoint Surface (beyond chat/responses)
- Debugging Rules
- Validation
SKILL.md preview
---
name: server-inference
description: Use this skill when the user wants to run or debug MLX-VLM server inference, including uv run mlx_vlm.server, /v1/models, /v1/chat/completions, /v1/responses, streaming, OpenAI-compatible clients, health checks, metrics, model unload/reload, adapters, trust-remote-code, and server request/response failures.
---
# Server Inference
Use this workflow for the FastAPI server and API-compatible inference.
## First Checks
1. Identify the server command, model, port, request endpoint, request body, and expected response.
2. Start with health/model-list checks before debugging generation.
3. Separate server startup failures from request-handling failures.
4. Keep streaming and non-streaming repros separate.
## Startup
```bash
uv run mlx_vlm.server \
--model <model-or-path> \
--port 8080
```
Useful startup flags include `--adapter-path`, `--trust-remote-code`, `--log-level`, `--enable-thinking`, `--thinking-budget`, `--draft-model`, `--draft-kind`, `--kv-bits`, `--kv-quant-scheme`, `--max-kv-size`, and `--vision-cache-size`.
## Minimal Checks
```bash
curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/v1/models
curl http://127.0.0.1:8080/metrics
```
Minimal chat request:
```bash
curl -s http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "<model-or-path>",
"messages": [{"role": "user", "content": "Say hello."}],
"max_tokens": 32,
"stream": false
}'
```
Minimal Responses API request:
```bash
curl -s http://127.0.0.1:8080/v1/responses \
-H 'Content-Type: application/json' \
-d '{
"model": "<model-or-path>",
"input": "Say hello.",
"max_output_tokens": 32
}'
```
## Endpoint Surface (beyond chat/responses)
The server exposes more than chat — cover or route to these as needed:
- **Anthropic Messages API:** `/v1/messages`, plus `/v1/messages/count` for token counting.
- **Audio:** `/v1/audio/speech` (TTS), `/v1/audio/transcriptions` and `/v1/audio/translations` (STT).
- **Images:** `/v1/images/generations` and `/v1/images/edits` (diffusion image models).
- **Cache & metrics:** `/v1/cache/stats`, `/v1/cache/reset`, `/v1/metrics`.
- **Models:** `/v1/models` lists loaded models and model candidates in the shared Hugging Face cache, with a `loaded` boolean indicating which are in memory — see `Skill("mlx-vlm-skills:hf-cache-models")`.
Match the model kind to the endpoint (an image endpoint needs a diffusion/image model, audio endpoints need an audio model); a mismatch returns a clear 4xx, not a crash.
## Debugging Rules
- Capture the server command, server logs, request body, response status, and response body.
- Test `/v1/models` after preload/load changes.
- For OpenAI client issues, reproduce with `curl` before blaming the client SDK.
- For streaming bugs, save raw event chunks and compare with non-streaming.
- For structured outputs, isolate the JSON schema and confirm whether the same request works without schema constraints.
- For tool calls, inspect native support and the parser inferred from the loaded processor's actual chat template. Follow [Tool Calling](../add-new-model/SKILL.md#tool-calling) for parser registration and validation.
## Validation
- For route/schema changes, run `uv run pytest mlx_vlm/tests/test_server.py -q`.
- For structured output changes, include `uv run pytest mlx_vlm/tests/test_generate.py -q -k 'thinking_aware or json_schema'`.
- For tool parser changes, add compact regressions to `TestProcessToolCalls` in `mlx_vlm/tests/test_server.py` and run the shared tool streaming tests in that module, plus any existing tests for affected parsers. Reuse consolidated tests instead of creating a new test file.
- If the result is a user-facing bug report, switch to `Skill("mlx-vlm-skills:reproducible-github-issues")`.
Versions
The database has no other skills named server-inference.
Database snapshot: 13.11.2026 · package version ad6bafac0b78 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100cli-inference
Use this skill when the user wants to run or debug MLX-VLM inference from the command line, including uv run mlx_vlm.generate, image/audio/video inputs, local model paths, Hugging Face model IDs, deterministic repro commands, and CLI errors around processors, prompts, model… - 100mlx-vlm
Runs and fine-tunes vision language models (image, audio and video plus text) locally on Apple Silicon Macs with MLX, from a CLI, a Python API or an OpenAI-compatible server. Use when installing mlx-vlm, describing or extracting text from images with a local model, batch… - 100iblai-api-inference
Run inference against an ibl.ai deployment through its OpenAI-compatible API — same request/response shape as OpenAI's /v1/chat/completions, routed to any configured provider/model (e.g. openai/gpt-5). List models, then POST OpenAI-format messages for a single completion or a… - 100hf-cache-models
Use this skill when the user wants to list, inspect, or report cached, custom, or loaded MLX-VLM models through the server's model discovery endpoint. - 100minicpm5-deploy-lmstudio
Run MiniCPM5-1B or MiniCPM5-2B in LM Studio (desktop GUI) using either the GGUF runtime (cross-platform) or the MLX runtime (Apple Silicon, faster). Includes OpenAI-compatible local server. Use when the user mentions "LM Studio", desktop GUI inference, "lms" CLI, or wants a… - 100azure-ai-inference
Use when calling Azure-hosted AI models via the Azure.AI.Inference SDK. Covers chat completions, embeddings, streaming, model selection, and Azure AI model catalog integration. USE FOR: calling Azure AI model catalog models, Azure OpenAI chat completions, generating embeddings…