local-models
/local-models
Run quick, offline, private LLM tasks on local models via llama.cpp, reusing models already downloaded by Ollama. Use for cheap/bulk text work (summarize, classify, extract JSON, anonymize PII, translate, proofread, keywords), local embeddings, and offline image description — and prefer it over a cloud API whenever a task is privacy-sensitive, must run offline, is high-volume/low-stakes, or just needs a fast throwaway answer. Provides an `lm` CLI wrapper plus an OpenAI-compatible local server.
- Author
- glebis ↗
- Repository
- claude-skills ↗
- Licence
- MIT
- Last repository change
- 25.10.2026
Assessment and security
- Files in this package
- 4
- Scripts
- 1
- HTML comments
- 0
- URLs in SKILL.md
- 1
No signals detected. Repository changed on 25.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
- 1 scripts in this package. Review them before installing, as Claude may execute them.
Scripts: scripts/ollama_blob.py
What it does and when to use it
Run quick, offline, private LLM tasks on local models via llama.cpp, reusing models already downloaded by Ollama. Use for cheap/bulk text work (summarize, classify, extract JSON, anonymize PII, translate, proofread, keywords), local embeddings, and offline image description — and prefer it over a cloud API whenever a task is privacy-sensitive, must run offline, is high-volume/low-stakes, or just needs a fast throwaway answer. Provides an `lm` CLI wrapper plus an OpenAI-compatible local server.
Original author description
Run quick, offline, private LLM tasks on local models via llama.cpp, reusing models already downloaded by Ollama. Use for cheap/bulk text work (summarize, classify, extract JSON, anonymize PII, translate, proofread, keywords), local embeddings, and offline image description — and prefer it over a cloud API whenever a task is privacy-sensitive, must run offline, is high-volume/low-stakes, or just needs a fast throwaway answer. Provides an `lm` CLI wrapper plus an OpenAI-compatible local server.
How it works
- When to use this skill
- The core trick: reuse Ollama's models
- Usage
- Critical gotchas
- Reference material
- Requirements
SKILL.md preview
--- name: local-models description: Run quick, offline, private LLM tasks on local models via llama.cpp, reusing models already downloaded by Ollama. Use for cheap/bulk text work (summarize, classify, extract JSON, anonymize PII, translate, proofread, keywords), local embeddings, and offline image description — and prefer it over a cloud API whenever a task is privacy-sensitive, must run offline, is high-volume/low-stakes, or just needs a fast throwaway answer. Provides an `lm` CLI wrapper plus an OpenAI-compatible local server. --- # local-models Quick access to local LLMs through **llama.cpp**, reusing the GGUF models already pulled by Ollama (no re-download for text and embeddings). Everything runs on the machine — no API key, no network, no per-token cost. ## When to use this skill Reach for local models instead of a cloud API when the task is: - **Privacy-sensitive** — redacting PII, processing personal notes, health data, secrets-adjacent text. The data never leaves the machine. - **Offline** — no network available, or the user explicitly wants local-only. - **High-volume / low-stakes** — classifying or tagging hundreds of items, where a small model is good enough and cloud cost/latency would add up. - **A fast throwaway** — a quick summary, translation, or "what is this" where round-tripping to a frontier model is overkill. Prefer a frontier (Claude) model when the task needs strong reasoning, long context, careful code, or high accuracy — these local models are small (0.6–4B). ## The core trick: reuse Ollama's models Ollama stores model weights as extension-less GGUF blobs under `~/.ollama/models/blobs/`. **These are ordinary GGUF files** — llama.cpp loads them directly. `scripts/ollama_blob.py` reads Ollama's manifests and resolves a friendly name (e.g. `qwen2.5:3b`) to its weights blob path. No conversion, no duplicate downloads. ## Usage The entry point is `scripts/lm`. Run `scripts/lm help` for the full list. Invoke it with an absolute path, e.g. `~/ai_projects/claude-skills/local-models/scripts/lm`. ```bash lm models # list local models (text / vision / embed) lm ask [MODEL] "PROMPT" # one-shot prompt (default qwen2.5:3b) lm chat [MODEL] # interactive REPL # Text presets — accept a file path, inline text, OR stdin: lm summarize report.md cat notes.txt | lm tldr lm keywords article.txt lm anonymize transcript.txt # → [NAME] [EMAIL] [PHONE] [ADDRESS] ... lm proofread draft.md lm translate German "Good morning" lm classify "praise,complaint,question" feedback.txt # → one label lm extract "invoice_number, total, due_date" invoice.txt # → JSON # Vision (downloads model+projector once via HuggingFace — see note below): lm describe-image photo.jpg lm tag-image screenshot.png lm vision photo.jpg "What brand is the shoe?" # Embeddings & serving: lm embed "text to embed" # → OpenAI-style JSON vector lm serve qwen2.5:3b 8080 # OpenAI-compatible server on :8080 ``` Output is clean (just the answer) — the wrapper drives `llama-completion` in single-turn mode and strips the chat-template scaffolding and llama.cpp logs. ### Choosing a model Defaults are tuned for clean, fast output and can be overridden per call: - General text presets → `qwen2.5:3b` (`LM_TEXT_MODEL`) - Classify / extract → `qwen2.5:3b` (`LM_REASON_MODEL`), run at temperature 0 - Embeddings → `jeffh/intfloat-multilingual-e5-large:f16` (`LM_EMBED_MODEL`) - Other envs: `LM_NTOK` (max tokens), `LM_VISION_HF` (vision repo), `LM_DEBUG=1` (show llama.cpp logs) Pass an explicit model as the first argument to `ask`/`chat`/`embed`/`serve` (e.g. `lm ask qwen3:4b "..."`). ## Critical gotchas - **Ollama's `gemma3` GGUF does NOT load in stock llama.cpp.** It fails with `key not found in model: gemma3.attention.layer_norm_rms_epsilon` because Ollama writes custom metadata keys mainline llama.cpp doesn't read. Use a `qwen*` model instead, or pull a community gemma3 GGUF via `-hf`. This is why the defaults are qwen, not gemma3. - **`qwen3:4b` emits `<think>…</think>` reasoning blocks** before its answer. Fine for `ask`/`chat`, but it pollutes preset output (JSON, labels) — the presets default to `qwen2.5:3b` to avoid this. - **Vision has no Ollama blob to reuse.** Ollama did not store an `mmproj` (vision projector) for `qwen2.5vl`, and llama.cpp needs one. So the vision commands use `llama-mtmd-cli -hf ggml-org/Qwen2.5-VL-3B-Instruct-GGUF`, which downloads model+projector (~2–3 GB) into `~/.cache/llama.cpp` on first use, then runs offline. Warn the user before the first vision call. - **Each one-shot call reloads the model** (a few seconds for these small models). For many sequential calls, start a server once with `lm serve` and hit `http://localhost:8080/v1/chat/completions` — see [references/serving-and-embeddings.md](references/serving-and-embeddings.md). ## Reference material - [references/serving-and-embeddings.md](references/serving-and-embeddings.md) — running `llama-server` as an OpenAI-compatible endpoint (and pointing the `llm` CLI or any OpenAI client at it), plus local embeddings / RAG patterns with `llama-embedding`. ## Requirements - `llama.cpp` installed (`brew install llama.cpp`) — provides `llama-completion`, `llama-mtmd-cli`, `llama-embedding`, `llama-server`. - Ollama with at least one pulled model (for the blob-reuse path). `python3` for the resolver. No API keys.
Files in this package (4)
- SKILL.md 5 KB
- references/serving-and-embeddings.md 3 KB
- scripts/lm 7 KB
- scripts/ollama_blob.py 4 KB
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · glebis/claude-skills | 100 | MIT |
| haedolabs/local-models-hub | 96 | MIT |
Database snapshot: 13.11.2026 · package version ce639cd627e4 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100ollama
Run and manage local LLMs via Ollama REST API — text generation, chat completions, embeddings, tool calling, structured output, and model management. Use when code imports ollama, references localhost:11434, or user asks about local LLM inference. - 100gemma4-local-deploy
在本机 Mac 或 Apple Silicon 上部署 Gemma 4 12B。本地安装/升级 llama.cpp,下载 GGUF 量化模型,用 llama-server 暴露 OpenAI-compatible API,或用 Ollama 暴露本地模型服务;按用户需求在默认 Q4_K_M、64K/128K 长上下文、QAT Q4_0 @ 256K、左右对比演示之间选择,配置 tmux 后台运行,验证健康检查、问答接口、资源占用和常见故障。当用户说部署 Gemma 4、Gemma 4… - 100open-webui
Open WebUI is a self-hosted web chat interface for local and cloud language models that connects to Ollama and any OpenAI-compatible API. Use when a user asks to install or deploy Open WebUI with Docker or pip, connect Open WebUI to Ollama, llama.cpp, vLLM or OpenAI, configure… - 100llama-cpp
Guide for llama.cpp, the C/C++ LLM inference framework by ggml-org. Covers the C API (llama.h), GGUF format, quantization (Q4_K_M, Q8_0, IQ4_XS), CMake builds, GPU backends (CUDA, Vulkan, Metal, ROCm), HTTP server with OpenAI-compatible API, embeddings, grammar constraints,… - 100minicpm5-deploy-llama-cpp
Run MiniCPM5-1B or MiniCPM5-2B with llama.cpp using the released GGUF artifacts (F16 / Q8_0 / Q4_K_M). Use when the user wants CPU-only / consumer-GPU / cross-platform native deployment, asks for "llama.cpp", "llama-cli", "llama-server", "GGUF", or has no Python available. - 100llamacpp
Complete llama.cpp C/C++ API reference covering model loading, inference, text generation, embeddings, chat, tokenization, sampling, batching, KV cache, LoRA adapters, and state management. Triggers on: llama.cpp questions, LLM inference code, GGUF models, local AI/ML inference,…