Skip to content
Skills Library
AI & agents · Tool integration

docker-model

/docker-model

Use this skill when running local AI models with Docker Model Runner — the `docker model` CLI — e.g. "run an LLM locally with Docker", "pull a model from the ai/ namespace", "connect my app to a local model", "use a local model as backend for the Drupal AI module", or when wiring the `models:` top-level element into a compose.yaml. Covers pulling/running models, OpenAI-compatible endpoints, and Compose integration.

Licence
MIT
Last repository change
26.09.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
8
Deprecation

No signals detected. Repository changed on 26.09.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Use this skill when running local AI models with Docker Model Runner — the `docker model` CLI — e.g. "run an LLM locally with Docker", "pull a model from the ai/ namespace", "connect my app to a local model", "use a local model as backend for the Drupal AI module", or when wiring the `models:` top-level element into a compose.yaml. Covers pulling/running models, OpenAI-compatible endpoints, and Compose integration.

Triggers from the description

run an LLM locally with Dockerpull a model from the ai/ namespaceconnect my app to a local modeluse a local model as backend for the Drupal AI module
Original author description

Use this skill when running local AI models with Docker Model Runner — the `docker model` CLI — e.g. "run an LLM locally with Docker", "pull a model from the ai/ namespace", "connect my app to a local model", "use a local model as backend for the Drupal AI module", or when wiring the `models:` top-level element into a compose.yaml. Covers pulling/running models, OpenAI-compatible endpoints, and Compose integration.

How it works

  1. Enabling
  2. Core CLI
  3. OpenAI-compatible API
  4. Compose integration — models: top-level element
  5. Using DMR as a Drupal AI backend
  6. Troubleshooting
Original SKILL.md headings · 128 lines, 567 words
SKILL.md preview
---
name: docker-model
description: >
  Use this skill when running local AI models with Docker Model Runner — the
  `docker model` CLI — e.g. "run an LLM locally with Docker", "pull a model
  from the ai/ namespace", "connect my app to a local model", "use a local
  model as backend for the Drupal AI module", or when wiring the `models:`
  top-level element into a compose.yaml. Covers pulling/running models,
  OpenAI-compatible endpoints, and Compose integration.
---

# Docker Model Runner Skill

Docker Model Runner (DMR) manages and serves AI models through Docker Desktop
or Docker Engine, exposing **OpenAI-compatible APIs**. Models are pulled as
OCI artifacts from Docker Hub (`ai/` namespace), any OCI registry, or
Hugging Face, and stored locally. For Drupal work it provides a free, local,
keyless backend for the AI module ecosystem during development.

---

## Enabling

- **Docker Desktop:** Settings → enable *Docker Model Runner* (Beta features).
- **Docker Engine (Linux):** supported without Desktop; models are served on
  the host. GPU support: NVIDIA (CUDA), AMD (ROCm), Vulkan; Apple Silicon on
  macOS; CPU everywhere.

---

## Core CLI

```bash
docker model status                      # is the runner active?
docker model pull ai/smollm2             # fetch a model (Docker Hub ai/ namespace)
docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF  # from Hugging Face
docker model list                        # local models
docker model run ai/smollm2 "Hello"      # one-shot prompt
docker model run ai/smollm2              # interactive chat (exit with /bye)
docker model configure --context-size 8192 ai/smollm2   # adjust context window
docker model inspect ai/smollm2          # model metadata
docker model logs                        # runner logs
docker model rm ai/smollm2               # delete local model
```

Run `docker model --help` for the full, current command list — the CLI is
still evolving.

---

## OpenAI-compatible API

| Endpoint | Method |
|---|---|
| `/engines/v1/models` | GET |
| `/engines/v1/chat/completions` | POST |
| `/engines/v1/completions` | POST |
| `/engines/v1/embeddings` | POST |

Base URLs:

- **From the host:** `http://localhost:12434` (default TCP port)
- **From containers (Docker Desktop):** `http://model-runner.docker.internal`
- **From containers (Docker Engine):** `http://172.17.0.1:12434`

```bash
curl http://localhost:12434/engines/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "ai/smollm2", "messages": [{"role": "user", "content": "Hi"}]}'
```

Any OpenAI SDK works by pointing `base_url` at
`http://localhost:12434/engines/v1` — no API key required.

---

## Compose integration — `models:` top-level element

Declare models next to services; Compose pulls and provisions them:

```yaml
services:
  app:
    image: my-app
    models:
      - llm                      # short syntax

models:
  llm:
    model: ai/smollm2
    context_size: 4096
    runtime_flags:
      - "--no-prefill-assistant"
```

Short syntax injects environment variables into the service container, named
after the model key: `LLM_URL` and `LLM_MODEL`. Long syntax picks your own
variable names:

```yaml
services:
  app:
    image: my-app
    models:
      llm:
        endpoint_var: AI_MODEL_URL
        model_var: AI_MODEL_NAME
```

---

## Using DMR as a Drupal AI backend

The Drupal **AI module** (`drupal/ai`) talks to providers over the OpenAI
API. Point an OpenAI-compatible provider (e.g. `drupal/ai_provider_openai`)
at the Model Runner endpoint to develop AI features without cloud keys:

- **Base URL** (Drupal in a container, Docker Desktop):
  `http://model-runner.docker.internal/engines/v1`
- **Base URL** (Drupal on the host): `http://localhost:12434/engines/v1`
- **API key:** any non-empty placeholder — DMR does not check it.
- **Model name:** exactly as listed by `docker model list` (e.g. `ai/smollm2`).

This gives local, reproducible AI development for content generation,
embeddings/search experiments, and automated tests without external costs.

---

## Troubleshooting

| Symptom | Fix |
|---|---|
| `docker model: command not found` | Enable Model Runner in Docker Desktop settings, or install the plugin on Docker Engine |
| Connection refused on 12434 | Enable *host-side TCP support* in the Model Runner settings; check `docker model status` |
| Container cannot reach `model-runner.docker.internal` | On Docker Engine use `http://172.17.0.1:12434` instead |
| Responses truncated | Raise the context window: `docker model configure --context-size <n> <model>` |
| Model too slow / out of memory | Pull a smaller quantized variant from the `ai/` namespace; check GPU is actually used (`docker model logs`) |

Versions

The database has no other skills named docker-model.

Database snapshot: 13.11.2026 · package version 358735095767 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100docker-compose-basics
    Use when defining and running multi-container Docker applications with Docker Compose YAML configuration.
  • 100docker
    Manage Docker containers, images, volumes, and Compose stacks via the `docker` CLI — list, inspect, logs, run, build, stop, remove, compose up/down. Use for local container ops.
  • 100generating-docker-compose-files
    Execute use when you need to work with Docker Compose. This skill provides Docker Compose file generation with comprehensive guidance and automation. Trigger with phrases like "generate docker-compose", "create compose file", or "configure multi-container app".
  • 100generating-docker-compose-files
    Execute use when you need to work with Docker Compose. This skill provides Docker Compose file generation with comprehensive guidance and automation. Trigger with phrases like "generate docker-compose", "create compose file", or "configure multi-container app".
  • 100docker-compose
    Docker Compose for multi-container apps. Covers services, networks, and volumes. Use for local development environments. USE WHEN: user mentions "docker-compose", "docker-compose.yml", "multi-container", "service dependencies", "docker compose up", asks about "local dev…
  • 90docker-compose-helper
    Use when the user asks to create, debug, or modify Docker Compose files, compose.yaml, docker-compose.yml, service dependencies, healthchecks, ports, volumes, networks, environment variables, or local multi-container development stacks. Use this skill even if the user says…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.