Skip to content
Skills Library
AI & agents · Tool integration

replicate

/replicate

Use when running, packaging, deploying or scaling a model on the Replicate platform from code — blocking run versus async predictions, handling FileOutput, deployments with warm private endpoints and autoscaling, packaging with Cog, verifying webhook signatures, and cutting GPU spend. NOT crafting image prompts and parameters or picking image model families (that is `replicate-images`).

Repository
rsc-harness ↗
Licence
MIT
Last repository change
25.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
7
Scripts
1
HTML comments
0
URLs in SKILL.md
1
Deprecation

No signals detected. Repository changed on 25.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What to check before installing
  • 1 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: scripts/verify.sh

What it does and when to use it

Use when running, packaging, deploying or scaling a model on the Replicate platform from code — blocking run versus async predictions, handling FileOutput, deployments with warm private endpoints and autoscaling, packaging with Cog, verifying webhook signatures, and cutting GPU spend. NOT crafting image prompts and parameters or picking image model families (that is `replicate-images`).

Original author description

Use when running, packaging, deploying or scaling a model on the Replicate platform from code — blocking run versus async predictions, handling FileOutput, deployments with warm private endpoints and autoscaling, packaging with Cog, verifying webhook signatures, and cutting GPU spend. NOT crafting image prompts and parameters or picking image model families (that is `replicate-images`).

How it works

  1. Decision: how should this model run?
  2. Auth & install
  3. Run a model
  4. Long-running & async
  5. Webhooks
  6. Deployments
  7. Package a custom model with Cog
  8. Cost & scaling levers
  9. Anti-patterns
  10. References
Original SKILL.md headings · 204 lines, 1,459 words
SKILL.md preview
---
name: replicate
description: "Use when running, packaging, deploying or scaling a model on the Replicate platform from code — blocking run versus async predictions, handling FileOutput, deployments with warm private endpoints and autoscaling, packaging with Cog, verifying webhook signatures, and cutting GPU spend. NOT crafting image prompts and parameters or picking image model families (that is `replicate-images`)."
tags: [replicate, cog, deployments, webhooks, model-serving, gpu, ai-infra]
recommends: [replicate-images, modal, runpod, huggingface, webhooks, cost-tracking, docker]
origin: risco
---

# Replicate platform operations

This skill is about **how a model runs in production on Replicate** — clients, async, deployments,
Cog packaging, webhooks, scaling, and spend. It is the platform-engineering counterpart to image
prompt craft. If the question is *what prompt, aspect ratio, or model family produces a good image*,
that is `replicate-images`, not this skill. Here the mental model is: **a prediction is a job. You
either wait for it, poll it, or get pinged about it — and where it runs (shared cold pool vs a private
warm deployment) is a cost-and-latency dial you set deliberately.**

Pinned facts (verified 2026-06-02): Python client `replicate` 1.x (latest 1.0.7), Python 3.8+;
JS client `replicate` on npm; auth via `REPLICATE_API_TOKEN`. A `2.0.0aN` alpha exists on PyPI but
is NOT the default — pin `replicate>=1,<2` so a fresh install never silently pulls it.

## Decision: how should this model run?

Pick the row by latency tolerance and whether your process can block. Do not default to `run()` for
everything — a 10-minute job inside a web request will time out and burn a worker.

| Pattern | Latency | Blocks your process? | Cost shape | Use when |
|---|---|---|---|---|
| `replicate.run(...)` | seconds | Yes — waits to completion | per-prediction, shared pool | interactive/quick calls, scripts, CLIs |
| `predictions.create()` + poll | minutes | Yes, but you control the loop | per-prediction, shared pool | long job, a worker can babysit it |
| `predictions.create(webhook=...)` | minutes+ | No — fire and forget | per-prediction, shared pool | long job, the request must return now |
| Deployment (private endpoint) | low + steady | depends on call style above | warm floor + per-prediction | sustained traffic, need warm/private/autoscale cap |

Rule: if a human or HTTP request is waiting longer than a few seconds, do not block on `run()` —
switch to predictions + webhook. Why: synchronous timeouts kill the request but the GPU job keeps
running and billing.

## Auth & install

```bash
export REPLICATE_API_TOKEN=r8_...           # clients read this env var automatically
pip install 'replicate>=1,<2'               # pin: 2.0.0aN is alpha; unpinned can pull it
npm install replicate                        # Node client
```

Python 3.8+ is required. Never `pip install replicate` unpinned in a Dockerfile or requirements
file — a rebuild months later can resolve to the 2.0 alpha and break your imports. Why: the alpha is
a full Stainless/httpx rewrite with a different surface.

## Run a model

```python
import replicate

output = replicate.run(
    "black-forest-labs/flux-schnell",
    input={"prompt": "a red bicycle", "num_outputs": 1},
)
```

Since client 1.0.0, file outputs come back as **`FileOutput`** objects, not URL strings. Treating one
as a string is the single most common bug.

```python
# Bad — output[0] is a FileOutput, not a str; this writes the repr, not the bytes
open("out.png", "w").write(output[0])

# Good — read the bytes, or take the URL explicitly
with open("out.png", "wb") as f:
    f.write(output[0].read())          # bytes
print(output[0].url)                    # hosted URL if you'd rather link
```

Rule: call `.read()` for bytes or `.url` for the link. Why: silently coercing a `FileOutput` to a
string corrupts the file and the error surfaces far from the cause.

## Long-running & async

For jobs over a few seconds, create a prediction instead of blocking:

```python
client = replicate.Client()
prediction = client.predictions.create(
    model="owner/model",
    input={"prompt": "..."},
)
prediction.reload()                      # refresh status from the API
while prediction.status not in ("succeeded", "failed", "canceled"):
    time.sleep(2)
    prediction.reload()
```

Set a **deadline** so a stuck job auto-cancels instead of billing forever, and `cancel()` on cleanup
paths. Why: a hung prediction with no deadline is silent, open-ended GPU spend. Poll with a small
backoff, not a tight loop — you are charged for the prediction, not the polling, but a tight loop
wastes your own process and rate budget. Full polling loop with backoff and 5xx handling is in
`references/webhooks-and-async.md`.

## Webhooks

For fire-and-forget, hand Replicate a URL and filter to the events you care about:

```python
client.predictions.create(
    model="owner/model",
    input={"prompt": "..."},
    webhook="https://your.app/hooks/replicate",
    webhook_events_filter=["completed"],   # not every intermediate "logs" event
)
```

Rule: ALWAYS verify the signature before trusting a webhook body. Why: the URL is public — anyone can
POST forged completions to it. Replicate signs each delivery; the secret has a **`whsec_`** prefix.
Reconstruct signed content as `{webhook-id}.{webhook-timestamp}.{body}`, HMAC-SHA256 with the
base64-decoded secret, base64-encode, and constant-time compare against the `webhook-signature`
header (a space-separated `v1,<sig>` list). The clients expose a verification helper; the full
Python and Node recipe (including idempotency via `webhook-id`) is in
`references/webhooks-and-async.md`.

## Deployments

A **deployment** is a private, dedicated API endpoint for one model version that autoscales from zero
to hundreds of instances. Reach for it when you need warm instances, a private endpoint, or a hard
spend cap — not for one-off runs.

- `min_instances` — the **warm floor**. Set >0 to kill cold starts for latency-sensitive traffic;
  every warm instance bills whether or not it serves a request.
- `max_instances` — the **spend cap**. The ceiling on concurrent instances; protects you from a
  traffic spike turning into a surprise bill.
- Hardware (NVIDIA T4, A100, H100, ...) is switchable **without touching code** — change the
  deployment config, not `predict.py`.
- Rolling updates ship a new version with no downtime; built-in monitoring covers latency,
  throughput, error rate, and GPU memory.

You still call a deployment with `run()` / `predictions.create()` — it just routes to your private
instances. Create/update via HTTP API, the clients, or CLI; fields and the rolling-update flow are in
`references/deployments-api.md`.

## Package a custom model with Cog

Cog packages a model into a production container. You need two files; Replicate builds the API server
for you on push.

```yaml
# cog.yaml
build:
  gpu: true
  python_version: "3.11"
  python_packages:
    - "torch==2.4.0"
predict: "predict.py:Predictor"
```

```python
# predict.py
from cog import BasePredictor, Input, Path

class Predictor(BasePredictor):
    def setup(self):
        # load weights ONCE here, not per request
        self.model = load_model("weights.pth")

    def predict(self, prompt: str = Input(description="text prompt")) -> Path:
        result = self.model(prompt)
        return Path(result)            # Cog uploads the file
```

```bash
cog predict -i prompt="hello"          # run locally (needs Docker)
cog push r8.im/owner/model             # build + push; Replicate hosts it
```

Rule: load weights in `setup()`, never in `predict()`. Why: `setup()` runs once per instance;
`predict()` runs every request — loading weights per request makes every call pay the model-load cost.
Full `cog.yaml` (system packages, run steps), typed `Input(...)`, GPU config, version pinning, and
common build failures are in `references/cog-packaging.md`. Building requires Docker.

## Cost & scaling levers

- **Scale to zero** is the default — idle deployments stop billing. Only raise `min_instances` above
  zero when cold-start latency actually hurts users, and treat the warm floor as a line item.
- **Deadlines** on predictions cap the worst case so a stuck job cannot bill open-ended.
- **Right-size the GPU**: do not run a 1B model on an H100. Hardware is a deployment-config change.
- **Cache weights in `setup()`** so per-request work is just inference.

This skill covers Replicate-specific levers only. Tracking total AI spend across many providers as a
discipline is `cost-tracking`.

## Anti-patterns

| Anti-pattern | Why it bites | Do instead |
|---|---|---|
| Treating a `FileOutput` as a URL string | Corrupts files / writes a repr; error surfaces far away | `.read()` for bytes, `.url` for the link |
| Blocking `run()` for a 10-min job in a web request | Request times out; the GPU job keeps running and billing | `predictions.create(webhook=...)`, return now |
| Webhook handler with no signature check | The URL is public; anyone can forge completions | Verify HMAC-SHA256 against `whsec_` secret |
| High `min_instances` "just in case" | Every warm instance bills 24/7 idle | Scale to zero; raise floor only when cold starts hurt |
| Loading weights inside `predict()` | Every request pays the model-load cost | Load once in `setup()` |
| `pip install replicate` unpinned | A rebuild can pull the 2.0 alpha and break imports | Pin `replicate>=1,<2` |
| A deployment for a one-off run | Pays for a private endpoint you call once | Use the shared pool via `run()` |
| No deadline on a long prediction | A hung job bills open-ended, silently | Set a deadline; `cancel()` on cleanup |

## References

- `references/cog-packaging.md` — full `cog.yaml` + `predict.py`, GPU config, build/push, version pinning, build failures.
- `references/webhooks-and-async.md` — signature verification (Python + Node), event filters, idempotency, polling with backoff, 5xx retries.
- `references/deployments-api.md` — create/update/get deployment via HTTP + clients, autoscaling fields, rolling updates, monitoring metrics, CLI.

`scripts/verify.sh` statically checks an emitted artifact dir: a `cog.yaml` declaring `build:` and
`predict:`, a `predict.py` `Predictor` with `setup` + `predict`, and warns on an unpinned `replicate`
dependency or a `FileOutput` written as a string. Presence + key checks only — it does not run Docker.
Files in this package (7)
  • SKILL.md 10 KB
  • evals/README.md 1 KB
  • evals/cases.yaml 3 KB
  • references/cog-packaging.md 4 KB
  • references/deployments-api.md 3 KB
  • references/webhooks-and-async.md 4 KB
  • scripts/verify.sh 6 KB

Versions

RepositoryScoreLicence
This page · ericrisco/rsc-harness100MIT
usemosio/replicate100Apache-2.0
illodev/drawn-by-code100MIT
openfilm/openfilm90MIT
openfilm/skill-replicate90MIT

Database snapshot: 13.11.2026 · package version 2aa859802dfa (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100build-models
    Package and build custom AI models with Cog for deployment on Replicate. Use when creating a cog.yaml or predict.py, defining model inputs and outputs, loading model weights at setup time, building Docker images for ML models, serving locally with cog serve or cog predict, or…
  • 100publish-models
    Push and publish custom AI models to Replicate, and set up CI/CD for releasing new model versions safely. Use when running cog push, deploying a model to Replicate, releasing a new version, validating a model with cog-safe-push before publishing, configuring a Replicate…
  • 100replicate-images
    Use when generating or editing images through the Replicate API, or when a modern image model keeps ignoring the prompt — choosing aspect ratio, resolution, seed and output format, image-to-image, multi-reference composition, text-driven inpainting, readable text inside a…
  • 100check-model-pricing
    Check what an image model costs on fal (default) or Replicate, search fal's library for a model, and review or update the preferred image models per use case. Use when asked "how much is this model", "what's cheapest for X", "check fal pricing", "review the fal library", "update…
  • 89image-generation-gateways
    Select, integrate, and operate multi-model image-generation gateways with model-specific schema discovery, version policy, asynchronous jobs, webhooks, spend approval, safe inputs and artifacts, data-governance review, and billing observability. Use for comparing or building…
  • 100replicate-webhooks
    Receive and verify Replicate webhooks. Use when setting up Replicate webhook handlers, debugging signature verification, or handling prediction events like start, output, logs, or completed.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.