celery-expert
/celery-expert
Pomaga konfigurować Celery z Redisem, pisać zadania w tle, harmonogramy beat, ponowienia, monitoring i testy workerów w projektach Pythona.
- Autor
- alex-kopylov ↗
- Repozytorium
- zweihander ↗
- Licencja
- Apache-2.0
- Ostatnia zmiana repozytorium
- 19.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 8
- Skrypty
- 3
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 19.10.2026, bez odwołań do wycofanych modeli i API.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
- 3 skrypty w paczce. Przejrzyj je przed instalacją, bo Claude może je uruchomić.
Skrypty: examples/celery_app.py, examples/celery_beat_schedule.py, examples/tasks.py
Co robi i kiedy
Pomaga konfigurować Celery z Redisem, pisać zadania w tle, harmonogramy beat, ponowienia, monitoring i testy workerów w projektach Pythona.
Wyzwalacze z opisu
Oryginalny opis (EN)
This skill should be used when the user asks to "configure Celery", "set up Celery with Redis", "write Celery tasks", "add background tasks with Celery", "celery beat schedule", "celery task retry", "celery performance tuning", "celery TDD", "test Celery tasks", "celery worker config", "celery monitoring", "celery security", or when code imports celery or the project uses a celeryconfig/celery.py module.
Jak działa
- Overview
- When to Apply
- Task Design Principles
- Redis Broker Quick Config
- Retry Strategy Summary
- Workflow Patterns
- Pre-Implementation Checklist
- Critical Reminders
- Additional Resources
Podgląd SKILL.md
---
name: celery-expert
description: This skill should be used when the user asks to "configure Celery", "set up Celery with Redis", "write Celery tasks", "add background tasks with Celery", "celery beat schedule", "celery task retry", "celery performance tuning", "celery TDD", "test Celery tasks", "celery worker config", "celery monitoring", "celery security", or when code imports celery or the project uses a celeryconfig/celery.py module.
metadata:
skills:
"python-dev-workflow:tests-manager": "Use for Celery test planning, pytest fixtures, eager-mode decisions, live-worker integration tests, and test isolation."
---
# Celery Expert (Redis Broker)
## Overview
Guide production Celery with Redis as broker/result backend, covering task
design, workflows, Beat scheduling, performance, monitoring, and security.
**Core Principles:**
1. **Reliability** — task retries, acknowledgment strategies, no task loss
2. **Performance Aware** — optimize throughput with chunking, pooling, and proper prefetch
3. **Security** — safe serialization (JSON only, never pickle), broker authentication
4. **Observable** — monitoring, metrics, tracing, alerting
**Pydantic models in tasks:** Do not manually serialize Pydantic models (`.dict()`, `.model_dump()`, `**kwargs`). Use `celery-pydantic` so tasks can accept models directly.
## When to Apply
Activate when any of the following appear:
- `from celery import Celery` or `@app.task` / `@shared_task` in code
- `celeryconfig.py`, `celery.py`, or `beat_schedule` in project files
- User asks about background tasks, task queues, async processing, or distributed workers
- User mentions Flower, Celery Beat, task retries, or task routing
## Task Design Principles
Every task should follow these rules:
- **Idempotent** — safe to retry without side effects. Set values, don't increment them.
- **`bind=True`** — access `self.request.id`, `self.retry()`, and task metadata
- **JSON serializer** — never use pickle (remote code execution risk)
- **Time limits** — always set both `soft_time_limit` and `time_limit`
- **`acks_late=True`** for critical tasks — acknowledge after completion, not before
- **Small results** — store large data externally, return only references
```python
@app.task(
bind=True,
max_retries=3,
acks_late=True,
reject_on_worker_lost=True,
time_limit=300,
soft_time_limit=240,
rate_limit='100/m',
)
def process_order(self, order_id: int):
...
```
## Redis Broker Quick Config
```python
from celery import Celery
app = Celery('myapp')
app.conf.update(
broker_url='redis://localhost:6379/0',
broker_connection_retry_on_startup=True,
broker_pool_limit=10,
result_backend='redis://localhost:6379/1',
result_expires=3600,
task_serializer='json',
result_serializer='json',
accept_content=['json'],
task_acks_late=True,
task_reject_on_worker_lost=True,
task_time_limit=300,
task_soft_time_limit=240,
worker_prefetch_multiplier=4,
worker_max_tasks_per_child=1000,
)
```
See `references/production-config.md` for complete production setup with TLS, beat, and deployment.
## Retry Strategy Summary
| Pattern | Use When | Config |
|---------|----------|--------|
| Manual retry | Custom backoff logic | `self.retry(exc=exc, countdown=2 ** self.request.retries)` |
| Auto-retry | Standard HTTP/network errors | `autoretry_for=(RequestException,), retry_backoff=True` |
| Auto-retry + jitter | High-concurrency retries | Add `retry_jitter=True, retry_backoff_max=600` |
| Reject (no retry) | Permanent failures, poison messages | `raise Reject(reason, requeue=False)` |
## Workflow Patterns
```python
from celery import chain, group, chord
# CHAIN: Sequential (A -> B -> C)
workflow = chain(fetch_data.s(url), process_item.s(), send_notification.s())
# GROUP: Parallel execution
job = group(fetch_data.s(url) for url in urls)
# CHORD: Map-Reduce (parallel + callback)
workflow = chord(group(process_item.s(item) for item in items))(aggregate_results.s())
```
## Pre-Implementation Checklist
**Before writing code:**
- [ ] Write a failing test for task behavior (use `python-dev-workflow:tests-manager`)
- [ ] Define idempotency strategy
- [ ] Choose queue routing for task priority
- [ ] Plan retry strategy and error handling
**Before deploying:**
- [ ] Time limits set (soft + hard)
- [ ] `acks_late=True` on critical tasks
- [ ] JSON serialization confirmed (not pickle)
- [ ] Redis broker authentication configured
- [ ] Result expiration set
- [ ] Monitoring active (Flower/Prometheus)
- [ ] Dead letter queue handling implemented
## Critical Reminders
**NEVER:** use pickle serialization, run without time limits, store large data in results, create non-idempotent tasks, run without broker auth, expose Flower without auth, manually serialize Pydantic models (use `celery-pydantic`)
**ALWAYS:** use JSON serialization, set time limits (soft + hard), make tasks idempotent, use `acks_late=True` for critical tasks, set result expiration, implement retry with backoff, monitor with Flower/Prometheus, log with correlation IDs
## Additional Resources
### Reference Files
- **`references/performance-patterns.md`** — Task chunking, prefetch tuning, result backend optimization, connection pooling, queue routing
- **`references/production-config.md`** — Complete Redis production config, beat scheduling, worker deployment, health checks
- **`references/security-standards.md`** — JSON serializer enforcement, Redis TLS/auth, input validation, Flower security
- **`references/common-mistakes.md`** — Anti-pattern table with fixes: pickle, idempotency, time limits, large results, pydantic serialization
### Example Files
Working code examples in `examples/`:
- **`examples/celery_app.py`** — App factory with Redis broker and recommended defaults
- **`examples/tasks.py`** — Task definitions: retry, chunking, idempotency patterns
- **`examples/celery_beat_schedule.py`** — Crontab and interval periodic task config
Pliki w paczce (8)
- SKILL.md 6 KB
- examples/celery_app.py 1 KB
- examples/celery_beat_schedule.py 1 KB
- examples/tasks.py 3 KB
- references/common-mistakes.md 4 KB
- references/performance-patterns.md 4 KB
- references/production-config.md 6 KB
- references/security-standards.md 3 KB
Wersje
W bazie nie ma innych skilli o nazwie celery-expert.
Migawka bazy z 13.11.2026 · wersja paczki c32d2a4d41e6 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100python-celery-task-queues
Uruchamia asynchroniczne zadania w tle i harmonogramy w Pythonie przez Celery. - 100django-celery
Pomaga dodawać do aplikacji Django zadania w tle, zadania cykliczne i przetwarzanie asynchroniczne z Celery, od konfiguracji i beat po ponowienia, monitoring i testy. - 100django-celery
Wzorce zadań asynchronicznych w Django z Celery: konfiguracja, harmonogram beat, ponowienia, przepływy canvas i monitoring. - 100django-celery
Opisuje wzorce asynchronicznych zadań w Django z Celery: konfiguracja, projekt zadań, harmonogram beat, ponowienia, workflow canvas, monitoring i testy. - 100django-celery
Opisuje wzorce zadań asynchronicznych Django z Celery: konfigurację, projekt zadań, harmonogram Beat, ponowienia, przepływy Canvas, monitoring i testy. - 100using-celery
Opisuje rozproszoną kolejkę zadań Celery 5.3+ z harmonogramem Beat, brokerami Redis i RabbitMQ, wzorcami workflow i integracją z FastAPI, do zadań w tle, okresowych i przetwarzania asynchronicznego.