Skip to content
Skills Library
AI & agents · Planning

rag-engineering

/rag-engineering

Design, audit and evaluate provider-neutral retrieval-augmented generation pipelines from ingest through answer quality, latency and drift. Use for planning and local evaluation; do not upload, deploy or publish automatically.

Repository
Ai-Stack ↗
Licence
Apache-2.0
Last repository change
03.11.2026

Assessment and security

98/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene13 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
23
Scripts
4
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 03.11.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What to check before installing
  • 4 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: scripts/evaluate_retrieval.py, scripts/hub_quality.py, scripts/test_evaluate_retrieval.py, scripts/test_hub_quality.py

What it does and when to use it

Design, audit and evaluate provider-neutral retrieval-augmented generation pipelines from ingest through answer quality, latency and drift. Use for planning and local evaluation; do not upload, deploy or publish automatically.

Original author description

Design, audit and evaluate provider-neutral retrieval-augmented generation pipelines from ingest through answer quality, latency and drift. Use for planning and local evaluation; do not upload, deploy or publish automatically.

How it works

  1. Workflow
  2. Local evaluation harness
Original SKILL.md headings · 52 lines, 269 words
SKILL.md preview
---
name: rag-engineering
description: Design, audit and evaluate provider-neutral retrieval-augmented generation pipelines from ingest through answer quality, latency and drift. Use for planning and local evaluation; do not upload, deploy or publish automatically.
---

# RAG Engineering

Use this skill for a grounded retrieval system or evaluation covering documents,
chunking, embeddings, retrieval, reranking, generation and monitoring.

## Workflow

1. Define corpus authority, user questions, answer contract and access boundary.
2. Inventory formats, provenance, revisions, permissions, PII and freshness.
3. Specify ingest, normalization, chunking, indexing, retrieval, reranking,
   context assembly, citation and abstention behavior.
4. Build a versioned eval set with gold evidence, adversarial cases and leakage checks.
5. Compare a simple baseline before tuning; report quality, grounding, latency,
   cost and failure modes.
6. Define drift, reindex, rollback and human-review gates. Keep upload, provider
   jobs, deployment and production writes explicitly approved.

## Local evaluation harness

The skill ships a deterministic, **stdlib-only** evaluator that runs against
declared predictions and never executes the system under test.

```powershell
python "<skill-dir>\scripts\evaluate_retrieval.py" validate-corpus --corpus "<corpus.json>"
python "<skill-dir>\scripts\evaluate_retrieval.py" evaluate --corpus "<corpus.json>" --predictions "<predictions.json>" --output "<report.json>"
python "<skill-dir>\scripts\evaluate_retrieval.py" compare --corpus "<corpus.json>" --baseline "<baseline.json>" --candidate "<candidate.json>" --output-dir "<out>"
python "<skill-dir>\scripts\evaluate_retrieval.py" selftest --output-dir "<out>"
python "<skill-dir>\scripts\test_evaluate_retrieval.py" --artifact-root "<fixtures-dir>"
```

The bundled synthetic corpus and stub predictions live under
[`fixtures/`](fixtures/README.md); they cover current-vs-historical,
superseded source, alias collision, unknown answer, conflicting evidence,
forbidden project citation and duplicate advisory proposals.

See:

- [references/contract.md](references/contract.md) — design / evaluation contract.
- [references/json-schema.md](references/json-schema.md) — JSON schema for the
  corpus, predictions and comparison report.
- [references/sample-report.md](references/sample-report.md) — preserved sample
  of an evaluator run on the bundled corpus.
- [references/trigger-cases.json](references/trigger-cases.json) — activation
  triggers and anti-triggers.
- [scripts/evaluate_retrieval.py](scripts/evaluate_retrieval.py) — the evaluator.
- [scripts/test_evaluate_retrieval.py](scripts/test_evaluate_retrieval.py) — the
  deterministic test suite.
- For a Hub pilot preparation or repair task explicitly assigned in scope, see
  [references/hub-quality.md](references/hub-quality.md). This conditional
  pointer does not create a general retrieval-evaluation trigger.
Files in this package (23)
  • SKILL.md 3 KB
  • agents/openai.yaml 1 KB
  • fixtures/README.md 3 KB
  • fixtures/baseline-predictions.json 7 KB
  • fixtures/candidate-predictions.json 6 KB
  • fixtures/gold-corpus.json 11 KB
  • fixtures/hub-quality-pilot/materials.json 3 KB
  • fixtures/hub-quality-pilot/questions.json 1 KB
  • fixtures/malformed/bad-timestamp-zone.json 1 KB
  • fixtures/malformed/citation-bad-source-id.json 1 KB
  • fixtures/malformed/duplicate-case-id.json 1 KB
  • fixtures/malformed/missing-schema-version.json 1 KB
  • fixtures/malformed/not-json.txt 1 KB
  • fixtures/malformed/wrong-schema-version.json 1 KB
  • references/contract.md 1 KB
  • references/hub-quality.md 4 KB
  • references/json-schema.md 9 KB
  • references/sample-report.md 7 KB
  • references/trigger-cases.json 1 KB
  • scripts/evaluate_retrieval.py 49 KB
  • scripts/hub_quality.py 40 KB
  • scripts/test_evaluate_retrieval.py 54 KB
  • scripts/test_hub_quality.py 44 KB

Versions

The database has no other skills named rag-engineering.

Database snapshot: 13.11.2026 · package version b2c289d8291c (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100rag-architecture-designer
    Design a Retrieval-Augmented Generation (RAG) pipeline architecture: ingestion, indexing, retrieval, generation, and evaluation components. Use when working on RAG architecture, retrieval pipelines, vector stores.
  • 90rag-evaluation
    Evaluate retrieval-augmented-generation pipelines with reference-free metrics to detect silent retrieval and grounding failures. Use when: evaluate rag pipeline, measure faithfulness, context precision recall, detect rag hallucination, reference-free rag metrics, score retrieval quality.
  • 100evaluate-rag
    Guides evaluation of RAG pipeline retrieval and generation quality. Use when evaluating a retrieval-augmented generation system, measuring retrieval quality, assessing generation faithfulness or relevance, generating synthetic QA pairs for retrieval testing, or optimizing chunking strategies.
  • 90monitor-rag-quality
    Use this to measure and monitor the quality of a RAG (retrieval-augmented generation) pipeline - whether it retrieves the right context and answers faithfully. Trigger on "my RAG gives wrong answers", "is my retrieval any good", "the chatbot makes things up", "evaluate my RAG",…
  • 100dspy-rag-pipeline
    Use for RAG pipelines, retrieval augmented generation, ColBERTv2, context retrieval, multi-hop RAG, and grounded DSPy answers.
  • 95rag-eval
    Use when the user has a RAG (Retrieval-Augmented Generation) system and wants to evaluate its quality — separating retrieval issues from generation issues. Also use when the user mentions RAG evaluation, faithfulness checking, hallucination detection in RAG, retrieval quality,…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.