rag-engineering
/rag-engineering
Design, audit and evaluate provider-neutral retrieval-augmented generation pipelines from ingest through answer quality, latency and drift. Use for planning and local evaluation; do not upload, deploy or publish automatically.
- Author
- PapeMati98 ↗
- Repository
- Ai-Stack ↗
- Licence
- Apache-2.0
- Last repository change
- 03.11.2026
Assessment and security
- Files in this package
- 23
- Scripts
- 4
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 03.11.2026, without references to retired models or APIs.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
- 4 scripts in this package. Review them before installing, as Claude may execute them.
Scripts: scripts/evaluate_retrieval.py, scripts/hub_quality.py, scripts/test_evaluate_retrieval.py, scripts/test_hub_quality.py
What it does and when to use it
Design, audit and evaluate provider-neutral retrieval-augmented generation pipelines from ingest through answer quality, latency and drift. Use for planning and local evaluation; do not upload, deploy or publish automatically.
Original author description
Design, audit and evaluate provider-neutral retrieval-augmented generation pipelines from ingest through answer quality, latency and drift. Use for planning and local evaluation; do not upload, deploy or publish automatically.
How it works
- Workflow
- Local evaluation harness
SKILL.md preview
--- name: rag-engineering description: Design, audit and evaluate provider-neutral retrieval-augmented generation pipelines from ingest through answer quality, latency and drift. Use for planning and local evaluation; do not upload, deploy or publish automatically. --- # RAG Engineering Use this skill for a grounded retrieval system or evaluation covering documents, chunking, embeddings, retrieval, reranking, generation and monitoring. ## Workflow 1. Define corpus authority, user questions, answer contract and access boundary. 2. Inventory formats, provenance, revisions, permissions, PII and freshness. 3. Specify ingest, normalization, chunking, indexing, retrieval, reranking, context assembly, citation and abstention behavior. 4. Build a versioned eval set with gold evidence, adversarial cases and leakage checks. 5. Compare a simple baseline before tuning; report quality, grounding, latency, cost and failure modes. 6. Define drift, reindex, rollback and human-review gates. Keep upload, provider jobs, deployment and production writes explicitly approved. ## Local evaluation harness The skill ships a deterministic, **stdlib-only** evaluator that runs against declared predictions and never executes the system under test. ```powershell python "<skill-dir>\scripts\evaluate_retrieval.py" validate-corpus --corpus "<corpus.json>" python "<skill-dir>\scripts\evaluate_retrieval.py" evaluate --corpus "<corpus.json>" --predictions "<predictions.json>" --output "<report.json>" python "<skill-dir>\scripts\evaluate_retrieval.py" compare --corpus "<corpus.json>" --baseline "<baseline.json>" --candidate "<candidate.json>" --output-dir "<out>" python "<skill-dir>\scripts\evaluate_retrieval.py" selftest --output-dir "<out>" python "<skill-dir>\scripts\test_evaluate_retrieval.py" --artifact-root "<fixtures-dir>" ``` The bundled synthetic corpus and stub predictions live under [`fixtures/`](fixtures/README.md); they cover current-vs-historical, superseded source, alias collision, unknown answer, conflicting evidence, forbidden project citation and duplicate advisory proposals. See: - [references/contract.md](references/contract.md) — design / evaluation contract. - [references/json-schema.md](references/json-schema.md) — JSON schema for the corpus, predictions and comparison report. - [references/sample-report.md](references/sample-report.md) — preserved sample of an evaluator run on the bundled corpus. - [references/trigger-cases.json](references/trigger-cases.json) — activation triggers and anti-triggers. - [scripts/evaluate_retrieval.py](scripts/evaluate_retrieval.py) — the evaluator. - [scripts/test_evaluate_retrieval.py](scripts/test_evaluate_retrieval.py) — the deterministic test suite. - For a Hub pilot preparation or repair task explicitly assigned in scope, see [references/hub-quality.md](references/hub-quality.md). This conditional pointer does not create a general retrieval-evaluation trigger.
Files in this package (23)
- SKILL.md 3 KB
- agents/openai.yaml 1 KB
- fixtures/README.md 3 KB
- fixtures/baseline-predictions.json 7 KB
- fixtures/candidate-predictions.json 6 KB
- fixtures/gold-corpus.json 11 KB
- fixtures/hub-quality-pilot/materials.json 3 KB
- fixtures/hub-quality-pilot/questions.json 1 KB
- fixtures/malformed/bad-timestamp-zone.json 1 KB
- fixtures/malformed/citation-bad-source-id.json 1 KB
- fixtures/malformed/duplicate-case-id.json 1 KB
- fixtures/malformed/missing-schema-version.json 1 KB
- fixtures/malformed/not-json.txt 1 KB
- fixtures/malformed/wrong-schema-version.json 1 KB
- references/contract.md 1 KB
- references/hub-quality.md 4 KB
- references/json-schema.md 9 KB
- references/sample-report.md 7 KB
- references/trigger-cases.json 1 KB
- scripts/evaluate_retrieval.py 49 KB
- scripts/hub_quality.py 40 KB
- scripts/test_evaluate_retrieval.py 54 KB
- scripts/test_hub_quality.py 44 KB
Versions
The database has no other skills named rag-engineering.
Database snapshot: 13.11.2026 · package version b2c289d8291c (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100rag-architecture-designer
Design a Retrieval-Augmented Generation (RAG) pipeline architecture: ingestion, indexing, retrieval, generation, and evaluation components. Use when working on RAG architecture, retrieval pipelines, vector stores. - 90rag-evaluation
Evaluate retrieval-augmented-generation pipelines with reference-free metrics to detect silent retrieval and grounding failures. Use when: evaluate rag pipeline, measure faithfulness, context precision recall, detect rag hallucination, reference-free rag metrics, score retrieval quality. - 100evaluate-rag
Guides evaluation of RAG pipeline retrieval and generation quality. Use when evaluating a retrieval-augmented generation system, measuring retrieval quality, assessing generation faithfulness or relevance, generating synthetic QA pairs for retrieval testing, or optimizing chunking strategies. - 90monitor-rag-quality
Use this to measure and monitor the quality of a RAG (retrieval-augmented generation) pipeline - whether it retrieves the right context and answers faithfully. Trigger on "my RAG gives wrong answers", "is my retrieval any good", "the chatbot makes things up", "evaluate my RAG",… - 100dspy-rag-pipeline
Use for RAG pipelines, retrieval augmented generation, ColBERTv2, context retrieval, multi-hop RAG, and grounded DSPy answers. - 95rag-eval
Use when the user has a RAG (Retrieval-Augmented Generation) system and wants to evaluate its quality — separating retrieval issues from generation issues. Also use when the user mentions RAG evaluation, faithfulness checking, hallucination detection in RAG, retrieval quality,…