Przejdź do treści
Biblioteka skilli
AI i agenci · Budowanie i kod

ai-integration

/ai-integration

Projektuje, wdraża, ocenia i zabezpiecza funkcje LLM, systemy RAG i agentów korzystających z narzędzi, w tym dane wyjściowe w strukturze i kontrolę kosztów.

Repozytorium
agent-forge ↗
Licencja
MIT
Ostatnia zmiana repozytorium
04.11.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
3
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 04.11.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Projektuje, wdraża, ocenia i zabezpiecza funkcje LLM, systemy RAG i agentów korzystających z narzędzi, w tym dane wyjściowe w strukturze i kontrolę kosztów.

Oryginalny opis (EN)

Design, implement, evaluate, and secure LLM-powered features, RAG systems, tool-using agents, and model-provider integrations. Use for model APIs, structured outputs, retrieval, prompt injection defenses, orchestration, evaluation, privacy, and cost control.

Jak działa

  1. Mission
  2. 1. Define the product behavior
  3. 2. Choose the right pattern
  4. 3. Model/provider abstraction
  5. 4. Prompt and instruction design
  6. 5. Structured output and tool use
  7. 6. Retrieval-augmented generation
  8. 7. Agent orchestration and state
  9. 8. Evaluation and quality assurance
  10. 9. Privacy and safety
Sekcje z nagłówków SKILL.md, w oryginale · 143 linii, 1 103 słów
Podgląd SKILL.md
---
name: ai-integration
description: Design, implement, evaluate, and secure LLM-powered features, RAG systems, tool-using agents, and model-provider integrations. Use for model APIs, structured outputs, retrieval, prompt injection defenses, orchestration, evaluation, privacy, and cost control.
license: MIT
---

# AI application and agent engineering

## Mission

Build AI features that are useful, bounded, measurable, secure, and operable. Choose the simplest approach that meets the product requirement. A model call is not a substitute for deterministic application logic, and an agent is not automatically better than a workflow.

## 1. Define the product behavior

Clarify:
- User and job-to-be-done.
- Expected output, accepted error rate, and unacceptable failure modes.
- Whether answers must be grounded in supplied data or current sources.
- Latency, availability, cost, throughput, and context-size budgets.
- Data sensitivity, retention, region, and provider constraints.
- Whether the feature can take actions or only provide information.
- Required human review, auditability, and recovery path.

Define success with representative examples and measurable evaluation criteria before choosing a model.

## 2. Choose the right pattern

Prefer, in order, the least complex viable approach:
1. Deterministic code or standard search when the task is fully specified.
2. A single model call for interpretation, transformation, summarization, or generation.
3. Retrieval-augmented generation when answers need private or changing source material.
4. Tool calling when the model must request bounded application capabilities.
5. A multi-step agent/workflow when the task requires conditional planning, iteration, or stateful orchestration that cannot be expressed cleanly as a deterministic pipeline.

Do not use multi-agent orchestration just to parallelize trivial work. Account for the increased latency, cost, coordination failures, and review burden.

## 3. Model/provider abstraction

Inspect provider APIs, supported models, limits, data policies, pricing, tool semantics, structured-output behavior, streaming, and region availability. Isolate provider-specific request/response formats when portability is a real requirement, but do not hide important capability differences behind a lowest-common-denominator interface.

Implement bounded:
- Connect/read timeouts and cancellation.
- Rate/concurrency limits and retry budget.
- Token/context budget and truncation policy.
- Error classification and provider fallback.
- Request IDs, usage/cost accounting, and safe observability.
- Model/version selection and rollback.

Do not log full prompts or completions when they may contain sensitive user data.

## 4. Prompt and instruction design

Separate trusted application policy from user content and retrieved material. State task, context, constraints, output contract, and refusal/escalation conditions clearly. Provide examples only when they resolve ambiguity. Avoid contradictory requirements and bloated global instructions.

Treat model output as untrusted. Validate structured output against a schema and semantic constraints. Never trust the model to enforce identity, permission, spending, data retention, or irreversible action policy.

## 5. Structured output and tool use

For structured output:
- Define a minimal schema with required fields, enums, bounds, and clear null semantics.
- Validate syntactic and semantic correctness after generation.
- Handle refusal, incomplete output, truncation, and schema mismatch explicitly.
- Avoid assuming the model will always return valid JSON or obey a schema perfectly.

For tools:
- Give each tool one narrow purpose and a strict input schema.
- Validate every argument in application code.
- Authenticate and authorize the user/action/resource independently.
- Enforce tenant boundaries, rate limits, quotas, and timeouts.
- Return bounded, redacted outputs with stable errors.
- Require explicit human confirmation before irreversible, destructive, financial, access-control, or production actions.
- Record audit events for side effects without logging secrets.

Tool descriptions must state when the tool should and should not be used, its side effects, and expected result.

## 6. Retrieval-augmented generation

Define the data sources and their access policies before designing indexing. Preserve document permissions, tenant identity, provenance, timestamps, and source links throughout ingestion, chunking, embedding, retrieval, and answer generation.

Evaluate separately:
- Ingestion completeness and freshness.
- Chunking and metadata quality.
- Retrieval recall/precision and ranking.
- Grounding and citation correctness.
- Answer faithfulness and abstention when evidence is insufficient.

Prevent cross-tenant retrieval and stale access after permission revocation or deletion. Treat retrieved text as untrusted evidence, not privileged instructions. The model must not follow instructions embedded in documents that request secrets, policy changes, or actions outside the user request.

If source evidence is missing, conflicting, or stale, communicate uncertainty rather than inventing a confident answer.

## 7. Agent orchestration and state

Define the agent's allowed tools, step/time/token/cost limits, stopping conditions, state model, and owner of each transition. Make side-effecting tools idempotent when retries are possible. Keep deterministic validation and authorization outside the model loop.

For multi-agent work:
- Assign bounded tasks with explicit inputs and expected outputs.
- Avoid delegating secrets or unnecessary user data.
- Define how outputs are verified and reconciled.
- Prevent parallel agents from editing the same resources without coordination.
- Keep a parent-level owner responsible for final correctness.
- Stop loops that repeat without progress or exceed budget.

## 8. Evaluation and quality assurance

Build a versioned evaluation set from representative user tasks, edge cases, adversarial inputs, and known failures. Measure:
- Task success and completeness.
- Factuality/faithfulness and citation accuracy.
- Tool selection, argument correctness, and authorization outcomes.
- Refusal correctness and escalation quality.
- Robustness to prompt injection and data leakage.
- Latency, cost, and resource usage.
- User correction rate and downstream outcome.

Use deterministic graders for objective properties, model-based graders only with calibration and sampling, and human review for high-impact decisions. Evaluate before model, prompt, retriever, or tool changes.

## 9. Privacy and safety

Minimize data sent to providers. Redact or transform sensitive fields when possible. Document provider retention, training use, region, subprocessors, and contractual controls. Apply access control before retrieval and before tool execution. Do not expose hidden prompts, credentials, private files, or other tenants' data.

For high-impact domains, define human oversight, appeals, logging, incident response, and safe fallback. Avoid presenting generated content as verified fact without evidence.

## 10. Operations and cost

Track provider/model, prompt version, input/output tokens, latency, status, retry, cost estimate, and evaluation version using privacy-safe telemetry. Set per-user and per-feature quotas. Monitor timeouts, refusal spikes, schema failures, empty retrieval, tool errors, and unusual cost growth.

Define fallback behavior for provider outage: cached/known data, deterministic path, reduced capability, or clear unavailability. Do not silently fabricate results during outages.

## Deliverables

Provide architecture/pattern choice, model/provider decision, prompt/output schema, tool permission model, retrieval design (if any), evaluation plan, privacy/security review, budget controls, and operational runbook.

## Anti-patterns

Avoid direct model authority over permissions, arbitrary shell/database tools, unrestricted multi-agent loops, trusting retrieved instructions, unvalidated JSON, logging sensitive prompts, using LLMs for simple deterministic transformations, and claims of guaranteed correctness.

## Supporting reference material

Load these resources when the task needs a reusable template or deeper checklist:
- [ai evaluation plan template](references/ai-evaluation-plan-template.md)
- [rag security checklist](references/rag-security-checklist.md)

The procedure and quality gates in this SKILL.md remain the primary workflow.


## Completion report

State the selected pattern, provider assumptions, tools/permissions, evaluation results, privacy safeguards, cost/latency limits, failure behavior, and known limitations.
Pliki w paczce (3)
  • SKILL.md 9 KB
  • references/ai-evaluation-plan-template.md 1 KB
  • references/rag-security-checklist.md 1 KB

Wersje

RepozytoriumWynikLicencja
Ta strona · mhingan/agent-forge100MIT
datenoio/iterabledata100MIT

Migawka bazy z 13.11.2026 · wersja paczki 661317f18c9c (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100llm-integration
    Buduje produkcyjne funkcje oparte na API LLM: strukturę promptów, walidowane dane wyjściowe, strumieniowanie i obronę przed wstrzyknięciami.
  • 100llm-app-engineering
    Buduje niezawodne funkcje oparte na LLM: prompty, dane wyjściowe, RAG, narzędzia, ewaluacje, koszty i bezpieczeństwo.
  • 100ai
    Projektuje, buduje i przegląda funkcje LLM: wybór modelu, prompty, dane wyjściowe, RAG, agentów, ewaluacje i zabezpieczenia przed wstrzykiwaniem poleceń.
  • 100senior-prompt-engineer
    Optymalizuje prompty i szablony, ocenia odpowiedzi LLM, projektuje RAG i architektury agentów oraz analizuje zużycie tokenów.
  • 90agentic-architecture
    Projektuje architekturę systemów agentów LLM: topologie orkiestracji, strategię pamięci i kontekstu, warstwę narzędzi oraz kontrolę człowieka, na poziomie architekta.
  • 100senior-ml-engineer
    Przygotowuje modele ML i funkcje LLM do produkcji: specyfikacje wdrożenia, monitoring dryfu, RAG i kontrolę kosztów.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.