agent-evaluation
/agent-evaluation
Evaluate LLM agents and tool-using workflows—task success, tool accuracy, latency/cost, safety, and regression suites. Use when shipping agent features, comparing prompts/models, or debugging agent failures. Triggers: "agent eval", "benchmark agent", "tool accuracy", "agent regression".
- Author
- charlieviettq ↗
- Repository
- awesome-agent-skill ↗
- Licence
- MIT
- Last repository change
- 20.07.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 20.07.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Evaluate LLM agents and tool-using workflows—task success, tool accuracy, latency/cost, safety, and regression suites. Use when shipping agent features, comparing prompts/models, or debugging agent failures. Triggers: "agent eval", "benchmark agent", "tool accuracy", "agent regression".
Triggers from the description
Original author description
Evaluate LLM agents and tool-using workflows—task success, tool accuracy, latency/cost, safety, and regression suites. Use when shipping agent features, comparing prompts/models, or debugging agent failures. Triggers: "agent eval", "benchmark agent", "tool accuracy", "agent regression".
How it works
- What to measure
- Workflow
- Automated checks
- Human rubric (when needed)
- Anti-patterns
- Output
SKILL.md preview
--- name: agent-evaluation description: > Evaluate LLM agents and tool-using workflows—task success, tool accuracy, latency/cost, safety, and regression suites. Use when shipping agent features, comparing prompts/models, or debugging agent failures. Triggers: "agent eval", "benchmark agent", "tool accuracy", "agent regression". --- # Agent evaluation ## What to measure | Dimension | Examples | |-----------|----------| | Task success | End state matches spec (binary or rubric) | | Tool use | Correct tool, valid args, no spurious calls | | Safety | No policy violations, no secret leakage | | Efficiency | Tokens, latency, tool call count | | Stability | Same input -> consistent outcome across runs | ## Workflow 1. **Define tasks** — realistic user intents with clear pass/fail or scored rubric. 2. **Build dataset** — golden set + edge cases (errors, ambiguous input, empty context). 3. **Run baseline** — fixed model/settings; log traces (inputs, tools, outputs). 4. **Score** — automated checks first; human review for ambiguous cases. 5. **Compare** — A/B prompts, models, or tool schemas; report deltas with confidence notes. 6. **Gate** — block release on regression in must-pass tasks. ## Automated checks - Schema validation on tool arguments. - Assert final answer contains required fields or avoids forbidden content. - Snapshot tests for deterministic sub-steps where possible. ## Human rubric (when needed) Score 1-5 on: correctness, completeness, tone, safety. Document disagreements. ## Anti-patterns - Eval only on cherry-picked happy paths. - Changing task and model simultaneously without isolation. - No trace logs when debugging tool failures. ## Output Summary table: variant | success rate | avg tools | avg latency | notes.
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · charlieviettq/awesome-agent-skill | 100 | MIT |
| FrancoStino/opencode-skills-collection | 100 | MIT |
| helx0071/skillsmo | 100 | MIT |
| alebgl77/claude-inc | 100 | MIT |
| guia-matthieu/clawfu-skills | 100 | MIT |
| seb1n/awesome-ai-agent-skills | 100 | MIT |
| viktorbezdek/skillstack | 100 | MIT |
| mlflow/skills | 95 | Apache-2.0 |
| NeoLabHQ/context-engineering-kit | 91 | GPL-3.0 |
Identical content appears in 1 other GitHub folders. We show one copy with the highest score.
Database snapshot: 13.11.2026 · package version c0429053f4d0 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100agent-evaluation-and-guardrails
Guides building evaluation harnesses, regression test suites, and runtime guardrails for LLM agents. Use when a user asks to "evaluate this agent," "write test cases for a prompt change," "set up an eval harness," "add guardrails to prevent unsafe output," "detect prompt… - 100llm-evals
Builds evaluation suites for LLM features and agents - error analysis on real outputs, pass criteria, code-based checks, calibrated LLM-as-judge graders, dataset splits, CI regression runs, and noise-aware comparisons with cost and latency. Use when testing prompts or models,… - 100agent-harness
Test and evaluation harness for AI agents — scenario suites, deterministic replay, regression diffing, cost and latency budgets. Use when agent quality is vibe-checked, before shipping a prompt or model change, or when evals drift. - 100agent-evaluation-harness
Design an evaluation harness for an LLM agent before shipping it. Use when the user is building or rewriting an agent, deciding ship/no-ship, debugging regressions, or mentions golden sets, eval suites, regression tests, trace-level evals, LLM-as-judge, scoring rubrics, or asks… - 100evaluate-agent
Sets up evaluation of AI agents using Dokimos. Use this skill when the user wants to evaluate, test, or benchmark an AI agent that uses tools, or assess tool call correctness, task completion, argument hallucination, or tool definition quality. Also use when the user mentions… - 90skill-evaluation-workbench
Use when designing, running, debugging, or hardening deterministic eval suites for agent skills, prompts, tool workflows, or MCP-backed cases.