agent-eval-trigger-accuracy
/agent-eval-trigger-accuracy
Measure whether a skill activates when it should and stays quiet when it should not (precision, recall, F1, collisions with sibling and distractor skills). Use when the user asks why a skill does not trigger, triggers too often, or how to improve its description. Not for task success.
- Author
- SayandipBagchi ↗
- Repository
- agent-evaluator ↗
- Licence
- MIT
- Last repository change
- 04.11.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 04.11.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Measure whether a skill activates when it should and stays quiet when it should not (precision, recall, F1, collisions with sibling and distractor skills). Use when the user asks why a skill does not trigger, triggers too often, or how to improve its description. Not for task success.
Original author description
Measure whether a skill activates when it should and stays quiet when it should not (precision, recall, F1, collisions with sibling and distractor skills). Use when the user asks why a skill does not trigger, triggers too often, or how to improve its description. Not for task success.
How it works
SKILL.md has no level-two headings · 9 lines, 160 words
SKILL.md preview
--- name: agent-eval-trigger-accuracy description: Measure whether a skill activates when it should and stays quiet when it should not (precision, recall, F1, collisions with sibling and distractor skills). Use when the user asks why a skill does not trigger, triggers too often, or how to improve its description. Not for task success. license: MIT --- # Trigger accuracy (weight 9%) 1. Prompt set: `<skill>/evals/trigger_prompts.yaml` (`positive`, `negative`, `near_miss` lists) if the author has one. Otherwise the host generates them, 30 per skill when sampled and up to 60 otherwise. 2. Run `agent-eval evaluate <src> --only trigger-accuracy --yes --provider host --host-dir <run>/host --out <run>/out`, then agent-eval-host-runner. A router prompt lists the target skill, its siblings in the same plugin, and 15 bundled distractor skills, and asks which one should handle each request. 3. Metrics per skill: precision (good ≥ 0.9), recall (good ≥ 0.85), F1, collision rate (positives routed to another skill, good ≤ 5%). Score = 5 × mean F1. 4. Without host answers the engine reports a static proxy (description has what/when/when-not, specificity, lexical overlap with siblings and distractors) at low confidence. Label it as a proxy. 5. Fixes to suggest: add the user phrasings that missed, add "not for X (use Y)" clauses for collisions, and keep name plus description under about 100 tokens.
Versions
The database has no other skills named agent-eval-trigger-accuracy.
Database snapshot: 13.11.2026 · package version 08a422e626c0 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 82skill-creator
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description /… - 100skill-eval
Measure whether a skill actually helps: build a `claude plugin eval` suite, run it with and without the skill under a required spend cap, and read the delta. Use when adding evals to a skill, converting evals.json, re-checking skills after a model upgrade, or deciding if a skill… - 100hk-skill-creator
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's… - 100skill-center-creator
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's… - 100skill-eval
Measure whether a skill helps by comparing runs with and without it. Use when: reading skill A/B results or deciding to keep, revise or remove one. - 100using-agent-skills
发现并调用 agent skills。用于开始会话,或需要判断当前任务适用哪个 skill 时。这个 meta-skill 负责治理所有其他 skills 的发现与调用方式。