Skip to content
Skills Library
AI & agents · Analysis

agent-eval-trigger-accuracy

/agent-eval-trigger-accuracy

Measure whether a skill activates when it should and stays quiet when it should not (precision, recall, F1, collisions with sibling and distractor skills). Use when the user asks why a skill does not trigger, triggers too often, or how to improve its description. Not for task success.

Licence
MIT
Last repository change
04.11.2026

Assessment and security

86/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content21 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 04.11.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Measure whether a skill activates when it should and stays quiet when it should not (precision, recall, F1, collisions with sibling and distractor skills). Use when the user asks why a skill does not trigger, triggers too often, or how to improve its description. Not for task success.

Original author description

Measure whether a skill activates when it should and stays quiet when it should not (precision, recall, F1, collisions with sibling and distractor skills). Use when the user asks why a skill does not trigger, triggers too often, or how to improve its description. Not for task success.

How it works

SKILL.md has no level-two headings · 9 lines, 160 words

SKILL.md preview
---
name: agent-eval-trigger-accuracy
description: Measure whether a skill activates when it should and stays quiet when it should not (precision, recall, F1, collisions with sibling and distractor skills). Use when the user asks why a skill does not trigger, triggers too often, or how to improve its description. Not for task success.
license: MIT
---

# Trigger accuracy (weight 9%)

1. Prompt set: `<skill>/evals/trigger_prompts.yaml` (`positive`, `negative`, `near_miss` lists) if the author has one. Otherwise the host generates them, 30 per skill when sampled and up to 60 otherwise.
2. Run `agent-eval evaluate <src> --only trigger-accuracy --yes --provider host --host-dir <run>/host --out <run>/out`, then agent-eval-host-runner. A router prompt lists the target skill, its siblings in the same plugin, and 15 bundled distractor skills, and asks which one should handle each request.
3. Metrics per skill: precision (good ≥ 0.9), recall (good ≥ 0.85), F1, collision rate (positives routed to another skill, good ≤ 5%). Score = 5 × mean F1.
4. Without host answers the engine reports a static proxy (description has what/when/when-not, specificity, lexical overlap with siblings and distractors) at low confidence. Label it as a proxy.
5. Fixes to suggest: add the user phrasings that missed, add "not for X (use Y)" clauses for collisions, and keep name plus description under about 100 tokens.

Versions

The database has no other skills named agent-eval-trigger-accuracy.

Database snapshot: 13.11.2026 · package version 08a422e626c0 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 82skill-creator
    Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description /…
  • 100skill-eval
    Measure whether a skill actually helps: build a `claude plugin eval` suite, run it with and without the skill under a required spend cap, and read the delta. Use when adding evals to a skill, converting evals.json, re-checking skills after a model upgrade, or deciding if a skill…
  • 100hk-skill-creator
    Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's…
  • 100skill-center-creator
    Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's…
  • 100skill-eval
    Measure whether a skill helps by comparing runs with and without it. Use when: reading skill A/B results or deciding to keep, revise or remove one.
  • 100using-agent-skills
    发现并调用 agent skills。用于开始会话,或需要判断当前任务适用哪个 skill 时。这个 meta-skill 负责治理所有其他 skills 的发现与调用方式。

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.