optimize-skill
/optimize-skill
Eval-driven skill optimization. Use when skill needs improvement — wrong tool selection, missing use cases, verbose instructions, stale rules. Structured eval cycles with parallel subagents measure skill quality, iterate until 100% pass rate.
- Author
- artk0de ↗
- Repository
- tearags-mcp ↗
- Licence
- MIT
- Last repository change
- 18.10.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 18.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Eval-driven skill optimization. Use when skill needs improvement — wrong tool selection, missing use cases, verbose instructions, stale rules. Structured eval cycles with parallel subagents measure skill quality, iterate until 100% pass rate.
Arguments
/optimize-skill [skill path or name to optimize]
Original author description
Eval-driven skill optimization. Use when skill needs improvement — wrong tool selection, missing use cases, verbose instructions, stale rules. Structured eval cycles with parallel subagents measure skill quality, iterate until 100% pass rate.
How it works
- Non-Negotiables (MUST honor in every run)
- Top Anti-Patterns (DO NOT)
- Prerequisites
- Phase 1: AUDIT
- Phase 2: BASELINE EVAL (before fixing)
- Phase 3: FIX
- Phase 4: VERIFY
- Phase 5: ITERATE (if needed)
- Phase 6: PERSIST (MANDATORY — execute inline, not deferred)
- Key Principles
SKILL.md preview
---
name: optimize-skill
description:
Eval-driven skill optimization. Use when skill needs improvement — wrong tool
selection, missing use cases, verbose instructions, stale rules. Structured
eval cycles with parallel subagents measure skill quality, iterate until 100%
pass rate.
argument-hint: [skill path or name to optimize]
---
# Optimize Skill
## Non-Negotiables (MUST honor in every run)
- **PERSIST results to .benchmarks/<skill-name>/** after Phase 4/5. Skipping =
work unverifiable.
- **MEASURE before fixing** — Phase 2 baseline eval confirms problems exist
before any change.
- **DELTA is the metric** — skill scoring 100% but baseline 90% adds little.
Target +50pp minimum.
- **NEVER skip Phase 6** — benchmark.md and evals.json are committed artifacts;
without them optimization unrepeatable.
🛑 STOP — these survive over individual phase sequence. Cross-check every Phase
4/5 completion against Non-Negotiables.
## Top Anti-Patterns (DO NOT)
- **Skipping PERSIST** — Phase 6 not optional, execute inline immediately after
verify
- **Fixing before measuring** — audit findings are hypotheses; eval-confirm
first
- **Skipping baseline** — without it, can't measure delta
Structured eval-driven methodology for optimizing Claude Code skills. Measures
actual agent behavior against expected tool selection, identifies gaps, fixes
them, verifies fixes.
## Prerequisites
1. **Load skill-creator toolkit** — invoke `/example-skills:skill-creator` via
Skill tool BEFORE any phase. Provides scripts, agents, eval-viewer used in
Phases 6-8. Keep loaded throughout session.
2. Skill file to optimize (SKILL.md path)
3. Understanding what skill should do (read it first)
## Phase 1: AUDIT
Read skill. Identify problems by category:
| Category | What to look for |
| --------------------- | -------------------------------------------------------------- |
| **Stale rules** | References to removed tools, old patterns, outdated stats |
| **Conflicts** | Contradicts search-cascade or other skills |
| **Verbosity** | Repeated instructions, keyword lists that could be tables |
| **Missing use cases** | User intents the skill doesn't handle |
| **Wrong routing** | Skill sends agent to wrong tool for a given intent |
| **Anti-patterns** | Read after search, ripgrep for semantic queries, Glob for code |
Present findings as numbered list. Get user approval on each finding (interview
style, one at a time, with recommended resolution).
## Phase 2: BASELINE EVAL (before fixing)
**Measure first, fix second.** Audit findings are hypotheses — eval confirms. Do
NOT apply fixes until baseline proves skill actually broken.
### 2.1 Design eval cases
Create eval cases targeting:
1. **Each audit finding** — one case per problem. Case FAILS if problem exists,
PASSES if skill handles correctly
2. **Regression controls** — unchanged behaviors that must still work
3. **Edge cases** — non-English input, code snippet input, ambiguous intent
4. **Subagent routing (MANDATORY)** — ALWAYS include 2+ cases simulating skill
invocation from different agent contexts:
- `[subagent: explore]` — skill called during codebase exploration
- `[subagent: task-agent]` — skill called during autonomous task execution
- `[subagent: plan-executor]` — skill called as a plan step
- `[subagent: other]` — any other agent type that might use the skill These
cases test whether skill instructions work when calling agent has limited
context (no CLAUDE.md, no rules, no search-cascade). Frame prompts as
subagent would phrase them — explicit paths, not vague intents.
Each case has:
```
Eval-N: "<user prompt>"
Expected: <tool sequence>. NOT <wrong tools>.
Failure mode: <what we're testing against>
Audit finding: <N or "control">
```
**Minimum 8 cases.** Balance: ~50% audit findings, ~30% controls, ~20% edges.
### 2.2 Run with-rule eval
Spawn ONE subagent with **full skill text injected** into prompt. Present ALL
eval cases in single prompt. Agent describes tool selection plan per case — does
NOT execute tools.
Grade each case: PASS / FAIL against expected behavior.
### 2.3 Run without-rule baseline
Spawn ONE subagent with NO skill text — only list of available MCP tools. Same
cases. Agent describes natural tool selection.
Establishes **delta** — how much skill improves behavior.
### 2.4 Grade and triage
```
With-rule: N/M PASS (X%)
Without-rule: N/M PASS (Y%)
Delta: +Zpp
```
**Triage audit findings by eval results:**
- Finding FAILS in with-rule eval → **confirmed problem**, proceed to fix
- Finding PASSES in with-rule eval → **not broken**, drop from fix list
- Finding PASSES in both with-rule and without-rule → **skill adds no value
here**, consider if instruction is dead weight
Present triage to user. Only confirmed problems proceed to Phase 3.
## Phase 3: FIX
Apply fixes ONLY for confirmed problems (eval-proven failures from Phase 2).
Track:
- Lines before/after (target: -30% or more for verbose skills)
- Each fix with reason and linked eval case
## Phase 4: VERIFY
Re-run full eval suite against fixed skill.
### 4.1 Run with-rule eval (fixed skill)
Same cases from Phase 2. All previously failing cases must now PASS. Previously
passing cases must not regress.
### 4.2 Grade
```
Before fix: N/M PASS (X%)
After fix: N/M PASS (X'%)
Baseline: N/M PASS (Y%)
Delta: +Zpp
```
**Target: 100% with-rule pass rate.** Not met → iterate (Phase 5).
## Phase 5: ITERATE (if needed)
For each remaining FAIL:
1. Identify **why** agent chose wrong — ambiguous instruction? missing rule?
conflicting guidance?
2. Fix the specific instruction
3. Re-run ONLY failed cases + 2 random controls (not full suite)
4. Repeat until 100%
**Max 3 iterations.** Still failing after 3 → skill design needs rethinking, not
tweaking. Report to user.
## Phase 6: PERSIST (MANDATORY — execute inline, not deferred)
**NOT optional.** Save results immediately after Phase 4/5. Don't wait for user
to ask. Don't skip because "session is ending". Benchmark artifacts = proof
optimization happened — without them, work unverifiable.
**Applies to feature-driven updates too.** Skill updated as part of feature
implementation (not standalone optimization) → eval cases and results MUST still
persist. Eval may be smaller (only new functionality), but
`evals/<feature>-evals.json` + benchmark.md appendix required. Eval cases =
reusable regression tests — skipping persistence means next optimization has no
baseline for new functionality.
Save results to `.claude-plugin/.benchmarks/<skill-name>/`:
```
.benchmarks/<skill-name>/
├── benchmark.md — committed. Permanent optimization record.
├── evals.json — committed. Eval cases + assertions + results.
└── workspace/ — gitignored. Recreatable by this skill.
├── skill-snapshot/ — frozen copy of skill before/after optimization
└── iteration-N/ — subagent outputs per iteration
```
**Execute these steps in order:**
1. **Create workspace/** dir (gitignored —
`.claude-plugin/.benchmarks/**/workspace/`)
2. **Save evals.json** to `.benchmarks/<skill-name>/evals.json` (NOT inside
workspace/) — must include ALL eval cases with: prompt, expected output,
assertions, audit finding references, and final pass/fail results per
iteration. This file is committed — eval cases are reusable test artifacts.
3. **Copy SKILL.md** before and after to `workspace/skill-snapshot/`
4. **Write benchmark.md** (committed) with: summary, changes table, key design
decisions, metrics (lines before/after), iterations table (pass rates per
iteration), per-eval detail table (prompt + before/after grade), integration
test results if Phase 7/8 was run
5. **Update README.md** in `.benchmarks/` with new skill row
Present final state to user:
```
Skill: [name]
Lines: [before] → [after] ([delta]%)
Eval cases: [N] (with-rule [X]%, baseline [Y]%, delta +[Z]pp)
Iterations: [count]
Benchmark: .claude-plugin/.benchmarks/[name]/benchmark.md
```
## Key Principles
**Eval tests INSTRUCTION QUALITY, not tool correctness.** Verify skill text
unambiguously guides agent to right tool. Whether tool itself works = separate
concern (integration tests).
**Parallel subagents for with/without.** Always run both in parallel when
establishing baseline. Re-runs (iterations) only need with-rule.
**One subagent per eval suite.** All cases in one prompt — cheaper and shows
whether instructions scale when agent handles multiple intents in context.
**Delta is the metric.** Skill scoring 100% but baseline also 90% adds little
value. Target: +50pp minimum delta.
**Minimum viable eval.** Don't over-test. 8-15 cases covers most skills. Add
cases only for discovered failure modes.
## Phase 7: DESCRIPTION OPTIMIZATION (optional)
After skill body stable, optimize `description` field in frontmatter for better
triggering accuracy. Follow **Description Optimization** section from
`/example-skills:skill-creator` (loaded in Prerequisites step 1):
1. Generate 20 trigger eval queries (10 should-trigger, 10 should-not-trigger)
2. Review with user via HTML template (`assets/eval_review.html` from
skill-creator)
3. Run optimization loop from the skill-creator directory:
```bash
python -m scripts.run_loop \
--eval-set <path-to-trigger-eval.json> \
--skill-path <path-to-skill> \
--model <model-id> \
--max-iterations 5 --verbose
```
4. Apply `best_description` to SKILL.md frontmatter
Separate from body optimization — description controls **when** skill triggers,
body controls **what it does** once triggered.
## Phase 8: FULL INTEGRATION EVAL (optional)
When lightweight tool-selection eval not enough (complex multi-step skills,
skills producing files), follow **Running and evaluating test cases** section
from `/example-skills:skill-creator` (loaded in Prerequisites step 1):
1. Save eval cases to `evals/evals.json` (see skill-creator's
`references/schemas.md` for schema)
2. Spawn with-skill and baseline subagents that **execute the skill** on real
tasks
3. Save outputs to `<skill-name>-workspace/iteration-N/`
4. Grade using `agents/grader.md` from skill-creator, then review via:
```bash
python <skill-creator-path>/eval-viewer/generate_review.py \
<workspace>/iteration-N \
--skill-name "<name>" \
--benchmark <workspace>/iteration-N/benchmark.json
```
5. Aggregate benchmarks:
```bash
python -m scripts.aggregate_benchmark <workspace>/iteration-N \
--skill-name <name>
```
Key difference from Phase 2: integration eval tests **end-to-end behavior**
(tool calls, outputs, quality). Phase 2 tests **instruction clarity** (does
agent know which tool to pick). Use Phase 2 first — faster. Escalate to Phase 8
only when tool selection correct but output quality uncertain.
## Anti-patterns
Top 3 are duplicated near the top — read both.
- **Skipping PERSIST** — benchmark artifacts are proof of work. Without
evals.json and benchmark.md, optimization unverifiable and unrepeatable.
Execute Phase 6 inline, immediately after verify passes
- **Fixing before measuring** — audit findings are hypotheses. Run baseline eval
(Phase 2) to confirm problems exist before applying fixes. A finding that
passes eval is not broken — drop from fix list
- **Running real MCP calls in Phase 2 eval** — unnecessary, tests tool not skill
- **One subagent per eval case** — wasteful, use one subagent for all cases
- **Skipping baseline** — can't measure delta without it
- **Fixing skill without re-eval** — any fix can break other cases
- **Inflating eval count** — 20+ cases with no new failure modes is waste
- **Skipping Phase 2, jumping to Phase 8** — Phase 8 is 10x slower. Always start
with lightweight tool-selection eval
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · artk0de/tearags-mcp | 100 | MIT |
| Bennyoooo/Airbot | 100 | MIT |
Database snapshot: 13.11.2026 · package version 04b48c41ca82 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100skill-eval
Runs and analyzes eval test cases for Claude skills. Use when asked to "test a skill", "check if this skill works", "run evals", "verify skill quality", or after creating/modifying a skill. - 100eval-optimize
Automated skill improvement loop. Runs eval, identifies judge failures, reads traces and rationale, edits the SKILL.md to fix issues, re-runs to verify, and checks for regressions. Use when the user wants to automatically improve a skill based on eval results, fix failing… - 100skill-evals
Run the eval loop that measures whether a skill actually improves an agent's output — design test cases, run them with the skill and against a baseline, grade against assertions, aggregate a benchmark, analyze patterns, and iterate. Use when the user wants to evaluate or test a… - 90ck:skill-creator
Create or update Claude skills with eval-driven iteration. Use for new skills, skill scripts, references, benchmark optimization, description optimization, eval testing, extending Claude's capabilities. - 91stack-ai-agent
Conventions for the AI agent stack (MCP tools per task, eval cases per tool, offline evals in the gate). Use when adding an MCP tool, an eval case, a prompt or a model call in this repo. - 95skill-optimizer
Guide creating and reviewing skills that extend Claude with specialized knowledge, workflows, or tool integrations. Covers SKILL.md structure, frontmatter quality, trigger design, eval pipelines, and packaging. Triggers on create a skill, review this skill, new skill, optimize skill, skill quality.