Skip to content
Skills Library
AI & agents · Testing

optimize-skill

/optimize-skill

Eval-driven skill optimization. Use when skill needs improvement — wrong tool selection, missing use cases, verbose instructions, stale rules. Structured eval cycles with parallel subagents measure skill quality, iterate until 100% pass rate.

Repository
tearags-mcp ↗
Licence
MIT
Last repository change
18.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 18.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Eval-driven skill optimization. Use when skill needs improvement — wrong tool selection, missing use cases, verbose instructions, stale rules. Structured eval cycles with parallel subagents measure skill quality, iterate until 100% pass rate.

Arguments

/optimize-skill [skill path or name to optimize]
Original author description

Eval-driven skill optimization. Use when skill needs improvement — wrong tool selection, missing use cases, verbose instructions, stale rules. Structured eval cycles with parallel subagents measure skill quality, iterate until 100% pass rate.

How it works

  1. Non-Negotiables (MUST honor in every run)
  2. Top Anti-Patterns (DO NOT)
  3. Prerequisites
  4. Phase 1: AUDIT
  5. Phase 2: BASELINE EVAL (before fixing)
  6. Phase 3: FIX
  7. Phase 4: VERIFY
  8. Phase 5: ITERATE (if needed)
  9. Phase 6: PERSIST (MANDATORY — execute inline, not deferred)
  10. Key Principles
Original SKILL.md headings · 297 lines, 1,602 words
SKILL.md preview
---
name: optimize-skill
description:
  Eval-driven skill optimization. Use when skill needs improvement — wrong tool
  selection, missing use cases, verbose instructions, stale rules. Structured
  eval cycles with parallel subagents measure skill quality, iterate until 100%
  pass rate.
argument-hint: [skill path or name to optimize]
---

# Optimize Skill

## Non-Negotiables (MUST honor in every run)

- **PERSIST results to .benchmarks/<skill-name>/** after Phase 4/5. Skipping =
  work unverifiable.
- **MEASURE before fixing** — Phase 2 baseline eval confirms problems exist
  before any change.
- **DELTA is the metric** — skill scoring 100% but baseline 90% adds little.
  Target +50pp minimum.
- **NEVER skip Phase 6** — benchmark.md and evals.json are committed artifacts;
  without them optimization unrepeatable.

🛑 STOP — these survive over individual phase sequence. Cross-check every Phase
4/5 completion against Non-Negotiables.

## Top Anti-Patterns (DO NOT)

- **Skipping PERSIST** — Phase 6 not optional, execute inline immediately after
  verify
- **Fixing before measuring** — audit findings are hypotheses; eval-confirm
  first
- **Skipping baseline** — without it, can't measure delta

Structured eval-driven methodology for optimizing Claude Code skills. Measures
actual agent behavior against expected tool selection, identifies gaps, fixes
them, verifies fixes.

## Prerequisites

1. **Load skill-creator toolkit** — invoke `/example-skills:skill-creator` via
   Skill tool BEFORE any phase. Provides scripts, agents, eval-viewer used in
   Phases 6-8. Keep loaded throughout session.
2. Skill file to optimize (SKILL.md path)
3. Understanding what skill should do (read it first)

## Phase 1: AUDIT

Read skill. Identify problems by category:

| Category              | What to look for                                               |
| --------------------- | -------------------------------------------------------------- |
| **Stale rules**       | References to removed tools, old patterns, outdated stats      |
| **Conflicts**         | Contradicts search-cascade or other skills                     |
| **Verbosity**         | Repeated instructions, keyword lists that could be tables      |
| **Missing use cases** | User intents the skill doesn't handle                          |
| **Wrong routing**     | Skill sends agent to wrong tool for a given intent             |
| **Anti-patterns**     | Read after search, ripgrep for semantic queries, Glob for code |

Present findings as numbered list. Get user approval on each finding (interview
style, one at a time, with recommended resolution).

## Phase 2: BASELINE EVAL (before fixing)

**Measure first, fix second.** Audit findings are hypotheses — eval confirms. Do
NOT apply fixes until baseline proves skill actually broken.

### 2.1 Design eval cases

Create eval cases targeting:

1. **Each audit finding** — one case per problem. Case FAILS if problem exists,
   PASSES if skill handles correctly
2. **Regression controls** — unchanged behaviors that must still work
3. **Edge cases** — non-English input, code snippet input, ambiguous intent
4. **Subagent routing (MANDATORY)** — ALWAYS include 2+ cases simulating skill
   invocation from different agent contexts:
   - `[subagent: explore]` — skill called during codebase exploration
   - `[subagent: task-agent]` — skill called during autonomous task execution
   - `[subagent: plan-executor]` — skill called as a plan step
   - `[subagent: other]` — any other agent type that might use the skill These
     cases test whether skill instructions work when calling agent has limited
     context (no CLAUDE.md, no rules, no search-cascade). Frame prompts as
     subagent would phrase them — explicit paths, not vague intents.

Each case has:

```
Eval-N: "<user prompt>"
Expected: <tool sequence>. NOT <wrong tools>.
Failure mode: <what we're testing against>
Audit finding: <N or "control">
```

**Minimum 8 cases.** Balance: ~50% audit findings, ~30% controls, ~20% edges.

### 2.2 Run with-rule eval

Spawn ONE subagent with **full skill text injected** into prompt. Present ALL
eval cases in single prompt. Agent describes tool selection plan per case — does
NOT execute tools.

Grade each case: PASS / FAIL against expected behavior.

### 2.3 Run without-rule baseline

Spawn ONE subagent with NO skill text — only list of available MCP tools. Same
cases. Agent describes natural tool selection.

Establishes **delta** — how much skill improves behavior.

### 2.4 Grade and triage

```
With-rule:    N/M PASS (X%)
Without-rule: N/M PASS (Y%)
Delta:        +Zpp
```

**Triage audit findings by eval results:**

- Finding FAILS in with-rule eval → **confirmed problem**, proceed to fix
- Finding PASSES in with-rule eval → **not broken**, drop from fix list
- Finding PASSES in both with-rule and without-rule → **skill adds no value
  here**, consider if instruction is dead weight

Present triage to user. Only confirmed problems proceed to Phase 3.

## Phase 3: FIX

Apply fixes ONLY for confirmed problems (eval-proven failures from Phase 2).

Track:

- Lines before/after (target: -30% or more for verbose skills)
- Each fix with reason and linked eval case

## Phase 4: VERIFY

Re-run full eval suite against fixed skill.

### 4.1 Run with-rule eval (fixed skill)

Same cases from Phase 2. All previously failing cases must now PASS. Previously
passing cases must not regress.

### 4.2 Grade

```
Before fix:   N/M PASS (X%)
After fix:    N/M PASS (X'%)
Baseline:     N/M PASS (Y%)
Delta:        +Zpp
```

**Target: 100% with-rule pass rate.** Not met → iterate (Phase 5).

## Phase 5: ITERATE (if needed)

For each remaining FAIL:

1. Identify **why** agent chose wrong — ambiguous instruction? missing rule?
   conflicting guidance?
2. Fix the specific instruction
3. Re-run ONLY failed cases + 2 random controls (not full suite)
4. Repeat until 100%

**Max 3 iterations.** Still failing after 3 → skill design needs rethinking, not
tweaking. Report to user.

## Phase 6: PERSIST (MANDATORY — execute inline, not deferred)

**NOT optional.** Save results immediately after Phase 4/5. Don't wait for user
to ask. Don't skip because "session is ending". Benchmark artifacts = proof
optimization happened — without them, work unverifiable.

**Applies to feature-driven updates too.** Skill updated as part of feature
implementation (not standalone optimization) → eval cases and results MUST still
persist. Eval may be smaller (only new functionality), but
`evals/<feature>-evals.json` + benchmark.md appendix required. Eval cases =
reusable regression tests — skipping persistence means next optimization has no
baseline for new functionality.

Save results to `.claude-plugin/.benchmarks/<skill-name>/`:

```
.benchmarks/<skill-name>/
├── benchmark.md        — committed. Permanent optimization record.
├── evals.json          — committed. Eval cases + assertions + results.
└── workspace/          — gitignored. Recreatable by this skill.
    ├── skill-snapshot/  — frozen copy of skill before/after optimization
    └── iteration-N/    — subagent outputs per iteration
```

**Execute these steps in order:**

1. **Create workspace/** dir (gitignored —
   `.claude-plugin/.benchmarks/**/workspace/`)
2. **Save evals.json** to `.benchmarks/<skill-name>/evals.json` (NOT inside
   workspace/) — must include ALL eval cases with: prompt, expected output,
   assertions, audit finding references, and final pass/fail results per
   iteration. This file is committed — eval cases are reusable test artifacts.
3. **Copy SKILL.md** before and after to `workspace/skill-snapshot/`
4. **Write benchmark.md** (committed) with: summary, changes table, key design
   decisions, metrics (lines before/after), iterations table (pass rates per
   iteration), per-eval detail table (prompt + before/after grade), integration
   test results if Phase 7/8 was run
5. **Update README.md** in `.benchmarks/` with new skill row

Present final state to user:

```
Skill: [name]
Lines: [before] → [after] ([delta]%)
Eval cases: [N] (with-rule [X]%, baseline [Y]%, delta +[Z]pp)
Iterations: [count]
Benchmark: .claude-plugin/.benchmarks/[name]/benchmark.md
```

## Key Principles

**Eval tests INSTRUCTION QUALITY, not tool correctness.** Verify skill text
unambiguously guides agent to right tool. Whether tool itself works = separate
concern (integration tests).

**Parallel subagents for with/without.** Always run both in parallel when
establishing baseline. Re-runs (iterations) only need with-rule.

**One subagent per eval suite.** All cases in one prompt — cheaper and shows
whether instructions scale when agent handles multiple intents in context.

**Delta is the metric.** Skill scoring 100% but baseline also 90% adds little
value. Target: +50pp minimum delta.

**Minimum viable eval.** Don't over-test. 8-15 cases covers most skills. Add
cases only for discovered failure modes.

## Phase 7: DESCRIPTION OPTIMIZATION (optional)

After skill body stable, optimize `description` field in frontmatter for better
triggering accuracy. Follow **Description Optimization** section from
`/example-skills:skill-creator` (loaded in Prerequisites step 1):

1. Generate 20 trigger eval queries (10 should-trigger, 10 should-not-trigger)
2. Review with user via HTML template (`assets/eval_review.html` from
   skill-creator)
3. Run optimization loop from the skill-creator directory:
   ```bash
   python -m scripts.run_loop \
     --eval-set <path-to-trigger-eval.json> \
     --skill-path <path-to-skill> \
     --model <model-id> \
     --max-iterations 5 --verbose
   ```
4. Apply `best_description` to SKILL.md frontmatter

Separate from body optimization — description controls **when** skill triggers,
body controls **what it does** once triggered.

## Phase 8: FULL INTEGRATION EVAL (optional)

When lightweight tool-selection eval not enough (complex multi-step skills,
skills producing files), follow **Running and evaluating test cases** section
from `/example-skills:skill-creator` (loaded in Prerequisites step 1):

1. Save eval cases to `evals/evals.json` (see skill-creator's
   `references/schemas.md` for schema)
2. Spawn with-skill and baseline subagents that **execute the skill** on real
   tasks
3. Save outputs to `<skill-name>-workspace/iteration-N/`
4. Grade using `agents/grader.md` from skill-creator, then review via:
   ```bash
   python <skill-creator-path>/eval-viewer/generate_review.py \
     <workspace>/iteration-N \
     --skill-name "<name>" \
     --benchmark <workspace>/iteration-N/benchmark.json
   ```
5. Aggregate benchmarks:
   ```bash
   python -m scripts.aggregate_benchmark <workspace>/iteration-N \
     --skill-name <name>
   ```

Key difference from Phase 2: integration eval tests **end-to-end behavior**
(tool calls, outputs, quality). Phase 2 tests **instruction clarity** (does
agent know which tool to pick). Use Phase 2 first — faster. Escalate to Phase 8
only when tool selection correct but output quality uncertain.

## Anti-patterns

Top 3 are duplicated near the top — read both.

- **Skipping PERSIST** — benchmark artifacts are proof of work. Without
  evals.json and benchmark.md, optimization unverifiable and unrepeatable.
  Execute Phase 6 inline, immediately after verify passes
- **Fixing before measuring** — audit findings are hypotheses. Run baseline eval
  (Phase 2) to confirm problems exist before applying fixes. A finding that
  passes eval is not broken — drop from fix list
- **Running real MCP calls in Phase 2 eval** — unnecessary, tests tool not skill
- **One subagent per eval case** — wasteful, use one subagent for all cases
- **Skipping baseline** — can't measure delta without it
- **Fixing skill without re-eval** — any fix can break other cases
- **Inflating eval count** — 20+ cases with no new failure modes is waste
- **Skipping Phase 2, jumping to Phase 8** — Phase 8 is 10x slower. Always start
  with lightweight tool-selection eval

Versions

RepositoryScoreLicence
This page · artk0de/tearags-mcp100MIT
Bennyoooo/Airbot100MIT

Database snapshot: 13.11.2026 · package version 04b48c41ca82 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100skill-eval
    Runs and analyzes eval test cases for Claude skills. Use when asked to "test a skill", "check if this skill works", "run evals", "verify skill quality", or after creating/modifying a skill.
  • 100eval-optimize
    Automated skill improvement loop. Runs eval, identifies judge failures, reads traces and rationale, edits the SKILL.md to fix issues, re-runs to verify, and checks for regressions. Use when the user wants to automatically improve a skill based on eval results, fix failing…
  • 100skill-evals
    Run the eval loop that measures whether a skill actually improves an agent's output — design test cases, run them with the skill and against a baseline, grade against assertions, aggregate a benchmark, analyze patterns, and iterate. Use when the user wants to evaluate or test a…
  • 90ck:skill-creator
    Create or update Claude skills with eval-driven iteration. Use for new skills, skill scripts, references, benchmark optimization, description optimization, eval testing, extending Claude's capabilities.
  • 91stack-ai-agent
    Conventions for the AI agent stack (MCP tools per task, eval cases per tool, offline evals in the gate). Use when adding an MCP tool, an eval case, a prompt or a model call in this repo.
  • 95skill-optimizer
    Guide creating and reviewing skills that extend Claude with specialized knowledge, workflows, or tool integrations. Covers SKILL.md structure, frontmatter quality, trigger design, eval pipelines, and packaging. Triggers on create a skill, review this skill, new skill, optimize skill, skill quality.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.