optimize-skill
/optimize-skill
Optymalizuje skill w cyklach ewaluacji: równoległe subagenty mierzą jakość, wskazują błędny wybór narzędzi, brakujące przypadki i rozwlekłe instrukcje, a iteracje trwają do 100 proc. zaliczeń.
- Autor
- artk0de ↗
- Repozytorium
- tearags-mcp ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 18.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 18.10.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Optymalizuje skill w cyklach ewaluacji: równoległe subagenty mierzą jakość, wskazują błędny wybór narzędzi, brakujące przypadki i rozwlekłe instrukcje, a iteracje trwają do 100 proc. zaliczeń.
Argumenty
/optimize-skill [skill path or name to optimize]
Oryginalny opis (EN)
Eval-driven skill optimization. Use when skill needs improvement — wrong tool selection, missing use cases, verbose instructions, stale rules. Structured eval cycles with parallel subagents measure skill quality, iterate until 100% pass rate.
Jak działa
- Non-Negotiables (MUST honor in every run)
- Top Anti-Patterns (DO NOT)
- Prerequisites
- Phase 1: AUDIT
- Phase 2: BASELINE EVAL (before fixing)
- Phase 3: FIX
- Phase 4: VERIFY
- Phase 5: ITERATE (if needed)
- Phase 6: PERSIST (MANDATORY — execute inline, not deferred)
- Key Principles
Podgląd SKILL.md
---
name: optimize-skill
description:
Eval-driven skill optimization. Use when skill needs improvement — wrong tool
selection, missing use cases, verbose instructions, stale rules. Structured
eval cycles with parallel subagents measure skill quality, iterate until 100%
pass rate.
argument-hint: [skill path or name to optimize]
---
# Optimize Skill
## Non-Negotiables (MUST honor in every run)
- **PERSIST results to .benchmarks/<skill-name>/** after Phase 4/5. Skipping =
work unverifiable.
- **MEASURE before fixing** — Phase 2 baseline eval confirms problems exist
before any change.
- **DELTA is the metric** — skill scoring 100% but baseline 90% adds little.
Target +50pp minimum.
- **NEVER skip Phase 6** — benchmark.md and evals.json are committed artifacts;
without them optimization unrepeatable.
🛑 STOP — these survive over individual phase sequence. Cross-check every Phase
4/5 completion against Non-Negotiables.
## Top Anti-Patterns (DO NOT)
- **Skipping PERSIST** — Phase 6 not optional, execute inline immediately after
verify
- **Fixing before measuring** — audit findings are hypotheses; eval-confirm
first
- **Skipping baseline** — without it, can't measure delta
Structured eval-driven methodology for optimizing Claude Code skills. Measures
actual agent behavior against expected tool selection, identifies gaps, fixes
them, verifies fixes.
## Prerequisites
1. **Load skill-creator toolkit** — invoke `/example-skills:skill-creator` via
Skill tool BEFORE any phase. Provides scripts, agents, eval-viewer used in
Phases 6-8. Keep loaded throughout session.
2. Skill file to optimize (SKILL.md path)
3. Understanding what skill should do (read it first)
## Phase 1: AUDIT
Read skill. Identify problems by category:
| Category | What to look for |
| --------------------- | -------------------------------------------------------------- |
| **Stale rules** | References to removed tools, old patterns, outdated stats |
| **Conflicts** | Contradicts search-cascade or other skills |
| **Verbosity** | Repeated instructions, keyword lists that could be tables |
| **Missing use cases** | User intents the skill doesn't handle |
| **Wrong routing** | Skill sends agent to wrong tool for a given intent |
| **Anti-patterns** | Read after search, ripgrep for semantic queries, Glob for code |
Present findings as numbered list. Get user approval on each finding (interview
style, one at a time, with recommended resolution).
## Phase 2: BASELINE EVAL (before fixing)
**Measure first, fix second.** Audit findings are hypotheses — eval confirms. Do
NOT apply fixes until baseline proves skill actually broken.
### 2.1 Design eval cases
Create eval cases targeting:
1. **Each audit finding** — one case per problem. Case FAILS if problem exists,
PASSES if skill handles correctly
2. **Regression controls** — unchanged behaviors that must still work
3. **Edge cases** — non-English input, code snippet input, ambiguous intent
4. **Subagent routing (MANDATORY)** — ALWAYS include 2+ cases simulating skill
invocation from different agent contexts:
- `[subagent: explore]` — skill called during codebase exploration
- `[subagent: task-agent]` — skill called during autonomous task execution
- `[subagent: plan-executor]` — skill called as a plan step
- `[subagent: other]` — any other agent type that might use the skill These
cases test whether skill instructions work when calling agent has limited
context (no CLAUDE.md, no rules, no search-cascade). Frame prompts as
subagent would phrase them — explicit paths, not vague intents.
Each case has:
```
Eval-N: "<user prompt>"
Expected: <tool sequence>. NOT <wrong tools>.
Failure mode: <what we're testing against>
Audit finding: <N or "control">
```
**Minimum 8 cases.** Balance: ~50% audit findings, ~30% controls, ~20% edges.
### 2.2 Run with-rule eval
Spawn ONE subagent with **full skill text injected** into prompt. Present ALL
eval cases in single prompt. Agent describes tool selection plan per case — does
NOT execute tools.
Grade each case: PASS / FAIL against expected behavior.
### 2.3 Run without-rule baseline
Spawn ONE subagent with NO skill text — only list of available MCP tools. Same
cases. Agent describes natural tool selection.
Establishes **delta** — how much skill improves behavior.
### 2.4 Grade and triage
```
With-rule: N/M PASS (X%)
Without-rule: N/M PASS (Y%)
Delta: +Zpp
```
**Triage audit findings by eval results:**
- Finding FAILS in with-rule eval → **confirmed problem**, proceed to fix
- Finding PASSES in with-rule eval → **not broken**, drop from fix list
- Finding PASSES in both with-rule and without-rule → **skill adds no value
here**, consider if instruction is dead weight
Present triage to user. Only confirmed problems proceed to Phase 3.
## Phase 3: FIX
Apply fixes ONLY for confirmed problems (eval-proven failures from Phase 2).
Track:
- Lines before/after (target: -30% or more for verbose skills)
- Each fix with reason and linked eval case
## Phase 4: VERIFY
Re-run full eval suite against fixed skill.
### 4.1 Run with-rule eval (fixed skill)
Same cases from Phase 2. All previously failing cases must now PASS. Previously
passing cases must not regress.
### 4.2 Grade
```
Before fix: N/M PASS (X%)
After fix: N/M PASS (X'%)
Baseline: N/M PASS (Y%)
Delta: +Zpp
```
**Target: 100% with-rule pass rate.** Not met → iterate (Phase 5).
## Phase 5: ITERATE (if needed)
For each remaining FAIL:
1. Identify **why** agent chose wrong — ambiguous instruction? missing rule?
conflicting guidance?
2. Fix the specific instruction
3. Re-run ONLY failed cases + 2 random controls (not full suite)
4. Repeat until 100%
**Max 3 iterations.** Still failing after 3 → skill design needs rethinking, not
tweaking. Report to user.
## Phase 6: PERSIST (MANDATORY — execute inline, not deferred)
**NOT optional.** Save results immediately after Phase 4/5. Don't wait for user
to ask. Don't skip because "session is ending". Benchmark artifacts = proof
optimization happened — without them, work unverifiable.
**Applies to feature-driven updates too.** Skill updated as part of feature
implementation (not standalone optimization) → eval cases and results MUST still
persist. Eval may be smaller (only new functionality), but
`evals/<feature>-evals.json` + benchmark.md appendix required. Eval cases =
reusable regression tests — skipping persistence means next optimization has no
baseline for new functionality.
Save results to `.claude-plugin/.benchmarks/<skill-name>/`:
```
.benchmarks/<skill-name>/
├── benchmark.md — committed. Permanent optimization record.
├── evals.json — committed. Eval cases + assertions + results.
└── workspace/ — gitignored. Recreatable by this skill.
├── skill-snapshot/ — frozen copy of skill before/after optimization
└── iteration-N/ — subagent outputs per iteration
```
**Execute these steps in order:**
1. **Create workspace/** dir (gitignored —
`.claude-plugin/.benchmarks/**/workspace/`)
2. **Save evals.json** to `.benchmarks/<skill-name>/evals.json` (NOT inside
workspace/) — must include ALL eval cases with: prompt, expected output,
assertions, audit finding references, and final pass/fail results per
iteration. This file is committed — eval cases are reusable test artifacts.
3. **Copy SKILL.md** before and after to `workspace/skill-snapshot/`
4. **Write benchmark.md** (committed) with: summary, changes table, key design
decisions, metrics (lines before/after), iterations table (pass rates per
iteration), per-eval detail table (prompt + before/after grade), integration
test results if Phase 7/8 was run
5. **Update README.md** in `.benchmarks/` with new skill row
Present final state to user:
```
Skill: [name]
Lines: [before] → [after] ([delta]%)
Eval cases: [N] (with-rule [X]%, baseline [Y]%, delta +[Z]pp)
Iterations: [count]
Benchmark: .claude-plugin/.benchmarks/[name]/benchmark.md
```
## Key Principles
**Eval tests INSTRUCTION QUALITY, not tool correctness.** Verify skill text
unambiguously guides agent to right tool. Whether tool itself works = separate
concern (integration tests).
**Parallel subagents for with/without.** Always run both in parallel when
establishing baseline. Re-runs (iterations) only need with-rule.
**One subagent per eval suite.** All cases in one prompt — cheaper and shows
whether instructions scale when agent handles multiple intents in context.
**Delta is the metric.** Skill scoring 100% but baseline also 90% adds little
value. Target: +50pp minimum delta.
**Minimum viable eval.** Don't over-test. 8-15 cases covers most skills. Add
cases only for discovered failure modes.
## Phase 7: DESCRIPTION OPTIMIZATION (optional)
After skill body stable, optimize `description` field in frontmatter for better
triggering accuracy. Follow **Description Optimization** section from
`/example-skills:skill-creator` (loaded in Prerequisites step 1):
1. Generate 20 trigger eval queries (10 should-trigger, 10 should-not-trigger)
2. Review with user via HTML template (`assets/eval_review.html` from
skill-creator)
3. Run optimization loop from the skill-creator directory:
```bash
python -m scripts.run_loop \
--eval-set <path-to-trigger-eval.json> \
--skill-path <path-to-skill> \
--model <model-id> \
--max-iterations 5 --verbose
```
4. Apply `best_description` to SKILL.md frontmatter
Separate from body optimization — description controls **when** skill triggers,
body controls **what it does** once triggered.
## Phase 8: FULL INTEGRATION EVAL (optional)
When lightweight tool-selection eval not enough (complex multi-step skills,
skills producing files), follow **Running and evaluating test cases** section
from `/example-skills:skill-creator` (loaded in Prerequisites step 1):
1. Save eval cases to `evals/evals.json` (see skill-creator's
`references/schemas.md` for schema)
2. Spawn with-skill and baseline subagents that **execute the skill** on real
tasks
3. Save outputs to `<skill-name>-workspace/iteration-N/`
4. Grade using `agents/grader.md` from skill-creator, then review via:
```bash
python <skill-creator-path>/eval-viewer/generate_review.py \
<workspace>/iteration-N \
--skill-name "<name>" \
--benchmark <workspace>/iteration-N/benchmark.json
```
5. Aggregate benchmarks:
```bash
python -m scripts.aggregate_benchmark <workspace>/iteration-N \
--skill-name <name>
```
Key difference from Phase 2: integration eval tests **end-to-end behavior**
(tool calls, outputs, quality). Phase 2 tests **instruction clarity** (does
agent know which tool to pick). Use Phase 2 first — faster. Escalate to Phase 8
only when tool selection correct but output quality uncertain.
## Anti-patterns
Top 3 are duplicated near the top — read both.
- **Skipping PERSIST** — benchmark artifacts are proof of work. Without
evals.json and benchmark.md, optimization unverifiable and unrepeatable.
Execute Phase 6 inline, immediately after verify passes
- **Fixing before measuring** — audit findings are hypotheses. Run baseline eval
(Phase 2) to confirm problems exist before applying fixes. A finding that
passes eval is not broken — drop from fix list
- **Running real MCP calls in Phase 2 eval** — unnecessary, tests tool not skill
- **One subagent per eval case** — wasteful, use one subagent for all cases
- **Skipping baseline** — can't measure delta without it
- **Fixing skill without re-eval** — any fix can break other cases
- **Inflating eval count** — 20+ cases with no new failure modes is waste
- **Skipping Phase 2, jumping to Phase 8** — Phase 8 is 10x slower. Always start
with lightweight tool-selection eval
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · artk0de/tearags-mcp | 100 | MIT |
| Bennyoooo/Airbot | 100 | MIT |
Migawka bazy z 13.11.2026 · wersja paczki 04b48c41ca82 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100skill-eval
Uruchamia i analizuje przypadki testowe ewaluacji skilli Claude, sprawdza, czy skill działa i jaka jest jego jakość, także po stworzeniu lub zmianie skilla. - 100eval-optimize
Automatycznie poprawia skill w pętli: uruchamia ewaluację, znajduje porażki sędziów, czyta ślady i uzasadnienia, edytuje SKILL.md, uruchamia ponownie i sprawdza regresje. - 100skill-evals
Prowadzi pętlę ewaluacji, która mierzy, czy skill poprawia wynik agenta: projektuje przypadki, ocenia je i porównuje z bazą. - 90ck:skill-creator
Tworzy i poprawia umiejętności Claude przez iteracje oparte na testach, skrypty, referencje i optymalizację opisów. - 91stack-ai-agent
Opisuje konwencje stosu agentów AI w repozytorium: narzędzia MCP dla zadań, przypadki ewaluacji dla narzędzi i ewaluacje offline w bramce. - 95skill-optimizer
Prowadzi tworzenie i przegląd skilli rozszerzających Claude: struktura SKILL.md, jakość frontmattera, projekt wyzwalaczy, potoki ewaluacji i pakowanie.