Skip to content
Skills Library
AI & agents · Testing

loop

/loop

Use when benchmarking and evolving prompt assets through a fixed-suite optimization loop

Licence
MIT
Last repository change
22.10.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 22.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Use when benchmarking and evolving prompt assets through a fixed-suite optimization loop

Original author description

Use when benchmarking and evolving prompt assets through a fixed-suite optimization loop

How it works

  1. Iron Law
  2. Red Flags
  3. When to Use
  4. Subcommands
  5. Workflow
  6. Hard Gates
  7. State
  8. Gotchas
Original SKILL.md headings · 63 lines, 545 words
SKILL.md preview
---
name: loop
description: "Use when benchmarking and evolving prompt assets through a fixed-suite optimization loop"
effort: high
disable-model-invocation: true
user-invocable: false
---

[Resolve runtime paths](../runtime-paths.md) before file or script operations.

## Iron Law

> **A loop without a benchmark is meaningless.**

## Red Flags

- "Let's just run it and see what happens" → STOP, because the Iron Law says a loop without a benchmark is meaningless — a fixed suite with numeric scoring must exist before any run.
- "The new variant feels better" → STOP, because evaluator output must expose a numeric score that can be parsed deterministically — subjective improvement is not a valid promotion signal.
- "Baseline is close enough, let's promote the candidate" → STOP, because winners are promoted only when the score beats baseline by at least `min_delta` — close is not enough.
- "I'll mutate the docs and test files too" → STOP, because `.mjs`, `docs/`, `README*`, and `tests/` are excluded from mutation scope — only allowed targets in the suite allowlist may be changed.
- "One more generation might improve it" → STOP, because the loop must stop when budget is exhausted, `min_delta` is not met, or scores plateau — open-ended iteration wastes tokens without measurable gain.

# Loop

Run a Karpathy-style optimization loop over the plugin's own prompt assets.

## When to Use

- Maintainers want to improve `skills/**/SKILL.md`, `agents/*.md`, `commands/*.md`, or `templates/*.md`
- A fixed benchmark suite with numeric scoring already exists
- Candidate variants should compete under the same time budget and the best variant should be promoted to an isolated branch

## Subcommands

| Command | Purpose |
|---------|---------|
| `list-suites` | List bundled loop suites in `benchmarks/loop/` |
| `show-suite <name>` | Inspect the validated suite manifest |
| `run <name>` | Create an isolated run branch, score a baseline, generate candidates, and promote a winner if `min_delta` is met |
| `resume <run_id>` | Reload the saved loop state from `.data/state/loop-active.json` |

## Workflow

1. Use `node scripts/loop-runner.mjs show-suite <name>` to load the suite and verify the allowed targets.
2. Use `node scripts/loop-runner.mjs run <name> ...` to create the isolated `codex/loop-...` branch and run worktree.
3. Inside that run worktree, create 3-5 prompt variants only within the selected targets.
4. Evaluate every candidate with the suite's fixed case commands. Parse scores from JSON or review-style output only.
5. Keep the top 1-2 elite candidates per generation. Stop when budget is exhausted, `min_delta` is not met, or the loop plateaus.
6. Only promote a winner when all hard gates pass and the score beats baseline by at least `min_delta`.

## Hard Gates

- Selected targets must stay within the suite allowlist and the global v1 mutation scope.
- Contract/runtime smoke checks must still pass when those files exist in the repo.
- Evaluator output must expose a numeric score that can be parsed deterministically.
- Winners are promoted only into the run branch/worktree, never into the current workspace.

## State

- Active state: `<plugin-data>/state/loop-active.json`
- Artifacts: `.captures/loop-<run_id>/`
- Bundled suites: `benchmarks/loop/*.json`

## Gotchas

- Do not add routing for this skill in `prompt-detect.mjs` for v1. It is slash-command only.
- Do not mutate `.mjs`, `docs/`, `README*`, or `tests/` as loop targets.
- Do not declare a winner when baseline remains best or when `min_delta` is not met.
- Do not write to the main worktree directly. The isolated loop branch is the output surface.

Versions

RepositoryScoreLicence
This page · unclejobs-ai/second-claude-code90MIT
mickeyyaya/evolve-loop100Apache-2.0
relai-ai/agent-eval-loop100Apache-2.0
SethGammon/Citadel100MIT
alirezarezvani/claude-skills100MIT
x-cmd/x-cmd100Apache-2.0
in-the-loop-labs/pair-review100Apache-2.0
railly/skills100MIT
slogsdon/loop-and-gate-foundation100MIT

Database snapshot: 13.11.2026 · package version 8679fd7fda25 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100prompt-iteration
    Improve a prompt through controlled changes measured against a fixed set of cases, rather than by rewriting until an example looks good. Use when a prompt underperforms and each edit is a guess.
  • 100benchmark-model-test
    用于批量测试多个大模型对同一组 prompt 的响应,自动化对比不同模型的输出质量并将结果保存到 Excel 表格中。当用户需要对比模型性能、收集测试数据或进行 benchmark 测试时使用。
  • 95evolving-ai-agents
    Provides guidance for automatically evolving and optimizing AI agents across any domain using LLM-driven evolution algorithms. Use when building self-improving agents, optimizing agent prompts and skills against benchmarks, or implementing automated agent evaluation loops.
  • 100prompt-optimization
    Use when auditing an existing prompt for clarity, structure, or completeness issues, fixing vague instructions, or producing a before/after optimization report.
  • 100prompt-engineering
    Use when designing, optimizing, testing, or deploying robust prompt systems for AI agents. This skill provides frameworks for structured prompt engineering, meta-prompting, and automated optimization workflows.
  • 100skillopt
    Improve a skill, agent file, or prompt through bounded patches and a held-out, per-task acceptance gate. Use when you say `optimize this skill`, `make this skill better`, `compress these skills safely`, `tune this prompt against an eval`, or a skill keeps failing the same way.…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.