AI & agents · Testing
loop
/loop
Use when benchmarking and evolving prompt assets through a fixed-suite optimization loop
For whom
- Author
- unclejobs-ai ↗
- Repository
- second-claude-code ↗
- Licence
- MIT
- Last repository change
- 22.10.2026
Assessment and security
90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
Deprecation
No signals detected. Repository changed on 22.10.2026, without references to retired models or APIs.
Licence
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Use when benchmarking and evolving prompt assets through a fixed-suite optimization loop
Original author description
Use when benchmarking and evolving prompt assets through a fixed-suite optimization loop
How it works
- Iron Law
- Red Flags
- When to Use
- Subcommands
- Workflow
- Hard Gates
- State
- Gotchas
Original SKILL.md headings · 63 lines, 545 words
SKILL.md preview
--- name: loop description: "Use when benchmarking and evolving prompt assets through a fixed-suite optimization loop" effort: high disable-model-invocation: true user-invocable: false --- [Resolve runtime paths](../runtime-paths.md) before file or script operations. ## Iron Law > **A loop without a benchmark is meaningless.** ## Red Flags - "Let's just run it and see what happens" → STOP, because the Iron Law says a loop without a benchmark is meaningless — a fixed suite with numeric scoring must exist before any run. - "The new variant feels better" → STOP, because evaluator output must expose a numeric score that can be parsed deterministically — subjective improvement is not a valid promotion signal. - "Baseline is close enough, let's promote the candidate" → STOP, because winners are promoted only when the score beats baseline by at least `min_delta` — close is not enough. - "I'll mutate the docs and test files too" → STOP, because `.mjs`, `docs/`, `README*`, and `tests/` are excluded from mutation scope — only allowed targets in the suite allowlist may be changed. - "One more generation might improve it" → STOP, because the loop must stop when budget is exhausted, `min_delta` is not met, or scores plateau — open-ended iteration wastes tokens without measurable gain. # Loop Run a Karpathy-style optimization loop over the plugin's own prompt assets. ## When to Use - Maintainers want to improve `skills/**/SKILL.md`, `agents/*.md`, `commands/*.md`, or `templates/*.md` - A fixed benchmark suite with numeric scoring already exists - Candidate variants should compete under the same time budget and the best variant should be promoted to an isolated branch ## Subcommands | Command | Purpose | |---------|---------| | `list-suites` | List bundled loop suites in `benchmarks/loop/` | | `show-suite <name>` | Inspect the validated suite manifest | | `run <name>` | Create an isolated run branch, score a baseline, generate candidates, and promote a winner if `min_delta` is met | | `resume <run_id>` | Reload the saved loop state from `.data/state/loop-active.json` | ## Workflow 1. Use `node scripts/loop-runner.mjs show-suite <name>` to load the suite and verify the allowed targets. 2. Use `node scripts/loop-runner.mjs run <name> ...` to create the isolated `codex/loop-...` branch and run worktree. 3. Inside that run worktree, create 3-5 prompt variants only within the selected targets. 4. Evaluate every candidate with the suite's fixed case commands. Parse scores from JSON or review-style output only. 5. Keep the top 1-2 elite candidates per generation. Stop when budget is exhausted, `min_delta` is not met, or the loop plateaus. 6. Only promote a winner when all hard gates pass and the score beats baseline by at least `min_delta`. ## Hard Gates - Selected targets must stay within the suite allowlist and the global v1 mutation scope. - Contract/runtime smoke checks must still pass when those files exist in the repo. - Evaluator output must expose a numeric score that can be parsed deterministically. - Winners are promoted only into the run branch/worktree, never into the current workspace. ## State - Active state: `<plugin-data>/state/loop-active.json` - Artifacts: `.captures/loop-<run_id>/` - Bundled suites: `benchmarks/loop/*.json` ## Gotchas - Do not add routing for this skill in `prompt-detect.mjs` for v1. It is slash-command only. - Do not mutate `.mjs`, `docs/`, `README*`, or `tests/` as loop targets. - Do not declare a winner when baseline remains best or when `min_delta` is not met. - Do not write to the main worktree directly. The isolated loop branch is the output surface.
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · unclejobs-ai/second-claude-code | 90 | MIT |
| mickeyyaya/evolve-loop | 100 | Apache-2.0 |
| relai-ai/agent-eval-loop | 100 | Apache-2.0 |
| SethGammon/Citadel | 100 | MIT |
| alirezarezvani/claude-skills | 100 | MIT |
| x-cmd/x-cmd | 100 | Apache-2.0 |
| in-the-loop-labs/pair-review | 100 | Apache-2.0 |
| railly/skills | 100 | MIT |
| slogsdon/loop-and-gate-foundation | 100 | MIT |
Database snapshot: 13.11.2026 · package version 8679fd7fda25 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100prompt-iteration
Improve a prompt through controlled changes measured against a fixed set of cases, rather than by rewriting until an example looks good. Use when a prompt underperforms and each edit is a guess. - 100benchmark-model-test
用于批量测试多个大模型对同一组 prompt 的响应,自动化对比不同模型的输出质量并将结果保存到 Excel 表格中。当用户需要对比模型性能、收集测试数据或进行 benchmark 测试时使用。 - 95evolving-ai-agents
Provides guidance for automatically evolving and optimizing AI agents across any domain using LLM-driven evolution algorithms. Use when building self-improving agents, optimizing agent prompts and skills against benchmarks, or implementing automated agent evaluation loops. - 100prompt-optimization
Use when auditing an existing prompt for clarity, structure, or completeness issues, fixing vague instructions, or producing a before/after optimization report. - 100prompt-engineering
Use when designing, optimizing, testing, or deploying robust prompt systems for AI agents. This skill provides frameworks for structured prompt engineering, meta-prompting, and automated optimization workflows. - 100skillopt
Improve a skill, agent file, or prompt through bounded patches and a held-out, per-task acceptance gate. Use when you say `optimize this skill`, `make this skill better`, `compress these skills safely`, `tune this prompt against an eval`, or a skill keeps failing the same way.…