benchmark-run
/benchmark-run
Zapisuje benchmark zakończonego uruchomienia Ork i wszystkich wywołanych przebiegów: koszt, tokeny, czasy i porównanie z Claude Code.
- Autor
- mimshins ↗
- Repozytorium
- ork ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 13.11.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 13.11.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Zapisuje benchmark zakończonego uruchomienia Ork i wszystkich wywołanych przebiegów: koszt, tokeny, czasy i porównanie z Claude Code.
Argumenty
/benchmark-run <run> <folder> [name]
Oryginalny opis (EN)
Save a benchmark of a finished Ork run, and every run it called, into a folder - cost, tokens, working and waiting time, the Ork version, by node, what went wrong - compare it with an estimate of the same change made in plain Claude Code sessions, and publish its safe parts to the repository's public benchmarks/. Use when asked to benchmark a run. Arguments - the run id, the folder to save into, and optionally a name.
Jak działa
- Rules
- Steps
Podgląd SKILL.md
---
name: benchmark-run
description: Save a benchmark of a finished Ork run, and every run it called, into a folder - cost, tokens, working and waiting time, the Ork version, by node, what went wrong - compare it with an estimate of the same change made in plain Claude Code sessions, and publish its safe parts to the repository's public benchmarks/. Use when asked to benchmark a run. Arguments - the run id, the folder to save into, and optionally a name.
argument-hint: <run> <folder> [name]
---
# Benchmark a run
Arguments: `$ARGUMENTS`, which are a run id (or prefix), the folder that holds
the benchmarks, and an optional name. If the run or the folder is missing, ask
for it. Never guess the folder.
The numbers come from scripts, not from you: `pnpm benchmark` measures and
`pnpm benchmark:control` estimates the control (`tools/benchmark/`, whose files
say how). Your part is the name, the result commits when a run committed none
itself, and the README's account of what happened.
## Rules
- **Working time** leaves out every minute the run waited on the maintainer:
gates, `workflow.call` nodes waiting on their run, a node blocked on its own
open question or permission prompt, and a leader waiting on a child that is
blocked. Wall time and waiting time are reported beside it, never instead of
it.
- The run's whole call tree is one benchmark: a `start` run with the `plan` and
`build` runs it called.
- The scripts read a consistent copy of `~/.ork/state.db`. Never write to the
store.
- A benchmark folder is never overwritten. Pick another name.
- **The README is public.** The publish step copies it to `benchmarks/` in this
public repository, so write it for someone who has never seen Ork: no local
paths, no names but "the maintainer", nothing from a prompt or a plan that
isn't already public, and commits linked on GitHub.
## Steps
1. **Check that it ended.** Run `pnpm ork ps --project all` and find the run. If it is
still `running` (or waiting at a gate), say so and stop; benchmark it once it
ends.
2. **Name it.** Use the name given. Otherwise take a short kebab-case name for
the work: the plan's file name in `.ai/plans/`, or a few words from the
brief, such as `cli-drivers-only` or `for-each-run-2`. Say which name you
chose.
3. **Measure:**
```sh
pnpm benchmark <run> <folder> --name <name>
```
This writes `<folder>/<name>/` with `summary.json` (with `ork`, the Ork
version the runs recorded, or the one at their start commit), `usage.csv`,
`brief.md`,
`spec.md` (the plan or spec that was built) and `workflow/` (the workflows,
`project.yaml` and agents at the commit the run started from).
4. **Result commits.** `summary.json`'s `result_commits` lists the commits the
run's tool nodes printed (a `publish` step). If it is empty because the
maintainer committed the change by hand, find that commit with
`git log --since=<run start>` and its diff, and confirm it with the
maintainer. Then add it to `result_commits` in `summary.json`. If nothing was
kept, leave it empty: the control then has no row for this run.
5. **Control:**
```sh
pnpm benchmark:control <folder>
```
This rewrites `<folder>/CONTROL.md` and `control.json` for every benchmark
in the folder, with this run's row among them.
6. **What happened.** Read `pnpm ork logs <id>` for each run in the tree
(`summary.json`'s `runs`), and `summary.json`'s `activations`, to find:
failed sessions and why (turn cap, budget, an error, an Ork bug), review
rounds, what each permission request asked, conflicts, and how builders
overlapped (from the activations' starts and minutes, by owner).
7. **Write `<folder>/<name>/README.md`** for the public, in the shape of the
published ones in `benchmarks/` (read one first):
- `# <what was built>`, then one paragraph: what Ork built, the run id, the
date, the Ork version from `summary.json`'s `ork`, the driver, and the
result commit as a GitHub link with its stat (`git show --shortstat`).
- `## At a glance`: status, cost and the workflow's `limits.budgetUsd`,
working, waiting and wall minutes, sessions (failed), turns, all tokens
by kind, questions and permission requests.
- `## By node`: from `by_node`: node, model, sessions, cost, output, cache
read and cache write.
- `## How it ran`: the shape of the run; for a leader, when builders ran and
how many at once; review rounds and what they found.
- `## What went wrong`: one bullet per problem, each saying whether Ork, the
agent or the workflow caused it, and the fix if one was made. "Nothing
failed" when nothing did.
- `## Against a manual session`: this benchmark's row from `CONTROL.md`, and
one sentence reading it, linking `../method.md#the-manual-baseline`.
- `## Files`: `summary.json`, `usage.csv`, and `inputs/` (the brief, and
the workflow, `project.yaml` and agents at the start commit).
8. **Publish** with `pnpm benchmark:publish <folder> <name>` from this
repository. It copies only what may be public into `benchmarks/<name>/`,
rewrites `benchmarks/results.json` and the README's results table, and
refuses (writing nothing) on a path, an email, a token or a local name.
If the overview in `benchmarks/README.md` ("What they show so far") no
longer matches the table, update it with the new totals. Run
`pnpm benchmarks:check`, and commit only when the maintainer asks.
9. **Report** the headline numbers to the maintainer (cost, working, waiting and
wall minutes, the control's median and range), and anything that went wrong.
Wersje
W bazie nie ma innych skilli o nazwie benchmark-run.
Migawka bazy z 13.11.2026 · wersja paczki ba8addae7ad4 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100setup
Konfiguruje Ork w repozytorium: sprawdza instalację, przygotowuje projekt, udostępnia narzędzia i prowadzi pierwszy przepływ pracy. - 100principles
Dzieli pracę na kroki workflow Ork: które kroki to programy, które sesje Claude, a które decyzje należą do człowieka. - 100benchmark-models
Porównuje ten sam prompt w Claude, GPT i Gemini: opóźnienie, tokeny, koszt, wywołania narzędzi i jakość, z liderami wskaźników. - 100benchmark-runner
Uruchamia próby benchmarku ai-coding-lang-bench dla AILANG, monitoruje sesje, analizuje koszt, tokeny, czas i odsetek zaliczonych prób oraz porównuje strategie promptów. - 100benchmark-runner
Uruchamia próby benchmarku ai-coding-lang-bench dla AILANG, monitoruje sesje, analizuje koszt, tokeny, czas i odsetek zaliczonych prób oraz porównuje strategie promptów. - 100benchmark-models
Uruchamia jeden prompt lub skill w Claude, GPT i Gemini obok siebie i porównuje opóźnienia, tokeny, koszt, wywołania narzędzi oraz opcjonalnie jakość wyniku przez sędziego z API Anthropic.