benchmark-checklist
/benchmark-checklist
Check whether a measured performance claim supports a comparison or decision. Use when running benchmarks, reporting a measured speedup or regression, or choosing between implementations from timing results. Do NOT use for interpreting published benchmarks without running them or for general debugging.
- Author
- wilbeibi ↗
- Repository
- claude-toolkit ↗
- Licence
- MIT
- Last repository change
- 26.10.2026
Assessment and security
- Files in this package
- 2
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 1
No signals detected. Repository changed on 26.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Check whether a measured performance claim supports a comparison or decision. Use when running benchmarks, reporting a measured speedup or regression, or choosing between implementations from timing results. Do NOT use for interpreting published benchmarks without running them or for general debugging.
Original author description
Check whether a measured performance claim supports a comparison or decision. Use when running benchmarks, reporting a measured speedup or regression, or choosing between implementations from timing results. Do NOT use for interpreting published benchmarks without running them or for general debugging.
How it works
- Before measuring
- Check the result
- Decision
- Output
SKILL.md preview
--- name: benchmark-checklist description: Check whether a measured performance claim supports a comparison or decision. Use when running benchmarks, reporting a measured speedup or regression, or choosing between implementations from timing results. Do NOT use for interpreting published benchmarks without running them or for general debugging. --- # benchmark-checklist Test the sentence the measurement is supposed to support. ## Before measuring Write the proposed claim with its metric, workload, and scope. Read the harness to see what it times, counts, awaits, and ignores. Define a correctness check independently of the timing result. Inspect machine load and resource limits. If competing work cannot be avoided, interleave the alternatives and report the interference; do not stop unrelated processes. ## Check the result | Question | Required evidence | |---|---| | **What limits it?** | Profile separately from reported timing runs. Map the bottleneck to code or a resource; check that the load generator is not the bottleneck. | | **Is the comparison fair?** | Comparable versions, data, builds, cache state, concurrency, and production settings. Tune avoidable configuration bottlenecks before recommending an implementation. If tuning is out of scope, bound the claim to the configurations tested and leave the adoption decision open. | | **Is the magnitude possible?** | Check bandwidth, core capacity, and the affected share of total time. Removing 10% of a run saves at most 10% of its time, about a 1.11x speedup. A result beyond a physical or arithmetic bound needs investigation. | | **Did it succeed?** | Count failures, retries, timeouts, and non-success responses. Validate actual outputs; fast rejection is not fast completion. | | **Does it repeat?** | For a comparison, start with at least five runs per side in alternating order, unless the user sets a smaller budget. Record warmup, raw results, median, and range. Mark the conclusion inconclusive when the budget or variation cannot support the claimed difference. | | **Does it affect the user?** | Measure the relevant end-to-end path with realistic data and concurrency. State a microbenchmark's share of the whole; without that check, keep the claim at microbenchmark scope. | | **Did the work happen inside the timer?** | Observe completion counts or effects: consumed iterators, awaited work, returned bytes, written rows, or used results. Setup, asynchronous tail work, caching, and no-ops can invalidate the timed region. | For an explicitly requested ballpark, one run is enough. Still verify successful output and work inside the timer, label it a single run, and do not turn it into a winner recommendation. A choice between alternatives is a comparison. ## Decision - **Faster / slower:** the successful, comparable runs support a difference beyond observed variation; explain the limiting work. - **No measurable difference:** the valid experiment cannot distinguish the alternatives at its observed resolution. - **Inconclusive:** correctness, completion, comparability, repetitions, or the proposed explanation is unresolved. Name the missing evidence. Distinguish elapsed-time reduction from throughput increase. For example, 100 ms to 80 ms is 20% less time and, for serial work, 1.25x throughput. Preserve units and the statistic being compared. ## Output Lead with the verdict and its scope. Give the main before/after number with units, run count, median and range, correctness/work counts, and bottleneck evidence. Link the harness and raw results when available. State limits that could change the decision; do not generalize a configuration result into an implementation ranking. Adapted from [pstack/benchmark-checklist](https://github.com/cursor/plugins/blob/850d7f2e9f5e55dff7eb00a0a479337fe79eb22f/pstack/skills/benchmark-checklist/SKILL.md), under the [MIT license](LICENSE.pstack).
Files in this package (2)
- LICENSE.pstack 1 KB
- SKILL.md 4 KB
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · wilbeibi/claude-toolkit | 100 | MIT |
| arjitj2/open-pstack | 100 | MIT |
| Calvin-LLC/skills | 100 | MIT |
| xinhi6716/axiom-atropos | 100 | MIT |
| michael-denyer/pstack-claude | 100 | MIT |
| theonlyfusioncube/potetos-for-everyone | 100 | MIT |
| robansuini/pstack-codex | 100 | MIT |
| Macro-Harder/pstack-muse | 100 | MIT |
| vahaah/astack | 100 | MIT |
Database snapshot: 13.11.2026 · package version 78e64a2afa3e (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100benchmark
Use when 要產出、查核或依據一個量測出來的效能數字:報 speedup/regression、挑 library 或設定、凍結 benchmark harness、對單一指標做 hillclimb。NOT for 找出變慢的原因(/diagnose)、估工時或交期(/estimate-hours)、CI runner 規格與排隊(/ci-runners、/delivery-capacity)。 - 100eng-benchmark-checklist
Vet a performance measurement for limiter, tuning, limits, errors, repeatability, relevance, and whether the work happened before you report or act on it. Use when you run a benchmark or report a measured speedup or regression. - 95performance-investigation
Measure, diagnose, and fix performance problems with evidence instead of code reading. Use when something is slow, a latency, throughput, memory, startup, render, or query time regressed, a user asks to "make it faster", "speed this up", "why is this slow", "profile this", or… - 95jmh-benchmarks
Expert guidance for writing Java microbenchmarks using JMH (Java Microbenchmark Harness). Use this skill when writing performance benchmarks, measuring method execution time, comparing algorithm implementations, profiling code performance, or debugging benchmark issues. Trigger… - 90android-benchmark-comparison
Use when comparing physical Android benchmark configurations, investigating inconsistent rankings, or selecting an Android default from measured results. Do not use for code-level Compose performance diagnosis without a configuration comparison. - 100golang-benchmark
Golang benchmarking, profiling, and performance measurement. Use when writing, running, or comparing Go benchmarks, profiling hot paths with pprof, interpreting CPU/memory/trace profiles, analyzing results with benchstat, setting up CI benchmark regression detection, or…