Przejdź do treści
Biblioteka skilli
Programowanie · Przegląd i audyt

benchmark-checklist

/benchmark-checklist

Sprawdza pomiar wydajności przed raportowaniem lub działaniem: ogranicznik, strojenie, błędy, powtarzalność i to, czy praca się wykonała.

Repozytorium
dabaige-skills ↗
Licencja
MIT
Ostatnia zmiana repozytorium
02.11.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
3
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 02.11.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Sprawdza pomiar wydajności przed raportowaniem lub działaniem: ogranicznik, strojenie, błędy, powtarzalność i to, czy praca się wykonała.

Oryginalny opis (EN)

Vet a perf measurement (limiter, tuning, limits, errors, repeatability, relevance, and whether the work happened) before you report or act on it. Use when you run a benchmark or report a speedup or regression you measured.

Jak działa

  1. Before you run anything
  2. The questions
  3. Report
  4. Applying the checklist
Sekcje z nagłówków SKILL.md, w oryginale · 36 linii, 913 słów
Podgląd SKILL.md
---
name: benchmark-checklist
description: "Vet a perf measurement (limiter, tuning, limits, errors, repeatability, relevance, and whether the work happened) before you report or act on it. Use when you run a benchmark or report a speedup or regression you measured."
disable-model-invocation: false
---

# Benchmark checklist

Use this when you produce a performance number: a PR's before and after, a regression claim, a hillclimb harness, or a library or config choice. Answer each question with evidence from a run, not a guess about the code.

For a quick ballpark the user asked for, one run is enough. Still check questions 4 and 7, and say that it is one run. Skip the rest unless that run looks wrong. A choice between options is never a ballpark.

## Before you run anything

- Write down the claim you expect to make, in the words you would ship ("export is 30% faster at p50 on the 60k-row dataset"). The questions test that sentence.
- Read the measurement script. Note what it times, counts, and ignores.
- Check the load average with `uptime`. On macOS, get logical core count with `sysctl -n hw.logicalcpu`; on Linux use `nproc` or `getconf _NPROCESSORS_ONLN`. Inspect busy processes without stopping other work. If the machine is busy, interleave the sides so both see the same noise, and say so in the report.

## The questions

1. **Why not double?** Name the limiter. Profile in a run you do not report, because profilers and tracers slow the work down. Use an existing runtime profiler, such as `node --cpu-prof` or an installed `py-spy`, and map the hot spot to source. On macOS, inspect process CPU with `top -l 1 -stats pid,cpu,time,command`, and use `iostat` or `sample <pid> <seconds>` when relevant and available. Linux may use `pidstat`, `perf` and `strace -c`. Choose metrics the platform actually exposes; do not substitute a different metric or claim an unavailable measurement. Installation and extra privileges require the task's authorization, not this checklist. Watch the load generator too: if it saturates first, you measured the load generator. If a change did not move the number, the limiter explains why, so find it before you call the change useless.
2. **Was it tuned?** Run every side the way production runs it: release builds, production flags and env, batching and transaction settings, connection pools, caches as warm or cold as production sees them, and the same versions and data. If one side runs on defaults, you compared configurations, not implementations. A limiter that is a setting (a commit per row, a debug build, a missing index) means that side is untuned. Tune it and measure again before you pick a winner. If you cannot tune it, do not pick a winner from that run. When the user is choosing what to adopt, narrowing the claim to the code as it ships today does not fix this, because they adopt the option, not today's settings.
3. **Did it break limits?** Do the arithmetic. Compare bytes per second with disk and network bandwidth, operations per second times cost per operation with your cores, and the time saved with the time the changed piece took. Removing a piece that takes 10% of the run can make the run at most about 11% faster. A result past a limit measured something other than the work, such as a cache, a no-op, or a bug.
4. **Did it error?** Count failures and non-success responses, and check that outputs are correct, not just present. Rejections are often fast, and timeouts and retries are slow. If the script does not count errors, add the count.
5. **Does it reproduce?** Run each side at least 5 times, alternating (A, B, A, B, and so on) so warmup, lazy initialization, caches, and drift favor neither. Report the median and range. A gap smaller than the run-to-run variation is no measurable difference. When the call is close, use a rank-sum test or the harness's own statistics.
6. **Does it matter?** Next to any micro result, measure the end-to-end path a user waits on, with realistic data sizes and concurrency, and report the micro result as a share of the whole. A helper that takes 1% of a request can make the request at most 1% faster, however fast the helper gets.
7. **Did it even happen?** Confirm the work ran inside the timed region: the request reached the server, the rows were written, the bytes were read, and the code used the result. Lazy code (generators nobody iterates, promises nobody awaits, results the JIT can discard) and timeouts produce numbers for work that never happened.

## Report

- Lead with the verdict: faster, slower, no measurable difference, or inconclusive.
- Give the number with its unit, the run count, the range, and the limiter. For example, "p50 41 ms → 33 ms, median of 7 runs per side, range 32 to 35 ms after, bound by JSON parsing on one core."
- Call the verdict inconclusive when you claim a difference but cannot name the limiter, when a side ran untuned, or when you could not check questions 4 and 7. Name the gap.
- Keep a PR body to one primary number, with detailed runs, range and limiter evidence in a linked artifact or notes file. No separate playbook is required.

## Applying the checklist

- Check the baseline before planning a performance fix and check each later claim against measured results.
- For iterative optimization, validate the measurement harness before freezing it. Include error and completed-work counts in its output so each keep-or-revert checks questions 4 and 7.
- Apply the current task's scope and verification requirements. This checklist does not authorize production load tests, stopping services or changing shared settings.
Pliki w paczce (3)
  • LICENSE 1 KB
  • SKILL.md 5 KB
  • agents/openai.yaml 1 KB

Wersje

RepozytoriumWynikLicencja
Ta strona · dabaige53/dabaige-skills100MIT
arjitj2/open-pstack100MIT
Calvin-LLC/skills100MIT
xinhi6716/axiom-atropos100MIT
michael-denyer/pstack-claude100MIT
wilbeibi/claude-toolkit100MIT
theonlyfusioncube/potetos-for-everyone100MIT
robansuini/pstack-codex100MIT
Macro-Harder/pstack-muse100MIT

Migawka bazy z 13.11.2026 · wersja paczki bc8418bce200 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100eng-benchmark-checklist
    Sprawdza pomiar wydajności pod kątem limitera, dostrajania, błędów, powtarzalności i trafności oraz czy praca faktycznie się odbyła przed zgłoszeniem wyniku.
  • 100steady-flame
    Sprawdza wiarygodność wyniku pomiaru wydajności: ograniczenia, strojenie, błędy i powtarzalność, zanim zgłosi się przyspieszenie lub regresję.
  • 100vet-measurement
    Sprawdza pomiar lub benchmark przed podjęciem decyzji: ograniczenia, błędy, powtarzalność i trafność wyniku.
  • 95performance-investigation
    Mierzy i diagnozuje problemy wydajności na podstawie danych, gdy coś działa wolno lub regresuje czas, pamięć albo zapytania.
  • 86principle-explain-the-number
    Sprawdza przed zaufaniem, zgłoszeniem lub działaniem na zmierzonej liczbie, np. na przyspieszeniu, regresji albo wyniku ewaluacji, co ją ogranicza i czy mierzy właściwą pracę.
  • 100benchmark
    Dba o wiarygodność liczb z pomiarów wydajności: raporty przyspieszeń, wybór bibliotek i ustawień oraz dostrajanie jednej metryki.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.