Przejdź do treści
Biblioteka skilli
Nauka i badania · Przegląd i audyt

rigor

/rigor

Pilnuje rzetelności wyniku badawczego: sprawdza pomiar, ten sam benchmark, pełne mianowniki i brak przecieku danych.

Repozytorium
sliu-skill ↗
Licencja
MIT
Ostatnia zmiana repozytorium
13.11.2026

Ocena i bezpieczeństwo

90/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść25 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
1
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 13.11.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Pilnuje rzetelności wyniku badawczego: sprawdza pomiar, ten sam benchmark, pełne mianowniki i brak przecieku danych.

Oryginalny opis (EN)

Rules for a trustworthy research answer, followed while you work: verify the measurement, same benchmark, numbers from files with the full denominator, no leakage, every difference recorded. Use when setting up, running or writing up a result.

Jak działa

  1. What each ticket needs
  2. The rules
  3. Rules of the result
Sekcje z nagłówków SKILL.md, w oryginale · 27 linii, 675 słów
Podgląd SKILL.md
---
name: rigor
description: "Rules for a trustworthy research answer, followed while you work: verify the measurement, same benchmark, numbers from files with the full denominator, no leakage, every difference recorded. Use when setting up, running or writing up a result."
---

# Rigor

An answer counts when it can be trusted. This skill is the reference for producing one: what makes a result trustworthy, and the rules to follow while you work, not after.

## What each ticket needs

| ticket | apply |
|---|---|
| `experiment` (compares runs, or computes numbers from our own data or results) | all of the rules below |
| `task` that builds something later tickets use (a data version, an encoded store, a tokenizer, a launcher) | the measurement, numbers from files |
| `research` (reading only), `grilling`, `prototype`, any other `task` (getting access, setting up an environment) | none |

## The rules

1. **Verify the measurement first.** A score depends on the input, the reference, the prediction and the scorer. Check all four before judging the model: labels present and aligned with the inputs, units and scale, the scorer on a known case, and the reference itself: ground truth, or named a proxy when a model made it (another model's predictions, a model's own confidence). A score is never normalised by the evaluated model's own outputs (its best run, its maximum). A new benchmark or metric, or a new or changed pipeline (any code on the path that produces the number), needs a positive control (a known answer it must reproduce) and a null (what an information-free input scores). Code that has already passed does not: on a repeat, check only what varies between repeats and would fail silently. Add a control when a result looks too good, sits at the floor or ceiling, or cannot be explained. A speed or cost number gets the same care: name what bounds it (GPU, data loading, I/O, the scheduler) and rule out failed, skipped or cached work and code that never ran.
2. **Same benchmark.** The runs compared are scored on the same benchmark: eval set and N, metric, scorer, decoding (the map's Contract names it; with `Benchmark: none`, the eval the ticket's pre-registration fixes, the same for every arm). A number from another benchmark is not comparable until re-scored on this one. When the benchmark itself changes, re-score the reference too.
3. **Numbers from files, with the full denominator.** Every number is read from a saved result file and written with its source: `0.712 (mean TM, test-v2 N=338, greedy; runs/arm-b/eval/scores.json)`. Failed, invalid and unscored items stay in the count; a score on the valid subset never stands in for the full one. Missing is not zero. Validity (parsed, complete, well-formed) is reported apart from accuracy. Never average different metrics or scales.
4. **No leakage.** The eval items, or near-duplicates of them, are not in the training data, and no setting was chosen on the eval set. A set you choose settings on or iterate against becomes a selection set and stops being independent evidence: final claims use a held-out set.
5. **Differences recorded.** A new kind of experiment records its full setup (data, code, config, launch). Otherwise record the reference run and every difference from it; a reference other tickets compare against has its full setup recorded (in a run card, where the project uses them). When several things changed together (data, budget, model, metric), the result belongs to all of them: credit one only if it was varied alone.

## Rules of the result

- **Screen, then confirm.** A single run or a fixed cheap budget is a **screen**: its answer is `[tentative]`. A claim the map stands on is **confirmed**: before results, the pre-registration fixes the repeats (seeds or draws; 3 by default), the comparison (paired where possible), the smallest gap that matters, and how the spread is reported. It is `[established]` when the gap clears both that threshold and the spread between repeats.
- **Criteria before results.** The pre-registration's success and stop rules are fixed before anything runs; amendments are dated and come before the numbers they touch.
- **Classify a failure before calling it negative.** Execution, measurement, data, method or hypothesis: only the last two are negative; the rest are inconclusive, with the fix.

Wersje

W bazie nie ma innych skilli o nazwie rigor.

Migawka bazy z 13.11.2026 · wersja paczki 11cec3c2acb6 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 90evaluation-leakage-audit
    Sprawdza ewaluację, benchmark albo harness oceniania pod kątem wycieku danych i wyników, które potwierdzają same siebie.
  • 90benchmark-research-skill
    Przegląda istniejące benchmarki dla pytania badawczego i wyciąga zbiory danych, metryki oraz bazowe wyniki z artykułów.
  • 100briefbound-research-rigor-review
    Ocenia rygor dowodowy wyniku badania AI/ML, awansu baseline, wniosku z ablacji, twierdzenia z benchmarku lub kosztownego kolejnego eksperymentu, zanim zostanie zaakceptowany, uogólniony lub opublikowany.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.