Przejdź do treści
Biblioteka skilli
AI i agenci · Testowanie

evaluate-skill

/evaluate-skill

Pomaga oceniać i porównywać zachowanie skilli agentów oraz uzasadniać twierdzenia o ich gotowości.

Repozytorium
freeflow ↗
Licencja
MIT
Ostatnia zmiana repozytorium
28.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
36
Skrypty
29
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 28.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co sprawdzić przed instalacją
  • 29 skryptów w paczce. Przejrzyj je przed instalacją, bo Claude może je uruchomić.

Skrypty: scripts/lib/args.mjs, scripts/lib/capabilities.mjs, scripts/lib/codex-adapter.mjs, scripts/lib/compact-evidence.mjs, scripts/lib/constants.mjs, scripts/lib/coordinator.mjs, scripts/lib/decision.mjs, scripts/lib/doctor.mjs, scripts/lib/evaluate.mjs, scripts/lib/feasibility.mjs, scripts/lib/fixture-oracle.mjs, scripts/lib/grade.mjs, scripts/lib/hash.mjs, scripts/lib/integrity.mjs, scripts/lib/materi

Co robi i kiedy

Pomaga oceniać i porównywać zachowanie skilli agentów oraz uzasadniać twierdzenia o ich gotowości.

Oryginalny opis (EN)

Use when evaluating, comparing, or making readiness claims about agent-skill behavior.

Jak działa

  1. Route First
  2. Name The Question
  3. Smallest Valid Loop
  4. Execution
  5. Grade And Diagnose
  6. Stop
Sekcje z nagłówków SKILL.md, w oryginale · 83 linii, 758 słów
Podgląd SKILL.md
---
name: evaluate-skill
description: Use when evaluating, comparing, or making readiness claims about agent-skill behavior.
---

# Evaluate Skill

Judge whether an exact skill changes agent behavior as intended under natural pressure. Evidence outranks confident prose.

A **variant** is one fixed skill-and-context condition being compared. The **subject** is the agent performing the natural task under that variant.

Inline examples teach the subject. They are part of the candidate, not evaluation evidence. Behavioral evals keep criteria outside the natural prompt and test what the agent actually does.

## Route First

- Grade adequate saved evidence before proposing a rerun or wording change.
- Reuse an existing case unchanged when it already preserves the failure and fixed criteria.
- For a draft-only request, use [Write Skill](../write-skill/SKILL.md), record the result as Draft or Unverified in the eval artifact, report, or response, and keep that status out of the subject `SKILL.md`; do not fake a run.
- Require evidence relevant to every readiness or host-support claim.
- After infrastructure failure, discard partial outcomes and rerun every variant as one whole case.
- Follow explicit constraints. Do not mutate adequate evidence merely to demonstrate process order.

## Name The Question

Separate the behavior being claimed:

- **First activation:** does a natural prompt cause the host to load the exact description and skill snapshot at the earliest useful moment?
- **First-read comprehension:** can the delivered body guide correct behavior with only guaranteed prior context?
- **Nearby behavior:** does a close case avoid activation or load without hijacking the task?
- **Dependency composition:** does the skill work with the exact declared base stack and resources?
- **Retained use:** does the method remain available across the claimed later turns?
- **Artifact outcome:** do files, commands, state, and reports match the required behavior?
- **Cross-host behavior:** does the same case hold on every named host?

One evidence class does not prove another. Direct body delivery cannot prove native activation. One turn cannot prove retained use. Ambient context cannot prove declared composition.

Read [evaluation architecture](references/evaluation-architecture.md) when selecting evidence classes. Read [eval patterns](references/eval-patterns.md) when choosing the smallest case shape.

## Smallest Valid Loop

1. Fix criteria before candidate output exists.
2. Preserve the earliest natural prompt or strongest pressure that should change behavior.
3. Choose a fair variant: no skill, exact old snapshot, release, candidate, description-only change, or declared composition.
4. Plan before spending model requests.
5. Keep prompt, fixture, tools, host, model, and thinking settings fixed across variants.
6. Grade objective artifacts first; use fresh semantic judgment only for unresolved meaning.
7. Classify the failure before editing.
8. Revise one measured pressure point and rerun the whole case.

For a nearby prompt, define whether success means no activation or safe behavior after activation. Do not treat every extra read as a behavioral failure.

## Execution

Use direct child processes for ordinary cases. Do not spend parent or subagent context merely to run a subject.

Use the bundled [skill evaluator](scripts/skill-eval.mjs). Resolve the link relative to this skill directory, then run `node <resolved-path> doctor|init|evaluate`. A complete `evaluate` invocation declares the skill, case, timeout, retained-output limit, and model/process limits. Use `--plan-only` before model execution so the owner can inspect the case, limits, and spend.

The default source lives under `.skill-eval/<skill-name>/`. Subjects receive only the natural prompt, isolated fixture, declared immutable skill resources, and allowed tools. Do not supply author discussion, desired reasoning, assertions, expected outcomes, reports, or another variant unless that material naturally belongs to the user task.

Use [portable execution](references/portable-execution.md) for capability fallbacks and isolation. Use [token-efficient execution](references/token-efficient-execution.md) before expanding cases, models, turns, or evidence scope.

## Grade And Diagnose

- Files, diffs, exit status, events, usage, and protocol fields are objective evidence.
- Final response is lower priority than contradictory artifacts.
- A subject never grades its own run in the same conversation.
- Semantic graders use fresh context, fixed criteria, opaque labels, sanitized paths, and visible uncertainty.
- Missing usage or cost is unavailable, not zero.
- Infrastructure failure publishes diagnostics only.

Classify failures precisely:

- activated too early, too late, or not at all;
- description relies on unavailable vocabulary;
- body fails on first read;
- undeclared or undelivered dependency;
- normal-path guidance hidden in a reference;
- nearby activation hijacks behavior;
- retained-context or multi-turn failure;
- wording, placement, missing stop, structure, fixture, host, or grader failure.

Read [grading priority](references/grading-priority.md) when evidence conflicts. Read [grading and revision](references/grading-and-revision.md) before deciding a revision or rerun.

## Stop

Stop rather than overclaim when required activation, first-read, composition, isolation, multi-turn, or host evidence is unavailable. Label a cheaper fallback Diagnostic, Reduced-Fidelity, or Unsupported when it changes the question. The owner decides disputed behavior and readiness.
Pliki w paczce (36)
  • SKILL.md 5 KB
  • references/eval-patterns.md 3 KB
  • references/evaluation-architecture.md 2 KB
  • references/grading-and-revision.md 2 KB
  • references/grading-priority.md 1 KB
  • references/portable-execution.md 2 KB
  • references/token-efficient-execution.md 1 KB
  • scripts/pi-composition-runtime.mjs 3 KB
  • scripts/pi-root-guard.mjs 3 KB
  • scripts/skill-eval.mjs 5 KB
  • scripts/lib/args.mjs 1 KB
  • scripts/lib/capabilities.mjs 8 KB
  • scripts/lib/codex-adapter.mjs 10 KB
  • scripts/lib/compact-evidence.mjs 10 KB
  • scripts/lib/constants.mjs 1 KB
  • scripts/lib/coordinator.mjs 7 KB
  • scripts/lib/decision.mjs 2 KB
  • scripts/lib/doctor.mjs 4 KB
  • scripts/lib/evaluate.mjs 31 KB
  • scripts/lib/feasibility.mjs 9 KB
  • scripts/lib/fixture-oracle.mjs 2 KB
  • scripts/lib/grade.mjs 8 KB
  • scripts/lib/hash.mjs 4 KB
  • scripts/lib/integrity.mjs 1 KB
  • scripts/lib/materialize.mjs 8 KB
  • scripts/lib/outcome.mjs 4 KB
  • scripts/lib/path-policy.mjs 3 KB
  • scripts/lib/pi-adapter.mjs 21 KB
  • scripts/lib/plan.mjs 28 KB
  • scripts/lib/process-outcome.mjs 2 KB
  • scripts/lib/process.mjs 4 KB
  • scripts/lib/publication.mjs 2 KB
  • scripts/lib/rpc-client.mjs 12 KB
  • scripts/lib/semantic-evidence.mjs 12 KB
  • scripts/lib/semantic.mjs 15 KB
  • scripts/lib/workspace.mjs 29 KB

Wersje

RepozytoriumWynikLicencja
Ta strona · hassan-mohiddin/freeflow100MIT
every-app/open-seo100MIT
laurigates/claude-plugins100MIT
ANcpLua/ancplua-claude-plugins100MIT
Thisisjuke/skills100MIT

Migawka bazy z 13.11.2026 · wersja paczki c8c725c4b059 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 90evaluate-behavior
    Testuje i porównuje zachowanie agenta: czy skill się uruchamia, jak zmiany promptu lub modelu wpływają na wyniki i jakie wnioski są uprawnione.
  • 100agent-evaluation
    Testuje jakość skilli, komend i agentów: po utworzeniu nowego, przed wdrożeniem agenta lub gdy zachowanie jest niespójne.
  • 100agent-skills-review
    Ocenia jakość SKILL.md pod kątem gotowości do Waza i zgodności z zasadami tworzenia skilli agentów, np. przed wydaniem albo w recenzji PR.
  • 100make-skill
    Pomaga tworzyć, ulepszać, porównywać, oceniać i pakować Agent Skills według specyfikacji agentskills.io: pisanie SKILL.md, frontmatter, opisy, progresywne ujawnianie, dystrybucję i wybór między skillem a MCP lub AGENTS.md.
  • 90write-skill
    Tworzy, edytuje, ocenia i testuje skille oraz powiązane pliki, takie jak skrypty i zasoby, sprawdzając ich skuteczność.
  • 100agent-skills-review
    Recenzuje jakość SKILL.md pod kątem gotowości Waza i zgodności ze standardami pisania skilli, przed wydaniem i przy przeglądzie pull requestów ze skillami.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.