Skip to content
Skills Library
AI & agents · Testing

eval

/eval

Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v3 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.

Licence
MIT
Last repository change
26.10.2026

Assessment and security

81/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content16 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
2
Scripts
0
HTML comments
1
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 26.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What to check before installing
  • SKILL.md zawiera komentarze HTML (1). Nie widać ich w podglądzie Markdown, a model je czyta.

What it does and when to use it

Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v3 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.

Original author description

Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v3 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.

How it works

SKILL.md has no level-two headings · 14 lines, 159 words

SKILL.md preview
---
name: eval
description: Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v3 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.
metadata:
  user-invocable: 'true'
  argument-hint: '[--session <id>] [--no-write] [--verify <run-id>]'
  tags: eval, measurement, quality, meta, standard
  model: sonnet
  model-preference: sonnet
  model-preference-codex: gpt-5.4-mini
  model-preference-cursor: claude-sonnet-4-6
  args-schema: '[{"flag":"--session","description":"session_id to evaluate (default: last completed session via the resolution cascade)"},{"flag":"--no-write","description":"Evaluate + report without appending to the eval journal (.orchestrator/metrics/eval.jsonl)"},{"flag":"--verify","description":"Re-evaluate a stored run-id and diff per-dimension for scoring drift (exit 1 on drift)"}]'
---

<!-- Generated by scripts/generate-codex-skills.mjs; do not edit. -->

# eval

Read [`skills/eval/SKILL.md`](../../../skills/eval/SKILL.md) in full before taking any action, then follow its complete workflow.
The linked document is authoritative, including its prechecks, argument rules, and stop conditions.

Resolve this link relative to this SKILL.md, not the project working directory. The plugin root is three directories above this file. Resolve package paths such as `skills/` and `scripts/` from that root; resolve relative links inside the canonical document from its own directory. Keep the user’s project as the target of project operations.

`$ARGUMENTS` means the trailing user input after the selected command skill, or an empty string when absent. Preserve flags, quoted text, and Unicode as data. Do not perform global substitution in the command document, shell expansion on the argument string, or execution of that string as shell code. When the workflow needs a command, pass its arguments through structured tool parameters or safely quoted individual arguments.

Regenerate with `node scripts/generate-codex-skills.mjs`.
Files in this package (2)
  • SKILL.md 2 KB
  • agents/openai.yaml 1 KB

Versions

RepositoryScoreLicence
This page · Kanevry/session-orchestrator81MIT
relai-ai/agent-eval-loop100Apache-2.0
jisuzzzz/haechi100MIT
alirezarezvani/claude-skills100MIT
ai-analyst-lab/ai-analyst100MIT
ai-analyst-lab/ai-analyst-plugin100MIT
prashantxo/codeforge100MIT
pspdfkit-labs/agentic-usability100Apache-2.0
netanel-abergel/pa-skills100MIT

Database snapshot: 13.11.2026 · package version 77393b63586c (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100agentic-evaluation-framework
    This skill should be used when the user asks to "evaluate LLM output quality", "set up LLM-as-judge", "build an eval rubric", "compare model outputs pairwise", or "measure agent quality".
  • 90om-judge-agent-session
    Judge generated Open Mercato code from a standalone harness eval or a user-shared agent session bundle. Use for LLM-as-judge validation, generative eval review, session/artifact analysis, harness-quality diagnosis, "judge this session", "analyze this eval", or "oceń…
  • 100llm-feature-eval
    Design and run an evaluation suite for a product feature built on an LLM (summary, classification, extraction, chat assistant): golden dataset, rubric and graders (exact match, schema, LLM-as-judge with calibration), regression thresholds, cost and latency, plus edge and…
  • 90llm-as-judge
    Score open-ended LLM/agent outputs with another LLM as evaluator while controlling for known judge biases. Use when: score open-ended output, llm as judge, evaluation rubric, pairwise comparison eval, judge bias mitigation, calibrate judge against humans.
  • 100overfit-skill-eval
    Build and run an eval harness for an agent skill or prompt LOCALLY with Overfit — deterministic (seeded/greedy), offline, zero API cost, with schema-guaranteed rubric grading. Use when asked to test/evaluate/score a skill or prompt, catch prompt regressions, or measure whether a…
  • 100llm-as-judge
    Use an LLM as an evaluator for open-ended outputs — rubrics, pairwise comparison, calibration with human labels, bias mitigation. Covers when LLM-judge works, when it fails, and how to trust its scores. Use this skill when evaluating generative outputs at scale, building eval…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.