eval
/eval
Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v3 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.
- Author
- Kanevry ↗
- Repository
- session-orchestrator ↗
- Licence
- MIT
- Last repository change
- 26.10.2026
Assessment and security
- Files in this package
- 2
- Scripts
- 0
- HTML comments
- 1
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 26.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
- SKILL.md zawiera komentarze HTML (1). Nie widać ich w podglądzie Markdown, a model je czyta.
What it does and when to use it
Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v3 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.
Original author description
Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v3 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.
How it works
SKILL.md has no level-two headings · 14 lines, 159 words
SKILL.md preview
---
name: eval
description: Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v3 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.
metadata:
user-invocable: 'true'
argument-hint: '[--session <id>] [--no-write] [--verify <run-id>]'
tags: eval, measurement, quality, meta, standard
model: sonnet
model-preference: sonnet
model-preference-codex: gpt-5.4-mini
model-preference-cursor: claude-sonnet-4-6
args-schema: '[{"flag":"--session","description":"session_id to evaluate (default: last completed session via the resolution cascade)"},{"flag":"--no-write","description":"Evaluate + report without appending to the eval journal (.orchestrator/metrics/eval.jsonl)"},{"flag":"--verify","description":"Re-evaluate a stored run-id and diff per-dimension for scoring drift (exit 1 on drift)"}]'
---
<!-- Generated by scripts/generate-codex-skills.mjs; do not edit. -->
# eval
Read [`skills/eval/SKILL.md`](../../../skills/eval/SKILL.md) in full before taking any action, then follow its complete workflow.
The linked document is authoritative, including its prechecks, argument rules, and stop conditions.
Resolve this link relative to this SKILL.md, not the project working directory. The plugin root is three directories above this file. Resolve package paths such as `skills/` and `scripts/` from that root; resolve relative links inside the canonical document from its own directory. Keep the user’s project as the target of project operations.
`$ARGUMENTS` means the trailing user input after the selected command skill, or an empty string when absent. Preserve flags, quoted text, and Unicode as data. Do not perform global substitution in the command document, shell expansion on the argument string, or execution of that string as shell code. When the workflow needs a command, pass its arguments through structured tool parameters or safely quoted individual arguments.
Regenerate with `node scripts/generate-codex-skills.mjs`.
Files in this package (2)
- SKILL.md 2 KB
- agents/openai.yaml 1 KB
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · Kanevry/session-orchestrator | 81 | MIT |
| relai-ai/agent-eval-loop | 100 | Apache-2.0 |
| jisuzzzz/haechi | 100 | MIT |
| alirezarezvani/claude-skills | 100 | MIT |
| ai-analyst-lab/ai-analyst | 100 | MIT |
| ai-analyst-lab/ai-analyst-plugin | 100 | MIT |
| prashantxo/codeforge | 100 | MIT |
| pspdfkit-labs/agentic-usability | 100 | Apache-2.0 |
| netanel-abergel/pa-skills | 100 | MIT |
Database snapshot: 13.11.2026 · package version 77393b63586c (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100agentic-evaluation-framework
This skill should be used when the user asks to "evaluate LLM output quality", "set up LLM-as-judge", "build an eval rubric", "compare model outputs pairwise", or "measure agent quality". - 90om-judge-agent-session
Judge generated Open Mercato code from a standalone harness eval or a user-shared agent session bundle. Use for LLM-as-judge validation, generative eval review, session/artifact analysis, harness-quality diagnosis, "judge this session", "analyze this eval", or "oceń… - 100llm-feature-eval
Design and run an evaluation suite for a product feature built on an LLM (summary, classification, extraction, chat assistant): golden dataset, rubric and graders (exact match, schema, LLM-as-judge with calibration), regression thresholds, cost and latency, plus edge and… - 90llm-as-judge
Score open-ended LLM/agent outputs with another LLM as evaluator while controlling for known judge biases. Use when: score open-ended output, llm as judge, evaluation rubric, pairwise comparison eval, judge bias mitigation, calibrate judge against humans. - 100overfit-skill-eval
Build and run an eval harness for an agent skill or prompt LOCALLY with Overfit — deterministic (seeded/greedy), offline, zero API cost, with schema-guaranteed rubric grading. Use when asked to test/evaluate/score a skill or prompt, catch prompt regressions, or measure whether a… - 100llm-as-judge
Use an LLM as an evaluator for open-ended outputs — rubrics, pairwise comparison, calibration with human labels, bias mitigation. Covers when LLM-judge works, when it fails, and how to trust its scores. Use this skill when evaluating generative outputs at scale, building eval…