Skip to content
Skills Library
AI & agents · Review & audit

answer-grader

/answer-grader

Grade a written answer (release notes, summaries, commit messages, JSON payloads) against checks using the shared evaluators of the agent infra, store every run in the evals UI and track the score trend per suite. Use when the user says "grade this answer", "check this output", "does this meet the criteria", "score the release notes" or "show grading history".

Licence
Unlicense
Last repository change
13.11.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
7
Scripts
2
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 13.11.2026, without references to retired models or APIs.

Licence

Unlicense, otwarta.

What to check before installing
  • 2 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: scripts/agentinfra.py, scripts/grade.py

What it does and when to use it

Grade a written answer (release notes, summaries, commit messages, JSON payloads) against checks using the shared evaluators of the agent infra, store every run in the evals UI and track the score trend per suite. Use when the user says "grade this answer", "check this output", "does this meet the criteria", "score the release notes" or "show grading history".

Triggers from the description

grade this answercheck this outputdoes this meet the criteriascore the release notesshow grading history
Original author description

Grade a written answer (release notes, summaries, commit messages, JSON payloads) against checks using the shared evaluators of the agent infra, store every run in the evals UI and track the score trend per suite. Use when the user says "grade this answer", "check this output", "does this meet the criteria", "score the release notes" or "show grading history".

How it works

  1. Commands
  2. Rules
Original SKILL.md headings · 31 lines, 202 words
SKILL.md preview
---
name: answer-grader
description: Grade a written answer (release notes, summaries, commit messages, JSON payloads) against checks using the shared evaluators of the agent infra, store every run in the evals UI and track the score trend per suite. Use when the user says "grade this answer", "check this output", "does this meet the criteria", "score the release notes" or "show grading history".
---

# Answer Grader

Turns acceptance criteria into checks, grades an answer with the common evaluators of the
agent infra evals service, and remembers the score history per suite in this skill's own state
tenant (`answer-grader`). Every run shows up in the evals UI and in the centralized logs.

All commands run `scripts/grade.py` from this skill's base directory with Python 3.15 and print JSON.

## Commands

| Goal | Command |
|---|---|
| See the evaluators | `python3.15 scripts/grade.py evaluators` |
| Grade an answer | `python3.15 scripts/grade.py grade --suite <suite> --answer <file> --checks '<json list>'` |
| See the trend | `python3.15 scripts/grade.py history --suite <suite>` |

A check is `{"type": "<evaluator>", "value": <expected>}`, such as:

```json
[{"type": "contains", "value": "Breaking changes"}, {"type": "max_length", "value": 1200}, {"type": "not_contains", "value": "TODO"}]
```

## Rules

1. Write the answer to a file first, then grade the file.
2. Derive checks only from criteria the user gave. Do not invent criteria.
3. Use the same suite name for the same kind of answer so the trend is meaningful.
4. When `failed_checks` is not empty, fix the answer and grade again, at most 3 times.
5. Report the score, the trend and each failed check's `detail`.
Files in this package (7)
  • SKILL.md 2 KB
  • evals/suite.json 2 KB
  • evals/fixtures/bad.md 1 KB
  • evals/fixtures/checks.json 1 KB
  • evals/fixtures/good.md 1 KB
  • scripts/agentinfra.py 1 KB
  • scripts/grade.py 3 KB

Versions

The database has no other skills named answer-grader.

Database snapshot: 13.11.2026 · package version 54018ba0928a (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100run-iteration-eval
    Run the Logic-Lens content-eval pipeline for one iteration and produce a scored summary.json — use to measure a skill change. Wraps scripts/run-content-evals.sh (runner, costs tokens) and scripts/grade-iteration.py (grader, free, re-runnable). ALWAYS sync the plugin cache first.…
  • 100skill-grader
    Evaluates Claude Agent Skills on 10 quality axes with letter grades (A+ through F) and specific improvement recommendations. Use when auditing a skill, comparing skills, prioritizing improvements, or performing quality control on a skill library. Activate on "grade skill",…
  • 100skill-grader
    Evaluates Claude Agent Skills on 10 quality axes with letter grades (A+ through F) and specific improvement recommendations. Use when auditing a skill, comparing skills, prioritizing improvements, or performing quality control on a skill library. Activate on "grade skill",…
  • 100model-grader
    Use this skill when someone needs to define what correct output means for an AI feature so that a grader can be built from the answers. Triggers on "build a grader", "define good", "what should the eval check", "how do we know the agent got it right", "write the acceptance…
  • 100hf-release-notes
    Generate Hugging Face Hub (huggingface_hub) release notes from cached PR JSON files. Use when asked to draft release notes from PR files.
  • 100evals
    Designing and maintaining evaluations for LLM prompts, agents, and Agent Skills: success criteria, test-set composition, code/LLM-judge/human graders, trajectory and tool-use grading, trigger evals, regression suites, eval saturation. WHEN: \"eval\", \"evaluation suite\",…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.