Skip to content
Skills Library
AI & agents · Testing

bounded-evaluation

/bounded-evaluation

Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform.

Licence
MIT
Last repository change
14.10.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
3
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 14.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform.

Original author description

Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform.

How it works

  1. Use When
  2. Evaluation Contract
  3. Pairwise And Judge Rules
  4. Workflow
  5. Output Shape
  6. Guardrails
  7. Provenance
Original SKILL.md headings · 67 lines, 340 words
SKILL.md preview
---
name: bounded-evaluation
description: Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform.
license: MIT
metadata:
  abvx_status: experimental
  abvx_origin: adapted
---

# Bounded Evaluation

Use this skill when a change needs a small, explicit evaluation gate.

## Use When

- reviewing skill changes, AGENTS.md rules, prompt playbooks, or SET bundle changes;
- comparing two agent outputs or proposals;
- using an LLM judge where bias or weak rubrics could mislead;
- creating activation tests, regression cases, or ship gates for agent behavior;
- deciding whether a repeated workflow improved after a bounded edit.

## Evaluation Contract

A bounded eval must state:

- target artifact or behavior;
- candidate outputs or versions;
- rubric dimensions and weights, if any;
- pass/fail threshold;
- fixtures or scenarios;
- judge type: deterministic check, human review, LLM judge, or hybrid;
- known bias risks and mitigation.

## Pairwise And Judge Rules

- Compare A/B outputs against the same rubric.
- Swap candidate order when using an LLM judge to detect position bias.
- Require reasons before scores, but treat reasons as evidence to inspect, not truth.
- Mark the eval invalid when swapped order changes the winner without a defensible reason.
- Prefer deterministic checks for syntax, schema, file existence, and command success.
- Keep human approval for high-stakes or ambiguous decisions.

## Workflow

1. Define the smallest behavior being evaluated.
2. Pick 1-5 representative fixtures or scenarios.
3. Choose deterministic checks before LLM judging.
4. If using pairwise judging, run both candidate orders.
5. Record results, invalidations, and decision.
6. Feed accepted/rejected outcomes back into the relevant ledger.

## Output Shape

Use:

- eval target;
- fixtures;
- rubric;
- judge type;
- results;
- bias checks;
- decision: `accept`, `reject`, `revise`, or `invalid`;
- follow-up artifact or ledger update.

## Guardrails

- Do not publish benchmark claims without committed fixtures, arms, model, host, run count, and captured artifacts.
- Do not let an LLM judge approve its own rubric changes.
- Do not use pairwise preference as proof of correctness.
- Do not hide invalid or inconsistent judge results.

## Provenance

Adapted from advanced-evaluation patterns in `muratcankoylan/Agent-Skills-for-Context-Engineering`, aligned with ABVX SkillOpt validation gates.
Files in this package (3)
  • SKILL.md 3 KB
  • SKILL_CARD.md 1 KB
  • agents/openai.yaml 1 KB

Versions

The database has no other skills named bounded-evaluation.

Database snapshot: 13.11.2026 · package version d139456fe0ca (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100llm-as-judge
    Use an LLM as an evaluator for open-ended outputs — rubrics, pairwise comparison, calibration with human labels, bias mitigation. Covers when LLM-judge works, when it fails, and how to trust its scores. Use this skill when evaluating generative outputs at scale, building eval…
  • 100agent-evaluation
    This skill should be used when the user asks to "evaluate agent performance", "build test framework", "measure agent quality", "create evaluation rubrics", "implement LLM-as-judge", "compare model outputs", "mitigate evaluation bias", or mentions multi-dimensional evaluation,…
  • 90llm-as-judge
    Score open-ended LLM/agent outputs with another LLM as evaluator while controlling for known judge biases. Use when: score open-ended output, llm as judge, evaluation rubric, pairwise comparison eval, judge bias mitigation, calibrate judge against humans.
  • 100advanced-evaluation
    This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
  • 100advanced-evaluation
    This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
  • 100advanced-evaluation
    Design LLM-as-judge rubrics, direct scores and pairwise comparisons. Use for judge models, evals, position bias and calibration against human labels.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.