bounded-evaluation
/bounded-evaluation
Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform.
- Author
- markoblogo ↗
- Repository
- abvx-agent-skills ↗
- Licence
- MIT
- Last repository change
- 14.10.2026
Assessment and security
- Files in this package
- 3
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 14.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform.
Original author description
Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform.
How it works
- Use When
- Evaluation Contract
- Pairwise And Judge Rules
- Workflow
- Output Shape
- Guardrails
- Provenance
SKILL.md preview
--- name: bounded-evaluation description: Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform. license: MIT metadata: abvx_status: experimental abvx_origin: adapted --- # Bounded Evaluation Use this skill when a change needs a small, explicit evaluation gate. ## Use When - reviewing skill changes, AGENTS.md rules, prompt playbooks, or SET bundle changes; - comparing two agent outputs or proposals; - using an LLM judge where bias or weak rubrics could mislead; - creating activation tests, regression cases, or ship gates for agent behavior; - deciding whether a repeated workflow improved after a bounded edit. ## Evaluation Contract A bounded eval must state: - target artifact or behavior; - candidate outputs or versions; - rubric dimensions and weights, if any; - pass/fail threshold; - fixtures or scenarios; - judge type: deterministic check, human review, LLM judge, or hybrid; - known bias risks and mitigation. ## Pairwise And Judge Rules - Compare A/B outputs against the same rubric. - Swap candidate order when using an LLM judge to detect position bias. - Require reasons before scores, but treat reasons as evidence to inspect, not truth. - Mark the eval invalid when swapped order changes the winner without a defensible reason. - Prefer deterministic checks for syntax, schema, file existence, and command success. - Keep human approval for high-stakes or ambiguous decisions. ## Workflow 1. Define the smallest behavior being evaluated. 2. Pick 1-5 representative fixtures or scenarios. 3. Choose deterministic checks before LLM judging. 4. If using pairwise judging, run both candidate orders. 5. Record results, invalidations, and decision. 6. Feed accepted/rejected outcomes back into the relevant ledger. ## Output Shape Use: - eval target; - fixtures; - rubric; - judge type; - results; - bias checks; - decision: `accept`, `reject`, `revise`, or `invalid`; - follow-up artifact or ledger update. ## Guardrails - Do not publish benchmark claims without committed fixtures, arms, model, host, run count, and captured artifacts. - Do not let an LLM judge approve its own rubric changes. - Do not use pairwise preference as proof of correctness. - Do not hide invalid or inconsistent judge results. ## Provenance Adapted from advanced-evaluation patterns in `muratcankoylan/Agent-Skills-for-Context-Engineering`, aligned with ABVX SkillOpt validation gates.
Files in this package (3)
- SKILL.md 3 KB
- SKILL_CARD.md 1 KB
- agents/openai.yaml 1 KB
Versions
The database has no other skills named bounded-evaluation.
Database snapshot: 13.11.2026 · package version d139456fe0ca (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100llm-as-judge
Use an LLM as an evaluator for open-ended outputs — rubrics, pairwise comparison, calibration with human labels, bias mitigation. Covers when LLM-judge works, when it fails, and how to trust its scores. Use this skill when evaluating generative outputs at scale, building eval… - 100agent-evaluation
This skill should be used when the user asks to "evaluate agent performance", "build test framework", "measure agent quality", "create evaluation rubrics", "implement LLM-as-judge", "compare model outputs", "mitigate evaluation bias", or mentions multi-dimensional evaluation,… - 90llm-as-judge
Score open-ended LLM/agent outputs with another LLM as evaluator while controlling for known judge biases. Use when: score open-ended output, llm as judge, evaluation rubric, pairwise comparison eval, judge bias mitigation, calibrate judge against humans. - 100advanced-evaluation
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment. - 100advanced-evaluation
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment. - 100advanced-evaluation
Design LLM-as-judge rubrics, direct scores and pairwise comparisons. Use for judge models, evals, position bias and calibration against human labels.