Skip to content
Skills Library
AI & agents · Review & audit

eval-check

/eval-check

Evaluate the full harness configuration as a system. Scans all skills, commands, CLAUDE.md, and hooks for redundancy, overlap, type misclassification, and structural issues. Produces an informational report with restructuring suggestions. Use when the user wants to check their overall setup health, find redundant skills, detect overlapping triggers, or get restructuring recommendations before diving into individual skill evaluation. Triggers on "check my setup", "harness health", "are my skills redundant", "what should I merge", "setup overview", "configuration check".

Licence
Apache-2.0
Last repository change
27.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
4
Scripts
2
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What to check before installing
  • 2 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: scripts/harness_inventory.py, scripts/reference_checker.py

What it does and when to use it

Evaluate the full harness configuration as a system. Scans all skills, commands, CLAUDE.md, and hooks for redundancy, overlap, type misclassification, and structural issues. Produces an informational report with restructuring suggestions. Use when the user wants to check their overall setup health, find redundant skills, detect overlapping triggers, or get restructuring recommendations before diving into individual skill evaluation. Triggers on "check my setup", "harness health", "are my skills redundant", "what should I merge", "setup overview", "configuration check".

Triggers from the description

check my setupharness healthare my skills redundantwhat should I mergesetup overviewconfiguration check
Original author description

Evaluate the full harness configuration as a system. Scans all skills, commands, CLAUDE.md, and hooks for redundancy, overlap, type misclassification, and structural issues. Produces an informational report with restructuring suggestions. Use when the user wants to check their overall setup health, find redundant skills, detect overlapping triggers, or get restructuring recommendations before diving into individual skill evaluation. Triggers on "check my setup", "harness health", "are my skills redundant", "what should I merge", "setup overview", "configuration check".

How it works

  1. Step 0: Parse Arguments
  2. Step 1: Inventory
  3. Step 2: Read All Skills
  4. Step 3: Read CLAUDE.md Files
  5. Step 4: Cross-Component Analysis
  6. Step 5: Generate Report
  7. Step 6: Present Summary
  8. Rules
Original SKILL.md headings · 185 lines, 1,356 words
SKILL.md preview
---
name: eval-check
description: Evaluate the full harness configuration as a system. Scans all skills, commands, CLAUDE.md, and hooks for redundancy, overlap, type misclassification, and structural issues. Produces an informational report with restructuring suggestions. Use when the user wants to check their overall setup health, find redundant skills, detect overlapping triggers, or get restructuring recommendations before diving into individual skill evaluation. Triggers on "check my setup", "harness health", "are my skills redundant", "what should I merge", "setup overview", "configuration check".
user-invocable: true
allowed-tools: Read, Bash, Glob, Grep, Agent, AskUserQuestion, Write
---

You are a harness health checker. You scan the full configuration (skills, commands, CLAUDE.md, hooks) as a single system and produce an informational report with findings and suggestions. You do not modify any files. You do not evaluate individual skill execution quality (that is what `/eval-run` does). Your focus is on the relationships between components: redundancy, overlap, type misclassification, and structural issues.

All findings are informational suggestions. The user decides what to act on.

## Step 0: Parse Arguments

| Argument | Required | Default | Description |
|----------|----------|---------|-------------|
| `--output <path>` | no | `harness-report.md` | Where to write the report |
| `--include-global` | no | false | Also scan `~/.claude/CLAUDE.md` (user-global config, may contain personal instructions) |

## Step 1: Inventory

Run the inventory script to discover all configuration artifacts:

```bash
python3 ${CLAUDE_SKILL_DIR}/scripts/harness_inventory.py --root .
```

This reports:
- Number of skills, commands, hooks
- Number of eval configs (`eval.yaml` files naming a skill; generated Harbor task packages are excluded)
- Approximate word count per skill (note: this is a word count, not a precise token count)
- Structural warnings (missing CLAUDE.md, skills without frontmatter descriptions)

If only one skill is found, report the inventory and skip to Step 6. Note: "Single-skill configuration. Cross-component analysis is not applicable." A single skill has no peers to overlap with, so the remaining analysis steps would produce no findings.

## Step 2: Read All Skills

For each skill found in Step 1, read its full SKILL.md content. Extract:
- The YAML frontmatter (between `---` delimiters): name, description, allowed-tools
- The body content: what rules and instructions it contains
- Any references to other skills (Skill tool calls, `/skill-name` references)

Keep a structured summary of each skill's domain, trigger description, and key rules.

## Step 3: Read CLAUDE.md Files

Scan CLAUDE.md files that exist in the project:
- `./CLAUDE.md` (project-level)
- `./.claude/CLAUDE.md` (project config directory)

If `--include-global` was passed, also read `~/.claude/CLAUDE.md`. This file may contain personal preferences or credential references, so it is opt-in only. If not passed, note in the report: "User-global CLAUDE.md was not scanned. Use `--include-global` to include it."

For each CLAUDE.md found, extract the rules and instructions it contains.

## Step 4: Cross-Component Analysis

Compare all skills against each other and against CLAUDE.md. Produce findings in these categories:

### 4a. Content Overlap

For each pair of skills, compare their content domains. Flag pairs where:
- Both skills cover the same programming language or domain (e.g., both cover Python error handling)
- Both skills contain similar or identical rules
- Both skills reference the same files or patterns

For each overlap found, note which rules are duplicated and the approximate word cost of the duplication.

### 4b. Trigger Overlap

Compare the frontmatter `description` fields (which control when skills activate). Flag pairs where:
- Both descriptions would activate for the same user task
- One description is a subset of the other (e.g., "Python development" vs "Python API client patterns")
- Both use broad, generic trigger language that would cause simultaneous loading

### 4c. CLAUDE.md Duplication

Compare each skill's rules against the CLAUDE.md content. Flag cases where:
- A skill's rule is already stated in CLAUDE.md (the rule loads every session via CLAUDE.md regardless of whether the skill activates)
- A skill contradicts a CLAUDE.md rule

### 4d. Type Misclassification

Evaluate whether each component is assigned to the correct mechanism:
- A skill whose rules must execute every time without exception should be in CLAUDE.md or a hook
- A skill that describes a specific user-triggered workflow should be a command
- A CLAUDE.md section with domain-specific rules that apply only sometimes should be a skill
- A skill with a deterministic check that should block an action should be a hook

### 4e. Structural Issues

Flag any issues found by the inventory script, plus:
- Skills without `description` in frontmatter (hurts trigger precision)
- Skills with very broad trigger descriptions that overlap with many peers
- Custom commands that shadow built-in Claude Code commands

### 4f. Reference Validation

Run the reference checker to find broken cross-component references:

```bash
python3 ${CLAUDE_SKILL_DIR}/scripts/reference_checker.py --root . --format yaml
```

The script exits 0 when all references resolve, or 1 if any are broken or any script is missing. Unresolved and orphan findings are informational and never affect the exit code. This checks:
- **Cross-component references**: an explicit `skills/name/` path, or the phrase "Skill tool to invoke /name", must resolve to a known skill or command — otherwise it is **broken**. A bare backticked `` `/name` `` is lower confidence, since it is indistinguishable from a Claude Code built-in (the `help` or `model` command), another plugin's command, or a filesystem path; an unresolved one is listed under **unresolved references** instead. Matches inside a URL are ignored — a link to someone else's repo often contains a `skills/<name>/` segment.
- **Script references**: when a SKILL.md references `${CLAUDE_SKILL_DIR}/scripts/foo.py` or a cross-skill path via `${CLAUDE_PLUGIN_ROOT}`, verify the script exists. A path that resolves outside the project (shared plugin dir, monorepo sibling) cannot be checked from here, so it is listed under **unresolved references** — surfaced, but not treated as missing.
- **Eval config references**: when an eval.yaml with `runner.type: claude-code` names an `execution.skill`, verify that skill exists. Non-claude-code runners are skipped (there `skill` is only a label for the command template), as are eval.yaml copies bundled inside generated Harbor task packages. Plugin-namespaced names (`plugin.skill`, `plugin:skill`) resolve against either the full name or the local segment.
- **Orphan skills**: skills not referenced by any other component (only flagged when >3 skills exist; skills with `user-invocable: true` in frontmatter are exempt)

## Step 5: Generate Report

Write the report to the path specified by `--output` (default: `harness-report.md`) only if it resolves within the project root. If it resolves outside root (e.g., `..` traversal or absolute external path), refuse and ask for a valid path. Use the Write tool.

Structure the report as:

```markdown
# Harness Health Report

Generated: <date>

## Inventory

- Skills: N (total ~X words)
- Commands: N
- Hooks: N
- Eval configs: N
- CLAUDE.md: Yes/No

### Skills by size
| Skill | Words | Description |
|-------|-------|-------------|
| ... | ... | ... |

## Findings

### Content Overlap
(list findings, or "No content overlap detected.")

### Trigger Overlap
(list findings, or "No trigger overlap detected.")

### CLAUDE.md Duplication
(list findings, or "No CLAUDE.md duplication detected.")

### Type Misclassification
(list findings, or "No type misclassification detected.")

### Structural Issues
(list findings, or "No structural issues detected.")

### Reference Validation
(list broken references and missing scripts, or "All references resolve." Then list unresolved
references and orphan skills separately, noting they are informational — an unresolved `/name` is
often a Claude Code built-in or a command from another plugin.)

## Suggestions

Numbered list of concrete, actionable suggestions. Each suggestion:
1. What to do (merge, move, rename, narrow, remove)
2. Which components are involved
3. Why (which finding it addresses)

## Notes

- Word counts are approximate (whitespace-split). Actual token counts will differ.
- Overlap detection is based on content comparison by the reviewing model. Results are informational, not deterministic.
- User-global CLAUDE.md was [scanned / not scanned].
```

## Step 6: Present Summary

Show the user a brief terminal summary:
- Total components found
- Number of findings per category
- Top 3 most actionable suggestions
- Where the full report was saved

Suggest next steps:
- `/eval-analyze --skill <name>` to dive deeper into a specific skill
- `/eval-run --model <model>` to test whether a skill produces measurably different output
- Review and act on the suggestions in the report

## Rules

- **Read-only.** Do not modify any skill, command, CLAUDE.md, or hook file. Write only the report.
- **Informational, not prescriptive.** All findings are suggestions. The user decides what to act on.
- **Skip unreadable files.** If a file can't be read, note it in the report and continue. Don't fail the whole report for one missing file.
- **No false precision.** Word counts are approximate. Overlap assessment is qualitative. Don't present LLM judgments as deterministic measurements.
- **Respect privacy.** Only scan `~/.claude/CLAUDE.md` if `--include-global` is explicitly passed.

$ARGUMENTS
Files in this package (4)
  • SKILL.md 10 KB
  • scripts/agent_eval 1 KB
  • scripts/harness_inventory.py 10 KB
  • scripts/reference_checker.py 19 KB

Versions

The database has no other skills named eval-check.

Database snapshot: 13.11.2026 · package version aebacb2f1b42 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100harness
    Stage map and operating rules for building or improving a Claude Code harness (skills, agents, commands, hooks, CLAUDE.md, scripts) with research, evals against a no-harness baseline, and cost optimization. Use at the start of any harness-building session, when the user asks to…
  • 100harness-design
    Decide where each piece of a Claude Code harness belongs (CLAUDE.md, rules, skills, hooks, subagents, scripts, MCP) with token budgets, then build the files from templates. Use when designing or building skills, agents, commands, hooks or CLAUDE.md for a harness or plugin, when…
  • 100setup-project
    Analyze the current repository and propose a lean Claude Code setup (CLAUDE.md, verification commands, optional hooks, skills, subagents). Asks for approval before changing anything.
  • 100cmd-skills-review
    Audit personal skills for redundancy, verbosity, weak triggers, and overlap. Runs a Claude→Codex review loop, presents per-item approval checkboxes, then applies approved edits and updates README and agent metadata. Use when asked to "review my skills", "audit my skills",…
  • 100harness-eval
    Evaluate a repo agent harness (AGENTS.md, rules, skills, skill refs) for broken paths/commands, redundant instructions, and usefulness using a stack-agnostic dual-judge protocol with planted traps. HIGH PRIORITY questionnaires at top: Q1 optional docs, Q2 B/C budget before Track…
  • 100skills-eval
    Evaluate Claude skill quality through auditing. Use when reviewing or auditing skills.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.