Skip to content
Skills Library
AI & agents · Testing

dev-prompt-evaluation

/dev-prompt-evaluation

Use to design, test, compare, version, and validate prompts or LLM behavior with measurable criteria and datasets.

Licence
GPL-3.0
Last repository change
17.09.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 17.09.2026, without references to retired models or APIs.

Licence

GPL-3.0, otwarta.

What it does and when to use it

Use to design, test, compare, version, and validate prompts or LLM behavior with measurable criteria and datasets.

Original author description

Use to design, test, compare, version, and validate prompts or LLM behavior with measurable criteria and datasets.

How it works

  1. Steps
  2. Output
  3. Do not
Original SKILL.md headings · 56 lines, 289 words
SKILL.md preview
---
name: dev-prompt-evaluation
description: Use to design, test, compare, version, and validate prompts or LLM behavior with measurable criteria and datasets.
metadata:
  version: "0.3"
  updated: "2026-06-12"
---

# Prompt evaluation workflow

Use this workflow when prompts, agent instructions, LLM outputs, or AI app behavior need systematic evaluation.

## Steps

1. Define the prompt surface.
   - System, developer, user, tool, retrieval, memory, and output-format instructions.
   - Runtime model, temperature or reasoning settings, tool access, and context limits when known.
   - Current failure modes or quality goals.

2. Define success criteria.
   - Correctness, completeness, citation quality, format compliance, tone, latency, cost, refusal behavior, or safety.
   - Pass/fail checks where possible.
   - Human review rubric when automated scoring is not enough.

3. Build an evaluation set.
   - Representative normal cases.
   - Edge cases and adversarial cases.
   - Regression cases from previous failures.
   - Expected outputs, constraints, or scoring notes.

4. Compare prompt variants.
   - Keep one controlled change per variant when feasible.
   - Track version, rationale, expected improvement, and observed outcome.
   - Measure token usage and latency if available.

5. Validate tool and retrieval behavior.
   - Check whether the prompt asks for unavailable tools or hidden context.
   - Verify citations, file references, and retrieved evidence.
   - Confirm that fallback behavior is explicit.

6. Decide deployment readiness.
   - Keep the simplest prompt that meets quality goals.
   - Document known weaknesses and monitoring signals.
   - Add regression examples for failures that should not return.

## Output

Return:

1. Prompt surface reviewed
2. Evaluation criteria
3. Test set or representative cases
4. Variant comparison
5. Recommended prompt version
6. Cost, latency, safety, and monitoring notes

## Do not

- Do not optimize by vibes only.
- Do not compare multiple changes without recording what changed.
- Do not claim robustness from a tiny or unrepresentative test set.
- Do not assume a model, tool, or retrieval source exists without checking runtime evidence.

Versions

The database has no other skills named dev-prompt-evaluation.

Identical content appears in 1 other GitHub folders. We show one copy with the highest score.

Database snapshot: 13.11.2026 · package version b83e5f496b34 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100prompt-engineer
    Design, test, and optimize prompts for LLM interactions. Cover prompt patterns (few-shot, chain-of-thought, ReAct), system prompt design, output formatting, prompt evaluation, and prompt optimization techniques. Triggers on "write prompt", "optimize prompt", "design system…
  • 100prompt-tester
    Design, test, and iterate on AI prompts systematically using structured evaluation criteria. Use when building AI features, optimizing agent instructions, comparing prompt variants, or evaluating output quality across edge cases. Trigger words: prompt engineering, prompt…
  • 100genai-prompt-versioning
    Use when the task is to version prompts and compare changes with repeatable examples.
  • 100llm-application-dev-prompt-optimize
    Evaluate and improve LLM prompts using examples, structured instructions, and measured comparisons. Use when refining prompt reliability, output quality, or model-specific behavior.
  • 100senior-prompt-engineer
    This skill should be used when the user asks to "optimize prompts", "design prompt templates", "evaluate LLM outputs", "build agentic systems", "implement RAG", "create few-shot examples", "analyze token usage", or "design AI workflows". Use for prompt engineering patterns, LLM…
  • 100prompt-engineering-expert
    Expert prompt engineering, custom instruction, system prompt, and agent instruction design. Use when Codex needs to review, generate, refactor, debug, optimize, document, or test AI prompts; design reusable prompt templates; create or improve system prompts, custom instructions,…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.