dev-prompt-evaluation
/dev-prompt-evaluation
Use to design, test, compare, version, and validate prompts or LLM behavior with measurable criteria and datasets.
- Author
- blueskyxn ↗
- Repository
- codex-is-all-you-need ↗
- Licence
- GPL-3.0
- Last repository change
- 17.09.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 17.09.2026, without references to retired models or APIs.
GPL-3.0, otwarta.
What it does and when to use it
Use to design, test, compare, version, and validate prompts or LLM behavior with measurable criteria and datasets.
Original author description
Use to design, test, compare, version, and validate prompts or LLM behavior with measurable criteria and datasets.
How it works
- Steps
- Output
- Do not
SKILL.md preview
--- name: dev-prompt-evaluation description: Use to design, test, compare, version, and validate prompts or LLM behavior with measurable criteria and datasets. metadata: version: "0.3" updated: "2026-06-12" --- # Prompt evaluation workflow Use this workflow when prompts, agent instructions, LLM outputs, or AI app behavior need systematic evaluation. ## Steps 1. Define the prompt surface. - System, developer, user, tool, retrieval, memory, and output-format instructions. - Runtime model, temperature or reasoning settings, tool access, and context limits when known. - Current failure modes or quality goals. 2. Define success criteria. - Correctness, completeness, citation quality, format compliance, tone, latency, cost, refusal behavior, or safety. - Pass/fail checks where possible. - Human review rubric when automated scoring is not enough. 3. Build an evaluation set. - Representative normal cases. - Edge cases and adversarial cases. - Regression cases from previous failures. - Expected outputs, constraints, or scoring notes. 4. Compare prompt variants. - Keep one controlled change per variant when feasible. - Track version, rationale, expected improvement, and observed outcome. - Measure token usage and latency if available. 5. Validate tool and retrieval behavior. - Check whether the prompt asks for unavailable tools or hidden context. - Verify citations, file references, and retrieved evidence. - Confirm that fallback behavior is explicit. 6. Decide deployment readiness. - Keep the simplest prompt that meets quality goals. - Document known weaknesses and monitoring signals. - Add regression examples for failures that should not return. ## Output Return: 1. Prompt surface reviewed 2. Evaluation criteria 3. Test set or representative cases 4. Variant comparison 5. Recommended prompt version 6. Cost, latency, safety, and monitoring notes ## Do not - Do not optimize by vibes only. - Do not compare multiple changes without recording what changed. - Do not claim robustness from a tiny or unrepresentative test set. - Do not assume a model, tool, or retrieval source exists without checking runtime evidence.
Versions
The database has no other skills named dev-prompt-evaluation.
Identical content appears in 1 other GitHub folders. We show one copy with the highest score.
Database snapshot: 13.11.2026 · package version b83e5f496b34 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100prompt-engineer
Design, test, and optimize prompts for LLM interactions. Cover prompt patterns (few-shot, chain-of-thought, ReAct), system prompt design, output formatting, prompt evaluation, and prompt optimization techniques. Triggers on "write prompt", "optimize prompt", "design system… - 100prompt-tester
Design, test, and iterate on AI prompts systematically using structured evaluation criteria. Use when building AI features, optimizing agent instructions, comparing prompt variants, or evaluating output quality across edge cases. Trigger words: prompt engineering, prompt… - 100genai-prompt-versioning
Use when the task is to version prompts and compare changes with repeatable examples. - 100llm-application-dev-prompt-optimize
Evaluate and improve LLM prompts using examples, structured instructions, and measured comparisons. Use when refining prompt reliability, output quality, or model-specific behavior. - 100senior-prompt-engineer
This skill should be used when the user asks to "optimize prompts", "design prompt templates", "evaluate LLM outputs", "build agentic systems", "implement RAG", "create few-shot examples", "analyze token usage", or "design AI workflows". Use for prompt engineering patterns, LLM… - 100prompt-engineering-expert
Expert prompt engineering, custom instruction, system prompt, and agent instruction design. Use when Codex needs to review, generate, refactor, debug, optimize, document, or test AI prompts; design reusable prompt templates; create or improve system prompts, custom instructions,…