uxd-prototype-evaluate
/uxd-prototype-evaluate
Evaluate a running prototype against a Jira ticket's acceptance criteria, automatically fix what fails, then run persona-based usability walkthroughs. Produces an HTML evidence report with screenshots, scores, and findings. Use when you want to validate a prototype, check usability, or generate evidence for an MR review.
- Author
- rh-uxd ↗
- Repository
- ai-helpers ↗
- Licence
- MIT
- Last repository change
- 27.10.2026
Assessment and security
- Files in this package
- 114
- Scripts
- 32
- HTML comments
- 0
- URLs in SKILL.md
- 6
No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
- 32 scripts in this package. Review them before installing, as Claude may execute them.
Scripts: scripts/append-iteration-log.js, scripts/artifact-readiness-gate.py, scripts/bootstrap-consistency-checker.sh, scripts/bootstrap-usability-testing.sh, scripts/build-leaderboard.js, scripts/classify-ac-tier.js, scripts/compute-patience-drain.js, scripts/csv-utils.js, scripts/eval_state.py, scripts/generate-dashboard.js, scripts/generate-journey-script.js, scripts/log-run.js, scripts/mlflow-compare-
What it does and when to use it
Evaluate a running prototype against a Jira ticket's acceptance criteria, automatically fix what fails, then run persona-based usability walkthroughs. Produces an HTML evidence report with screenshots, scores, and findings. Use when you want to validate a prototype, check usability, or generate evidence for an MR review.
Original author description
Evaluate a running prototype against a Jira ticket's acceptance criteria, automatically fix what fails, then run persona-based usability walkthroughs. Produces an HTML evidence report with screenshots, scores, and findings. Use when you want to validate a prototype, check usability, or generate evidence for an MR review.
How it works
- Inputs
- Outputs
- Flags
- Artifact locations
- Prerequisites
- Workflow (two-phase)
- Review Mode
- Error Handling
- What's Next
- Guardrails
SKILL.md preview
---
name: uxd-prototype-evaluate
version: 0.1.0
description: >-
Evaluate a running prototype against a Jira ticket's acceptance criteria,
automatically fix what fails, then run persona-based usability walkthroughs.
Produces an HTML evidence report with screenshots, scores, and findings.
Use when you want to validate a prototype, check usability, or generate
evidence for an MR review.
---
# Evaluate Prototype
Two-phase eval: **Phase A (x-ray)** validates acceptance criteria with source access and can fix FAILs; **Phase B (discovery)** always runs persona Playwright walkthroughs. Writes a self-contained HTML report with screenshots, think-aloud traces, and AC verdicts.
Family: `uxd-prototype-create` → **evaluate** → `uxd-prototype-publish`. Re-run after changes. Full loop: [references/orchestration.md](references/orchestration.md). Phase procedures: `references/phases/` — follow each file when that phase runs.
## Inputs
| Input | Example | Required |
|-------|---------|----------|
| Jira story key | `PROJ-298` | **Yes** |
| Prototype URL | `http://localhost:3000` | Conditional |
| MR URL | `https://gitlab.example.com/…/-/merge_requests/42` | No |
| `--workspace` | Path to prototype repo | No |
Need a **reachable URL** or `--workspace` with a built `dist/`. If the key or URL is missing, **stop and ask** — do not start Playwright.
**Remote-only (blocking):** URL but no `--workspace` and no MR URL → warn (no fix loop, no source-mode consistency, heuristic navigation only) and wait for confirm before proceeding.
**URL resolution:** probe remote URLs; if unreachable and `dist/` exists, serve with `sirv` (`--single` for SPAs). Hybrid: MR URL + remote prototype clones source into `.artifacts/<KEY>/code/` for route/component-map data; Playwright still uses the remote URL.
## Outputs
Per-key eval files under `${UXD_PROJECT_ROOT}/.artifacts/<KEY>/eval/` (`ARTIFACTS_DIR`):
| File | Description |
|------|-------------|
| `evaluation-report.html` | Final HTML report (both phases) |
| `evaluation-report.csv` | AC verdicts + usability dimensions |
| `iteration-log.json` | Per-iteration counts + Phase B usability |
| `journey-log.json` | Playwright step log + usability overlays |
| `scripts/journey-test.mjs` / `persona-walkthrough.mjs` | Generated Playwright scripts |
| `evaluation-report-iter-N.csv`, `screenshots-iter-N/` | Phase A archives |
| `screenshots/persona-<id>-step-N.png` | Phase B screenshots |
| `usability-thinkaloud-<id>.md` | Phase B traces |
| `runs/<timestamp>/` | Archived copy of this run |
| `report-url.txt` | Hosted eval URL after `publish-report.sh` |
Cross-key (`.artifacts/eval/`, not deleted by `--fresh`): `runs/run-log.csv`, `pain-leaderboard.html`.
Create-owned (key root, not deleted by `--fresh`): `.artifacts/<KEY>/prototype-bar.json` — sync with `--artifacts ${KEY_DIR}` after the report. Local Eval browsing: export skill `export-helper.mjs` on port 9417. Static Pages: `copy-eval-for-pages.sh` / `install-prototype-bar.sh --artifacts` into `public/evals/<KEY>/`.
## Flags
$ARGUMENTS
Parse as: `<KEY> <prototype-url> [mr-url] [--workspace=…]` or `review <KEY>`
| Flag | Default | Description |
|------|---------|-------------|
| `--workspace` | — | Prototype repo (enables the fix loop) |
| `--max-iterations` | `3` | Max Phase A fix-loop iterations |
| `--no-iterate` | Off | Single Phase A pass, no loop |
| `--no-fix` | Off | Evaluate only — FAILs remain; Phase B still runs |
| `--reset` | Off | Hard-reset workspace to origin HEAD before eval |
| `--fresh` | Off | Delete `.artifacts/<KEY>/eval/` only (not the key root, not `.artifacts/eval/`) |
| `--no-report` | Off | Compact chat summary; run `review` later for the full report |
```
/uxd-prototype-evaluate PROJ-298 http://localhost:3000 --workspace=/path/to/prototype
/uxd-prototype-evaluate PROJ-298 http://localhost:4200 --max-iterations=2
/uxd-prototype-evaluate PROJ-298 https://pages.example.com/mr-42/ https://gitlab.example.com/group/repo/-/merge_requests/42
/uxd-prototype-evaluate review PROJ-298
```
## Artifact locations
Eval runtime outputs live in the **consumer project**, never under `${CLAUDE_SKILL_DIR}` (the skill install). Mixing them into the create/publish key root also breaks `--fresh` and publish.
| Variable | Path | Role |
|----------|------|------|
| `UXD_PROJECT_ROOT` | Consumer project root (git toplevel, never the skill install, never a nested clone) | Pin first |
| `KEY_DIR` | `${UXD_PROJECT_ROOT}/.artifacts/<KEY>` | Create/publish key root |
| `ARTIFACTS_DIR` | `${KEY_DIR}/eval` | **All** per-key eval outputs |
Pinned by `scripts/pipeline-setup.sh` at pipeline start.
1. After any `cd`, use absolute `${ARTIFACTS_DIR}` — never a relative `.artifacts/<KEY>/eval/…`.
2. `cd "${CLAUDE_SKILL_DIR}"` only for `npm install` / Playwright browsers, then return to `${UXD_PROJECT_ROOT}`.
3. Generated Playwright scripts go in `${ARTIFACTS_DIR}/scripts/`.
4. `--fresh` deletes only `${ARTIFACTS_DIR}` (`.artifacts/<KEY>/eval/`). Never the key root, never `.artifacts/eval/`, never `rm -rf .artifacts` from an unknown cwd.
5. Create-owned siblings (`decisions/`, `prototype-bar.json`, …) stay at `${KEY_DIR}`.
6. Node helpers resolve via `scripts/resolve-root.js` (honors `UXD_PROJECT_ROOT`).
BAD: `cd .artifacts/<KEY>/code && echo "result" > eval/report.csv` — relative write lands in the clone.
In phase docs, `.artifacts/<KEY>/eval/…` means `${UXD_PROJECT_ROOT}/.artifacts/<KEY>/eval/…`.
## Prerequisites
Marketplace install copies files only — it does not run `npm install` or `package.json` `postinstall`. Before proceeding, install Node deps and Chromium from the skill directory:
```bash
export UXD_PROJECT_ROOT="$(node -e "console.log(require('${CLAUDE_SKILL_DIR}/scripts/resolve-root').resolveProjectRoot())" 2>/dev/null || git rev-parse --show-toplevel 2>/dev/null || pwd)"
cd "${CLAUDE_SKILL_DIR}"
npm install
npx playwright install chromium
cd "${UXD_PROJECT_ROOT}"
```
Stop if Chromium install fails; do not start Playwright without it.
Context repos (`.context/consistency-checker/`, `.context/usability-testing/`) bootstrap on first run when `CONSISTENCY_CHECKER_REPO` / `USABILITY_TESTING_REPO` (or overlay `context_repos`) are set; otherwise those phases degrade. Product overlay: [references/skill-overlays.md](references/skill-overlays.md).
**Personas:** `${CLAUDE_PLUGIN_ROOT}/knowledge/personas/catalog.yaml` + overlays. Deep YAML from `.context/usability-testing/`. Internal study URLs: `node ${CLAUDE_SKILL_DIR}/scripts/overlay-get.js --knowledge-persona <id>` when internal-ai-helpers is present.
IDE auto-approve for the bundled scripts is optional and tool-specific — see this skill's README if the user is prompted on every script.
## Workflow (two-phase)
```
PHASE A (X-Ray — Informed AC Validation Loop):
eval-extract (--phase=core) → eval-consistency (--mode=source) → eval-classify → eval-journey
↓
Exit condition met? → Phase B (ALWAYS)
FAIL + cycle ≤ max → eval-fix → loop from eval-classify
Exit conditions (any triggers Phase B):
all_pass — 0 FAIL, 0 FLAGGED
flagged_unfixable — 0 FAIL, FLAGGED items unfixable
max_iterations — still has FAILs after N loops
regression — fix loop broke a previously-passing AC
no_fix/no_iterate — user flag or single-run mode
POST-PHASE-A:
eval-consistency (--mode=visual) → eval-extract (--phase=enrichment) → eval-hint
PHASE B (Discovery — Per-Persona Usability Walkthroughs) — ALWAYS FIRES:
eval-usability → eval-report
Runs on whatever prototype state exists after Phase A exits.
```
**Phase A** exits on zero FAIL in `evaluation-report.csv` Section 1, or max iterations. FLAGGED items need human review; the loop only targets FAILs.
**Phase B always fires**, regardless of Phase A outcome (including `--no-fix`). When `exit_reason != all_pass`, usability scores may reflect missing features.
Do not improvise the loop from this overview — follow [references/orchestration.md](references/orchestration.md).
## Review Mode
`/uxd-prototype-evaluate review <KEY>` (or "show me the eval for …"): follow [references/phases/eval-review.md](references/phases/eval-review.md). Do not re-run Playwright unless asked.
## Error Handling
| Problem | Action |
|---------|--------|
| Prototype URL unreachable | Fall back to workspace `dist/` via `resolve-prototype-url.sh`; fail clearly if neither exists |
| eval-fix produces no changes | Stop Phase A; proceed to Phase B |
| Dev server crashes after fix | Stop Phase A; note suspect files; proceed to Phase B |
| Missing `.context/` | Phase A degrades (token-check fallback); Phase B uses bundled personas |
## What's Next
Report: `.artifacts/<KEY>/eval/evaluation-report.html`.
| Result | Next step |
|--------|-----------|
| All ACs pass, usability acceptable | `/uxd-prototype-publish` |
| FLAGGED only | Review, then publish |
| Major failures or low usability | `/uxd-prototype-create` with eval findings |
| Re-run after changes | `/uxd-prototype-evaluate <KEY> <URL>` |
After each run, offer: fix a specific issue, explain a finding, re-run, or accept.
## Guardrails
- **Do not start Playwright** until Chromium is installed (`npx playwright install chromium`), and the Jira key and a URL or `--workspace` are known.
- **Never write eval artifacts to `${CLAUDE_SKILL_DIR}`.**
- **`--fresh` deletes `.artifacts/<KEY>/eval/` only** — key root and `.artifacts/eval/` stay intact.
- **Phase B is not optional** and is not inference from Phase A screenshots.
- **`--no-fix` skips the fix loop;** FAILs remain; Phase B still runs.
## Reference Docs
| Doc | When to load |
|-----|-------------|
| [orchestration.md](references/orchestration.md) | Running a full eval |
| [phases/eval-extract.md](references/phases/eval-extract.md) | Jira context, ACs, MR delta |
| [phases/eval-classify.md](references/phases/eval-classify.md) | AC tier classification |
| [phases/eval-consistency.md](references/phases/eval-consistency.md) | PatternFly checks |
| [phases/eval-journey.md](references/phases/eval-journey.md) | Phase A Playwright |
| [phases/eval-fix.md](references/phases/eval-fix.md) | Applying fixes |
| [phases/eval-hint.md](references/phases/eval-hint.md) | Navigation hints |
| [phases/eval-usability.md](references/phases/eval-usability.md) | Phase B walkthroughs |
| [phases/eval-report.md](references/phases/eval-report.md) | HTML report |
| [phases/eval-review.md](references/phases/eval-review.md) | Reviewing a prior run |
| [playwright-rules.md](references/playwright-rules.md) | Writing Playwright scripts |
| [skill-overlays.md](references/skill-overlays.md) | Product-specific config |
Files in this package (114)
- .gitignore 1 KB
- README.md 4 KB
- SKILL.md 11 KB
- package.json 1 KB
- config/csv-schema.yaml 4 KB
- config/eval-settings.yaml 1 KB
- config/ground-truth.example.json 1 KB
- config/mlflow-skill-map.json 4 KB
- config/product-overlay.yaml 3 KB
- config/publish.yaml 1 KB
- config/report-style.yaml 14 KB
- eval/eval.yaml 15 KB
- eval/cases/artifact-path-contract/annotations.yaml 1 KB
- eval/cases/artifact-path-contract/input.yaml 1 KB
- eval/cases/asks-required-inputs/annotations.yaml 1 KB
- eval/cases/asks-required-inputs/input.yaml 1 KB
- eval/cases/fresh-flag-behavior/annotations.yaml 1 KB
- eval/cases/fresh-flag-behavior/input.yaml 1 KB
- eval/cases/max-iterations-planning/annotations.yaml 1 KB
- eval/cases/max-iterations-planning/input.yaml 1 KB
- eval/cases/no-fix-flag-respected/annotations.yaml 1 KB
- eval/cases/no-fix-flag-respected/input.yaml 1 KB
- eval/cases/review-mode-missing-artifacts/annotations.yaml 1 KB
- eval/cases/review-mode-missing-artifacts/input.yaml 1 KB
- eval/cases/review-mode-no-rerun/annotations.yaml 1 KB
- eval/cases/review-mode-no-rerun/input.yaml 1 KB
- eval/cases/review-mode-no-rerun/.artifacts/DEMO/evaluation-report.csv 1 KB
- eval/cases/review-mode-no-rerun/.artifacts/DEMO/extract-state.json 1 KB
- eval/cases/review-mode-no-rerun/.artifacts/DEMO/journey-log.json 1 KB
- eval/cases/review-multi-iteration/annotations.yaml 1 KB
- eval/cases/review-multi-iteration/input.yaml 1 KB
- eval/cases/review-multi-iteration/.artifacts/ITER-DEMO/evaluation-report.csv 2 KB
- eval/cases/review-multi-iteration/.artifacts/ITER-DEMO/extract-state.json 1 KB
- eval/cases/review-multi-iteration/.artifacts/ITER-DEMO/iteration-log.json 1 KB
- eval/cases/two-phase-orchestration/annotations.yaml 1 KB
- eval/cases/two-phase-orchestration/input.yaml 1 KB
- gitlab-pages/.gitlab-ci.yml 1 KB
- gitlab-pages/README.md 1 KB
- references/draft-phase-a-cli-workflow.md 7 KB
- references/mlflow-conventions.md 5 KB
- references/mlflow-logging.md 2 KB
- references/orchestration.md 36 KB
- references/persona-walkthrough-prompt.md 5 KB
- references/playwright-rules.md 18 KB
- references/skill-overlays.md 4 KB
- references/verdict-policy.md 8 KB
- references/phases/eval-classify.md 8 KB
- references/phases/eval-consistency.md 12 KB
- references/phases/eval-extract.md 17 KB
- references/phases/eval-fix.md 8 KB
- references/phases/eval-hint.md 5 KB
- references/phases/eval-journey.md 26 KB
- references/phases/eval-report.md 6 KB
- references/phases/eval-review.md 4 KB
- references/phases/eval-usability-personas.md 5 KB
- references/phases/eval-usability-score.md 12 KB
- references/phases/eval-usability.md 18 KB
- scripts/append-iteration-log.js 9 KB
- scripts/artifact-readiness-gate.py 2 KB
- scripts/bootstrap-consistency-checker.sh 2 KB
- scripts/bootstrap-usability-testing.sh 2 KB
- scripts/build-leaderboard.js 17 KB
- scripts/classify-ac-tier.js 4 KB
- scripts/compute-patience-drain.js 5 KB
- scripts/csv-utils.js 1 KB
- scripts/eval_state.py 4 KB
- scripts/generate-dashboard.js 6 KB
- scripts/generate-journey-script.js 21 KB
- scripts/log-run.js 7 KB
- scripts/mlflow-compare-models.py 24 KB
- scripts/mlflow-trace-eval.py 54 KB
- scripts/mlflow-trace-pipeline.py 12 KB
- scripts/overlay-get.js 11 KB
- scripts/pipeline-setup.sh 8 KB
- scripts/preflight-check.sh 4 KB
- scripts/propagate-exit-reason.py 1 KB
- scripts/publish-report.sh 15 KB
- scripts/render-mini-report.js 7 KB
- scripts/render-report.js 210 KB
- scripts/resolve-prototype-url.sh 8 KB
- scripts/resolve-root.js 4 KB
- scripts/validate-artifact-schemas.js 14 KB
- scripts/validate-classify.js 4 KB
- scripts/validate-consistency.js 5 KB
- scripts/validate-fix-loop.js 5 KB
- scripts/validate-phase-b-output.js 13 KB
- scripts/validate-report-rendering.js 19 KB
- scripts/validate-verdicts.js 2 KB
- templates/evaluation-report.html 108 KB
- templates/report-index.html 17 KB
- tests/run-script-tests.sh 3 KB
- tests/fixtures/manifest.json 4 KB
- tests/fixtures/audio/speech-sample.wav 284 KB
- tests/fixtures/documents/invoice-sample.pdf 2 KB
- tests/fixtures/images/chart-quarterly-revenue.png 10 KB
- tests/fixtures/images/diagram-architecture.png 8 KB
- tests/fixtures/images/photo-workspace.jpg 44 KB
- tests/fixtures/minimal/classify/extract-state.json 1 KB
- tests/fixtures/minimal/classify/tier-overrides.json 1 KB
- tests/fixtures/minimal/consistency/consistency-report.json 1 KB
- tests/fixtures/minimal/fix/fix-log.json 1 KB
- tests/fixtures/minimal/fix/iteration-log.json 1 KB
- tests/fixtures/minimal/journey/journey-log.json 3 KB
- tests/fixtures/minimal/report/consistency-report.json 1 KB
- tests/fixtures/minimal/report/evaluation-report.csv 1 KB
- tests/fixtures/minimal/report/evaluation-summary.json 1 KB
- tests/fixtures/minimal/report/extract-state.json 1 KB
- tests/fixtures/minimal/report/journey-log.json 1 KB
- tests/fixtures/minimal/report/persona-results.json 1 KB
- tests/fixtures/minimal/usability/journey-log.json 1 KB
- tests/fixtures/minimal/usability/persona-results.json 2 KB
- tests/fixtures/text/prompt-analyze-image.txt 1 KB
- tests/fixtures/text/prompt-code-explain.txt 1 KB
- tests/fixtures/text/prompt-summarize.txt 1 KB
Versions
The database has no other skills named uxd-prototype-evaluate.
Database snapshot: 13.11.2026 · package version 41dc9cf80ea1 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100uxd-prototype-create
Create or refine a UX prototype from a Jira ticket, Figma design, feature description, or rough idea. Enumerates user journeys and page states (empty/error/alternate conditions), wires mock data via ?state=, writes a design spec for QE test-plan ingest, and supports export of… - 100uxd-prototype-publish
Publish a prototype to a git merge request, GitHub Pages, GitLab Pages, or Vercel. Use when sharing a finished prototype for review, deploying a sanitized public copy, or updating Jira with submission links. - 100v1-reviewing-usability
Use when reviewing a UI, product flow, prototype, or form for usability and user-error risk. Triggers on "review this UI", "usability review", or "confusing interaction". - 90ux-evaluate
UX heuristic evaluation and usability assessment — reviews HTML wireframes against UX strategy artifacts and established principles. Read-only, produces scored reports. Use when "ux evaluate", "heuristic evaluation", "usability review", "ux audit", "accessibility check",… - 86make-a-prototype
Build a scoped interactive prototype to test a specific usability question. Use when static sketches cannot answer a flow or feedback question. - 90prototype
Build a throwaway HTML or in-app prototype that answers one design question. Use when the user wants to try a state model or UI idea before committing, or says "mock it up", "prototype", "what should this look like".