Skip to content
Skills Library
Design & UX · Testing

uxd-prototype-evaluate

/uxd-prototype-evaluate

Evaluate a running prototype against a Jira ticket's acceptance criteria, automatically fix what fails, then run persona-based usability walkthroughs. Produces an HTML evidence report with screenshots, scores, and findings. Use when you want to validate a prototype, check usability, or generate evidence for an MR review.

Repository
ai-helpers ↗
Licence
MIT
Last repository change
27.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
114
Scripts
32
HTML comments
0
URLs in SKILL.md
6
Deprecation

No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What to check before installing
  • 32 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: scripts/append-iteration-log.js, scripts/artifact-readiness-gate.py, scripts/bootstrap-consistency-checker.sh, scripts/bootstrap-usability-testing.sh, scripts/build-leaderboard.js, scripts/classify-ac-tier.js, scripts/compute-patience-drain.js, scripts/csv-utils.js, scripts/eval_state.py, scripts/generate-dashboard.js, scripts/generate-journey-script.js, scripts/log-run.js, scripts/mlflow-compare-

What it does and when to use it

Evaluate a running prototype against a Jira ticket's acceptance criteria, automatically fix what fails, then run persona-based usability walkthroughs. Produces an HTML evidence report with screenshots, scores, and findings. Use when you want to validate a prototype, check usability, or generate evidence for an MR review.

Original author description

Evaluate a running prototype against a Jira ticket's acceptance criteria, automatically fix what fails, then run persona-based usability walkthroughs. Produces an HTML evidence report with screenshots, scores, and findings. Use when you want to validate a prototype, check usability, or generate evidence for an MR review.

How it works

  1. Inputs
  2. Outputs
  3. Flags
  4. Artifact locations
  5. Prerequisites
  6. Workflow (two-phase)
  7. Review Mode
  8. Error Handling
  9. What's Next
  10. Guardrails
Original SKILL.md headings · 190 lines, 1,261 words
SKILL.md preview
---
name: uxd-prototype-evaluate
version: 0.1.0
description: >-
  Evaluate a running prototype against a Jira ticket's acceptance criteria,
  automatically fix what fails, then run persona-based usability walkthroughs.
  Produces an HTML evidence report with screenshots, scores, and findings.
  Use when you want to validate a prototype, check usability, or generate
  evidence for an MR review.
---

# Evaluate Prototype

Two-phase eval: **Phase A (x-ray)** validates acceptance criteria with source access and can fix FAILs; **Phase B (discovery)** always runs persona Playwright walkthroughs. Writes a self-contained HTML report with screenshots, think-aloud traces, and AC verdicts.

Family: `uxd-prototype-create` → **evaluate** → `uxd-prototype-publish`. Re-run after changes. Full loop: [references/orchestration.md](references/orchestration.md). Phase procedures: `references/phases/` — follow each file when that phase runs.

## Inputs

| Input | Example | Required |
|-------|---------|----------|
| Jira story key | `PROJ-298` | **Yes** |
| Prototype URL | `http://localhost:3000` | Conditional |
| MR URL | `https://gitlab.example.com/…/-/merge_requests/42` | No |
| `--workspace` | Path to prototype repo | No |

Need a **reachable URL** or `--workspace` with a built `dist/`. If the key or URL is missing, **stop and ask** — do not start Playwright.

**Remote-only (blocking):** URL but no `--workspace` and no MR URL → warn (no fix loop, no source-mode consistency, heuristic navigation only) and wait for confirm before proceeding.

**URL resolution:** probe remote URLs; if unreachable and `dist/` exists, serve with `sirv` (`--single` for SPAs). Hybrid: MR URL + remote prototype clones source into `.artifacts/<KEY>/code/` for route/component-map data; Playwright still uses the remote URL.

## Outputs

Per-key eval files under `${UXD_PROJECT_ROOT}/.artifacts/<KEY>/eval/` (`ARTIFACTS_DIR`):

| File | Description |
|------|-------------|
| `evaluation-report.html` | Final HTML report (both phases) |
| `evaluation-report.csv` | AC verdicts + usability dimensions |
| `iteration-log.json` | Per-iteration counts + Phase B usability |
| `journey-log.json` | Playwright step log + usability overlays |
| `scripts/journey-test.mjs` / `persona-walkthrough.mjs` | Generated Playwright scripts |
| `evaluation-report-iter-N.csv`, `screenshots-iter-N/` | Phase A archives |
| `screenshots/persona-<id>-step-N.png` | Phase B screenshots |
| `usability-thinkaloud-<id>.md` | Phase B traces |
| `runs/<timestamp>/` | Archived copy of this run |
| `report-url.txt` | Hosted eval URL after `publish-report.sh` |

Cross-key (`.artifacts/eval/`, not deleted by `--fresh`): `runs/run-log.csv`, `pain-leaderboard.html`.

Create-owned (key root, not deleted by `--fresh`): `.artifacts/<KEY>/prototype-bar.json` — sync with `--artifacts ${KEY_DIR}` after the report. Local Eval browsing: export skill `export-helper.mjs` on port 9417. Static Pages: `copy-eval-for-pages.sh` / `install-prototype-bar.sh --artifacts` into `public/evals/<KEY>/`.

## Flags

$ARGUMENTS

Parse as: `<KEY> <prototype-url> [mr-url] [--workspace=…]` or `review <KEY>`

| Flag | Default | Description |
|------|---------|-------------|
| `--workspace` | — | Prototype repo (enables the fix loop) |
| `--max-iterations` | `3` | Max Phase A fix-loop iterations |
| `--no-iterate` | Off | Single Phase A pass, no loop |
| `--no-fix` | Off | Evaluate only — FAILs remain; Phase B still runs |
| `--reset` | Off | Hard-reset workspace to origin HEAD before eval |
| `--fresh` | Off | Delete `.artifacts/<KEY>/eval/` only (not the key root, not `.artifacts/eval/`) |
| `--no-report` | Off | Compact chat summary; run `review` later for the full report |

```
/uxd-prototype-evaluate PROJ-298 http://localhost:3000 --workspace=/path/to/prototype
/uxd-prototype-evaluate PROJ-298 http://localhost:4200 --max-iterations=2
/uxd-prototype-evaluate PROJ-298 https://pages.example.com/mr-42/ https://gitlab.example.com/group/repo/-/merge_requests/42
/uxd-prototype-evaluate review PROJ-298
```

## Artifact locations

Eval runtime outputs live in the **consumer project**, never under `${CLAUDE_SKILL_DIR}` (the skill install). Mixing them into the create/publish key root also breaks `--fresh` and publish.

| Variable | Path | Role |
|----------|------|------|
| `UXD_PROJECT_ROOT` | Consumer project root (git toplevel, never the skill install, never a nested clone) | Pin first |
| `KEY_DIR` | `${UXD_PROJECT_ROOT}/.artifacts/<KEY>` | Create/publish key root |
| `ARTIFACTS_DIR` | `${KEY_DIR}/eval` | **All** per-key eval outputs |

Pinned by `scripts/pipeline-setup.sh` at pipeline start.

1. After any `cd`, use absolute `${ARTIFACTS_DIR}` — never a relative `.artifacts/<KEY>/eval/…`.
2. `cd "${CLAUDE_SKILL_DIR}"` only for `npm install` / Playwright browsers, then return to `${UXD_PROJECT_ROOT}`.
3. Generated Playwright scripts go in `${ARTIFACTS_DIR}/scripts/`.
4. `--fresh` deletes only `${ARTIFACTS_DIR}` (`.artifacts/<KEY>/eval/`). Never the key root, never `.artifacts/eval/`, never `rm -rf .artifacts` from an unknown cwd.
5. Create-owned siblings (`decisions/`, `prototype-bar.json`, …) stay at `${KEY_DIR}`.
6. Node helpers resolve via `scripts/resolve-root.js` (honors `UXD_PROJECT_ROOT`).

BAD: `cd .artifacts/<KEY>/code && echo "result" > eval/report.csv` — relative write lands in the clone.

In phase docs, `.artifacts/<KEY>/eval/…` means `${UXD_PROJECT_ROOT}/.artifacts/<KEY>/eval/…`.

## Prerequisites

Marketplace install copies files only — it does not run `npm install` or `package.json` `postinstall`. Before proceeding, install Node deps and Chromium from the skill directory:

```bash
export UXD_PROJECT_ROOT="$(node -e "console.log(require('${CLAUDE_SKILL_DIR}/scripts/resolve-root').resolveProjectRoot())" 2>/dev/null || git rev-parse --show-toplevel 2>/dev/null || pwd)"
cd "${CLAUDE_SKILL_DIR}"
npm install
npx playwright install chromium
cd "${UXD_PROJECT_ROOT}"
```

Stop if Chromium install fails; do not start Playwright without it.

Context repos (`.context/consistency-checker/`, `.context/usability-testing/`) bootstrap on first run when `CONSISTENCY_CHECKER_REPO` / `USABILITY_TESTING_REPO` (or overlay `context_repos`) are set; otherwise those phases degrade. Product overlay: [references/skill-overlays.md](references/skill-overlays.md).

**Personas:** `${CLAUDE_PLUGIN_ROOT}/knowledge/personas/catalog.yaml` + overlays. Deep YAML from `.context/usability-testing/`. Internal study URLs: `node ${CLAUDE_SKILL_DIR}/scripts/overlay-get.js --knowledge-persona <id>` when internal-ai-helpers is present.

IDE auto-approve for the bundled scripts is optional and tool-specific — see this skill's README if the user is prompted on every script.

## Workflow (two-phase)

```
PHASE A (X-Ray — Informed AC Validation Loop):
  eval-extract (--phase=core) → eval-consistency (--mode=source) → eval-classify → eval-journey
                                                                                     ↓
                                                                             Exit condition met? → Phase B (ALWAYS)
                                                                             FAIL + cycle ≤ max → eval-fix → loop from eval-classify

  Exit conditions (any triggers Phase B):
    all_pass          — 0 FAIL, 0 FLAGGED
    flagged_unfixable — 0 FAIL, FLAGGED items unfixable
    max_iterations    — still has FAILs after N loops
    regression        — fix loop broke a previously-passing AC
    no_fix/no_iterate — user flag or single-run mode

POST-PHASE-A:
  eval-consistency (--mode=visual) → eval-extract (--phase=enrichment) → eval-hint

PHASE B (Discovery — Per-Persona Usability Walkthroughs) — ALWAYS FIRES:
  eval-usability → eval-report
  Runs on whatever prototype state exists after Phase A exits.
```

**Phase A** exits on zero FAIL in `evaluation-report.csv` Section 1, or max iterations. FLAGGED items need human review; the loop only targets FAILs.

**Phase B always fires**, regardless of Phase A outcome (including `--no-fix`). When `exit_reason != all_pass`, usability scores may reflect missing features.

Do not improvise the loop from this overview — follow [references/orchestration.md](references/orchestration.md).

## Review Mode

`/uxd-prototype-evaluate review <KEY>` (or "show me the eval for …"): follow [references/phases/eval-review.md](references/phases/eval-review.md). Do not re-run Playwright unless asked.

## Error Handling

| Problem | Action |
|---------|--------|
| Prototype URL unreachable | Fall back to workspace `dist/` via `resolve-prototype-url.sh`; fail clearly if neither exists |
| eval-fix produces no changes | Stop Phase A; proceed to Phase B |
| Dev server crashes after fix | Stop Phase A; note suspect files; proceed to Phase B |
| Missing `.context/` | Phase A degrades (token-check fallback); Phase B uses bundled personas |

## What's Next

Report: `.artifacts/<KEY>/eval/evaluation-report.html`.

| Result | Next step |
|--------|-----------|
| All ACs pass, usability acceptable | `/uxd-prototype-publish` |
| FLAGGED only | Review, then publish |
| Major failures or low usability | `/uxd-prototype-create` with eval findings |
| Re-run after changes | `/uxd-prototype-evaluate <KEY> <URL>` |

After each run, offer: fix a specific issue, explain a finding, re-run, or accept.

## Guardrails

- **Do not start Playwright** until Chromium is installed (`npx playwright install chromium`), and the Jira key and a URL or `--workspace` are known.
- **Never write eval artifacts to `${CLAUDE_SKILL_DIR}`.**
- **`--fresh` deletes `.artifacts/<KEY>/eval/` only** — key root and `.artifacts/eval/` stay intact.
- **Phase B is not optional** and is not inference from Phase A screenshots.
- **`--no-fix` skips the fix loop;** FAILs remain; Phase B still runs.

## Reference Docs

| Doc | When to load |
|-----|-------------|
| [orchestration.md](references/orchestration.md) | Running a full eval |
| [phases/eval-extract.md](references/phases/eval-extract.md) | Jira context, ACs, MR delta |
| [phases/eval-classify.md](references/phases/eval-classify.md) | AC tier classification |
| [phases/eval-consistency.md](references/phases/eval-consistency.md) | PatternFly checks |
| [phases/eval-journey.md](references/phases/eval-journey.md) | Phase A Playwright |
| [phases/eval-fix.md](references/phases/eval-fix.md) | Applying fixes |
| [phases/eval-hint.md](references/phases/eval-hint.md) | Navigation hints |
| [phases/eval-usability.md](references/phases/eval-usability.md) | Phase B walkthroughs |
| [phases/eval-report.md](references/phases/eval-report.md) | HTML report |
| [phases/eval-review.md](references/phases/eval-review.md) | Reviewing a prior run |
| [playwright-rules.md](references/playwright-rules.md) | Writing Playwright scripts |
| [skill-overlays.md](references/skill-overlays.md) | Product-specific config |
Files in this package (114)
  • .gitignore 1 KB
  • README.md 4 KB
  • SKILL.md 11 KB
  • package.json 1 KB
  • config/csv-schema.yaml 4 KB
  • config/eval-settings.yaml 1 KB
  • config/ground-truth.example.json 1 KB
  • config/mlflow-skill-map.json 4 KB
  • config/product-overlay.yaml 3 KB
  • config/publish.yaml 1 KB
  • config/report-style.yaml 14 KB
  • eval/eval.yaml 15 KB
  • eval/cases/artifact-path-contract/annotations.yaml 1 KB
  • eval/cases/artifact-path-contract/input.yaml 1 KB
  • eval/cases/asks-required-inputs/annotations.yaml 1 KB
  • eval/cases/asks-required-inputs/input.yaml 1 KB
  • eval/cases/fresh-flag-behavior/annotations.yaml 1 KB
  • eval/cases/fresh-flag-behavior/input.yaml 1 KB
  • eval/cases/max-iterations-planning/annotations.yaml 1 KB
  • eval/cases/max-iterations-planning/input.yaml 1 KB
  • eval/cases/no-fix-flag-respected/annotations.yaml 1 KB
  • eval/cases/no-fix-flag-respected/input.yaml 1 KB
  • eval/cases/review-mode-missing-artifacts/annotations.yaml 1 KB
  • eval/cases/review-mode-missing-artifacts/input.yaml 1 KB
  • eval/cases/review-mode-no-rerun/annotations.yaml 1 KB
  • eval/cases/review-mode-no-rerun/input.yaml 1 KB
  • eval/cases/review-mode-no-rerun/.artifacts/DEMO/evaluation-report.csv 1 KB
  • eval/cases/review-mode-no-rerun/.artifacts/DEMO/extract-state.json 1 KB
  • eval/cases/review-mode-no-rerun/.artifacts/DEMO/journey-log.json 1 KB
  • eval/cases/review-multi-iteration/annotations.yaml 1 KB
  • eval/cases/review-multi-iteration/input.yaml 1 KB
  • eval/cases/review-multi-iteration/.artifacts/ITER-DEMO/evaluation-report.csv 2 KB
  • eval/cases/review-multi-iteration/.artifacts/ITER-DEMO/extract-state.json 1 KB
  • eval/cases/review-multi-iteration/.artifacts/ITER-DEMO/iteration-log.json 1 KB
  • eval/cases/two-phase-orchestration/annotations.yaml 1 KB
  • eval/cases/two-phase-orchestration/input.yaml 1 KB
  • gitlab-pages/.gitlab-ci.yml 1 KB
  • gitlab-pages/README.md 1 KB
  • references/draft-phase-a-cli-workflow.md 7 KB
  • references/mlflow-conventions.md 5 KB
  • references/mlflow-logging.md 2 KB
  • references/orchestration.md 36 KB
  • references/persona-walkthrough-prompt.md 5 KB
  • references/playwright-rules.md 18 KB
  • references/skill-overlays.md 4 KB
  • references/verdict-policy.md 8 KB
  • references/phases/eval-classify.md 8 KB
  • references/phases/eval-consistency.md 12 KB
  • references/phases/eval-extract.md 17 KB
  • references/phases/eval-fix.md 8 KB
  • references/phases/eval-hint.md 5 KB
  • references/phases/eval-journey.md 26 KB
  • references/phases/eval-report.md 6 KB
  • references/phases/eval-review.md 4 KB
  • references/phases/eval-usability-personas.md 5 KB
  • references/phases/eval-usability-score.md 12 KB
  • references/phases/eval-usability.md 18 KB
  • scripts/append-iteration-log.js 9 KB
  • scripts/artifact-readiness-gate.py 2 KB
  • scripts/bootstrap-consistency-checker.sh 2 KB
  • scripts/bootstrap-usability-testing.sh 2 KB
  • scripts/build-leaderboard.js 17 KB
  • scripts/classify-ac-tier.js 4 KB
  • scripts/compute-patience-drain.js 5 KB
  • scripts/csv-utils.js 1 KB
  • scripts/eval_state.py 4 KB
  • scripts/generate-dashboard.js 6 KB
  • scripts/generate-journey-script.js 21 KB
  • scripts/log-run.js 7 KB
  • scripts/mlflow-compare-models.py 24 KB
  • scripts/mlflow-trace-eval.py 54 KB
  • scripts/mlflow-trace-pipeline.py 12 KB
  • scripts/overlay-get.js 11 KB
  • scripts/pipeline-setup.sh 8 KB
  • scripts/preflight-check.sh 4 KB
  • scripts/propagate-exit-reason.py 1 KB
  • scripts/publish-report.sh 15 KB
  • scripts/render-mini-report.js 7 KB
  • scripts/render-report.js 210 KB
  • scripts/resolve-prototype-url.sh 8 KB
  • scripts/resolve-root.js 4 KB
  • scripts/validate-artifact-schemas.js 14 KB
  • scripts/validate-classify.js 4 KB
  • scripts/validate-consistency.js 5 KB
  • scripts/validate-fix-loop.js 5 KB
  • scripts/validate-phase-b-output.js 13 KB
  • scripts/validate-report-rendering.js 19 KB
  • scripts/validate-verdicts.js 2 KB
  • templates/evaluation-report.html 108 KB
  • templates/report-index.html 17 KB
  • tests/run-script-tests.sh 3 KB
  • tests/fixtures/manifest.json 4 KB
  • tests/fixtures/audio/speech-sample.wav 284 KB
  • tests/fixtures/documents/invoice-sample.pdf 2 KB
  • tests/fixtures/images/chart-quarterly-revenue.png 10 KB
  • tests/fixtures/images/diagram-architecture.png 8 KB
  • tests/fixtures/images/photo-workspace.jpg 44 KB
  • tests/fixtures/minimal/classify/extract-state.json 1 KB
  • tests/fixtures/minimal/classify/tier-overrides.json 1 KB
  • tests/fixtures/minimal/consistency/consistency-report.json 1 KB
  • tests/fixtures/minimal/fix/fix-log.json 1 KB
  • tests/fixtures/minimal/fix/iteration-log.json 1 KB
  • tests/fixtures/minimal/journey/journey-log.json 3 KB
  • tests/fixtures/minimal/report/consistency-report.json 1 KB
  • tests/fixtures/minimal/report/evaluation-report.csv 1 KB
  • tests/fixtures/minimal/report/evaluation-summary.json 1 KB
  • tests/fixtures/minimal/report/extract-state.json 1 KB
  • tests/fixtures/minimal/report/journey-log.json 1 KB
  • tests/fixtures/minimal/report/persona-results.json 1 KB
  • tests/fixtures/minimal/usability/journey-log.json 1 KB
  • tests/fixtures/minimal/usability/persona-results.json 2 KB
  • tests/fixtures/text/prompt-analyze-image.txt 1 KB
  • tests/fixtures/text/prompt-code-explain.txt 1 KB
  • tests/fixtures/text/prompt-summarize.txt 1 KB

Versions

The database has no other skills named uxd-prototype-evaluate.

Database snapshot: 13.11.2026 · package version 41dc9cf80ea1 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100uxd-prototype-create
    Create or refine a UX prototype from a Jira ticket, Figma design, feature description, or rough idea. Enumerates user journeys and page states (empty/error/alternate conditions), wires mock data via ?state=, writes a design spec for QE test-plan ingest, and supports export of…
  • 100uxd-prototype-publish
    Publish a prototype to a git merge request, GitHub Pages, GitLab Pages, or Vercel. Use when sharing a finished prototype for review, deploying a sanitized public copy, or updating Jira with submission links.
  • 100v1-reviewing-usability
    Use when reviewing a UI, product flow, prototype, or form for usability and user-error risk. Triggers on "review this UI", "usability review", or "confusing interaction".
  • 90ux-evaluate
    UX heuristic evaluation and usability assessment — reviews HTML wireframes against UX strategy artifacts and established principles. Read-only, produces scored reports. Use when "ux evaluate", "heuristic evaluation", "usability review", "ux audit", "accessibility check",…
  • 86make-a-prototype
    Build a scoped interactive prototype to test a specific usability question. Use when static sketches cannot answer a flow or feedback question.
  • 90prototype
    Build a throwaway HTML or in-app prototype that answers one design question. Use when the user wants to try a state model or UI idea before committing, or says "mock it up", "prototype", "what should this look like".

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.