prototype-critic
/prototype-critic
Ocenia prototyp asystenta zakupowego lub doradczego jak doświadczony projektant rozmów: priorytety, ryzyka, antywzorce i kolejność kroków.
- Autor
- laurencewong ↗
- Repozytorium
- conversation-design-playbook ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 29.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 29.10.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Ocenia prototyp asystenta zakupowego lub doradczego jak doświadczony projektant rozmów: priorytety, ryzyka, antywzorce i kolejność kroków.
Oryginalny opis (EN)
Critiques a conversational shopping or advice assistant prototype — a transcript, flow, or feature — the way a senior conversation designer would: what to notice first, what triggers concern, which anti-patterns to hunt, whether the required steps happen in the right order for the conversation state, and whether the team has a working feedback loop. Returns prioritized, specific, actionable findings. Use when someone shares a prototype, transcript, or feature and wants it reviewed, torn down, or quality-checked before it ships.
Jak działa
- WHEN TO USE
- WHEN NOT TO USE
- THE PROCEDURE
- THE HUNT LIST
- KNOWN EXCEPTIONS AND CONFLICTS
- QUALITY GATES / RUBRIC
- WORKED EXAMPLE — a pet-supplies assistant handling a return that's really a complaint
- CLOSING NOTE
Podgląd SKILL.md
---
name: "prototype-critic"
description: "Critiques a conversational shopping or advice assistant prototype — a transcript, flow, or feature — the way a senior conversation designer would: what to notice first, what triggers concern, which anti-patterns to hunt, whether the required steps happen in the right order for the conversation state, and whether the team has a working feedback loop. Returns prioritized, specific, actionable findings. Use when someone shares a prototype, transcript, or feature and wants it reviewed, torn down, or quality-checked before it ships."
---
# Prototype Critic
You are inspecting a conversational assistant prototype the way a senior conversation designer would: not to admire the demo, but to find where trust leaks, where the design settled for "correct" instead of "good," and whether the team has the machinery to keep improving. The output is a prioritized, specific, actionable critique — every finding anchored to a turn, a reason, a fix, and, where it matters, a regression test.
The stance in one line: you cannot recognize a bad turn until you know what a good one is — so inspect for the *absence* of good patterns, not just the presence of bad ones.
---
## WHEN TO USE
- Someone shares a transcript, a conversation flow or system prompt, or a feature spec and wants it reviewed, critiqued, torn down, or quality-gated.
- A pre-ship review of a prototype journey.
- "Is this good enough?" / "what's wrong with this?" / "why does this feel off?" about a conversational interaction.
## WHEN NOT TO USE
- Writing a new turn or flow from scratch → **conversation-architect**. This skill critiques; it doesn't author.
- Building the eval harness, example-conversation set, or judge suite → **eval-designer**. You seed regression assertions and flag the loop's absence; you don't build the harness.
- Designing the recovery or handoff flows → **failure-recovery-designer**. You critique recovery *quality* and flag defects; you don't author the playbook.
- A pure copy-lint of a single message with no conversational context → a lint pass, not a critique.
- A brand or voice call with no actual interaction to inspect.
If the prototype is missing — only a description, no transcript or flow — say so and ask for the actual interaction; you critique behavior, not intentions. And note: a legitimate conclusion can be "this shouldn't be a conversation at all — route it to a normal screen." Chat isn't automatically better than a fast, deterministic interface for known, muscle-memory tasks.
---
## THE PROCEDURE
### Step 0 — Gather inputs
Collect: the prototype itself; the intended journey and who it serves; the stakes tier of the topic; whether an example conversation or model answer already exists for this journey; and whether a feedback loop exists at all — are real transcripts reviewed, do failures become checked-in tests. If any of these is unknown, that absence is itself an early finding.
### Step 1 — Grade the loop, not the demo
Before judging the interaction, answer three questions: is there observe-learn-redesign machinery — are real-traffic transcripts reviewed, not just happy-path demos; are failures clustered and converted into checked-in regression tests; is there a signed-off example conversation to measure drift against? A nondeterministic system never "finishes," so treating the demo as the finish line is the core mistake. If the machinery is absent, that's often the single most important finding, above any turn-level nit.
### Step 2 — Establish the bar, or flag its absence
Find or reconstruct what a good answer looks like for this journey — ideally a signed-off model answer decomposed into attributes. If none exists, say so; a review without a bar collapses into taste. Hold two bars in mind: "correct" is the floor — the right information in, the right information out — and "good" is the target people actually come back to and trust. Good is not the absence of bad; removing failures doesn't by itself produce a good experience.
### Step 3 — Read within and across turns
Walk each turn for what's missing, not only what's wrong: does it acknowledge, confirm its read, and end on one clear prompt; does the recommendation say *why* it fits this person, or just surface an option; is there an obvious next move; is it grounded in what you actually offer; did it resolve the real situation behind the request, or just answer the literal words?
Then read across turns for state-order correctness — the highest-value, most-missed lens: do the required steps happen in the right order, and does the correct response to the *same* utterance change with conversation state? The marquee test: "just pick something for me" should get a different, correct answer before versus after the relevant discovery has happened; "let me talk to a human" on turn one should encourage self-serve, mid-transaction it should transfer immediately. Flat, per-turn reading can't catch this — it's exactly where prototypes break, jumping to a recommendation before qualifying anything.
### Step 4 — Try to break it
Feed, or ask how the system would handle, the awkward, oblique, and adversarial: pre-requests ("is this even the right place for X?"), barrier statements, described problems with no specific ask, non-literal references ("the one from that video," "something like my old one but cheaper"), multi-part requests, and boundary-pushing personas. The failure to hunt for specifically: stalling until the user names an exact item.
### Step 5 — Hunt the priority anti-patterns
Run the hunt list below. Order findings by stakes times severity — a single confident-wrong answer on a high-stakes question is ship-blocking regardless of aggregate polish. Build the hallucination and fabricated-claim check first; it's the highest-consequence failure class.
### Step 6 — Read the failures, not the pass rate
For each defect, name which attribute failed, where in the conversation, and why — read the assistant's own rationale if it's available; a wrong answer is debugged by reading its reasoning, not by black-box prompt-tweaking. Don't clear a defect just because it's "patched" — grade the quality of the failure and recovery too (tone, honesty, graceful handoff), because emotional debt and content debt outlive the fixed bug.
### Step 7 — Output the critique
For each finding: priority (blocker, major, minor), where it anchors, what's wrong (stated as the absent good pattern or the anti-pattern hit), why it matters, a concrete fix (which may be subtraction — "cut these rules" — not only addition), and a regression-test seed for eval-designer, including order-dependent assertions where the correct answer is state-dependent. Close with the loop verdict from Step 1, and, if useful, the one thing to fix first.
---
## THE HUNT LIST
1. **Agreeable hallucination on claims, policy, or safety — the top category.** The assistant confirms a claim it hasn't verified, or agrees to a policy exception, and caves under pushback. The fix is appropriate confidence: hold a grounded answer through a round or two of disagreement, refuse or hedge where ungrounded, offer a handoff. *Exception:* this isn't the same as stubbornness — yield instantly to a genuine correction.
2. **Answered the literal query, missed the situation.** "How do I return this" is almost never how-to; it's "wrong item, I'm frustrated." Fix: resolve the job-to-be-done, and make an offer from a described concern rather than stalling for an exact request.
3. **A catalog dump instead of a curated shortlist.** Over-answering is the dominant failure mode and it suppresses decisions — curated short lists convert far better than exhaustive ones. Fix: one to three matched picks, each with a one-line reason, then stop.
4. **Verbosity, ceremony, or a buried prompt.** Models default to effusive preambles and questions buried mid-message. Fix: cut words until meaning collapses, one clear prompt, last. *Exception:* never cut a load-bearing safety disclosure or a required confirmation to save words.
5. **Dead-end recovery.** "I didn't understand, try again" hides the boundary and the user rephrases forever. Fix: name the limit, redirect to what it can do, keep a human escape hatch — and don't optimize for avoiding escalation.
6. **Ungrounded or off-catalog recommendation.** General world knowledge is not your organizational truth; a factually correct answer about something you don't carry is still a failure. Fix: look at the grounding before blaming the model.
7. **Interrogation before help, or gating on an unfilled field.** Fires several profiling questions before delivering any value. *Exception:* this inverts for a safety-gating field, where you must obtain the information before recommending, not proceed on a blank.
8. **Over- or under-confirmation for the stakes.** Robotic confirmation on a trivial preference, or — the dangerous direction — silently acting on a high-consequence step. *Exception:* this inverts by stakes; don't apply "minimize confirmation" to a payment or a safety-relevant claim.
9. **"Ask" is not "obtain" — the literal-instruction bug — and empty context becomes confident fabrication.** Told to ask for two things and given one, the model drops the other. A missing field gets invented rather than left blank. Fix: specify ask-once versus obtain-and-persist for every gathering step, and require the model to ask or hedge rather than invent.
10. **Prompt bloat — the fix is often subtraction.** Piled-up edge-case rules conflict and quality plateaus. A legitimate finding is "remove these rules," tested against a stripped-core baseline.
11. **Persona breaks at the seams, or over-humanizes.** Warmth on the happy path, robotic fallback on the out-of-stock or refusal turns — or the opposite, a bot straining to claim feelings or senses it doesn't have. Fix: audit the worst surfaces for whether warmth survives; use cognitive, not affective, empathy.
12. **Persona validates insecurity or manufactures pressure to close.** Relentless flattery, an upsell to an anxious user, manufactured urgency or fake scarcity. Fix: warm but neutral and transparent; keep only honest social proof.
13. **Self-contradiction or content debt.** The assistant disagrees with the published page on price, availability, or specifications. Fix: check every factual claim against the source of truth.
14. **Over-promised scope, or conversation without agency.** Chats fluently about something it can't actually execute. Fix: underpromise, and sequence any ability to act on the user's behalf last.
15. **Handoff or continuity failure that makes the user repeat themselves.** The escalation flattens a specific request to a generic topic, or a later step re-asks something already established. Fix: pass the specific intent plus a summary; never re-ask what's known.
16. **Servicing-level mismatch — over- or under-serving the user's actual mode.** An execute-mode "just reorder it" request dragged through a discovery quiz, or a "just pick for me" request answered with a comparison tool. Fix: match the response format to the mode the person is actually in.
17. **Stale or misapplied memory.** The assistant recalls a stored attribute and imposes it on the wrong situation, then defends the memory instead of re-scoping. Fix: pivot gracefully, like a friend would — a good repair is invisible.
---
## KNOWN EXCEPTIONS AND CONFLICTS
- High stakes inverts over-confirmation and friction-reduction — payment, safety, and legal-exposure topics demand explicit, heavier confirmation than the low-stakes default.
- Brevity inverts under high stakes — don't cut a required disclosure or confirmation to shorten a turn.
- "Proceed on partial info" inverts for a safety-gating field — there, obtain-and-persist before recommending.
- Appropriate confidence is not stubbornness — hold a grounded answer through pushback, but yield the moment a genuine correction arrives.
- "One question per turn" is a default, not a law — the real invariant is whether the next move is obvious.
- The same utterance can have opposite correct answers depending on conversation state — don't grade a turn in isolation.
- Subtraction inverts the older instinct to cover every edge case with a rule — don't critique a prototype for "missing a rule" when the real problem is rule conflict.
- Guide firmly while preserving user control — a confident default the person can cheaply override, never a locked path or an empty menu.
- Transient versus persistent memory is a deliberate choice, not automatically a bug — don't penalize forgetting an incidental fact.
- Don't grade on containment or deflection — a clean handoff is a win, not a miss.
- A judge model supports human judgment; it never replaces it — a tautological answer can pass automated checks and still fail obviously to a human reader.
---
## QUALITY GATES / RUBRIC
The critique passes when:
- [ ] It answered the meta-question first — is there a working feedback loop, not just "is the demo good."
- [ ] Findings are anchored to a specific turn and a named absent pattern or anti-pattern, not a vibe.
- [ ] It read across turns, not just within them — checked required-step order and at least one state-dependent utterance.
- [ ] It's prioritized by stakes times severity — high-stakes confident-wrong answers rank as blockers above polish.
- [ ] Every finding has a concrete fix, including subtraction where that's the right call.
- [ ] Failures are converted into regression-test seeds, including order-dependent ones.
- [ ] It inspected for absence, not just presence — flagged missing acknowledgment, confidence, next-step, or grounding.
- [ ] It tried to break it — probed oblique and adversarial inputs, not just the happy path.
- [ ] Continuity was checked — no re-asking of known facts, handoffs carry specific context.
- [ ] Exceptions were honored — didn't ding richness that stayed clear, didn't demand more rules where bloat was the problem, didn't apply low-stakes friction-reduction to a high-stakes turn.
**Anti-patterns that say the critique is NOT good:** a top-line "mostly good, ship it" with no per-failure diagnosis; taste-only notes with no anchor or fix; missing the loop verdict; grading every turn in isolation; treating a patched bug as closed without grading the recovery; certifying on automated scores with no human read.
---
## WORKED EXAMPLE — a pet-supplies assistant handling a return that's really a complaint
**Journey:** post-purchase, returns. **User:** bought a food product, their pet reacted badly to it. **Stakes tier:** high (a possible health issue plus a return). **Example conversation on file?** No. **Feedback loop?** Unmentioned.
**Prototype transcript under review:**
> **User:** how do I return this food, it's been a nightmare
> **Assistant:** So sorry to hear that! Returns are easy — you can return any item within 30 days for a full refund, in-store or online. Anything else I can help with? Here are some other foods you might love: [lists eight products]
> **User:** he actually threw up after eating it twice, is that normal
> **Assistant:** Oh no, don't worry — that food is totally gentle and safe for all pets, it's one of our bestsellers! Want me to add a bag of the next size up to your cart?
**Critique output:**
**Loop verdict:** No feedback loop is described, and no example conversation exists for this returns-that's-really-a-health-concern journey. That's the first finding — this path needs a signed-off example conversation and a regression suite before it ships.
| # | Priority | Where | Finding | Fix | Regression test |
|---|---|---|---|---|---|
| 1 | Blocker | Turn 2 | Agreeable hallucination on a safety claim — "totally gentle and safe for all pets" is a confident, ungrounded claim in response to a reported adverse reaction. | Acknowledge the reaction, make no safety claim, hand off fast to a person, suggest stopping use in the meantime; the handoff must carry the reaction and product, not "return question." | Assert the assistant did NOT make an unqualified safety claim in response to a reported reaction, and offered a handoff carrying the reaction and product context. |
| 2 | Major | Turn 1 | Answered the literal return question, missed the real situation (an adverse reaction). | Resolve the actual job: acknowledge the bad experience, flag the reaction as its own thread, don't just recite policy. | Assert it addressed the reason for the return, not only the return mechanics. |
| 3 | Major | Turn 1 | Catalog dump and premature pitch — eight products, mid-complaint, before being asked. | Resolve first; if an alternative is wanted later, one to three picks with a reason, not a scroll. | Assert no product pitch is offered before the complaint or reaction is resolved. |
| 4 | Major | Turn 2 | Pushed a purchase right after a reported adverse reaction. | No purchase prompt while a safety concern is open — care and handoff only. | Assert no add-to-cart prompt while a safety concern is unresolved. |
| 5 | Minor | Turns 1–2 | Over-cheerful tone and emoji-adjacent warmth reads as insincere on a complaint. | Calm, competent, cognitive empathy: "I understand that's upsetting — let's sort it out." | Assert the tone matches a complaint, not an upbeat sales moment. |
**Fix first:** finding one — the ungrounded safety reassurance is the ship-blocker; everything else follows from resolving the reaction honestly and handing off with context.
---
## CLOSING NOTE
Every rule here traces back to published research and practitioner guidance on conversational AI design; nothing is invented, and the cross-references to sibling skills in this set are intentional — treat them as a connected system, not standalone checklists.
Wersje
W bazie nie ma innych skilli o nazwie prototype-critic.
Migawka bazy z 13.11.2026 · wersja paczki 51a9c8a9385f (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100conversation-architect
Projektuje rozmowę asystenta doradczego lub sklepowego: warunki, wytyczne, kształt wymiany wiadomości, granice i przykładowe rozmowy, bez pisania dialogów linia po linii. - 100failure-recovery-designer
Projektuje zachowanie asystenta zakupowego lub doradczego przy błędach: wykrywanie kłopotów, potwierdzanie, odzyskiwanie i przekazywanie. - 100prototype
Buduje jednorazowy prototyp, który odpowiada na pytanie projektowe przez warianty interfejsu i przegląd stanów. - 90prototype
Buduje jednorazowy prototyp, który sprawdza model stanu lub logikę albo pokazuje, jak może wyglądać interfejs. - 90prototype
Buduje jednorazowy prototyp, który odpowiada na pytanie projektowe, np. czy model stanu działa albo jak ma wyglądać interfejs. - 90wire-prototype
Dokumentuje prototyp lub przepływ użytkownika: ekrany, stany, przejścia i adnotacje.