test-case-executor
/test-case-executor
Execute supplied product test cases against web, H5, app, or mini-program builds; normalize a complete case ledger; collect case-level evidence; distinguish product defects, contract conflicts, flaky behavior, environment/data blockers, and device retest gaps; restore test data; and produce or sync an auditable report. Use when the user asks to run, verify, regress, or report cases from test platforms, spreadsheets, SMM, Markdown, JSON, screenshots, or exports.
- Author
- cristi1965 ↗
- Repository
- ai-test-skills ↗
- Licence
- MIT
- Last repository change
- 26.10.2026
Assessment and security
- Files in this package
- 9
- Scripts
- 3
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 26.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
- 3 scripts in this package. Review them before installing, as Claude may execute them.
Scripts: scripts/extract-smm-cases.mjs, scripts/skill-tests.mjs, scripts/validate-markdown-report.mjs
What it does and when to use it
Execute supplied product test cases against web, H5, app, or mini-program builds; normalize a complete case ledger; collect case-level evidence; distinguish product defects, contract conflicts, flaky behavior, environment/data blockers, and device retest gaps; restore test data; and produce or sync an auditable report. Use when the user asks to run, verify, regress, or report cases from test platforms, spreadsheets, SMM, Markdown, JSON, screenshots, or exports.
Original author description
Execute supplied product test cases against web, H5, app, or mini-program builds; normalize a complete case ledger; collect case-level evidence; distinguish product defects, contract conflicts, flaky behavior, environment/data blockers, and device retest gaps; restore test data; and produce or sync an auditable report. Use when the user asks to run, verify, regress, or report cases from test platforms, spreadsheets, SMM, Markdown, JSON, screenshots, or exports.
How it works
- Efficiency Default
- 1. Establish Scope and Evidence Boundary
- 2. Build the Case Ledger Before Execution
- 3. Compile the Expectation Oracle
- 4. Plan Coverage
- 5. Execute Deterministically
- 6. Classify Without Overclaiming
- 7. Report, Sync, and Verify
- 8. Fix and Retest
- Completion Gate
SKILL.md preview
---
name: test-case-executor
description: Execute supplied product test cases against web, H5, app, or mini-program builds; normalize a complete case ledger; collect case-level evidence; distinguish product defects, contract conflicts, flaky behavior, environment/data blockers, and device retest gaps; restore test data; and produce or sync an auditable report. Use when the user asks to run, verify, regress, or report cases from test platforms, spreadsheets, SMM, Markdown, JSON, screenshots, or exports.
---
# Test Case Executor
Use one immutable case ledger from intake through retest. Preserve source wording and IDs; add interpretation and evidence without overwriting history.
## Efficiency Default
Optimize for reliable coverage, elapsed time, and token cost together. Evidence requirements remain the acceptance floor; avoid work that does not improve a case decision.
- Read only relevant skill references and code paths. Parse large case exports with scripts; keep full ledgers/logs in files and return compact counts, failures, and needed excerpts to context.
- Order execution by critical path and risk: smoke-check the environment, then shared prerequisites, high-risk transitions, and remaining cases. Group cases by surface/fixture/state to reduce navigation and resets without leaking state between cases.
- When the user requests all cases, grouping never reduces assertion coverage. A single execution may support multiple cases only when every case's preconditions and assertions match; link the shared evidence to each case.
- Reuse authenticated sessions, browser handles, parsed inventories, and current source findings. Reuse prior acceptance evidence only when build, surface, fixture, and relevant dependencies remain valid and the user has not requested a fresh run; retain its original date and provenance.
- Prefer compact semantic snapshots and targeted assertions. Capture screenshots for visual claims, failures, or required evidence; avoid repeated full-page screenshots, DOM dumps, and unchanged logs.
- Batch independent reads and safe actions, but keep timing checks atomic and shared-state mutations sequential. Use bounded business-state waits instead of fixed sleeps or frequent unchanged polling.
- Use deterministic helpers for extraction, counting, report validation, and repeated transformations. Do not reread their implementation on every run.
- Investigate a failure with one targeted reproduction first. Repeat only to resolve a concrete uncertainty; preserve flaky history. Stop dependent cases at a confirmed shared blocker, record each affected case, and continue independent coverage.
- After a fix, run affected and adjacent regressions; broaden only for dependency impact, new failures, or an explicit full-regression request. Do not rerun unchanged successful checks without a reason.
- Keep progress updates delta-only and final reports focused on failures, blockers, coverage, and evidence links. When available, record elapsed time, tool calls, and token usage from actual telemetry; never invent savings or add measurement machinery that costs more than it helps.
## 1. Establish Scope and Evidence Boundary
Record the requested feature/release, case source, included runtimes, environment/build, account/fixture state, allowed side effects, and cleanup requirement.
- A scope reduction changes excluded rows to `OUT_OF_SCOPE`; it never deletes them from the inventory.
- Browser emulation, source inspection, API checks, deployed browser behavior, and real-device behavior are different evidence levels. Do not substitute one for another.
- Reuse an authenticated browser profile or open tab only after confirming the visible environment, organization, route, and login are still valid. Conversation history does not prove browser authentication.
- Never persist credentials, cookies, tokens, verification codes, or private account data.
## 2. Build the Case Ledger Before Execution
Normalize every source case with [references/case-model.md](references/case-model.md). For spreadsheets, use the spreadsheet skill to read/write structured cells, then export or maintain the canonical ledger as JSON. For screenshots, preserve unreadable text as a blocker instead of guessing.
For SMM:
```bash
TEST_EXECUTOR_SKILL="${CODEX_HOME:-$HOME/.codex}/skills/test-case-executor"
node "$TEST_EXECUTOR_SKILL/scripts/extract-smm-cases.mjs" /absolute/path/cases.smm --format json
```
Inventory totals must remain conserved: `source = in_scope + out_of_scope`. Record missing fields, duplicate IDs, contradictory expectations, and cases needing side-effect authorization.
Create a fixture ledger for answers, accounts, class/resource/question IDs, initial history state, dynamic API fields, and expected side effects. Do not infer the current answer or reward from old labels, prior attempts, or UI copy.
## 3. Compile the Expectation Oracle
Keep the original expected result, then record the active requirement source separately. Use the strongest confirmed source available:
1. user-confirmed product contract or current design;
2. accepted API/data semantics;
3. supplied test-case wording;
4. implementation behavior only as diagnostic evidence.
When sources conflict, classify the case contract; do not silently rewrite the expectation or change code to make the test green. A confirmed design can turn an apparent product failure into `FAIL_CASE` or a valid `PASS`, with provenance recorded.
## 4. Plan Coverage
Map every requirement, control, state transition, and visible claim to an assertion and expected evidence. Functional and visual checks share the same inventory but run as separate passes.
For stateful flows, add explicit combination cases instead of inferring combinations from isolated passes. Relevant dimensions may include natural progression, seek/drag, speed changes, refresh/re-entry, prior answer state, blockers, retry, completion, and recovery. Run at least one complete critical path.
For reversible state, verify initial state, mutation, persistence after reopen/reload, restoration, and restoration persistence.
## 5. Execute Deterministically
Before each action, inspect the current URL/screen, loading state, target identity, and preconditions. Prefer semantic roles, labels, and accessibility references. Use screenshots for visual evidence and coordinates only when the surface lacks a reliable semantic target.
For each assertion:
1. reset or isolate state left by earlier cases;
2. perform the documented user action;
3. read the cheapest authoritative result;
4. record actual result and evidence immediately;
5. record mutations and cleanup obligations.
Do not use `networkidle` as a universal readiness signal for video, polling, WebSocket, or analytics-heavy pages. Wait for the business signal: metadata/first frame, time progression, question visibility, submit response, persisted state, or the explicit failure state.
Time-sensitive behavior must be captured in one automation sequence with timestamps. Time elapsed between separate tool calls is not evidence for a countdown or lock interval.
For weak-network/media cases, record the actual injection point and request. Distinguish latency, timeout, HTTP failure, offline, `waiting/stalled`, image/GIF failure, and submit delay. A generic throttle preset does not prove all of them.
## 6. Classify Without Overclaiming
Use the terminal statuses defined in the case model. Important rules:
- `PASS` requires every assertion to pass at the required evidence level.
- `FLAKY` means an attempt failed and a retry passed; never collapse it into stable `PASS`.
- `FAIL_PRODUCT` requires a valid expectation plus observed contradictory behavior.
- `FAIL_CASE` requires requirement/design evidence showing the source case is wrong or inapplicable.
- `BLOCKED_ENV` and `BLOCKED_DATA` are not product failures.
- `DEVICE_RETEST_REQUIRED` records that browser/static evidence exists but the required app/WebView/device acceptance was not executed.
- `NOT_RUN` always has a reason.
Code inspection, tests, compilation, browser simulation, deployed WebUI, and real-device checks each support only their own layer.
## 7. Report, Sync, and Verify
Use [references/report-template.md](references/report-template.md). Lead with unresolved product failures and blockers, then coverage and evidence. Preserve original failures and add retest attempts rather than replacing them.
Validate the report with a Node 18+ runtime:
```bash
TEST_EXECUTOR_SKILL="${CODEX_HOME:-$HOME/.codex}/skills/test-case-executor"
node "$TEST_EXECUTOR_SKILL/scripts/validate-markdown-report.mjs" /absolute/path/report.md [expected-case-count]
```
When syncing to a test platform, use a two-phase flow:
1. prepare a reviewable case-ID/status/note write set;
2. apply it, reload/re-read, and verify exact case ID, status, and note.
A disappearing row or changed filter count does not prove persistence. Existing historical results may be synced only when labeled with their original evidence source, environment, and date; missing provenance is `BLOCKED_DATA`, not a guessed backfill.
## 8. Fix and Retest
Testing and fixing remain separate evidence phases. If the current request already authorizes fixes, freeze the original failing attempt and proceed without asking again. Otherwise report the proposed fixes before changing product code.
After a fix, rerun the failed assertions, the full affected case, adjacent regression cases, and any combination path touched by the change. Record the new attempt without erasing the original failure.
## Completion Gate
Do not claim completion unless totals reconcile; every in-scope case has a terminal status; every `PASS`, `FAIL_PRODUCT`, `FAIL_CASE`, and `FLAKY` has the required evidence; blockers and unrun items have reasons; required combination paths ran; reversible data was restored or handed off; external sync was re-read and verified; and product Git contains no test reports, screenshots, temporary harnesses, or other non-source artifacts.
For authenticated legacy admin pages, read [references/console-webui.md](references/console-webui.md). For video, point-question, timing, and network-failure work, read [references/media-timing-network.md](references/media-timing-network.md).
Files in this package (9)
- SKILL.md 10 KB
- agents/openai.yaml 1 KB
- references/case-model.md 5 KB
- references/console-webui.md 3 KB
- references/media-timing-network.md 2 KB
- references/report-template.md 2 KB
- scripts/extract-smm-cases.mjs 8 KB
- scripts/skill-tests.mjs 4 KB
- scripts/validate-markdown-report.mjs 6 KB
Versions
The database has no other skills named test-case-executor.
Database snapshot: 13.11.2026 · package version fe6dd8b44c0c (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100qa-run
Execute test cases on an environment the user names — a test-case sheet (.xlsx/.csv), the cases from ba-scenarios, or regression rows of .bdt/TESTS.md — recording Pass / Fail / Blocked / Not run with evidence for each case, writing the results into a copy of the sheet, drafting… - 100test-case-authoring
Write the test plan and test cases for a single work item: every acceptance criterion traced to a case, each case with test data, steps, an observable expected result, a level, and the evidence a pass must leave behind. Use this skill whenever the user wants to write test cases,… - 90test-cases
Format and rules for reusable test cases in tests/cases/ and for the evidence attached to PRs. Use when writing, updating or running a test case, or when collecting evidence. - 100test-case-export
Convert test cases written in Markdown or JSON into import-ready CSV for Azure DevOps Test Plans or Jira Xray (steps with action and expected result), with a bundled stdlib script. Use when test cases designed with Claude must land in the team's test management tool without retyping. - 100qa-test-case-generation
Use when generating manual QA test cases from requirements, BRDs, user stories, acceptance criteria, spreadsheets, live mockup/prototype URLs, uploaded mockups, screenshots, wireframes, or existing test case templates; especially when coverage, deduplication, traceability,… - 100test-execution
Run test specs against an application and report defects, routing each step to its driver (browser via playwright-cli, API calls, database checks, knowledge-base questions, UI design checks). Use whenever the user wants to test a website, web app, or API for defects (happy…