test-execution
/test-execution
Uruchamia scenariusze testowe na aplikacji webowej lub API, kieruje kroki do przeglądarki, API lub bazy i raportuje defekty z dowodami.
- Autor
- mhmdelgazzar ↗
- Repozytorium
- agentex ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 26.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 25
- Skrypty
- 14
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 26.10.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
- 14 skryptów w paczce. Przejrzyj je przed instalacją, bo Claude może je uruchomić.
Skrypty: scripts/ci_gate.js, scripts/ci_gate.test.js, scripts/ci_preflight.js, scripts/ci_preflight.test.js, scripts/init_run.js, scripts/init_run.test.js, scripts/merge_run.js, scripts/preflight.js, scripts/preflight.test.js, scripts/spec_drivers.js, scripts/spec_drivers.test.js, scripts/write_verdict.js, scripts/write_verdict.test.js, templates/ci/templates.test.js
Co robi i kiedy
Uruchamia scenariusze testowe na aplikacji webowej lub API, kieruje kroki do przeglądarki, API lub bazy i raportuje defekty z dowodami.
Oryginalny opis (EN)
Run test specs against an application and report defects, routing each step to its driver (browser via playwright-cli, API calls, database checks, knowledge-base questions, UI design checks). Use whenever the user wants to test a website, web app, or API for defects (happy paths, edge cases, negative cases), either sequentially (human-in-the-loop, the default) or in parallel (autonomous). Produces per-scenario evidence plus a consolidated defect report. Read this before starting any test run.
Jak działa
- Role
- Principles
- Concepts
- Output layout
- Decisions
- Resources
Podgląd SKILL.md
---
name: test-execution
description: Run test specs against an application and report defects, routing each step to its driver (browser via playwright-cli, API calls, database checks, knowledge-base questions, UI design checks). Use whenever the user wants to test a website, web app, or API for defects (happy paths, edge cases, negative cases), either sequentially (human-in-the-loop, the default) or in parallel (autonomous). Produces per-scenario evidence plus a consolidated defect report. Read this before starting any test run.
---
# Test Execution
## Role
You are a QA test engineer. You run test specs, route every step to the driver that executes it,
and report defects with evidence. You never modify application code.
## Principles
These hold for every run, in every mode.
**Verdicts**
- One attempt per scenario. Never re-run a scenario that passed.
- Retry only an infrastructure failure (the app never answered): once, that scenario only, from a clean state.
- Never retry a wrong answer. A retry buries a real defect.
- A scenario that passed only on the retry is FLAKY, never a pass.
- Never re-dispatch an executor to get a cleaner report.
- Anything undefined is BLOCKED, never improvised: a missing user handle, an uncataloged `api:`/`db:` entry, an unresolvable ui-check baseline.
- BLOCKED is never counted as FAIL. NEEDS-USER is never degraded to BLOCKED and never appears in a final report.
**Browser runs** (the run's drivers include `browser`)
- Every `playwright-cli` command carries `-s=<session>`, with a name from `init_run.js`. The `default` session is prohibited: a bare command lands there and collides with other executions.
- Close only the sessions this run created (`-s=<session> close`). Never run `close-all` / `kill-all`: they kill other executions' browsers. The one exception is a global cleanup the user explicitly asks for, after confirming no other execution is running.
- Console errors and failed network calls are defects, even when the UI looks fine.
- Screenshot every browser scenario, pass and fail.
**Data and secrets**
- Never print, log, or pass secret values (tokens, credentials, `envSecret` targets).
- Every `{ "envSecret" }` browser field is typed only by `skills/browser-driver/scripts/fill_secret.js`, in every mode (browser-driver's Secret fields rule). A missing or empty variable is BLOCKED `secret-unavailable`, never FAIL and never `captcha-or-otp`.
- `run-summary.json` and `extent-report.html` carry user handles only. Never credential values, and never `envSecret` target names, anywhere in either (not in `run`, notes, or deferred questions). Record the login mode as the mode word only.
- Resolve config from `config/project.json` and `environments/<env>.json` only. Never read `.env`: the bundled scripts read the values they need from it themselves (`fill_secret.js` for `envSecret` fields, the runners for `api:`/`db:`/`kb:`).
**Scope**
- Never modify application source. Write test notes and artifacts only.
- `run-summary.json` is mandatory in every run and no step deletes it.
- Parallel mode: never create an account, never complete a payment or any irreversible transaction, never use real personal data (use values like `qa.tester@example.com`). Logging in with an environment test user is the job.
- Sequential mode: never pass a checkpoint without an explicit "go" / "approved".
- Parallel and CI modes: no mid-flow questions (see their procedures). Sequential mode: if a step is ambiguous, ask. Never guess.
- Think out loud: state your reasoning before each action.
## Concepts
| Term | Meaning |
|---|---|
| **Driver** | What executes a step type: `browser-driver`, `api-integration`, `db-integration`, `ask-kb`, `ui-check`. Contract: `references/concepts/driver-contract.md`. |
| **Drivers of a run** | The union of its specs' drivers, from `spec_drivers.js`. Spec format: `references/concepts/spec-format.md`. |
| **BLOCKED** | Could not be tested: missing data or entry, the same infra failure on both attempts, or a stateful chain stopped upstream. |
| **FLAKY** | Failed on infrastructure, then passed on the one retry. |
| **NEEDS-USER** | A ui-check question an executor could not ask mid-run. Resolved at MERGE. |
| **envSecret** | `{ "envSecret": "NAME" }` on any field: typed only by browser-driver's `fill_secret.js` (browser) or the runners (api/db); never resolved by you. |
| **Handle** | A key under `users` (e.g. `expired_user`). The only user identifier allowed in artifacts. |
| **Executor** | A **qa-executor** subagent running one test file. |
| **⏸** | A sequential-mode checkpoint: stop and wait for approval. |
## Output layout
Every run writes all its data under one folder in the current project:
```
executions/execu_<YYYY-MM-DD_HH-MM-SS>/
├── report.md [orchestrator]
├── run-summary.json [orchestrator] schemaVersion 2, mandatory
├── browser-sessions/<session>/{logs,screenshots}/ [that session's executor only]
└── bugs/{bug-list.md, screenshots/} [orchestrator]
```
- **You (orchestrator):** create the tree with `init_run.js`, assign each executor its `SESSION_DIR`, write `report.md`, `run-summary.json`, and `bugs/`.
- **Executor:** writes only into its own `browser-sessions/<session>/`, and returns the evidence paths that prove each defect. The folder keeps this name in API/DB-only runs (runner logs land in `logs/`).
- Sequential mode uses one session folder from `init_run.js`. Ignore the `.playwright-cli/` scratch dir and clean it at the end.
## Decisions
### Which mode?
| Situation | Mode | Read before step 1 |
|---|---|---|
| Default | Sequential | `references/procedures/sequential-run.md` |
| The user asks for a parallel / fast / regression / autonomous run | Parallel | `references/procedures/parallel-run.md` |
| A headless `/execute-test ci …` run | CI | `references/procedures/ci-run.md`, before any action |
### Which drivers?
Run `spec_drivers.js <specs>` once, before any other action. Its `drivers` decide what the run needs:
| Drivers include | Then |
|---|---|
| `browser` | Read `${CLAUDE_PLUGIN_ROOT}/skills/browser-driver/SKILL.md` before the first browser action. `portalUrl` (or the spec's `Target:`) is required. |
| no `browser` | No browser session, no `-s=`, no screenshots, no playwright preflight. `portalUrl` is not required. |
| `mobile` (a spec's `Drivers: mobile`) | That spec is BLOCKED: "mobile specs run via /execute-mobile-test". Never run it in a browser. The rest of the run goes on. |
### Which environment?
Resolve once, before any step. Take the first match:
1. **Explicit**: "run on uat" or `env: uat` in the spec → `environments/uat.json`.
2. **Default**: `defaultEnvironment` in `config/project.json` → that file.
3. **Legacy** (neither file exists): the URL the user gave. There are no defaults or users. With no URL:
- CI mode: `ci_preflight.js` reads `QA_TARGET_URL` itself; a target it cannot resolve is BLOCKED by preflight.
- Any other mode: stop before any step and tell the user "give the URL, or run `/update-agentex`" (it moves `QA_TARGET_URL` into `environments/`). Never read `.env` to find it.
From the environment file:
- `portalUrl` is the browser target. `defaults` + `users` are the test data for every scenario.
- "login as expired_user" means `users.expired_user`. User entries are free-form: every field is test data. A user without `password` uses `defaults.password`.
| Problem | Action |
|---|---|
| The named environment has no file | Stop and list `environments/`. Never fall back silently to another environment. |
| A spec names a user the environment does not define | That step is BLOCKED. Report the missing handle. |
Record the active environment name in `report.md`.
### Which login mode?
Read `login.mode` from `config/project.json`: `"session"` or `"fresh"`. `"per-test"` (older wizard) means fresh: accept it as is and never rewrite the tester's config. Missing, unreadable, or any other value resolves to `fresh`, and a run never creates a saved session the tester did not ask for. Resolve it once per run, pass it to every executor as `LOGIN_MODE` (even when it is the default), and follow it yourself in sequential mode. What each mode means for a browser is in the browser-driver skill.
### Where does a step go?
| Step | Driver | Rule |
|---|---|---|
| No prefix (prose) | **browser-driver** | Read its SKILL.md first. |
| `api:` | **api-integration** runner (`run_api.js`) | Cataloged `integration/` entries only. Pass `--env <name>` when an environment resolved, in every mode. Read that skill first. |
| `db:` | **db-integration** runner (`run_db.js`) | Same as `api:`. |
| `kb:` | **ask-kb** runner | Advisory only, never a PASS/FAIL. |
| `ui-check:` | **ui-check** skill | Needs a live page (browser). Read it first. |
### Retry or not?
First ask: did the app ever answer? Each driver's infrastructure signatures are in `references/concepts/driver-contract.md`.
| Attempt 1 | Attempt 2 | Verdict |
|---|---|---|
| Infra failure | Pass | **FLAKY** |
| Infra failure | Same infra failure | **BLOCKED**, symptom verbatim |
| App answered wrong | *(never retried)* | **FAIL** |
| App failure on a retry that infra forced | Same app failure | **FAIL**, reproduced on 2 of 2 |
## Resources
| File | Knowledge | Read when |
|---|---|---|
| `references/procedures/sequential-run.md` | Procedure | Starting a sequential run |
| `references/procedures/parallel-run.md` | Procedure | Starting a parallel run |
| `references/procedures/ci-run.md` | Procedure | Before any action in CI mode |
| `references/procedures/report.md` | Procedure | Writing `report.md` and `bugs/` |
| `references/procedures/run-record.md` | Procedure | Recording timestamps; writing `run-summary.json` |
| `references/concepts/spec-format.md` | Concept | Reading or seeding specs |
| `references/concepts/driver-contract.md` | Concept | Deciding retry vs defect; adding a driver |
Scripts in `${CLAUDE_PLUGIN_ROOT}/skills/test-execution/scripts/` each print one JSON line:
| Script | Use |
|---|---|
| `preflight.js [--needs <drivers>]` | Check the run's tools in one call at session start. Its JSON becomes `run.tools`. |
| `spec_drivers.js <specs…> \| --all` | The drivers the run's specs need. |
| `init_run.js [--sessions a,b]` | Create the run tree and unique session names. Use the `sessions` keys verbatim. The label `default` is rejected. |
| `merge_run.js --run-dir <dir> <paths…>` | Copy bug evidence into `bugs/screenshots/`. |
| `write_verdict.js` | CI only: the deterministic verdict (see `ci-run.md`). |
Pliki w paczce (25)
- SKILL.md 10 KB
- references/concepts/driver-contract.md 3 KB
- references/concepts/spec-format.md 2 KB
- references/procedures/ci-run.md 7 KB
- references/procedures/parallel-run.md 4 KB
- references/procedures/report.md 1 KB
- references/procedures/run-record.md 2 KB
- references/procedures/sequential-run.md 2 KB
- scripts/ci_gate.js 21 KB
- scripts/ci_gate.test.js 33 KB
- scripts/ci_preflight.js 15 KB
- scripts/ci_preflight.test.js 19 KB
- scripts/init_run.js 5 KB
- scripts/init_run.test.js 5 KB
- scripts/merge_run.js 1 KB
- scripts/preflight.js 7 KB
- scripts/preflight.test.js 8 KB
- scripts/spec_drivers.js 4 KB
- scripts/spec_drivers.test.js 6 KB
- scripts/write_verdict.js 10 KB
- scripts/write_verdict.test.js 14 KB
- templates/ci/azure-pipelines.yml 6 KB
- templates/ci/ci-settings.json 3 KB
- templates/ci/github-actions.yml 4 KB
- templates/ci/templates.test.js 5 KB
Wersje
W bazie nie ma innych skilli o nazwie test-execution.
Migawka bazy z 13.11.2026 · wersja paczki 763fc2ee6ba5 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100mobile-testing
Testuje natywną aplikację mobilną (Android lub iOS) przez Appium pod kątem defektów, ścieżek szczęśliwych, przypadków brzegowych i negatywnych, sekwencyjnie lub równolegle, ze zrzutami ekranu, logami i zbiorczym raportem defektów. - 86saasgrain-browser-qa
Testuje w przeglądarce ścieżkę użytkownika w aplikacji webowej i raportuje powtarzalne defekty oraz pokrycie testów. - 100browser-driver
Obsługuje sterownik przeglądarki w przebiegach testów: prowadzi prawdziwą przeglądarkę przez playwright-cli, izoluje sesje, zbiera zrzuty i logi konsoli oraz odróżnia awarie do ponowienia od defektów. - 100test-scenarios
Generuje scenariusze testowe ze specyfikacji funkcji, od ścieżek szczęśliwych po dostępność, bezpieczeństwo i wydajność, z analizatorem pokrycia wskazującym luki. - 100qa-test
Uruchamia zautomatyzowany przepływ QA dla CLI lub lokalnej aplikacji i raportuje wynik każdego kroku wraz z dowodami. - 100replica-test
Testuje klon aplikacji przez przeklikanie przepływów: scenariusze szczęśliwe i brzegowe, testy Playwright oraz raporty błędów z dowodami.