edt-mcp-e2e-testing
/edt-mcp-e2e-testing
Prowadzi pisanie i uruchamianie zautomatyzowanych testów e2e black-box dla narzędzi EDT-MCP na żywym serwerze, z izolacją fixture w git i pokryciem przypadków negatywnych.
- Autor
- ditrixnew ↗
- Repozytorium
- edt-mcp ↗
- Licencja
- AGPL-3.0
- Ostatnia zmiana repozytorium
- 24.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 1
Brak sygnałów. Repozytorium zmienione 24.10.2026, bez odwołań do wycofanych modeli i API.
AGPL-3.0, copyleft: przeróbki udostępniane innym muszą mieć tę samą licencję.
Co robi i kiedy
Prowadzi pisanie i uruchamianie zautomatyzowanych testów e2e black-box dla narzędzi EDT-MCP na żywym serwerze, z izolacją fixture w git i pokryciem przypadków negatywnych.
Oryginalny opis (EN)
How to write/run the AUTOMATED black-box e2e suite (tests/e2e/) that covers every EDT-MCP tool (62 today) against a live server with git-fixture isolation, happy + negative + error-quality coverage, and an anti-cheat bar. Use when adding/editing a test in tests/e2e/, adding a new tool (the coverage ratchet requires a test), or running the suite. (For the older MANUAL per-tool reference checklists see edt-mcp-testing; for Java unit/build see edt-mcp-build-test.)
Jak działa
- What this is
- Run it
- Launch-preset wire checks
- The rules in brief (details in tests/e2e/SKILL.md)
- Don't
- Protocol conformance (a separate gate)
Podgląd SKILL.md
--- name: edt-mcp-e2e-testing description: How to write/run the AUTOMATED black-box e2e suite (tests/e2e/) that covers every EDT-MCP tool (62 today) against a live server with git-fixture isolation, happy + negative + error-quality coverage, and an anti-cheat bar. Use when adding/editing a test in tests/e2e/, adding a new tool (the coverage ratchet requires a test), or running the suite. (For the older MANUAL per-tool reference checklists see edt-mcp-testing; for Java unit/build see edt-mcp-build-test.) --- # EDT-MCP — automated black-box e2e suite (tests/e2e/) > **The full, authoritative, always-current guide is `tests/e2e/SKILL.md` in the repo. READ IT before writing or running a test** — it has the harness API, the worked examples, and the anti-cheat anti-pattern list. This skill is the entrypoint + the rules in brief; the repo file wins on any disagreement (it lives with the harness). ## What this is A Python (stdlib-only) **black-box** suite: a real MCP client hits the **live** server on `:8765` and asserts the **real** effect. One file per tool: `tests/e2e/tools/test_<tool>.py`. Shared base: `harness.py`. Runner: `run_all.py` (serial, `--junit-xml`, per-test timeout). Every registered tool covered (62 today; the coverage ratchet `test_coverage_ratchet.py` fails the suite if a `tools/list` tool has no test). Different layer from Java/JUnit (which is headless and can't touch a real project). ## Run it ``` python tests/e2e/run_all.py --project TestConfiguration --junit-xml tests/e2e/e2e-results.xml ``` Needs a LIVE server on `:8765` + `TestConfiguration` loaded (the dev EDT copy — see the project memory / edt-redeploy loop). Execution is **serial** (one fixture, one git tree); authoring can be parallel (one agent per tool file). `--filter <substr>` runs a subset. ## Launch-preset wire checks For `EDT_MCP_TOOL_PRESET=analysis-only` or `code-review`, restart EDT for each value and pass the same variable to `python -m tools.test_preset_launch` from `tests/e2e`. Set `EDT_MCP_PROGRESSIVE_DISCLOSURE=false` on EDT. IDs are case-insensitive and trimmed. The override is resolved once on first use, including bridge calls and manual starts without early startup; stored Tools preferences are untouched by resolution and registry reads, including migration state. Predefined override IDs are `all-tools`, `analysis-only`, `code-review`, and `development`. Unknown IDs produce one WARNING listing only those IDs and the stored-selection hint, then use stored settings. Unset/blank values and `custom` use the stored selection; explicit `custom` logs one INFO line explaining that there is no launch override. The Tools tab loads saved settings with their normal migrations and shows a read-only launch note. Unchanged OK/Apply preserves the saved selection and preset ID; deliberate edits apply on a launch without the override. Use a fresh workspace for each CI preset launch; wire checks prove the catalogue, and unit tests prove that the override is not persisted. The module skips without a selected read-only preset. Use its direct entry point: the full runner's mandatory `clean_project` setup/cleanup is disabled by these presets. See the launch-preset section of `tests/e2e/SKILL.md` and `tests/TESTING.md` for the wire assertions and clean-fixture recipe. ## The rules in brief (details in tests/e2e/SKILL.md) 1. **One file per tool**, `tests/e2e/tools/test_<tool>.py`, importing ONLY from `harness`. Adding a new MCP tool? add its test file — the **coverage ratchet** fails the suite otherwise. 2. **git-fixture isolation:** `TestConfiguration` is a committed fixture. The orchestrator resets it before every test (`git reset` + `git checkout HEAD --` + `git clean -fd`). Read tools must leave it clean (`assert_no_diff()`). 3. **Verify the effect the RIGHT way per `kind=`:** - `read` → `assert_no_diff()` + content. - `write` (file-write, write_module_source) → `assert_diff_contains()` (on disk). - `write-metadata` (create/add/delete/rename) → **MODEL READ-BACK + on-disk structure.** Call a read tool and assert the change in the model, AND assert it landed on disk via `poll_diff_contains` of the **structural element** (`<name>X</name>`, `<catalogs>Type.X</catalogs>` — not the bare name). Metadata writes now persist to disk via forceExport (~1-2s lag → poll). The orchestrator runs `reset_fixture`+`reset_model` after these. - `action` → usually `assert_no_diff()` + status; never run a destructive happy mutation that corrupts the fixture. - **env-dependent** (form-render JVM flag, LanguageTool, debug no-session) → env-robust branch: real result OR a clear actionable **sentinel**. 4. **Negative coverage is MANDATORY** (not just happy): invalid path/object, every XOR/conditional/enum, missing-required, boundaries. Assert **error QUALITY** (`assert_error_quality`): names the bad value + actionable next step. A vague/non-actionable tool error → `# AUDIT:` note + a fix-card; **do NOT weaken the test** to a bad error. 5. **Anti-cheat (mutation thinking):** "would this test FAIL if the tool were broken?" If not, it's a CHEAT. No trivial asserts, no `assert_no_diff()`-only, no swallowed exceptions, no tolerant matching. A separate anti-cheat reviewer rules REAL/CHEAT. ## Don't - Don't add pip dependencies (stdlib only; the client is kept spec-conformant by hand on purpose). - Don't assert a metadata-write effect with the BARE name in the diff (it false-matches the parent's collection reference) — assert the structural element via `poll_diff_contains`, paired with a model read-back. (Metadata writes DO persist now, with a ~1-2s lag.) - Don't parallelize the RUN (serial only). Don't call `reset_fixture`/`reset_model` in a test (the orchestrator owns them). ## Protocol conformance (a separate gate) This suite tests tool **business logic**. The MCP **protocol** itself (initialize handshake, capabilities/version negotiation, `Mcp-Session-Id`, `Accept`/`Content-Type`, `isError`, `ping`, SSE) is a different layer, checked by the official `modelcontextprotocol/conformance` suite — see `tests/conformance/` (`tests/conformance/README.md`). Run: `npx @modelcontextprotocol/conformance@latest server --url http://127.0.0.1:8765/mcp --spec-version 2025-11-25 --expected-failures tests/conformance/baseline.yml` → green when only the pinned (intentional) gaps fail. Don't mix the two gates. Related: `edt-mcp-tool-conventions` (param/error/output canon), `edt-mcp-bilingual` (ru/en), `edt-mcp-build-test` (Java unit/build), `edt-mcp-testing` (manual per-tool checklists), `edt-mcp-ready-to-deploy` (final pre-merge checklist).
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · ditrixnew/edt-mcp | 100 | AGPL-3.0 |
| infostart-hub/edt-mcp-aiadopted | 100 | AGPL-3.0 |
Migawka bazy z 13.11.2026 · wersja paczki 02e701709334 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 88edt-mcp-testing
Testuje ręcznie od początku do końca każde narzędzie serwera EDT-MCP na żywym środowisku EDT, z oczekiwanym wynikiem i pułapkami. - 90edt-mcp-testing
Opisuje, jak testować ręcznie każde narzędzie serwera EDT-MCP w działającym środowisku EDT, z przykładowym wywołaniem i oczekiwanym wynikiem. - 100edt-mcp-yaxunit
Pomaga pisać i uruchamiać testy jednostkowe YAXUnit dla konfiguracji 1C przez 1C:EDT i narzędzia EDT-MCP, z modułami ИсполняемыеСценарии i asercjami ЮТест. - 100edt-mcp-new-tool
Prowadzi przez checklistę dodawania nowego narzędzia MCP do wtyczki EDT-MCP, obejmującą interfejs, schemat, rejestrację, testy i wdrożenie. - 100testing
Podaje wskazówki do testów React Native: logiki, zachowania komponentów, ścieżek krytycznych E2E i przypadków brzegowych. - 100unit-testing
Pisze i aktualizuje testy jednostkowe dla nowego lub zmienionego kodu, uruchamia je z pokryciem i naprawia błędy.