Przejdź do treści
Biblioteka skilli
Programowanie · Testowanie

edt-mcp-e2e-testing

/edt-mcp-e2e-testing

Prowadzi pisanie i uruchamianie zautomatyzowanych testów e2e black-box dla narzędzi EDT-MCP na żywym serwerze, z izolacją fixture w git i pokryciem przypadków negatywnych.

Repozytorium
edt-mcp ↗
Licencja
AGPL-3.0
Ostatnia zmiana repozytorium
24.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
1
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
1
Przestarzałość

Brak sygnałów. Repozytorium zmienione 24.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

AGPL-3.0, copyleft: przeróbki udostępniane innym muszą mieć tę samą licencję.

Co robi i kiedy

Prowadzi pisanie i uruchamianie zautomatyzowanych testów e2e black-box dla narzędzi EDT-MCP na żywym serwerze, z izolacją fixture w git i pokryciem przypadków negatywnych.

Oryginalny opis (EN)

How to write/run the AUTOMATED black-box e2e suite (tests/e2e/) that covers every EDT-MCP tool (62 today) against a live server with git-fixture isolation, happy + negative + error-quality coverage, and an anti-cheat bar. Use when adding/editing a test in tests/e2e/, adding a new tool (the coverage ratchet requires a test), or running the suite. (For the older MANUAL per-tool reference checklists see edt-mcp-testing; for Java unit/build see edt-mcp-build-test.)

Jak działa

  1. What this is
  2. Run it
  3. Launch-preset wire checks
  4. The rules in brief (details in tests/e2e/SKILL.md)
  5. Don't
  6. Protocol conformance (a separate gate)
Sekcje z nagłówków SKILL.md, w oryginale · 55 linii, 814 słów
Podgląd SKILL.md
---
name: edt-mcp-e2e-testing
description: How to write/run the AUTOMATED black-box e2e suite (tests/e2e/) that covers every EDT-MCP tool (62 today) against a live server with git-fixture isolation, happy + negative + error-quality coverage, and an anti-cheat bar. Use when adding/editing a test in tests/e2e/, adding a new tool (the coverage ratchet requires a test), or running the suite. (For the older MANUAL per-tool reference checklists see edt-mcp-testing; for Java unit/build see edt-mcp-build-test.)
---

# EDT-MCP — automated black-box e2e suite (tests/e2e/)

> **The full, authoritative, always-current guide is `tests/e2e/SKILL.md` in the repo. READ IT before writing or running a test** — it has the harness API, the worked examples, and the anti-cheat anti-pattern list. This skill is the entrypoint + the rules in brief; the repo file wins on any disagreement (it lives with the harness).

## What this is
A Python (stdlib-only) **black-box** suite: a real MCP client hits the **live** server on `:8765` and asserts the **real** effect. One file per tool: `tests/e2e/tools/test_<tool>.py`. Shared base: `harness.py`. Runner: `run_all.py` (serial, `--junit-xml`, per-test timeout). Every registered tool covered (62 today; the coverage ratchet `test_coverage_ratchet.py` fails the suite if a `tools/list` tool has no test). Different layer from Java/JUnit (which is headless and can't touch a real project).

## Run it
```
python tests/e2e/run_all.py --project TestConfiguration --junit-xml tests/e2e/e2e-results.xml
```
Needs a LIVE server on `:8765` + `TestConfiguration` loaded (the dev EDT copy — see the project memory / edt-redeploy loop). Execution is **serial** (one fixture, one git tree); authoring can be parallel (one agent per tool file). `--filter <substr>` runs a subset.

## Launch-preset wire checks

For `EDT_MCP_TOOL_PRESET=analysis-only` or `code-review`, restart EDT for each value and
pass the same variable to `python -m tools.test_preset_launch` from `tests/e2e`. Set
`EDT_MCP_PROGRESSIVE_DISCLOSURE=false` on EDT. IDs are case-insensitive and trimmed.
The override is resolved once on first use, including bridge calls and manual starts without
early startup; stored Tools preferences are untouched by resolution and registry reads,
including migration state. Predefined override IDs are `all-tools`, `analysis-only`,
`code-review`, and `development`. Unknown IDs produce one WARNING listing only those IDs and
the stored-selection hint, then use stored settings. Unset/blank values and `custom` use the
stored selection; explicit `custom` logs one INFO line explaining that there is no launch override.
The Tools tab loads saved settings with their normal migrations and shows a read-only launch
note. Unchanged OK/Apply preserves the saved selection and preset ID; deliberate edits apply
on a launch without the override. Use a fresh workspace for each CI preset launch; wire checks
prove the catalogue, and unit tests prove that the override is not persisted.
The module skips without a selected read-only preset. Use its direct entry point: the full runner's mandatory
`clean_project` setup/cleanup is disabled by these presets. See the launch-preset section
of `tests/e2e/SKILL.md` and `tests/TESTING.md` for the wire assertions and clean-fixture recipe.

## The rules in brief (details in tests/e2e/SKILL.md)
1. **One file per tool**, `tests/e2e/tools/test_<tool>.py`, importing ONLY from `harness`. Adding a new MCP tool? add its test file — the **coverage ratchet** fails the suite otherwise.
2. **git-fixture isolation:** `TestConfiguration` is a committed fixture. The orchestrator resets it before every test (`git reset` + `git checkout HEAD --` + `git clean -fd`). Read tools must leave it clean (`assert_no_diff()`).
3. **Verify the effect the RIGHT way per `kind=`:**
   - `read` → `assert_no_diff()` + content.
   - `write` (file-write, write_module_source) → `assert_diff_contains()` (on disk).
   - `write-metadata` (create/add/delete/rename) → **MODEL READ-BACK + on-disk structure.** Call a read tool and assert the change in the model, AND assert it landed on disk via `poll_diff_contains` of the **structural element** (`<name>X</name>`, `<catalogs>Type.X</catalogs>` — not the bare name). Metadata writes now persist to disk via forceExport (~1-2s lag → poll). The orchestrator runs `reset_fixture`+`reset_model` after these.
   - `action` → usually `assert_no_diff()` + status; never run a destructive happy mutation that corrupts the fixture.
   - **env-dependent** (form-render JVM flag, LanguageTool, debug no-session) → env-robust branch: real result OR a clear actionable **sentinel**.
4. **Negative coverage is MANDATORY** (not just happy): invalid path/object, every XOR/conditional/enum, missing-required, boundaries. Assert **error QUALITY** (`assert_error_quality`): names the bad value + actionable next step. A vague/non-actionable tool error → `# AUDIT:` note + a fix-card; **do NOT weaken the test** to a bad error.
5. **Anti-cheat (mutation thinking):** "would this test FAIL if the tool were broken?" If not, it's a CHEAT. No trivial asserts, no `assert_no_diff()`-only, no swallowed exceptions, no tolerant matching. A separate anti-cheat reviewer rules REAL/CHEAT.

## Don't
- Don't add pip dependencies (stdlib only; the client is kept spec-conformant by hand on purpose).
- Don't assert a metadata-write effect with the BARE name in the diff (it false-matches the parent's collection reference) — assert the structural element via `poll_diff_contains`, paired with a model read-back. (Metadata writes DO persist now, with a ~1-2s lag.)
- Don't parallelize the RUN (serial only). Don't call `reset_fixture`/`reset_model` in a test (the orchestrator owns them).

## Protocol conformance (a separate gate)
This suite tests tool **business logic**. The MCP **protocol** itself (initialize handshake, capabilities/version negotiation, `Mcp-Session-Id`, `Accept`/`Content-Type`, `isError`, `ping`, SSE) is a different layer, checked by the official `modelcontextprotocol/conformance` suite — see `tests/conformance/` (`tests/conformance/README.md`). Run: `npx @modelcontextprotocol/conformance@latest server --url http://127.0.0.1:8765/mcp --spec-version 2025-11-25 --expected-failures tests/conformance/baseline.yml` → green when only the pinned (intentional) gaps fail. Don't mix the two gates.

Related: `edt-mcp-tool-conventions` (param/error/output canon), `edt-mcp-bilingual` (ru/en), `edt-mcp-build-test` (Java unit/build), `edt-mcp-testing` (manual per-tool checklists), `edt-mcp-ready-to-deploy` (final pre-merge checklist).

Wersje

RepozytoriumWynikLicencja
Ta strona · ditrixnew/edt-mcp100AGPL-3.0
infostart-hub/edt-mcp-aiadopted100AGPL-3.0

Migawka bazy z 13.11.2026 · wersja paczki 02e701709334 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 88edt-mcp-testing
    Testuje ręcznie od początku do końca każde narzędzie serwera EDT-MCP na żywym środowisku EDT, z oczekiwanym wynikiem i pułapkami.
  • 90edt-mcp-testing
    Opisuje, jak testować ręcznie każde narzędzie serwera EDT-MCP w działającym środowisku EDT, z przykładowym wywołaniem i oczekiwanym wynikiem.
  • 100edt-mcp-yaxunit
    Pomaga pisać i uruchamiać testy jednostkowe YAXUnit dla konfiguracji 1C przez 1C:EDT i narzędzia EDT-MCP, z modułami ИсполняемыеСценарии i asercjami ЮТест.
  • 100edt-mcp-new-tool
    Prowadzi przez checklistę dodawania nowego narzędzia MCP do wtyczki EDT-MCP, obejmującą interfejs, schemat, rejestrację, testy i wdrożenie.
  • 100testing
    Podaje wskazówki do testów React Native: logiki, zachowania komponentów, ścieżek krytycznych E2E i przypadków brzegowych.
  • 100unit-testing
    Pisze i aktualizuje testy jednostkowe dla nowego lub zmienionego kodu, uruchamia je z pokryciem i naprawia błędy.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.