Skip to content
Skills Library
Programming · Testing

edt-mcp-e2e-testing

/edt-mcp-e2e-testing

How to write/run the AUTOMATED black-box e2e suite (tests/e2e/) that covers every EDT-MCP tool (62 today) against a live server with git-fixture isolation, happy + negative + error-quality coverage, and an anti-cheat bar. Use when adding/editing a test in tests/e2e/, adding a new tool (the coverage ratchet requires a test), or running the suite. (For the older MANUAL per-tool reference checklists see edt-mcp-testing; for Java unit/build see edt-mcp-build-test.)

Repository
edt-mcp ↗
Licence
AGPL-3.0
Last repository change
24.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
1
Deprecation

No signals detected. Repository changed on 24.10.2026, without references to retired models or APIs.

Licence

AGPL-3.0, copyleft: przeróbki udostępniane innym muszą mieć tę samą licencję.

What it does and when to use it

How to write/run the AUTOMATED black-box e2e suite (tests/e2e/) that covers every EDT-MCP tool (62 today) against a live server with git-fixture isolation, happy + negative + error-quality coverage, and an anti-cheat bar. Use when adding/editing a test in tests/e2e/, adding a new tool (the coverage ratchet requires a test), or running the suite. (For the older MANUAL per-tool reference checklists see edt-mcp-testing; for Java unit/build see edt-mcp-build-test.)

Original author description

How to write/run the AUTOMATED black-box e2e suite (tests/e2e/) that covers every EDT-MCP tool (62 today) against a live server with git-fixture isolation, happy + negative + error-quality coverage, and an anti-cheat bar. Use when adding/editing a test in tests/e2e/, adding a new tool (the coverage ratchet requires a test), or running the suite. (For the older MANUAL per-tool reference checklists see edt-mcp-testing; for Java unit/build see edt-mcp-build-test.)

How it works

  1. What this is
  2. Run it
  3. Launch-preset wire checks
  4. The rules in brief (details in tests/e2e/SKILL.md)
  5. Don't
  6. Protocol conformance (a separate gate)
Original SKILL.md headings · 55 lines, 814 words
SKILL.md preview
---
name: edt-mcp-e2e-testing
description: How to write/run the AUTOMATED black-box e2e suite (tests/e2e/) that covers every EDT-MCP tool (62 today) against a live server with git-fixture isolation, happy + negative + error-quality coverage, and an anti-cheat bar. Use when adding/editing a test in tests/e2e/, adding a new tool (the coverage ratchet requires a test), or running the suite. (For the older MANUAL per-tool reference checklists see edt-mcp-testing; for Java unit/build see edt-mcp-build-test.)
---

# EDT-MCP — automated black-box e2e suite (tests/e2e/)

> **The full, authoritative, always-current guide is `tests/e2e/SKILL.md` in the repo. READ IT before writing or running a test** — it has the harness API, the worked examples, and the anti-cheat anti-pattern list. This skill is the entrypoint + the rules in brief; the repo file wins on any disagreement (it lives with the harness).

## What this is
A Python (stdlib-only) **black-box** suite: a real MCP client hits the **live** server on `:8765` and asserts the **real** effect. One file per tool: `tests/e2e/tools/test_<tool>.py`. Shared base: `harness.py`. Runner: `run_all.py` (serial, `--junit-xml`, per-test timeout). Every registered tool covered (62 today; the coverage ratchet `test_coverage_ratchet.py` fails the suite if a `tools/list` tool has no test). Different layer from Java/JUnit (which is headless and can't touch a real project).

## Run it
```
python tests/e2e/run_all.py --project TestConfiguration --junit-xml tests/e2e/e2e-results.xml
```
Needs a LIVE server on `:8765` + `TestConfiguration` loaded (the dev EDT copy — see the project memory / edt-redeploy loop). Execution is **serial** (one fixture, one git tree); authoring can be parallel (one agent per tool file). `--filter <substr>` runs a subset.

## Launch-preset wire checks

For `EDT_MCP_TOOL_PRESET=analysis-only` or `code-review`, restart EDT for each value and
pass the same variable to `python -m tools.test_preset_launch` from `tests/e2e`. Set
`EDT_MCP_PROGRESSIVE_DISCLOSURE=false` on EDT. IDs are case-insensitive and trimmed.
The override is resolved once on first use, including bridge calls and manual starts without
early startup; stored Tools preferences are untouched by resolution and registry reads,
including migration state. Predefined override IDs are `all-tools`, `analysis-only`,
`code-review`, and `development`. Unknown IDs produce one WARNING listing only those IDs and
the stored-selection hint, then use stored settings. Unset/blank values and `custom` use the
stored selection; explicit `custom` logs one INFO line explaining that there is no launch override.
The Tools tab loads saved settings with their normal migrations and shows a read-only launch
note. Unchanged OK/Apply preserves the saved selection and preset ID; deliberate edits apply
on a launch without the override. Use a fresh workspace for each CI preset launch; wire checks
prove the catalogue, and unit tests prove that the override is not persisted.
The module skips without a selected read-only preset. Use its direct entry point: the full runner's mandatory
`clean_project` setup/cleanup is disabled by these presets. See the launch-preset section
of `tests/e2e/SKILL.md` and `tests/TESTING.md` for the wire assertions and clean-fixture recipe.

## The rules in brief (details in tests/e2e/SKILL.md)
1. **One file per tool**, `tests/e2e/tools/test_<tool>.py`, importing ONLY from `harness`. Adding a new MCP tool? add its test file — the **coverage ratchet** fails the suite otherwise.
2. **git-fixture isolation:** `TestConfiguration` is a committed fixture. The orchestrator resets it before every test (`git reset` + `git checkout HEAD --` + `git clean -fd`). Read tools must leave it clean (`assert_no_diff()`).
3. **Verify the effect the RIGHT way per `kind=`:**
   - `read` → `assert_no_diff()` + content.
   - `write` (file-write, write_module_source) → `assert_diff_contains()` (on disk).
   - `write-metadata` (create/add/delete/rename) → **MODEL READ-BACK + on-disk structure.** Call a read tool and assert the change in the model, AND assert it landed on disk via `poll_diff_contains` of the **structural element** (`<name>X</name>`, `<catalogs>Type.X</catalogs>` — not the bare name). Metadata writes now persist to disk via forceExport (~1-2s lag → poll). The orchestrator runs `reset_fixture`+`reset_model` after these.
   - `action` → usually `assert_no_diff()` + status; never run a destructive happy mutation that corrupts the fixture.
   - **env-dependent** (form-render JVM flag, LanguageTool, debug no-session) → env-robust branch: real result OR a clear actionable **sentinel**.
4. **Negative coverage is MANDATORY** (not just happy): invalid path/object, every XOR/conditional/enum, missing-required, boundaries. Assert **error QUALITY** (`assert_error_quality`): names the bad value + actionable next step. A vague/non-actionable tool error → `# AUDIT:` note + a fix-card; **do NOT weaken the test** to a bad error.
5. **Anti-cheat (mutation thinking):** "would this test FAIL if the tool were broken?" If not, it's a CHEAT. No trivial asserts, no `assert_no_diff()`-only, no swallowed exceptions, no tolerant matching. A separate anti-cheat reviewer rules REAL/CHEAT.

## Don't
- Don't add pip dependencies (stdlib only; the client is kept spec-conformant by hand on purpose).
- Don't assert a metadata-write effect with the BARE name in the diff (it false-matches the parent's collection reference) — assert the structural element via `poll_diff_contains`, paired with a model read-back. (Metadata writes DO persist now, with a ~1-2s lag.)
- Don't parallelize the RUN (serial only). Don't call `reset_fixture`/`reset_model` in a test (the orchestrator owns them).

## Protocol conformance (a separate gate)
This suite tests tool **business logic**. The MCP **protocol** itself (initialize handshake, capabilities/version negotiation, `Mcp-Session-Id`, `Accept`/`Content-Type`, `isError`, `ping`, SSE) is a different layer, checked by the official `modelcontextprotocol/conformance` suite — see `tests/conformance/` (`tests/conformance/README.md`). Run: `npx @modelcontextprotocol/conformance@latest server --url http://127.0.0.1:8765/mcp --spec-version 2025-11-25 --expected-failures tests/conformance/baseline.yml` → green when only the pinned (intentional) gaps fail. Don't mix the two gates.

Related: `edt-mcp-tool-conventions` (param/error/output canon), `edt-mcp-bilingual` (ru/en), `edt-mcp-build-test` (Java unit/build), `edt-mcp-testing` (manual per-tool checklists), `edt-mcp-ready-to-deploy` (final pre-merge checklist).

Versions

RepositoryScoreLicence
This page · ditrixnew/edt-mcp100AGPL-3.0
infostart-hub/edt-mcp-aiadopted100AGPL-3.0

Database snapshot: 13.11.2026 · package version 02e701709334 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 88edt-mcp-testing
    How to manually e2e-test each EDT-MCP server tool against a live EDT workbench + TestConfiguration. One reference per tool under references/<family>/, each with the exact MCP call, the real expected result, and gotchas. Use when validating an MCP tool end-to-end by actually…
  • 90edt-mcp-testing
    How to manually e2e-test each EDT-MCP server tool against a live EDT workbench + TestConfiguration. One reference per tool under references/<family>/, each with the exact MCP call, the real expected result, and gotchas. Use when validating an MCP tool end-to-end by actually…
  • 100edt-mcp-yaxunit
    How to write and run YAXUnit unit tests for a 1C configuration through 1C:EDT + the EDT-MCP run_yaxunit_tests / debug_yaxunit_tests tools. Covers the three-piece setup (EDT plugin = IDE runner, YAxUnit.cfe engine in the infobase, a test extension that holds the tests), the…
  • 100edt-mcp-new-tool
    Canonical, battle-tested checklist for adding a new MCP tool to the EDT-MCP plugin the right way — the IMcpTool surface, shared resolvers, schema, registration, MANIFEST, the two MANDATORY test ratchets (unit + e2e), the golden snapshot, README, and the full…
  • 100testing
    Guidance for testing React Native code — unit tests for logic, component tests for behaviour, E2E for critical paths, and edge-case coverage. Use when adding tests, reviewing test quality, or setting a coverage bar.
  • 100unit-testing
    Writes and updates unit tests for new or changed code in the project's existing framework and style, covering happy, error and boundary paths, then runs them with coverage and fixes failures. Use when implementing a feature or bug fix that needs tests, when asked to add or…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.