Skip to content
Skills Library
Programming · Analysis

analyzing-test-coverage

/analyzing-test-coverage

Find where a Qase project's test coverage is thin. Use when the user asks about test coverage or coverage gaps; what isn't tested, untested areas, or what is missing tests; which cases are still manual, not automated, or yet to be automated; which cases have never been run or executed; which suites are empty, thin, or hold no cases; how much of the project is automated; which cases are stale or outdated; or wants test cases drafted to close a gap.

Licence
MIT
Last repository change
27.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Find where a Qase project's test coverage is thin. Use when the user asks about test coverage or coverage gaps; what isn't tested, untested areas, or what is missing tests; which cases are still manual, not automated, or yet to be automated; which cases have never been run or executed; which suites are empty, thin, or hold no cases; how much of the project is automated; which cases are stale or outdated; or wants test cases drafted to close a gap.

Original author description

Find where a Qase project's test coverage is thin. Use when the user asks about test coverage or coverage gaps; what isn't tested, untested areas, or what is missing tests; which cases are still manual, not automated, or yet to be automated; which cases have never been run or executed; which suites are empty, thin, or hold no cases; how much of the project is automated; which cases are stale or outdated; or wants test cases drafted to close a gap.

How it works

  1. When to use
  2. What "coverage" can and cannot mean here
  3. Prerequisites
  4. Tools
  5. Workflow
  6. Guardrails
Original SKILL.md headings · 272 lines, 1,774 words
SKILL.md preview
---
name: analyzing-test-coverage
description: >-
  Find where a Qase project's test coverage is thin. Use when the user asks about
  test coverage or coverage gaps; what isn't tested, untested areas, or what is
  missing tests; which cases are still manual, not automated, or yet to be
  automated; which cases have never been run or executed; which suites are empty,
  thin, or hold no cases; how much of the project is automated; which cases are
  stale or outdated; or wants test cases drafted to close a gap.
---

# Quality Supervisor — Coverage / Gap Analysis

Find where a project's coverage is thin, rank the gaps by what they actually
risk, and — on request — draft the missing cases.

**Read the plugin's `references/qql.md` before writing any query** — it sits two
levels up from this skill's directory (`../../references/qql.md`). Field names
differ per entity and a wrong name is a hard error, not an empty result.

## When to use

- "Where are our coverage gaps?" / "What isn't tested in project ABC?"
- "Which cases aren't automated?" / "Which suites are empty?"
- "What's never been run?" / "Generate the missing cases for <feature>."

## What "coverage" can and cannot mean here

Decide which question you are answering and say so — these are different gaps
with different fixes:

| Lens | Question | Available? |
|---|---|---|
| **Execution** | which cases have never run? | yes |
| **Automation** | which cases are still manual? | yes |
| **Structure** | which suites are empty or unfiled? | yes |
| **Freshness** | which cases are stale? | yes |
| **Requirements** | which requirements have no tests? | **partly — see below** |

**Requirements are readable; their coverage is not.** You can query them —

```
SELECT (status, COUNT(*)) entity = "requirement" and project = "CODE" GROUP BY status
entity = "requirement" and project = "CODE" and status = "Implemented"
```

— and the response carries `id`, `title`, `description`, `status`, `type`, and
`parent_id`, so requirement hierarchy (epic → user story → feature) is available
too.

What is **not** available is the link between a requirement and its test cases.
No field on either entity exposes it, in either direction; nothing is filterable
on it; `qase_get` doesn't support requirements; and there is no requirements
endpoint in the REST API. So "which requirements have no tests" cannot be
answered, only "which requirements exist, and in what state".

If the user asks for requirement coverage, say that directly. A useful honest
substitute — clearly labelled as a proxy, not coverage — is to list requirements
marked `Implemented` and let the team judge whether tests exist for them.
Never present it as measured coverage.

Note when reporting: QQL **filters** on labels (`type = "User story"`, which is
case-sensitive here) while the **response** returns slugs (`user-story`). Map
them rather than quoting raw slugs at the user.

## Prerequisites

- A **project code**. If the user didn't give one, ask — never guess.
- QQL access (Teams/Enterprise) for everything except the suite listing.

## Tools

- `qase_project_context` — project metadata. Call first.
- `qql_search` — all the counting below. Use aggregation, not row fetching.
- `qase_api` — the suite listing (`GET /v1/suite/{code}`), which is the only way
  to see empty suites.
- `qase_case_upsert` — create one case (core). `qase_suite_upsert` — create a
  suite (core).
- `qase_case_bulk_create` — many cases in one call. **Discoverable, not core**:
  activate it first with `qase_discover_tools` or the call will fail with
  "unknown tool". Verified against the hosted server on 2026-08-18; the split
  belongs to the server, so re-check it after a server upgrade.

## Workflow

### 1. Establish the denominator

```
entity = "case" and project = "CODE"
```

Take `total`. Every gap below is a fraction of this, and a gap count without it
is meaningless — "40 cases aren't automated" reads very differently against 50
cases than against 18,000.

Call `qase_project_context` too, for the suite tree and milestones. If it reports
a truncated collection, respect that: it caps at 100 per collection, so on a
large project treat its suite list as a sample, not the tree.

### 2. Execution coverage — what has never run

Two aggregate queries, no enumeration:

```
entity = "case" and project = "CODE"
SELECT (COUNT(id)) entity = "result" and project = "CODE" GROUP BY caseId
```

The second query's `total` is the number of **distinct cases that have at least
one result**. Subtract it from the case total and you have the never-executed
count — one number, two calls, no matter how large the project.

Two caveats to state rather than hide:

- results survive their cases, so a deleted case can still contribute a group.
  That inflates the executed count and makes the gap a **lower bound**.
- this counts "ever executed". For "executed recently", add
  `and ended >= now("-90d")` to the second query and label it accordingly — a
  case last run two years ago is a different kind of gap.

To **name** the never-executed cases rather than count them, scope down first —
one suite (`suiteTree = "…"`), one milestone, or one priority — and compare the
case list against that scope's executed IDs. Enumerating them project-wide means
paging both lists in full; don't attempt it on a large project, and say why.

### 3. Automation coverage

```
SELECT (automation, COUNT(*)) entity = "case" and project = "CODE" GROUP BY automation
```

One query, whole distribution. Aggregated responses return the enum as an
integer: 0 = Manual, 1 = To be automated, 2 = Automated.

**Caveat this number.** QQL reads an index that catches up asynchronously — minutes
normally, but eleven days on one real project whose analytics republish had stalled,
still calling cases manual long after they were automated. The distribution is
therefore a **floor** on automation, not a measurement: real coverage can be better
than it says, never worse. Before quoting a percentage, spot-check two or three cases
from the `Manual` bucket with `qase_get`. If they come back automated, say the figure
is indicative and that the project's analytics data looks stale — that is a finding in
itself, not just a caveat. See `references/qql.md`.

`To be automated` is the interesting bucket — someone already decided those
should be automated and it hasn't happened. Report it separately from `Manual`
rather than lumping both into "not automated".

### 4. Structure — empty and unfiled suites

A `GROUP BY suite` **cannot find empty suites**: a suite with no cases produces
no group. Use the suite listing, which carries `cases_count` per suite:

```
GET /v1/suite/{code}?limit=100
```

Paginate with `offset` until `total` is covered. Flag suites with
`cases_count == 0`, and thin ones (1–2 cases) where siblings have many.

Cases filed nowhere are the mirror-image gap:

```
entity = "case" and project = "CODE" and suite is empty
```

For per-area depth, `suiteTree` matches a suite **and all its descendants**,
which is what you want for "how covered is area X":

```
entity = "case" and project = "CODE" and suiteTree = "tests/api/billing"
```

### 5. Risk weighting — check it is possible first

```
SELECT (priority, COUNT(*)) entity = "case" and project = "CODE" GROUP BY priority
```

Integer codes: 0 = Not set, 1 = High, 2 = Medium, 3 = Low. **`priority` has no
"critical" value** — critical is a *severity*, and `priority = "critical"` fails
outright.

Look at how much is `Not set` before ranking anything by priority. If most cases
have no priority, then "high-priority cases that aren't automated" describes a
handful of outliers, not the real risk — and the honest finding is that the
project has no usable risk metadata. Report that instead of a confident ranking
built on 0.4% of the data.

When priority *is* populated, the risk-weighted gap is:

```
SELECT (automation, COUNT(*)) entity = "case" and project = "CODE" and priority = "High" GROUP BY automation
```

Grouped rather than one filtered query per bucket: aggregation summed exactly to an
independent `COUNT(*)` everywhere it was checked, and it sidesteps the row-duplication
seen on one messy project. Read the `Manual` and `To be automated` buckets from the
grouping, and apply the freshness caveat above — it applies to this number too.

### 6. Freshness

```
entity = "case" and project = "CODE" and updated <= now("-6m")
```

Cases untouched for a long time while the product moved on are a quieter gap
than a missing case — they may pass while testing behaviour that no longer
matters. Report as a count with a couple of examples; don't infer staleness is a
defect without knowing the area.

### 7. Rank and report

Rank by what a gap risks, not by count: an untested area covered by a milestone
in flight outranks a large pile of stale low-priority cases. Say what each gap
means, not just its size.

```
## Coverage Gaps — <PROJECT>
_Generated by Qase Quality Supervisor · analyzing-test-coverage skill_

Denominator: <N> cases · <M> suites

### Execution
Never executed: <n> (<x>% of cases, lower bound) · not run in 90d: <n>

### Automation
Manual: <n> · To be automated: <n> · Automated: <n> (<x>% automated)

### Structure
Empty suites: <n> of <M> · thin suites (1-2 cases): <n> · cases with no suite: <n>

### Risk weighting
Priority set on <n> of <N> cases (<x>%). <either the High+Manual gap, or:
"risk ranking isn't meaningful on this project — priority is unset for <x>%">

### Freshness
Not updated in 6 months: <n>

### Requirements
<n> requirements, by status: <breakdown>. Coverage per requirement is **not
measurable** — Qase exposes no link between requirements and cases.

### Top gaps (ranked)
1. <area/lens> — <why it matters> — <the number>
   QQL: <query>

### Recommended next steps
- <e.g. "Draft 6 cases for the empty billing/refunds suite — say the word.">

---
_Generated by Qase Quality Supervisor · analyzing-test-coverage skill_
```

Always show the queries. If any figure came from a page rather than the whole
set, mark it.

### 8. Close gaps (only after approval)

If the user asks for the missing cases:

1. Draft them with clear titles, preconditions, steps, expected results.
2. Ensure the target suite exists. Creating one needs `qase_suite_upsert`, a core
   tool — no discovery step required.
3. Create them: `qase_case_upsert` for one, `qase_case_bulk_create` for a batch
   (same case body, one request). Enum fields accept labels (`"High"`).
   `qase_case_bulk_create` is discoverable — activate it via
   `qase_discover_tools` first, or the batch fails as an unknown tool.
4. Tag them so a human can review what was generated, and report the IDs.

Never bulk-create dozens of cases without showing a sample and getting a yes.

## Guardrails

- Read-only until the user approves a write.
- Never call a `*_delete` tool. (A hook blocks them anyway.)
- Always report gaps against the denominator, never as bare counts.
- State which lens you used — "coverage" without a stated definition is noise.
- Never present requirement coverage as measured; it isn't available.
- Prefer aggregation over paging. If you paged, say how much you saw.
- Don't infer intent from metadata absence: an unset priority means nobody set
  it, not that the case is unimportant.

Versions

RepositoryScoreLicence
This page · qase-tms/qase-quality-supervisor100MIT
jeremylongshore/tons-of-skills-marketplace100MIT
HelixDevelopment/agent100MIT
saleor/configurator100BSD-3-Clause

Database snapshot: 13.11.2026 · package version 68041857099a (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100linking-test-cases
    Link a repository's autotests to Qase test cases by meaning: propose a mapping for confirmation, write the confirmed case IDs into the test code in each Qase reporter's own syntax, and mark the linked cases Automated. Use when the user asks to link, map, match, or connect…
  • 100test-coverage-gap-finder
    Find the meaningful test-coverage gaps in a .NET project — untested public types, uncovered branches, and risky code without tests — and prioritize what to test next. Use this skill whenever the user wants to find coverage gaps, what to test next, untested code, low-coverage…
  • 84analyze-test-coverage
    Analyze test coverage gaps and prioritize what to test next. Use this skill whenever the user says "analyze test coverage", "what's our test coverage", "find gaps in test coverage", "which code isn't tested", "improve test coverage", "show me a coverage report", "coverage…
  • 100test-coverage-improver
    Analyze test coverage gaps and generate tests to improve coverage. Use when improving test coverage, finding untested code, or writing missing tests.
  • 100coverage
    Analyze test coverage gaps. Use when user says "test coverage", "what's not tested", "coverage gaps", "missing tests", "coverage report", or "what needs testing".
  • 100test-coverage-analyzer
    Analyzes test coverage reports, identifies gaps, and recommends priority areas for testing. Use when reviewing coverage, finding untested code, or planning test improvements.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.