Skip to content
Skills Library
Programming · Testing

debugging-failures

/debugging-failures

Diagnose a failing Ginkgo suite as an agent — always run with --json-report into a predictable temp/gitignored location, read the terminal verdict line, then use jq to extract structured failure details (name, message, file:line, panic value, captured logs). Covers the panicked-vs-failed trap, panic locations pointing into the Go runtime, parallel output interleaving, reproducing with --seed, and progress reports for hangs. Use when a suite failed and you need to know why. Also invokable as /ginkgo:debugging-failures.

Author
onsi ↗
Repository
ginkgo ↗
Licence
MIT
Last repository change
09.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
1
Deprecation

No signals detected. Repository changed on 09.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Diagnose a failing Ginkgo suite as an agent — always run with --json-report into a predictable temp/gitignored location, read the terminal verdict line, then use jq to extract structured failure details (name, message, file:line, panic value, captured logs). Covers the panicked-vs-failed trap, panic locations pointing into the Go runtime, parallel output interleaving, reproducing with --seed, and progress reports for hangs. Use when a suite failed and you need to know why. Also invokable as /ginkgo:debugging-failures.

Original author description

Diagnose a failing Ginkgo suite as an agent — always run with --json-report into a predictable temp/gitignored location, read the terminal verdict line, then use jq to extract structured failure details (name, message, file:line, panic value, captured logs). Covers the panicked-vs-failed trap, panic locations pointing into the Go runtime, parallel output interleaving, reproducing with --seed, and progress reports for hangs. Use when a suite failed and you need to know why. Also invokable as /ginkgo:debugging-failures.

How it works

  1. The workflow: one run, JSON-first diagnosis
  2. Where the JSON lands (use --output-dir deliberately)
  3. The jq filters (tested against the real schema)
  4. Schema facts you need
  5. The three traps that bite agents
  6. Reproduce, then narrow
  7. When it's a hang, not a failure
  8. Other formats
Original SKILL.md headings · 93 lines, 817 words
SKILL.md preview
---
name: debugging-failures
description: Diagnose a failing Ginkgo suite as an agent — always run with --json-report into a predictable temp/gitignored location, read the terminal verdict line, then use jq to extract structured failure details (name, message, file:line, panic value, captured logs). Covers the panicked-vs-failed trap, panic locations pointing into the Go runtime, parallel output interleaving, reproducing with --seed, and progress reports for hangs. Use when a suite failed and you need to know why. Also invokable as /ginkgo:debugging-failures.
---

# Debugging Ginkgo failures (especially as an agent)

Ginkgo's terminal output is good for a human; for *programmatic* diagnosis, prefer a hybrid: **terminal for the verdict, JSON + `jq` for the details.** This is empirically the most reliable, lowest-token workflow. Docs: <https://onsi.github.io/ginkgo/#reporting-and-profiling-suites>.

## The workflow: one run, JSON-first diagnosis

```bash
# One invocation gives you BOTH the human verdict on the terminal AND a structured report.
# --json-report does NOT suppress console output, so you don't pay for two runs.
ginkgo -r -p --no-color --json-report=report.json --output-dir=.ginkgo-report
```

1. **Read the terminal tail** for the one-line verdict: `FAIL! -- 5 Passed | 6 Failed | 1 Pending | 1 Skipped`. Cheap, immediate.
2. **If there are failures, query the JSON with `jq`** (filters below) to get exactly the failing specs — stable, addressable, order-independent.
3. **Clean up** the report dir when done.

Why hybrid and not one or the other:
- **Terminal-only** is brittle to parse (free-form text) and, **under `-p`, interleaves failures in nondeterministic order**.
- **JSON-only** is wasteful if you dump it: the full `report.json` is ~9× the tokens of the terminal output. A `jq` failures-only extraction is a fraction of either. The discipline is *filter, don't dump*.

## Where the JSON lands (use --output-dir deliberately)

- `--json-report=report.json` alone writes `report.json` to the **current directory** — which scatters into package dirs under `-r` and risks getting committed.
- **Prefer `--output-dir=DIR`** to collect everything in one known place (auto-created). With `-r`, all suites **merge into one `DIR/report.json`** — ideal: one file, one set of `jq` filters.
- Use a **gitignored or temp** location: a repo-local `.ginkgo-report/` (add it to `.gitignore`) or an absolute temp path (`--output-dir=/tmp/ginkgo-report`). Don't leave `report.json` in the working tree.
- `--keep-separate-reports` (with `-r`) writes one `PACKAGE_report.json` per package instead of merging — only reach for it when you must attribute failures per package.

## The jq filters (tested against the real schema)

The report is an **array of suite reports**, each with a `.SpecReports[]` array.

**Verdict / counts per suite:**
```bash
jq -r '.[] | "\(.SuiteDescription): \([.SpecReports[].State]|group_by(.)|map("\(length) \(.[0])")|join(", "))"' .ginkgo-report/report.json
```

**All failures — name, location, message** (the workhorse). **Select `failed` *and* `panicked` *and* `timedout`** — see the trap below:
```bash
jq -r '.[].SpecReports[]
  | select(.State=="failed" or .State=="panicked" or .State=="timedout")
  | "[\(.State|ascii_upcase)] \((.ContainerHierarchyTexts+[.LeafNodeText])|join(" > "))\n  \(.Failure.Location.FileName):\(.Failure.Location.LineNumber)\n  \(.Failure.Message)"' .ginkgo-report/report.json
```

**Panics — the real user-code line** (the headline location lies; see below):
```bash
jq -r '.[].SpecReports[] | select(.State=="panicked")
  | "PANIC: \(.Failure.ForwardedPanic)\n\(.Failure.Location.FullStackTrace)"' .ginkgo-report/report.json
```

**Drill into one spec by name, with its captured logs:**
```bash
jq -r '.[].SpecReports[] | select(.LeafNodeText|test("SUBSTRING"))
  | "STATE: \(.State)\n\(.Failure.Message)\n--- GinkgoWriter ---\n\(.CapturedGinkgoWriterOutput // "(none)")"' .ginkgo-report/report.json
```

**Slowest specs:**
```bash
jq '.[0].SpecReports | map([(.ContainerHierarchyTexts+[.LeafNodeText])|join(" "), .RunTime/1e9]) | sort_by(.[1]) | reverse | .[0:10]' .ginkgo-report/report.json
```

## Schema facts you need

- `State`: `passed | failed | panicked | pending | skipped | timedout | interrupted | aborted`.
- Spec name = `ContainerHierarchyTexts` (array) + `LeafNodeText`.
- `Failure.Message` — the failure text (Gomega's full expected/got block for assertions).
- `Failure.Location.{FileName,LineNumber,FullStackTrace}`.
- `Failure.ForwardedPanic` — the panic value (only meaningful for `panicked`).
- `Failure.FailureNodeType` — e.g. `BeforeEach` vs `It`, so you know a *setup* node failed.
- `CapturedGinkgoWriterOutput` — `GinkgoWriter` logs (omitted if empty).
- `SpecEvents[]` — `By(...)` steps and node enter/exit (does **not** include raw GinkgoWriter text).
- `ParallelProcess` — which worker ran the spec (`-p`).

## The three traps that bite agents

1. **`panicked` is not `failed`.** `select(.State=="failed")` silently misses panics. Always include `"panicked"` (and `"timedout"`).
2. **A panic's `Location.FileName` points into the Go runtime**, not your code (e.g. `runtime_faststr.go`). The real line is in `Failure.ForwardedPanic` + `Failure.Location.FullStackTrace`.
3. **`--fail-fast` truncates the report** to whatever ran before the stop. For a full failure picture, *don't* use it — Ginkgo keeps going within a suite by default; use `--keep-going` with `-r` so one failing package doesn't stop the others.

## Reproduce, then narrow

- Every run prints its **random seed**. Reproduce the exact order with `ginkgo --seed=N`.
- Narrow to the failing spec with `--focus="<regex over the full name>"` or `--focus-file=file:line` (→ `ginkgo:filtering`). Avoid committing `FIt`.
- Re-run the one spec with `-v` (or `-vv --trace`) for the full inline **timeline** — `By` steps, `GinkgoWriter` output, and the failure point in execution order. Reserve this verbosity for a specific elusive spec; it's noise across a whole suite.

## When it's a hang, not a failure

A stuck spec emits no failure. Get a snapshot of what's running without stopping the suite: send **`SIGINFO` (`Ctrl+\` is `SIGQUIT`; macOS `Ctrl+T` = `SIGINFO`)** or **`SIGUSR1` on Linux** — Ginkgo prints the current node, goroutine stacks, and the last `GinkgoWriter` lines (a *progress report*). Make stuck specs auto-report with `--poll-progress-after=120s --poll-progress-interval=30s` (or per-node `PollProgressAfter`). Progress reports also appear in the JSON. → `ginkgo:timeouts-and-async` for making specs interrupt cleanly.

## Other formats

`--junit-report` / `--teamcity-report` / `--gojson-report` exist but lose Ginkgo metadata — use them only to feed an external CI system. For diagnosis, the native `--json-report` is richer. → `ginkgo:reporting`.

Versions

The database has no other skills named debugging-failures.

Database snapshot: 13.11.2026 · package version 4e0494017d29 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100reporting
    Generate, consume, and enrich Ginkgo reports — console verbosity (-v/-vv/--trace/--no-color/--succinct), machine-readable reports (--json-report/--junit-report with --output-dir/--keep-separate-reports), programmatic reporting nodes (ReportAfterEach, ReportAfterSuite,…
  • 100running
    Run Ginkgo suites with the ginkgo CLI — run, -r, -p, --dry-run, watch, build (precompiled .test binaries), generate, outline, unfocus, labels, version; spec randomization (--randomize-all/--randomize-suites/--seed); running multiple suites…
  • 87debug-failures
    See why a Go/Ginkgo Biloba spec failed or flaked — the on-failure artifacts (DOM outline + screenshots + poll trajectory of the timed-out read + the visual-regression diagnosis with its .actual.png/.diff.png, the "never settled" warning an update run prints + the detached-node…
  • 100engine-debugging
    Run Dagger repo tests and debug Dagger engine, core, dagql, filesync, cache, CI trace, panic, hang, leak, and performance issues. Use whenever an agent needs to run tests, choose a test command, or interpret test output in this repository, even before a failure is diagnosed;…
  • 100cluster-logging
    Use this skill to analyze failing CI tests for the OpenShift Cluster Logging Operator (ClusterLogForwarder, collector, LogFileMetricExporter), rerun the specific failing tests (Ginkgo/openshift-tests-extension, junit_openshift_logging_e2e_tests_cluster_logging_operator prefix),…
  • 100biloba-testing
    How to write and run Biloba's own Ginkgo test suite. Use when adding or modifying specs in this repo, asserting that a Biloba call should fail the test, working with the fixture server, or running the suite (including the driver, parity, e2e, and npm packaging lanes). Covers the…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.