Skip to content
Skills Library
Programming · Testing

debug-failures

/debug-failures

See why a Go/Ginkgo Biloba spec failed or flaked — the on-failure artifacts (DOM outline + screenshots + poll trajectory of the timed-out read + the visual-regression diagnosis with its .actual.png/.diff.png, the "never settled" warning an update run prints + the detached-node "matched then stopped matching" signal + the occluded-click diagnosis naming what covered the target), how Biloba auto-adapts to humans vs CI vs AI agents, the env vars and config knobs that surface them (BILOBA_SCREENSHOTS_DIR, BILOBA_SCREENSHOT_BASELINES_DIR, BILOBA_UPDATE_SCREENSHOTS, BILOBA_INLINE_SCREENSHOTS, BILOBA

Author
onsi ↗
Repository
biloba ↗
Licence
MIT
Last repository change
27.10.2026

Assessment and security

87/ 100Class: good
Format20 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability17 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
3
Deprecation

No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

See why a Go/Ginkgo Biloba spec failed or flaked — the on-failure artifacts (DOM outline + screenshots + poll trajectory of the timed-out read + the visual-regression diagnosis with its .actual.png/.diff.png, the "never settled" warning an update run prints + the detached-node "matched then stopped matching" signal + the occluded-click diagnosis naming what covered the target), how Biloba auto-adapts to humans vs CI vs AI agents, the env vars and config knobs that surface them (BILOBA_SCREENSHOTS_DIR, BILOBA_SCREENSHOT_BASELINES_DIR, BILOBA_UPDATE_SCREENSHOTS, BILOBA_INLINE_SCREENSHOTS, BILOBA

Triggers from the description

never settledmatched then stopped matching
Original author description

See why a Go/Ginkgo Biloba spec failed or flaked — the on-failure artifacts (DOM outline + screenshots + poll trajectory of the timed-out read + the visual-regression diagnosis with its .actual.png/.diff.png, the "never settled" warning an update run prints + the detached-node "matched then stopped matching" signal + the occluded-click diagnosis naming what covered the target), how Biloba auto-adapts to humans vs CI vs AI agents, the env vars and config knobs that surface them (BILOBA_SCREENSHOTS_DIR, BILOBA_SCREENSHOT_BASELINES_DIR, BILOBA_UPDATE_SCREENSHOTS, BILOBA_INLINE_SCREENSHOTS, BILOBA

How it works

  1. Zero config: what you already get
  2. Find the failure message first: it is at the bottom of each block
  3. Read the artifacts in this order
  4. Look at the page yourself, any time
  5. Two headless quirks that look like Biloba bugs
  6. Env vars
  7. Config knobs (ConnectToChrome)
  8. Watch it live
  9. A hang, not a failure
Original SKILL.md headings · 214 lines, 2,660 words
SKILL.md preview
---
name: debug-failures
description: See why a Go/Ginkgo Biloba spec failed or flaked — the on-failure artifacts (DOM outline + screenshots + poll trajectory of the timed-out read + the visual-regression diagnosis with its .actual.png/.diff.png, the "never settled" warning an update run prints + the detached-node "matched then stopped matching" signal + the occluded-click diagnosis naming what covered the target), how Biloba auto-adapts to humans vs CI vs AI agents, the env vars and config knobs that surface them (BILOBA_SCREENSHOTS_DIR, BILOBA_SCREENSHOT_BASELINES_DIR, BILOBA_UPDATE_SCREENSHOTS, BILOBA_INLINE_SCREENSHOTS, BILOBA_OUTLINE_MAX, BILOBA_INTERACTIVE, BilobaConfig*), attaching app/store state to a failure, headless quirks (stale innerText, unscheduled requestAnimationFrame), and using b.Outline()/b.A11yOutline() to understand why a selector did not match. Use when a browser spec is failing or flaky and you need visibility, or to configure failure output for CI/agents. For *preventing* flakes (single-shot reads, avoiding b.Immediate(), optimistic-UI) see this plugin's flaky-specs skill.
---

# Debugging Biloba failures

Reading artifacts after a spec failed. To *prevent* flakes → `flaky-specs`. Docs: <https://onsi.github.io/biloba/#failure-artifacts-humans-ci-and-agents>. Sibling skills are named here without a prefix; invoke one with the same plugin prefix you loaded this skill under.

## Zero config: what you already get

Biloba detects the environment in `ConnectToChrome`. "Automation" = `CI` is set **or** an AI coding agent is detected (`CLAUDECODE`/`AI_AGENT`/Cursor/Gemini CLI/Codex/…).

| | Interactive human | CI **or** AI agent |
|---|---|---|
| Failure screenshot | inline in the terminal | written to `./biloba-screenshots` |
| DOM outline on failure | no | yes |
| Inline image blob | yes (if the terminal supports it) | no |

So an agent or CI run needs nothing: `ginkgo -r -p`, then read the outline and the screenshot files. `BILOBA_SCREENSHOTS_DIR=./artifacts` points the directory elsewhere.

## Find the failure message first: it is at the bottom of each block

Ginkgo prints a failed spec's **Timeline** first and its full failure message **last**. Biloba attaches its artifacts to the timeline after the spec fails, so the outline (often 100+ lines), the poll trajectory and the screenshot paths all come *before* the message:

```
• [FAILED] [3.1 seconds]
Projects editing a project [It] …
  Timeline >>
  [FAILED] in [It] - projects_test.go:42 @ …     ← a marker, not the message
  [DOM Outline for: '…'] …                        ← Biloba's artifacts
  << Timeline

  [FAILED] Timed out after 1.000s.                ← the message
  Expected …
  In [It] at: projects_test.go:42
```

Read each block from the bottom. To pull out only the messages from a long run:

```bash
ginkgo --no-color -r -p 2>&1 | grep -A12 '^  \[FAILED\] [^i]'   # skips the "[FAILED] in [It]" markers
ginkgo -r -p --json-report=report.json; jq -r '.[].SpecReports[] | select(.State=="failed") | "\(.LeafNodeText)\n\(.Failure.Message)\n"' report.json
```

Then go to the artifacts for that spec, in the order below.

## Read the artifacts in this order

1. **Console errors** — any `console.error`/`console.assert` and any uncaught exception or unhandled rejection (`Uncaught …`) before the failure, replayed under "Console errors logged before this failure" at the **top** of the failure block. On a JS crash (a React error boundary) this is the root cause. Everything here (and in the streamed timeline, `AllPageErrors`, `AllConsoleMessages`, `AllRequests`) comes from this spec: what the previous spec's page raised as `Prepare()` navigated it away is dropped. Locations (`app.js:12:5`) are one-based, so they open at the right line in an editor. Chrome's own warnings and errors (`[network] Failed to load resource: … (url)`, `[intervention] …`) appear in the streamed timeline but not in this replay; read the timeline for a failed load, or `b.AllConsoleMessages()` / `b.AllRequests()` for the full record.
2. **`⚠` diagnostic notes** — they name the cause outright. "Page clock paused" (or the same line at the end of a poll's timeout: `This tab's page clock is paused at <t>; …`) means the page's timers were stopped by the spec's own `b.InstallClock`/`PauseAt` — advance the clock before asserting (`flaky-specs` §9).
3. **Poll trajectory** — what the read you failed on did over its whole deadline (absent when the failure had no value read behind it).
4. **Visual diagnosis** — only on a failed `b.HaveScreenshot`: pixel counts plus the *shape* of the change, in words, before you open any image.
5. **Screenshot** — `Read` the printed PNG path.
6. **DOM outline** — "DOM Outline for: '<title>'": indented DOM, `<script>/<style>/<svg>` bodies pruned, whitespace collapsed, capped at 32 KB. Past `... [truncated]`? Raise the cap with **`BILOBA_OUTLINE_MAX`** — a byte count (`=131072`), or `0`/`off` for the whole DOM.

### Poll trajectory

When an `Eventually` over a read that observes and compares a value (`b.HaveInnerText`/`b.HaveCount`/any value matcher, a geometry matcher, `b.EvaluateTo`, `b.GetJSValue`) times out, Biloba attaches the `(elapsed, value)` series **of that assertion**. Gomega's `Timed out … Expected <120>` shows only the final value; the shape is the diagnosis:

| Shape | Means | Do |
|---|---|---|
| **flat** (one row, `held ×N`) | the product computed the value once and never reconciled | fix the product (`flaky-specs` §4) — a wider timeout won't help |
| **monotone staircase** | latency; it nearly made it | widen the timeout |
| **dip-then-rebound** | a late reflow shoved it back | settle layout before asserting |

**No entry is a normal outcome, not a bug.** The series is claimed by the matcher Gomega asked for a failure message, so an entry always describes the read you failed on. Reads that passed, `b.Run` setup lines, and failures with no value read underneath (a `b.Click` whose selector never matched, a getter whose value was never there — that one gets the `AllowMissing` enrichment instead) simply produce nothing. Don't read a missing trajectory as a signal; go to the outline and the screenshot. To get a trajectory for an arbitrary expression, poll it with `b.EvaluateTo`/`b.GetJSValue` rather than wrapping `b.Run` in your own `Eventually` — Biloba only records reads it owns.

On by default; `BilobaConfigPollTrajectory(false)` disables it (and the detached-node signal). **Don't disable it for speed.** Recording takes a lock and renders the value per poll sample, so it isn't free — but measured on a 1,558-spec suite with 2,470 call sites across the instrumented matchers (including gates asserting on whole paragraphs of prose), turning it off saved under a second across the whole run, inside that suite's own run-to-run spread.

### Visual diagnosis (a failed `b.HaveScreenshot`) → `visual-assertions`

Two extra PNGs land in the screenshots dir — `<name>.actual.png` (what Biloba saw) and `<name>.diff.png` (the actual, washed out, differing pixels in magenta) — and the failure message says what moved:

```
screenshot "home-desktop" differs from baseline
  38,160 of 1,017,600 pixels differ (3.75%), max channel delta 221
  changed region: one box, (0,14)-(1272,44)  [100% of the width, 4% of the height, at its top edge]
  unchanged: everything below y=44
  baseline: /Users/you/app/biloba-baselines/home-desktop.png
  actual:   /Users/you/app/biloba-screenshots/home-desktop.actual.png
  diff:     /Users/you/app/biloba-screenshots/home-desktop.diff.png
```

The shape line (`one box`, `changed regions: N boxes`, `scattered`, `uniform shift`, a size change) and what to do about each → `visual-assertions`. When every differing pixel is off by only a few levels Biloba says `a rasterisation or compositing difference, not a content change` — nothing moved, so look for a shadow or gradient compositing into the capture. A **missing** baseline is a different failure: it says to re-run with `BILOBA_UPDATE_SCREENSHOTS=1`; never script your way past it. `Read` the `.diff.png` when the words aren't enough (inline images are off under an agent, so you get the path).

**"never settled" is printed during an update run, not on a failure.** The run stays green, but the baseline it just wrote was captured from a page still changing and the next normal run will fail against it. Mask the moving region (or stop it moving) and re-run the update; re-running alone changes nothing. → `visual-assertions`

**"Failed to restore the tab's prefers-color-scheme"** — a dropped `b.InColorSchemes` teardown. The override is target-level and survives navigation, so `b.Prepare()` clears the leak before the next spec; the spec that printed the warning, though, finished rendering in the emulated scheme. Read any odd-looking screenshot from that spec with that in mind.

### The `⚠` diagnostic notes

**"Selector matched, then stopped matching"** — the detached node. The selector resolved, then its node was replaced (list re-key, portal migration) or its identifying attribute swapped in place. Silent when the selector genuinely never matched.

```
⚠ Selector "#row-4" matched 6× during this poll (+0.00s to +0.41s) then stopped matching
  — the node was likely replaced, or its identifying attribute changed in place.
```

**"Click dispatched onto a covered element"** — fast `Click` stays occlusion-blind by design and still succeeds through an overlay, but records a hit-test so the downstream failure points somewhere. Diagnostic only; never changes whether a spec passes.

```
⚠ Click on "#submit" was dispatched while <div#overlay.modal-scrim> was the topmost
  element at its centre — the click may have been swallowed.
  Consider Eventually("#submit").Should(b.BeClickable()) or b.Realistic().Click("#submit").
```

**"Network handler never ran (shadowed by an earlier handler)"** — handlers are first-match-wins, so one registered for a URL an earlier handler claims is dead code. Deadly across an `Ordered` container, where `Prepare()` doesn't run between `It`s. Both call sites are named.

```
⚠ A ModifyResponse handler registered at network_test.go:231 never ran — an earlier ModifyResponse
  handler (registered at network_test.go:223) claimed 1 matching response(s) first.
```

Reported only when a handler **never fired** *and* was shadowed at least once. **Limit:** it's a failure artifact, so it cannot surface shadowing's other presentation — a leftover *stateful* handler claiming the response, passing it through untouched, spec **green**. Only your own `Eventually(hold.Count).Should(Equal(1))` catches that (`flaky-specs` §6).

**"Network handler could not read a response body"** — Chrome refuses to hand over some response bodies (a redirect's, notably), so a `ModifyResponse`/`HoldResponse` handler that matched never gets to run `Using`, apply a static override, or hold the response; Biloba continues it unmodified. Any other read failure (a timeout, a lost reply, a body over the limit) fails the request instead, since the body may already be gone. Either way the handler still counts the dispatch (`Count()` ticks at selection, before the read), and the note names the handler, its registration site, the URL, and which of the two Biloba did.

```
⚠ A ModifyResponse handler registered at checkout_test.go:42 could not read the response body for http://localhost:8080/login (status 302):
  Can only get response body on requests captured after headers received. (-32000)
  Biloba continued the response unmodified.
```

The same note reports a handler whose resolved response Chrome refused (an out-of-range status, say) as `could not deliver its response`; Biloba fails that request.

### When the failure is Chrome itself

Every command Biloba sends Chrome runs under a deadline, so an unresponsive browser produces a failing spec instead of a hung suite. Three shapes, each naming its cause on the first line:

| First line | Means | Do |
|---|---|---|
| `deadline_exceeded: Chrome did not <command> within 30s` | Chrome accepted the command and never answered | Chrome is wedged or the box is badly overloaded — look for a long synchronous script in the page, or too many parallel processes for the machine |
| `page_crashed: this tab's renderer crashed` | Chrome reported the crash (promptly on macOS, several seconds later on Linux — a command in between reports `deadline_exceeded` instead) | usually the page itself (OOM, a bad WASM/canvas path). Navigating clears the crash and gets a fresh renderer on macOS; on Linux the tab has stayed dead in our testing, so continue on `b.NewTab()` rather than trusting the crashed one |
| `frame_detached: the document this frame handle was found in is gone` | the frame handle's iframe was removed or navigated, or its tab navigated or `Prepare()`d, since `b.Frame` found it | a handle belongs to one document: call `b.Frame(...)` again after whatever replaces the frame, and don't carry a handle across that change |
| `browser_gone: the connection to Chrome is closed` | the browser process exited — crashed, OOM-killed, or reaped | not recoverable; the rest of the suite will fail too. Check the machine's memory and whether anything is killing Chrome |

The deadline is generous on purpose (a healthy command answers in milliseconds), so hitting it is a real signal, not a tight-timeout artifact. `WithTimeout` doesn't move it: that knob bounds how long Biloba keeps *retrying*, which is a different question from whether Chrome is alive.

**A suite that ends on Ginkgo's `--timeout` with no failing spec** is a command blocked inside a poll's callback, where Gomega can't interrupt it. Biloba's own commands can't do that, so look for a `chromedp` call of your own on `b.Context` without a deadline.

### Two failure *messages* that self-explain

- A two-axis getter timing out because the **property** never became defined says the element was present, names the property, and prints the `b.AllowMissing("disabled")` to paste (`flaky-specs` §5).
- A failed `BePrecededBy`/`BeFollowedBy` reports the order actually observed (`Actually: #o-first comes BEFORE #o-second.`) — enough to spot an inverted assertion.

## Look at the page yourself, any time

```go
fmt.Println(b.Outline())     // indented DOM
fmt.Println(b.A11yOutline()) // accessibility tree: role + accessible name per node
AddReportEntry("DOM before click", b.Outline(), ReportEntryVisibilityFailureOrVerbose)
b.Run("document.querySelectorAll('.card').length")   // quick count probe
```

`b.A11yOutline()` is **not** auto-attached — call it explicitly. It's often more useful than raw HTML for reasoning about what a page *means*.

**Attach app/store state to every failure.** For a state-heavy or optimistic-UI app the store beats the DOM (which may be the pre-confirmation copy):

```go
ReportAfterEach(func(report SpecReport) {
    if !report.Failed() { return }
    AddReportEntry("app state", b.Run(`JSON.stringify(window.__APP_STATE__ ?? null)`))
})
```

Keep the `?? null` so a crashed page doesn't turn the snapshot itself into a failure. (That's a *snapshot*. To **wait** on app state as part of a spec, use `b.GetJSValue` — `flaky-specs` §3.)

**Page `console.*` streams to the `GinkgoWriter`**, each argument rendered space-separated, and so do uncaught page errors (in red, with their stack). `b.AllConsoleMessages()`/`b.AllPageErrors()` hand you the same as data. Objects come from CDP's **shallow** preview, so nested/large objects log lossily. Build the string yourself when you need the whole value: `console.log('state ' + JSON.stringify(obj))`.

## Two headless quirks that look like Biloba bugs

- **`HaveInnerText`/`GetInnerText` timing out on text that's plainly in the outline.** `innerText` is computed from layout and can return a stale/partial value before a paint settles. Switch to `HaveTextContent`/`GetTextContent` (reads `textContent` off the tree) or to a plain existence assertion.
- **`requestAnimationFrame` never firing.** On a **fully static page**, `chrome-headless-shell` can leave rAF unscheduled after the first scroll (nothing animating ⇒ no frames), wedging app code driven off an rAF loop. Shape: a hang/timeout only in the default headless lane. Confirm with a counter read through a **coalescing `b.Run`** — `b.Run("window.__rafTicks ?? 0")`, **not** `b.GetJSValue`, which would sit waiting for a counter that never appears. Fix: drive the work off a real event, or run that spec under `HighFidelityHeadless()`/`BILOBA_INTERACTIVE=true`.

## Env vars

| Var | Effect |
|---|---|
| `BILOBA_SCREENSHOTS_DIR=./artifacts` | where failure screenshots — and the visual `.actual.png`/`.diff.png` artifacts — are written |
| `BILOBA_SCREENSHOT_BASELINES_DIR=./baselines` | where `b.HaveScreenshot`'s **committed** baselines live (default `./biloba-baselines`) |
| `BILOBA_UPDATE_SCREENSHOTS=1` | capture and (re)write every visual baseline the run touches instead of comparing; prints what changed. Accepts `1/t/true/y/yes/on` (off: `0/f/false/n/no/off`), case-insensitive; **any other value warns and is treated as off**. Suite-wide — scope it with `--focus`. **Never set it in CI**: every visual assertion then passes unconditionally |
| `BILOBA_OUTLINE_MAX=131072` | raise the outline byte cap; `0`/`off` = no truncation |
| `BILOBA_INLINE_SCREENSHOTS=iterm\|kitty\|sixel\|none` | force an inline-image protocol, or `none` to disable the blob (the file path is still printed — use `none` in CI and in Claude Code, where base64 is noise) |
| `BILOBA_PROBE_TERMINAL=true` | actively query the TTY for Sixel support when env detection finds nothing |
| `BILOBA_INTERACTIVE=true` | headful high-fidelity run that pauses on failure until `^C` |

Inline images are auto-detected (Kitty, iTerm2, Sixel/VS Code) and only emitted when the terminal supports them.

## Config knobs (`ConnectToChrome`)

Each boolean takes an optional bool (no arg = `true`). **Explicit settings win, per knob** — automation only fills knobs you left untouched.

- `BilobaConfigScreenshotsToDir(dir)` — write each tab's failure screenshot there (prints the absolute path).
- `BilobaConfigFailureOutlines(...bool)` / `BilobaConfigInlineScreenshots(...bool)` — force on/off.
- `BilobaConfigFailureScreenshots(...bool)` (default on) / `BilobaConfigPollTrajectory(...bool)` (default on) / `BilobaConfigProgressReportScreenshots(...bool)` (default on).
- `BilobaConfigFailureScreenshotsSize(w,h)` / `BilobaConfigProgressReportScreenshotSize(w,h)`.
- `BilobaConfigDebugLogging(...bool)` — stream all CDP traffic to the `GinkgoWriter` (verbose).
- `BilobaConfigScreenshotBaselinesDir(dir)` — where `b.HaveScreenshot` reads/writes committed baselines (default `./biloba-baselines`; commit that dir, gitignore the screenshots dir).
- `BilobaConfigScreenshotTolerance(fraction)` / `BilobaConfigScreenshotChannelTolerance(delta)` — suite-wide visual-comparison defaults, both `0` (exact) by default; `b.Tolerance(...)`/`b.ChannelTolerance(...)` override per assertion. → `visual-assertions`

```go
// CI that only redirects the directory still keeps the automation default of outlines-on:
b = biloba.ConnectToChrome(GinkgoT(), biloba.BilobaConfigScreenshotsToDir("./artifacts"))
```

## Watch it live

```bash
BILOBA_INTERACTIVE=true ginkgo --focus="..."
```

Headful, high fidelity, prints the failure and waits for `^C`. Use a small handful of focused specs, in serial. (`SpinUpChrome(GinkgoT(), biloba.ChromeFlags(chromedp.Flag("headless", false)))` does the same in code.)

## A hang, not a failure

Biloba screenshots Ginkgo [progress reports](https://onsi.github.io/ginkgo/#getting-visibility-into-long-running-specs) — on a spec timeout, a `PollProgressAfter` spec, or on demand: `^T` (SIGINFO) on macOS, `SIGUSR2` on Linux.

Versions

RepositoryScoreLicence
This page · onsi/biloba87MIT
onsi/biloba100MIT

Database snapshot: 13.11.2026 · package version a690f5219104 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100biloba-testing
    How to write and run Biloba's own Ginkgo test suite. Use when adding or modifying specs in this repo, asserting that a Biloba call should fail the test, working with the fixture server, or running the suite (including the driver, parity, e2e, and npm packaging lanes). Covers the…
  • 100explore-unfamiliar-page
    Orient to an unfamiliar page, then draft a starter Go/Ginkgo Biloba spec from its DOM outline, accessibility tree, and screenshot. Use when writing browser tests against a URL or fixture you have not seen. Covers the orient-author-cleanup loop and accepts a URL or fixture…
  • 90overview
    The Ginkgo mental model for writing Go tests — the one idea that explains everything (Ginkgo builds a spec tree at construction time, then runs it) and its consequences for how you write specs, plus spec independence and the node taxonomy. Use this first when you start working…
  • 100running
    Run Ginkgo suites with the ginkgo CLI — run, -r, -p, --dry-run, watch, build (precompiled .test binaries), generate, outline, unfocus, labels, version; spec randomization (--randomize-all/--randomize-suites/--seed); running multiple suites…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.