Skip to content
Skills Library
Programming · Analysis

bisect

/bisect

Bisect a ClickHouse regression using pre-built master binaries from CI. Use when the user wants to find the commit that introduced a bug.

Repository
ClickHouse ↗
Licence
Apache-2.0
Last repository change
25.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
1
Deprecation

No signals detected. Repository changed on 25.10.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Bisect a ClickHouse regression using pre-built master binaries from CI. Use when the user wants to find the commit that introduced a bug.

Arguments

/bisect <repro.sql> [good-ref] [bad-ref]
Original author description

Bisect a ClickHouse regression using pre-built master binaries from CI. Use when the user wants to find the commit that introduced a bug.

How it works

  1. Arguments
  2. How the auto-bisect script works
  3. Test script contract
  4. Process
  5. Notes
Original SKILL.md headings · 153 lines, 753 words
SKILL.md preview
---
name: bisect
description: Bisect a ClickHouse regression using pre-built master binaries from CI. Use when the user wants to find the commit that introduced a bug.
argument-hint: <repro.sql> [good-ref] [bad-ref]
disable-model-invocation: false
allowed-tools: Task, Bash(git:*), Bash(curl:*), Bash(chmod:*), Bash(mkdir:*), Bash(ls:*), Bash(/tmp/*), Bash(cat:*), Bash(utils/auto-bisect/*), Read, Write, Glob, Grep, WebFetch, LSP
---

# ClickHouse Bisect Skill

Bisect a ClickHouse regression using `utils/auto-bisect/bisect.sh`, which downloads pre-built CI binaries and manages the bisect loop automatically.

## Arguments

- `$0` (required): Path to a SQL repro file (e.g., `/tmp/repro.sql`)
- `$1` (optional): Good ref — a git tag, branch, or SHA known to be good (default: latest `v*.1.1-*` tag)
- `$2` (optional): Bad ref — a git tag, branch, or SHA known to be bad (default: `HEAD`)

## How the auto-bisect script works

`utils/auto-bisect/bisect.sh` orchestrates the full bisect:
1. For each commit, `helpers/download.sh` fetches the release binary from CI S3
2. `env/<option>.sh` installs configs and starts a ClickHouse server (configs go to `utils/auto-bisect/data/etc/`, never to `/etc/`)
3. Your test script is called with `$COMMIT_SHA`, `$CH_PATH` (binary), and `$GIT_WORK_TREE` in the environment
4. Exit code from your test determines good/bad/skip

The working tree is **never checked out** — `BISECT_HEAD` is written for each step so git history is untouched.

## Test script contract

The test script receives these environment variables:
- `$COMMIT_SHA` — full SHA of the commit being tested
- `$CH_PATH` — path to the downloaded binary (e.g., `utils/auto-bisect/data/clickhouse`)
- `$GIT_WORK_TREE` — path to the ClickHouse repository

Exit codes:
- `0` — good (bug not present)
- `1` — bad (bug present)
- `125` — skip (handled upstream by download.sh, not typically needed in the test itself)

## Process

### 1. Validate inputs

- Read and understand the repro SQL
- Resolve good/bad refs to full SHAs: `git rev-parse <ref>`
- If no good ref given, find the previous release branch point:
  ```bash
  git tag -l 'v*' --sort=-creatordate | grep '\.1\.1-' | head -5
  ```
  Pick the nearest `v<MAJOR>.<MINOR>.1.1-*` tag before the bad ref.

### 2. Write the test script

Create `/tmp/bisect_test.sh`. Use `$CH_PATH` for the binary — do **not** download it yourself, the framework handles that.

**For `clickhouse local` repros (no server needed — use `--env nothing`):**

```bash
#!/bin/bash
set -e

OUTPUT=$("$CH_PATH" local --multiquery 2>&1 <<'SQL'
-- paste repro SQL here
SQL
)

if echo "$OUTPUT" | grep -q "FAILURE_PATTERN"; then
    exit 1  # bad
else
    exit 0  # good
fi
```

**For server-based repros (use `--env single`, the default):**

```bash
#!/bin/bash
# Server is already running when this script is called.
# Use clickhouse-client on the default port (9000).
set -e

OUTPUT=$("$CH_PATH" client --multiquery 2>&1 <<'SQL'
-- paste repro SQL here
SQL
)

if echo "$OUTPUT" | grep -q "FAILURE_PATTERN"; then
    exit 1
else
    exit 0
fi
```

Failure pattern examples:
- Exception: `grep -q "LOGICAL_ERROR\|Code: [0-9]"`
- Wrong result: compare `OUTPUT` to expected string
- Segfault: `[ "${PIPESTATUS[0]}" = "139" ]`

### 3. Confirm the bug range

Before running the full bisect, verify the repro:
```bash
GOOD_SHA=$(git rev-parse <good-ref>)
BAD_SHA=$(git rev-parse <bad-ref>)
```

Optionally do a quick walker sanity check over just the two endpoints:
```bash
CH_PATH=/usr/local/bin/clickhouse /tmp/bisect_test.sh  # should exit 0 on good
```

### 4. Run bisect

```bash
cd utils/auto-bisect
./bisect.sh \
  --good <good-sha> \
  --bad  <bad-sha> \
  --path /path/to/ClickHouse \
  --test /tmp/bisect_test.sh \
  --env  nothing          # or: single (default), replicateddb, sharedcatalog
```

Use `--env nothing` when the test uses `clickhouse local` (no server).
Use `--env single` (default) when the test needs a running server.

**Walker mode** — walk every commit linearly instead of bisecting (useful when you want to see the progression or the range is small):
```bash
./bisect.sh --good <sha> --bad <sha> --path ... --test ... --walker
# Or provide explicit commits:
./bisect.sh --path ... --test ... --walker --walker-commits "sha1 sha2 sha3"
# Or limit to N evenly-spaced steps:
./bisect.sh --good <sha> --bad <sha> --path ... --test ... --walker --walker-steps 10
```

Set a generous timeout (up to 10 min) — each step downloads ~200 MB.

### 5. Report findings

After bisect completes, the output contains `<sha> is the first bad commit`.

```bash
git show --stat <bad-sha>
```

Report:
- First bad commit SHA and message
- PR number from message (e.g., "Merge pull request #NNNNN") → link to `https://github.com/ClickHouse/ClickHouse/pull/<N>`
- Brief description from `git show --stat`
- Any skipped commits (no CI binary) that fall in the range

## Notes

- Binaries are downloaded to `utils/auto-bisect/data/clickhouse` and overwritten each step (not cached between steps — use walker if you want to re-test)
- Only first-parent merge commits on master have CI binaries; `bisect.sh` uses `--first-parent` automatically
- Configs are installed to `utils/auto-bisect/data/etc/clickhouse-server/` — the host's `/etc/clickhouse-*` is never touched
- The working tree is never modified during bisect or walker runs; `BISECT_HEAD` is used instead of `git checkout`
- For private CI builds, set `CH_CI_USER` / `CH_CI_PASSWORD` and pass `--private`

Versions

RepositoryScoreLicence
This page · ClickHouse/ClickHouse100Apache-2.0
paultyng/skill-issue100MIT

Database snapshot: 13.11.2026 · package version 0e68145d87ba (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90git-bisect
    Find the exact commit that introduced a regression by driving git bisect with an automated pass/fail script. Use when a behavior worked in an older build and you need the first bad commit out of hundreds, not a guess.
  • 100git-bisect-hunting-regression-bugs
    Use this skill when pinpointing the exact commit introducing a software defect using automated binary search with git bisect.
  • 100git-bisect-regression
    Find which commit introduced a bug or behavior change using `git bisect`. Use this whenever the user says something "used to work", "worked last week", "stopped working", "broke at some point", asks "which commit broke this", "when did this start", "find the regression" or "who…
  • 100git-bisect-ai
    AI-guided git bisect for finding the exact commit that introduced a bug or regression. Automates the binary search, interprets test results, and identifies the responsible code change with full context. Use when you know a bug exists now but didn't exist before, or when a…
  • 100bisect-anything
    Find the exact commit (or dependency version, config value, or input) that caused a regression by turning it into an automatic pass/fail check and running git bisect run in a separate worktree, then shrink the bad commit to the hunk that matters and explain it. Handles failing…
  • 100regression-hunt
    Find the commit that introduced a regression by bisecting history in an isolated worktree. Use when a test, build, or behavior used to pass and now fails, when the user asks when something broke, which commit broke it, or asks to bisect. Requires a command that exits 0 on a…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.