codex-ab
/codex-ab
Run an A/B codex review experiment — holistic codex review vs 3 focused dimension passes (security, ecto, liveview) on the branch diff, classify findings, report a panel-value verdict. Use when the branch is fresh, before any codex review runs.
- Author
- oliver-kriska ↗
- Repository
- claude-elixir-phoenix ↗
- Licence
- MIT
- Last repository change
- 22.10.2026
Assessment and security
- Files in this package
- 3
- Scripts
- 1
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 22.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
- 1 scripts in this package. Review them before installing, as Claude may execute them.
Scripts: scripts/codex-panel-ab.sh
What it does and when to use it
Run an A/B codex review experiment — holistic codex review vs 3 focused dimension passes (security, ecto, liveview) on the branch diff, classify findings, report a panel-value verdict. Use when the branch is fresh, before any codex review runs.
Arguments
/codex-ab [base-branch]
Original author description
Run an A/B codex review experiment — holistic codex review vs 3 focused dimension passes (security, ecto, liveview) on the branch diff, classify findings, report a panel-value verdict. Use when the branch is fresh, before any codex review runs.
How it works
- Usage
- Iron Laws
- Workflow
- Integration
- References
SKILL.md preview
---
name: codex-ab
description: Run an A/B codex review experiment — holistic codex review vs 3 focused dimension passes (security, ecto, liveview) on the branch diff, classify findings, report a panel-value verdict. Use when the branch is fresh, before any codex review runs.
effort: medium
argument-hint: "[base-branch]"
---
# Codex Panel A/B (contributor instrument — verdict decided)
Answer one question with evidence: **do dimension-focused codex passes find
real issues that one holistic `codex exec review` misses?** Runs both on the
same diff, then classifies every focused finding against the holistic pass.
> **DECIDED 2026-07-10 after 4 runs (2 fresh): panel KILLED.** Fresh-only
> 1 real miss / 1 false positive plus one zero-value run at 4× cost —
> real misses did not outnumber FPs. Kept as contributor tooling (NOT
> distributed) for one possible retest: a UI-heavy diff with a single
> extra liveview-focused pass (2× cost). Scoreboard:
> `.claude/research/2026-07-03-codex-review-integration.md` §7.
## Usage
```
/codex-ab # A/B against main (~5 min, 4 codex runs)
/codex-ab develop # explicit base branch
```
## Iron Laws
1. **FRESH DIFF ONLY** — ask the user to confirm this branch has NOT been
codex-reviewed yet (cloud or `/phx:codex-loop`). A drained diff returns
NO FINDINGS everywhere and proves nothing — wasted quota
2. **Verify every REAL MISS in the code before counting it** — a focused
finding only scores if the issue actually exists at that file:line
3. **Read ONLY the findings `.md` files** — streams are diverted to `.log`
files; never cat a log into context (10k+ lines each)
4. **Exactly 4 codex runs, never re-run dimensions** — bounded quota
5. **Persist the verdict** — an unrecorded experiment is wasted quota
## Workflow
### Step 1: Preflight
Run `command -v codex` — missing → STOP with install hint. Then:
- `git status --short` dirty → warn (codex flags local dirt as findings)
- Ask: "Has codex already reviewed this branch (PR review or codex-loop)?"
If yes → STOP, explain the fresh-diff requirement (Iron Law 1)
### Step 2: Run the A/B (background, ~5 min)
```bash
bash ${CLAUDE_SKILL_DIR}/scripts/codex-panel-ab.sh {base} \
.claude/reviews/codex-ab-$(date +%Y-%m-%d-%H%M)
```
Use `run_in_background` — it runs 1 holistic `codex exec review` + 3
focused `codex exec` workers (security / ecto / liveview) in parallel,
all streams redirected. Do other work or wait; never poll.
### Step 3: Classify
Read the 4 findings files (`holistic.md`, `security.md`, `ecto.md`,
`liveview.md` — small). For EACH focused finding:
| Class | Meaning | Test |
|-------|---------|------|
| DUPLICATE | Holistic already found it | Same file + same defect |
| REAL MISS | Genuine issue holistic missed | Read the code at file:line — defect confirmed (Iron Law 2) |
| FALSE POSITIVE | Manufactured, pre-existing, or wrong | Code check fails, or issue exists on base branch too |
### Step 4: Verdict
Present:
```markdown
## Codex Panel A/B — {branch} vs {base}
| dimension | findings | duplicate | real miss | false positive |
Holistic-only findings: {n}
Verdict this run: {REAL MISS count} real miss vs {FP count} false positive
Decision rule: build --codex-panel only if real misses outnumber false
positives across 2-3 fresh branches.
```
Write the verdict table to `.claude/reviews/codex-ab-{date}/VERDICT.md`.
Suggest repeating on the next 1–2 fresh branches before deciding.
## Integration
```text
fresh branch → /codex-ab (YOU ARE HERE) → verdict logged
├─ real misses win across runs → build /phx:review --codex-panel
└─ duplicates/FPs win → keep holistic /phx:codex-loop, drop panel idea
└─ OUTCOME 2026-07-10: this branch won — panel dropped
```
## References
- `${CLAUDE_SKILL_DIR}/scripts/codex-panel-ab.sh` — the 4-run harness
- Related: `/phx:codex-loop` (holistic fix loop), `/phx:review --codex`
Files in this package (3)
- SKILL.md 4 KB
- triggers.json 1 KB
- scripts/codex-panel-ab.sh 3 KB
Versions
The database has no other skills named codex-ab.
Database snapshot: 13.11.2026 · package version 46690634c9d6 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100review
Review Elixir/Phoenix changes before committing — bugs, security, Ecto, LiveView, and Oban anti-patterns, via parallel specialist agents. Use when asked to review changes, a component, a diff, or a PR. - 100cross-review
Cross-model review — hands the current branch diff to Codex (OpenAI), a different model than the one that wrote the code, running the `delivery-reviewer` Codex skill against the base branch. Use as step 5 of /delivery:task, or when the user asks to "review this branch with… - 100codex-review
Run Codex to review a plan file and return structured feedback with a verdict. Triggers on: debate loop Codex review rounds. Returns VERDICT:APPROVED or VERDICT:NEEDS_CHANGES plus CODEX_SESSION token. - 100codex-review
Guides codex-review plugin workflows. Use when the user asks to review a plan or implementation with Codex, mentions codex-review, asks about review status or findings, or needs help with plan resolution or inline plans. - 100review
Delegate a read-only code review to a Codex subagent (GPT 6.1 Sol by default) that inspects code and reports findings without editing anything. Use when the user wants Codex — or "GPT-5.5" — to review, look over, audit, or give a second opinion on a diff, branch, pull request,… - 100codex-review
Get an independent second opinion from OpenAI Codex CLI on a plan, diff, spec, or Claude's last response. Use when the user asks to "get a second opinion", "have codex review", "cross-check with codex", or wants adversarial review of Claude's output. Argument is one of…