Skip to content
Skills Library
AI & agents · Analysis

regression-alert

/regression-alert

Compare this period's reliability against the prior period using Agent Monitor data — error rate (APIError/total) and tool-failure rate (PreToolUse→PostToolUse gap) — flag any regression where reliability got worse, and optionally wire a persistent alert rule so the dashboard catches the next regression automatically. Use when checking whether reliability degraded.

Licence
MIT
Last repository change
23.10.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
2
Scripts
0
HTML comments
0
URLs in SKILL.md
1
Deprecation

No signals detected. Repository changed on 23.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Compare this period's reliability against the prior period using Agent Monitor data — error rate (APIError/total) and tool-failure rate (PreToolUse→PostToolUse gap) — flag any regression where reliability got worse, and optionally wire a persistent alert rule so the dashboard catches the next regression automatically. Use when checking whether reliability degraded.

Original author description

Compare this period's reliability against the prior period using Agent Monitor data — error rate (APIError/total) and tool-failure rate (PreToolUse→PostToolUse gap) — flag any regression where reliability got worse, and optionally wire a persistent alert rule so the dashboard catches the next regression automatically. Use when checking whether reliability degraded.

How it works

  1. Input
  2. Data Sources
  3. Report Sections
  4. Output
Original SKILL.md headings · 56 lines, 511 words
SKILL.md preview
---
name: regression-alert
description: >
  Compare this period's reliability against the prior period using Agent Monitor
  data — error rate (APIError/total) and tool-failure rate (PreToolUse→PostToolUse
  gap) — flag any regression where reliability got worse, and optionally wire a
  persistent alert rule so the dashboard catches the next regression
  automatically. Use when checking whether reliability degraded.
---

# Regression Alert

Detect whether Claude Code reliability is getting worse period-over-period, and
optionally arm an alert so it never has to be checked by hand again. Scope is
reliability/failures only — for cache/cost/compaction drift, use ccam-insights'
`regression-watch` instead.

## Input

The user provides: **$ARGUMENTS**

This may be:
- empty or "all" — check error rate and tool-failure rate (default)
- "errors" — APIError-rate regression only
- "tools" — tool-failure-rate regression only
- a window like "7 vs 7" or "30 vs 30" — recent vs baseline window sizes (default: last 7 days vs the prior 7)
- "arm" — after reporting, also create an alert rule via `POST /api/alerts/rules` (only on explicit request)

## Data Sources

| Endpoint | Returns |
|----------|---------|
| `GET /api/analytics` | `daily_events` (365d), `daily_sessions` (365d), `event_types` — split into recent vs baseline windows to compute per-window failure rates |
| `GET /api/events?session_id=X` | Per-session stream — localize a regression to the sessions driving it |
| `GET /api/alerts/rules` | Existing alert rules — check whether a matching reliability rule already exists before arming a new one |
| `POST /api/alerts/rules` | Create a new alert rule (only when the user says "arm") |

## Report Sections

### 1. Windowing
Split history into a **recent window** (newer) and a **baseline window** (the equal-length period just before it). Default: recent = last 7 days, baseline = the prior 7. Use `daily_events`/`daily_sessions` to bucket counts by day.

### 2. Error-Rate Regression
- Per window: `error rate = APIError count / total events`.
- Compare recent vs baseline. Flag if recent is higher. Report absolute change (pp) and relative change (%), plus the recent sessions contributing the most `APIError` events.

### 3. Tool-Failure-Rate Regression
- Per window: `tool-failure rate = (PreToolUse − PostToolUse) / PreToolUse`.
- Compare recent vs baseline. Flag a rising rate as a reliability regression. Name the tools whose gap grew most.

### 4. Verdict
Roll up which rates regressed, rank by relative worsening, and name the most likely driver.

### 5. Optional — Arm an Alert
**Only if the user passed "arm".** First `GET /api/alerts/rules` to avoid duplicates. Then `POST /api/alerts/rules` with a rule that fires when the regressed metric crosses a threshold near the recent value (e.g., error rate > recent rate). Echo the created rule back; do not create webhooks or fire alerts.

## Output

- A Markdown table: metric | baseline | recent | Δ (pp) | Δ (%) | direction (▲ worse / ▼ better) | verdict.
- Tag each metric 🔴 (clear regression), 🟡 (within noise), or 🟢 (improved).
- Rates as percentages to 2 decimals; any currency in USD to 4 decimals.
- List the specific session IDs that contributed most to any regression.
- End with the single highest-priority regression and a concrete next step (and, if armed, the new rule's id/threshold).
- Read-only **except** the explicit "arm" path, which is the only write. Never mutate alert rules otherwise. If `curl` cannot reach `http://localhost:4820`, tell the user to start the dashboard with `npm start` from the repo root.
Files in this package (2)
  • SKILL.md 4 KB
  • agents/openai.yaml 1 KB

Versions

The database has no other skills named regression-alert.

Database snapshot: 13.11.2026 · package version 78c798b1408d (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90slo-check
    Define and check simple service-level objectives for Claude Code from Agent Monitor data — session completion rate, tool success rate (PostToolUse/PreToolUse), and error rate (APIError/total) — then compare each to its target and report the error budget remaining. Use when…
  • 90error-scan
    Scan recent Claude Code activity for errors and failure signals across all sessions using Agent Monitor data — APIError events and PreToolUse→PostToolUse gaps (tools that started but never completed) — then group failures by tool and model and rank them by frequency. Use when…
  • 100eval-regression-detector
    Use when implementing or triggering the component that compares current benchmark run scores to the previous run and raises alerts when quality drops below threshold. Defines the regression rules, alert thresholds, escalation actions, and investigation flow for quality…
  • 95monitor-reliability
    Delivers a reliability health check using auto-captured network request, JS error, and error click data. Use when the user asks for a "reliability check", "error rate", "quality metrics", "page health", "did the release break anything", "error budget", or wants a proactive product quality report.
  • 100regression-finder
    Regression check for coding agents: shows how the agent behaved before and after each harness update, model switch, or week in the user's own Claude Code or Codex history, and finds the point where it changed. Use when the user says the agent's quality dropped or it got worse,…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.