Skip to content
Skills Library
Data & analytics · Analysis

imbalanced-data

/imbalanced-data

Handle skewed classes with threshold moving, weighting, and PR-based evaluation instead of reflexive resampling. Use when the positive class is rare and accuracy looks deceptively high.

Repository
AI-SKILLS ↗
Licence
MIT
Last repository change
05.09.2026

Assessment and security

90/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 05.09.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Handle skewed classes with threshold moving, weighting, and PR-based evaluation instead of reflexive resampling. Use when the positive class is rare and accuracy looks deceptively high.

Original author description

Handle skewed classes with threshold moving, weighting, and PR-based evaluation instead of reflexive resampling. Use when the positive class is rare and accuracy looks deceptively high.

How it works

  1. Method
  2. Boundaries
Original SKILL.md headings · 56 lines, 386 words
SKILL.md preview
---
name: imbalanced-data
description: Handle skewed classes with threshold moving, weighting, and PR-based evaluation instead of reflexive resampling. Use when the positive class is rare and accuracy looks deceptively high.
---

# Imbalanced data

With 1% positives, "always predict negative" scores 99% accuracy. The
cure is rarely synthetic data; it is evaluating correctly, weighting
costs, and choosing the operating threshold deliberately.

## Method

1. **Fix the evaluation first.** PR curves and PR-AUC, precision and
   recall at the deployment threshold, per the cost table from
   ml-problem-framing; never accuracy, and treat ROC-AUC with
   suspicion at heavy skew (it barely moves when false positives
   grow by thousands against a huge negative pool). Keep the test
   set at the *natural* imbalance whatever you do to train (see
   train-test-discipline).
2. **Move the threshold before touching the data.** A well-calibrated
   model at natural imbalance plus a cost-derived threshold
   (minimize expected cost on validation) solves many "imbalance
   problems" outright. Sweep the threshold, plot cost vs threshold,
   pick the minimum; revisit when costs or base rates change.
3. **Weight before you resample.** Class weights
   (inverse-frequency or cost-ratio) in the loss are equivalent to
   oversampling without duplicating rows, supported natively by most
   learners (`class_weight`, `scale_pos_weight`). This is the
   default second move; it changes nothing about the data pipeline.
4. **Resample only with care, inside the fold.** Random undersampling
   of negatives works at massive scale (and speeds training);
   SMOTE-class synthesis helps sometimes on tabular data and harms
   whenever synthetic points cross the true boundary. All resampling
   fits inside cross-validation folds on train only; resampling
   before splitting leaks duplicated/synthetic neighbors into test
   (see cross-validation).
5. **Recalibrate after any reweighting.** Weights and resampling
   distort predicted probabilities away from true base rates; if
   probabilities feed thresholds or expected values, recalibrate on
   an untouched validation slice (see model-evaluation calibration).
6. **Buy more positives where it is cheapest.** Longer time windows,
   related-event labels, hard-negative mining to sharpen the
   boundary, or targeted labeling of model-uncertain cases (active
   learning); a hundred real positives beat ten thousand synthetic
   ones. At extreme rarity (<0.01%), reconsider the frame: anomaly
   detection or tiered rules-then-model funnels (see
   ml-problem-framing).

## Boundaries

- Imbalance is not the problem when both classes are plentiful in
  absolute terms; a 1:100 ratio with a million positives needs
  nothing special.
- If the minority class is undersampled *in collection* (label bias,
  censoring), no algorithm recovers what was never observed; that is
  a data acquisition problem.
- Base rates drift; a threshold tuned at 1% positives misfires at 3%
  (see drift-monitoring), so monitor the rate itself.

Versions

RepositoryScoreLicence
This page · Amey-Thakur/AI-SKILLS90MIT
param087/agent-ml-skills100MIT

Database snapshot: 13.11.2026 · package version 8f0adaec3cc3 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 85model-evaluation
    Model Evaluation. Evaluate classifiers and regressors honestly - confusion matrix, precision, recall, F1, F-beta, MCC, balanced accuracy, ROC-AUC versus PR-AUC on imbalanced data, accuracy paradox, threshold selection with a cost matrix, calibration (Platt, isotonic, Brier…
  • 100class-imbalance-handling-for-rare-signal-events
    Use when predicting rare events such as halts or flash crashes where one class is under a few percent; cost-sensitive weighting and undersampling with the probability recalibration that undersampling makes necessary.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.