dataset-curation
/dataset-curation
Use when the user wants to analyze dataset bias, create stratified samples, evaluate fairness, or plan dataset collection. Triggers on phrases like "dataset bias", "stratified sample", "class imbalance", "data distribution", "fairness analysis", or "ethical review".
- Author
- fcakyon ↗
- Repository
- phd-skills ↗
- Licence
- MIT
- Last repository change
- 11.10.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 11.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Use when the user wants to analyze dataset bias, create stratified samples, evaluate fairness, or plan dataset collection. Triggers on phrases like "dataset bias", "stratified sample", "class imbalance", "data distribution", "fairness analysis", or "ethical review".
Triggers from the description
Original author description
Use when the user wants to analyze dataset bias, create stratified samples, evaluate fairness, or plan dataset collection. Triggers on phrases like "dataset bias", "stratified sample", "class imbalance", "data distribution", "fairness analysis", or "ethical review".
How it works
- Step 1: Distribution Analysis
- Step 2: Bias Assessment
- Step 3: Stratified Sampling
- Step 4: Quality Assessment
- Step 5: Expansion Recommendations
- Step 6: Ethical Review Checklist
- Output Format
SKILL.md preview
--- name: dataset-curation description: > Use when the user wants to analyze dataset bias, create stratified samples, evaluate fairness, or plan dataset collection. Triggers on phrases like "dataset bias", "stratified sample", "class imbalance", "data distribution", "fairness analysis", or "ethical review". --- # Dataset Curation Methodology You are helping a researcher curate, analyze, or expand a dataset with attention to bias, fairness, and quality. ## Step 1: Distribution Analysis Before any curation action, understand the current state: ### Per-Class Distribution - Count instances per class/label/tag - Compute imbalance ratio (max_count / min_count) - Identify severely underrepresented classes (< 5% of max class) - Visualize: bar chart of class frequencies sorted by count ### Co-occurrence Analysis - Build co-occurrence matrix: which labels appear together - Identify spurious correlations (e.g., "violence" always co-occurs with "male") - Check for label leakage between splits ### Metadata Distribution - Source diversity: how many sources/movies/documents contribute - Temporal distribution: are all time periods represented? - Content diversity: genre, style, domain coverage ## Step 2: Bias Assessment For each identified imbalance or correlation: 1. **Is it real-world reflective?** Some imbalances reflect genuine phenomena 2. **Is it harmful?** Would a model trained on this data make unfair predictions? 3. **Is it fixable?** Can we collect more data, resample, or reweight? ### Fairness Dimensions Check for bias along relevant protected attributes: - Gender representation (if applicable) - Racial/ethnic representation (if applicable) - Age distribution (if applicable) - Geographic/cultural diversity (if applicable) ### Bias Metrics - Demographic parity: equal positive rates across groups - Equalized odds: equal TPR and FPR across groups - Representation ratio: group proportion in data vs population ## Step 3: Stratified Sampling When creating splits (train/val/test): 1. **Primary stratification**: by label/class distribution 2. **Secondary stratification**: by source (prevent source leakage across splits) 3. **Validation**: - Chi-squared test for label distribution similarity across splits - No source overlap between splits - Rare classes have minimum representation in each split Split ratios depend on dataset size: - Large (>50k): 80/10/10 or 90/5/5 - Medium (5k-50k): 70/15/15 or 80/10/10 - Small (<5k): k-fold cross-validation preferred ## Step 4: Quality Assessment For labeled datasets, assess annotation quality: - **Inter-annotator agreement**: Cohen's kappa, Fleiss' kappa, or Krippendorff's alpha - **Label noise estimation**: sample and manually verify N labels - **Edge cases**: identify ambiguous examples that annotators might disagree on - **Consistency checks**: automated rules for label validity ## Step 5: Expansion Recommendations If the dataset needs more data: 1. **Priority classes**: which classes benefit most from more data 2. **Source suggestions**: where to find more data for underrepresented classes 3. **Collection strategy**: active learning, targeted scraping, synthetic augmentation 4. **Cost estimation**: time and resources for each approach ## Step 6: Ethical Review Checklist Before using or publishing any dataset: - [ ] Content sensitivity: does the data contain sensitive material? - [ ] Consent: was data collected with appropriate consent? - [ ] Privacy: are individuals identifiable? Is anonymization needed? - [ ] Licensing: are data sources used within their license terms? - [ ] Potential harms: could the dataset be misused? - [ ] Documentation: is the dataset documented with a datasheet/data card? ## Output Format Produce: 1. **Distribution report**: per-class counts, imbalance ratios, co-occurrence matrix 2. **Bias findings**: identified biases with severity and actionability 3. **Split recommendation**: stratification strategy with validation results 4. **Expansion plan**: prioritized suggestions for addressing gaps 5. **Ethics checklist**: completed checklist with notes per item
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · fcakyon/phd-skills | 100 | MIT |
| wshobson/agents | 100 | MIT |
Database snapshot: 13.11.2026 · package version dcee46fd004c (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100validating-ai-ethics-and-fairness
Validate AI/ML models and datasets for bias, fairness, and ethical concerns. Use when auditing AI systems for ethical compliance, fairness assessment, or bias detection. Trigger with phrases like "evaluate model fairness", "check for bias", or "validate AI ethics". - 90bias-fairness-assessment
Assess an AI model, feature, or dataset for bias and fairness across groups — representational and allocative harms, disparate performance, and skewed refusals — using appropriate fairness metrics, and recommend mitigations. Use when evaluating whether an AI system treats people equitably. - 100validating-ai-ethics-and-fairness
Validate AI/ML models and datasets for bias, fairness, and ethical concerns. Use when auditing AI systems for ethical compliance, fairness assessment, or bias detection. Trigger with phrases like "evaluate model fairness", "check for bias", or "validate AI ethics". - 100dataset-loader-creator
Create dataset loader creator operations. Auto-activating skill for ML Training. Triggers on: dataset loader creator, dataset loader creator Part of the ML Training skill category. Use when working with dataset loader creator functionality. Trigger with phrases like "dataset… - 100golden-dataset
Golden dataset lifecycle patterns for curation, versioning, quality validation, and CI integration. Use when building evaluation datasets, managing dataset versions, validating quality scores, or integrating golden tests into pipelines. - 100grad-ai-ethics
Apply AI ethics frameworks (fairness, accountability, transparency, privacy) to evaluate AI systems for algorithmic bias, explainability gaps, and value alignment failures. Use this skill when the user needs to audit an AI system for ethical risks, design fairness constraints,…