Skip to content
Skills Library
Data & analytics · Analysis

data-analysis

/data-analysis

Analyze, explore, clean, and visualize datasets with statistical rigor. Use when user asks to analyze data, find patterns, compute statistics, create visualizations, clean messy data, or explore a dataset. Trigger when user says things like "analyze this data", "what trends do you see", "find patterns in", "create a chart", "clean this dataset", "run statistics on", "what does this data tell us", or provides CSV/Excel/JSON data for exploration. Also trigger for A/B test analysis, cohort analysis, and data quality assessments. Do NOT trigger for simple data format conversions, database query wr

Repository
upsonic ↗
Licence
MIT
Last repository change
18.06.2026

Assessment and security

98/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene13 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
3
Scripts
1
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 18.06.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What to check before installing
  • 1 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: scripts/profile_data.py

What it does and when to use it

Analyze, explore, clean, and visualize datasets with statistical rigor. Use when user asks to analyze data, find patterns, compute statistics, create visualizations, clean messy data, or explore a dataset. Trigger when user says things like "analyze this data", "what trends do you see", "find patterns in", "create a chart", "clean this dataset", "run statistics on", "what does this data tell us", or provides CSV/Excel/JSON data for exploration. Also trigger for A/B test analysis, cohort analysis, and data quality assessments. Do NOT trigger for simple data format conversions, database query wr

Triggers from the description

analyze this datawhat trends do you seefind patterns increate a chartclean this datasetrun statistics onwhat does this data tell us
Original author description

Analyze, explore, clean, and visualize datasets with statistical rigor. Use when user asks to analyze data, find patterns, compute statistics, create visualizations, clean messy data, or explore a dataset. Trigger when user says things like "analyze this data", "what trends do you see", "find patterns in", "create a chart", "clean this dataset", "run statistics on", "what does this data tell us", or provides CSV/Excel/JSON data for exploration. Also trigger for A/B test analysis, cohort analysis, and data quality assessments. Do NOT trigger for simple data format conversions, database query wr

How it works

  1. Before You Analyze
  2. Reference Materials and Scripts
  3. Analysis Workflow
  4. Common Pitfalls
  5. Handling Common Requests
Original SKILL.md headings · 287 lines, 1,516 words
SKILL.md preview
---
name: data-analysis
description: Analyze, explore, clean, and visualize datasets with statistical rigor. Use when user asks to analyze data, find patterns, compute statistics, create visualizations, clean messy data, or explore a dataset. Trigger when user says things like "analyze this data", "what trends do you see", "find patterns in", "create a chart", "clean this dataset", "run statistics on", "what does this data tell us", or provides CSV/Excel/JSON data for exploration. Also trigger for A/B test analysis, cohort analysis, and data quality assessments. Do NOT trigger for simple data format conversions, database query writing without analysis, or ETL pipeline design.
metadata:
  version: "2.0.0"
  author: Upsonic
  tags: [data, analysis, statistics, visualization]
---

# Data Analysis

Explore, clean, analyze, and communicate findings from data. The goal is always to answer a question — start with what the user wants to know and work backward to the analysis that answers it.

## Before You Analyze

### Understand the Question

Before touching the data, clarify:

1. **What question are we answering?** ("Is our conversion rate improving?" is an answerable question. "Analyze this data" is not — help the user sharpen it.)
2. **Who needs the answer?** (Engineer debugging an issue? Executive making a budget decision? Researcher testing a hypothesis?)
3. **What decisions will this inform?** (This determines how precise you need to be and what format the answer should take.)
4. **What's the timeline?** (A quick sanity check and a thorough statistical analysis require different approaches.)

If the user says "analyze this data" without a specific question, help them formulate one:
- "What would be most useful to know from this data?"
- "Are you looking for trends over time, comparisons between groups, or something else?"
- "Is there a specific business question this should answer?"

## Reference Materials and Scripts

- Execute `profile_data.py` with a data file path to get a quick profile of any CSV, Excel, or JSON dataset — it reports shape, types, missing values, stats, and value distributions. Run with `--help` for usage.
- Load `statistical-tests-guide.md` when choosing statistical tests — it has a decision matrix for test selection, effect size interpretation tables, and sample size guidelines.

### Understand the Data

Before analysis, get your bearings:

1. **Source and context**: Where did this data come from? How was it collected? What time period does it cover?
2. **Schema**: What are the columns/fields? What do they represent? What are the data types?
3. **Scale**: How many rows/records? What's the granularity? (Per user? Per day? Per transaction?)
4. **Known issues**: Is the data known to be incomplete, biased, or have quality problems?

```python
# First look at any dataset
import pandas as pd

df = pd.read_csv("data.csv")  # or read_excel, read_json, etc.
print(f"Shape: {df.shape}")
print(f"\nColumn types:\n{df.dtypes}")
print(f"\nFirst rows:\n{df.head()}")
print(f"\nMissing values:\n{df.isnull().sum()}")
print(f"\nBasic stats:\n{df.describe()}")
```

## Analysis Workflow

### Step 1: Clean and Validate

Data quality determines analysis quality. Don't skip this.

#### Handle Missing Values
- **Count them first**: What percentage of each column is missing?
- **Understand why**: Are they random? Systematic? (e.g., optional fields vs data collection failures)
- **Choose a strategy and document it**:
  - Drop rows: When missing data is rare and random (less than 5%)
  - Impute with median/mode: When missing data is moderate and the distribution is known
  - Flag as separate category: When missingness itself is informative
  - Leave as-is: When the analysis method handles nulls natively

```python
# Document your decisions
missing_pct = df.isnull().sum() / len(df) * 100
print("Missing data percentage per column:")
print(missing_pct[missing_pct > 0].sort_values(ascending=False))
```

#### Handle Outliers
- **Detect**: Use IQR method, z-scores, or domain knowledge
- **Investigate**: Are they errors or legitimate extreme values?
- **Document your decision**: Keep, cap, or remove — and explain why

```python
# IQR method for outlier detection
Q1 = df['value'].quantile(0.25)
Q3 = df['value'].quantile(0.75)
IQR = Q3 - Q1
outliers = df[(df['value'] < Q1 - 1.5 * IQR) | (df['value'] > Q3 + 1.5 * IQR)]
print(f"Found {len(outliers)} outliers ({len(outliers)/len(df)*100:.1f}%)")
```

#### Validate Data Types and Ranges
- Dates should be dates, numbers should be numbers
- Check for impossible values (negative ages, future dates, percentages over 100)
- Verify categorical values are consistent (watch for "USA", "US", "United States")

### Step 2: Explore

Start broad, then focus on what's interesting.

#### Descriptive Statistics
Always start here — understand the basics before going deeper.

```python
# Numerical columns
print(df.describe())

# Categorical columns
for col in df.select_dtypes(include='object').columns:
    print(f"\n{col}: {df[col].nunique()} unique values")
    print(df[col].value_counts().head(10))
```

#### Distributions
Understanding shape matters for choosing the right tests.

```python
import matplotlib.pyplot as plt

# Distribution of key metrics
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
for i, col in enumerate(['metric_a', 'metric_b', 'metric_c']):
    df[col].hist(ax=axes[i], bins=30)
    axes[i].set_title(col)
    axes[i].axvline(df[col].median(), color='red', linestyle='--', label='median')
    axes[i].legend()
plt.tight_layout()
plt.savefig("distributions.png")
```

#### Correlations and Relationships
Look for patterns between variables.

```python
# Correlation matrix for numerical columns
corr = df.select_dtypes(include='number').corr()
print("Strong correlations (|r| > 0.5):")
for i in range(len(corr.columns)):
    for j in range(i+1, len(corr.columns)):
        if abs(corr.iloc[i, j]) > 0.5:
            print(f"  {corr.columns[i]} vs {corr.columns[j]}: {corr.iloc[i,j]:.3f}")
```

#### Trends Over Time
If the data has a time dimension, always look at trends.

```python
# Time series analysis
df['date'] = pd.to_datetime(df['date'])
daily = df.groupby('date')['metric'].agg(['mean', 'count'])
daily['mean'].plot(figsize=(12, 4), title='Daily Average')
plt.savefig("trend.png")
```

### Step 3: Analyze

Choose the right method for the question.

#### Comparison Questions ("Is A different from B?")

Use the right statistical test:
- **Two groups, continuous outcome**: t-test (if normal) or Mann-Whitney U (if not)
- **Multiple groups**: ANOVA (if normal) or Kruskal-Wallis (if not)
- **Two groups, categorical outcome**: Chi-squared test
- **Before/after with same subjects**: Paired t-test or Wilcoxon signed-rank

Always report:
- Sample sizes for each group
- Effect size (not just p-value) — Cohen's d, odds ratio, or percentage difference
- Confidence intervals
- Practical significance, not just statistical significance

```python
from scipy import stats

# Example: comparing two groups
group_a = df[df['variant'] == 'A']['metric']
group_b = df[df['variant'] == 'B']['metric']

# Check normality first
_, p_normal_a = stats.shapiro(group_a.sample(min(5000, len(group_a))))
_, p_normal_b = stats.shapiro(group_b.sample(min(5000, len(group_b))))

if p_normal_a > 0.05 and p_normal_b > 0.05:
    stat, p_value = stats.ttest_ind(group_a, group_b)
    test_name = "t-test"
else:
    stat, p_value = stats.mannwhitneyu(group_a, group_b)
    test_name = "Mann-Whitney U"

# Effect size (Cohen's d)
pooled_std = ((group_a.std()**2 + group_b.std()**2) / 2) ** 0.5
cohens_d = (group_a.mean() - group_b.mean()) / pooled_std

print(f"Test: {test_name}")
print(f"Group A: mean={group_a.mean():.3f}, n={len(group_a)}")
print(f"Group B: mean={group_b.mean():.3f}, n={len(group_b)}")
print(f"Difference: {group_a.mean() - group_b.mean():.3f} ({(group_a.mean() - group_b.mean())/group_b.mean()*100:.1f}%)")
print(f"Cohen's d: {cohens_d:.3f}")
print(f"p-value: {p_value:.4f}")
```

#### Trend Questions ("Is this changing over time?")

- Use rolling averages to smooth noise
- Look for seasonality (day-of-week, monthly, quarterly patterns)
- Segment by key dimensions — overall trends can mask group-level differences
- Compare period-over-period (WoW, MoM, YoY)

#### Composition Questions ("What's the breakdown?")

- Use percentages and proportions
- Show both absolute numbers and percentages
- Look for the Pareto principle (80/20 rule)
- Segment into meaningful groups

#### A/B Test Analysis

For experiment analysis, follow this checklist:

1. **Sample size**: Was the test adequately powered?
2. **Duration**: Did it run long enough for weekly patterns?
3. **Randomization**: Were groups balanced on key dimensions?
4. **Metric definition**: Is the metric clearly defined and correctly computed?
5. **Statistical test**: Use the appropriate test for the metric type
6. **Multiple comparisons**: Correct for multiple testing if checking many metrics
7. **Practical significance**: Is the effect large enough to matter?

### Step 4: Communicate Findings

Structure matters as much as the analysis itself.

#### Lead With the Answer
Start with what the user asked. Then support it.

**Pattern:**
```
[Answer to the question in 1-2 sentences]

[Key supporting evidence: 2-4 bullet points with specific numbers]

[Methodology note: How you arrived at this, in 1-2 sentences]

[Caveats and limitations: What could affect this conclusion]

[Recommended next steps: What to do with this information]
```

#### Visualization Guidelines

Choose the right chart type:
- **Trend over time**: Line chart
- **Comparison between categories**: Bar chart (horizontal for many categories)
- **Distribution**: Histogram or box plot
- **Relationship between two variables**: Scatter plot
- **Part-of-whole**: Stacked bar or pie chart (only for 2-5 categories)
- **Multiple dimensions**: Heatmap or small multiples

For every chart:
- Title that states the finding, not just the data ("Conversion drops 40% on weekends" not "Conversion by day")
- Label axes with units
- Include sample size or time period
- Use consistent colors across related charts

#### What to Always Include
- **Sample sizes**: How much data backs each claim
- **Time periods**: When the data is from
- **Definitions**: How key metrics are calculated
- **Confidence**: How certain you are (confidence intervals, p-values where appropriate)
- **Limitations**: What the data can't tell you

## Common Pitfalls

- **Survivorship bias**: Only analyzing data from things that succeeded/persisted
- **Simpson's paradox**: A trend that appears in groups but reverses when combined
- **Confounding variables**: Correlation doesn't mean causation — look for third variables
- **Small sample sizes**: Don't draw strong conclusions from small N. State the limitation.
- **Cherry-picking timeframes**: Make sure the time period is representative
- **Precision theater**: Reporting "23.847%" when the data only supports "roughly 24%"
- **Missing baseline**: A number without context is meaningless ("10,000 errors" — is that a lot?)

## Handling Common Requests

### "Just give me a quick look"
Provide: shape, column summary, missing data count, top-level descriptive stats, one key insight. Keep it to one screen of output.

### "What's interesting in this data?"
Run the full exploration workflow (Step 2) and surface the 3-5 most notable patterns: unexpected distributions, strong correlations, outliers, trends.

### "Is this statistically significant?"
Clarify what comparison they mean, choose the right test, report effect size and p-value, and explain what it means in practical terms.

### "Create a dashboard / visualization"
Ask what decisions the dashboard supports. Design 3-5 charts that answer those questions. Use consistent styling and clear titles that state findings.
Files in this package (3)
  • SKILL.md 12 KB
  • references/statistical-tests-guide.md 4 KB
  • scripts/profile_data.py 4 KB

Versions

RepositoryScoreLicence
This page · upsonic/upsonic98MIT
DomynAI/domyn-agents100Apache-2.0
bytedance/deer-flow100MIT
quantummetric/qm-plugin-skills100MIT
Sellyew/Workbuddy-workflow100MIT
DougTrajano/pydantic-ai-skills100MIT
microclaw/microclaw100MIT
lynricsy/hyperskills100MIT
DvJ4Y/shipwright100MIT

Database snapshot: 13.11.2026 · package version 503de6a4626b (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100csv-data-analysis
    This skill should be used when users need to analyze CSV or Excel files, understand data patterns, generate statistical summaries, or create data visualizations. Trigger keywords include "analyze CSV", "analyze Excel", "data analysis", "CSV analysis", "Excel analysis", "data…
  • 90data-scientist
    Explore, clean, analyze, and model data in the workspace (CSV, JSON, Parquet, SQL, notebooks) with reproducible scripts and honest statistics. Use for data analysis, metrics questions, experiment analysis, and quick models.
  • 100analyze-data
    This skill should be used when the user asks to "analyze this dataset", "run datavoyager", "run data voyager", "explore this data", "what's in this CSV", "statistical analysis of", "find patterns in", or wants an AI-driven, code-executing analysis of a tabular dataset that…
  • 90data-analyzer
    Guidance for analyzing structured data, generating statistics and producing data-driven insights. Use when the user asks to analyze data, compute statistics, find patterns, or generate analytical reports.
  • 87explore
    Quick, interactive data exploration without the full pipeline. Use this skill whenever a user wants to explore data, understand what's in a dataset, browse tables or columns, check data distributions, spot patterns, or get familiar with data before analysis. Trigger on phrases…
  • 100data-analyst
    Cleans, analyzes and visualizes data from CSV, Excel or JSON files and explains the findings in plain language. Use when the user shares a spreadsheet or dataset, asks for trends, charts, summaries, or wants to know what the numbers say.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.