Skip to content

skillData and AIpython

Exploratory Data Analysis

Profiles a new dataset, finds quality problems and summarizes findings with plots. Use it before modeling or reporting on data you have not seen yet.

npx workspacetemplates add exploratory-data-analysis --tool claude-code

Writes 1 file to .claude/skills/exploratory-data-analysis/

.claude/skills/exploratory-data-analysis/SKILL.md28 lines · 1.6 KBFile
---
name: exploratory-data-analysis
description: Profiles a new dataset, finds quality problems and summarizes findings with plots. Use it before modeling or reporting on data you have not seen yet.
---

# Exploratory data analysis

Understand the data before drawing a conclusion from it. Report what you saw, separate from what you suspect.

## Steps

1. State the unit of a row ("one order line", "one user per day") and the target question. If you cannot name the row unit, find out before going on.
2. Inspect structure: row and column counts, dtypes, memory, a random sample of 10 rows (not just `head`), and the date range if any.
3. Check quality: nulls per column as a percentage, duplicate rows and duplicate keys, constant columns, impossible values (negative ages, future dates), and inconsistent categories.
4. Describe each numeric column: min, quartiles, max, skew, and outliers by the 1.5 IQR rule. Draw a histogram or box plot for the important ones.
5. Describe categoricals: cardinality, top values, and rare levels. Use a bar chart for the leading categories.
6. Look at relationships: a correlation matrix for numerics, group means against the target, and a time series plot for anything dated. Remember correlation is not cause.
7. Check for leakage: any column that would not exist at prediction time.
8. Save plots to `eda/` as PNG files with titles and axis labels.

## Rules

- Never modify the source data; work on a copy.
- Report numbers with their denominators.

## Report

Open with the 3 findings that change what you will do next, then data quality problems ranked by impact, then plots with one sentence each, then open questions.

What it does

Jumping straight into a model or dashboard on unfamiliar data is how wrong conclusions ship. This skill walks the agent through a fixed pass: shape, types, missing values, distributions, relationships and oddities, with a plot where a plot says more than a number.

The output is a short written summary with the three or four things that matter, the problems that must be fixed first, and the questions the data can and cannot answer.

Example

you › Explore sales_2025.parquet and tell me if it is safe to forecast from.

agent › 412,300 rows, one per order line, Jan 2024 to Oct 2025. Findings: 3.1% of unit_price is zero (free samples, exclude); 14 duplicate order_line_id; March 2025 has no rows for store 17. Revenue is strongly seasonal with a weekly cycle. Not safe to forecast store 17 until the gap is explained. Six plots saved to eda/.

Questions

Does it need a notebook?
No. It runs as scripts and saves PNG files, though you can paste the code into a notebook if you prefer.
Can it handle files too large for memory?
It samples or reads columns selectively and says when a conclusion rests on a sample rather than the whole file.
  • Data Pipeline Design

    SkillData and AI

    Free
  • dbt Modeling

    SkillData and AI

    Free
  • LLM Cost and Latency

    SkillData and AI

    Free