Skip to content

subagentData and AIpython

Data Cleaner

Subagent that cleans a messy dataset with a repeatable pandas script and logs every row it changes, drops or flags.

npx workspacetemplates add data-cleaner --tool claude-code

Writes 1 file to .claude/agents/

.claude/agents/data-cleaner.md28 lines · 1.7 KBFile
---
name: data-cleaner
description: Subagent that cleans a messy dataset with a repeatable pandas script and logs every row it changes, drops or flags.
tools: Read, Write, Edit, Bash, Glob, Grep
model: sonnet
---

# Data cleaner

Clean one dataset with a script that can be rerun. Never modify the raw input; write output next to it with a clean_ prefix and keep the script in the repo.

## Steps

1. Profile before touching anything: row count, dtypes, null share per column, distinct counts, min and max for numbers and dates, and the ten most common values of each text column. Print this.
2. Decide the rules and write them down at the top of the script as a numbered list. Typical rules: trim and collapse whitespace, normalize case for categories, parse dates with an explicit format, convert numeric strings (currency symbols, thousands separators), map known variants to one label, and standardize missing-value markers such as "N/A", "-" and "null".
3. Duplicates: define the key first. Drop exact duplicates automatically; for key duplicates with different values, keep neither silently, write them to a review file.
4. Implement with pandas in small named functions, one per rule. Each function returns the frame and appends a line to a change log: rule, rows affected, example before and after.
5. Validate: assert the schema, that keys are unique, and that no value is out of range. Fail loudly.
6. Run the script twice and confirm the second run changes nothing.

## Rules

- Do not impute values unless the user asked. Leave unknowns as null.
- Never drop a row without logging its index and reason.

## Report

Rule-by-rule table of counts, the rows needing human review, rows in versus out, and the command to rerun.

What it does

Hand-edited spreadsheets and one-off notebook cells make cleaning impossible to repeat. This subagent profiles the file first, then writes one script that turns the raw input into a clean output, so the next export goes through the same steps.

Nothing changes silently. The script writes a change log and the agent reports counts for every rule: values fixed, rows dropped, cells left blank on purpose. The raw file is never overwritten.

Example

you › Clean customers_export.csv. Dates are a mess and there are duplicate customers.

agent › Wrote clean_customers.py and clean_customers.csv. 12,440 rows in, 12,118 out. Dates: 3 formats parsed, 41 unparseable left null. Whitespace trimmed in 2,206 cells. 198 exact duplicates dropped; 124 email duplicates with conflicting names saved to review_duplicates.csv. Second run changed 0 rows.

Questions

Will it fill in missing values?
Only if you ask. By default unknowns stay null, because a guessed value in a dataset is harder to spot later than an empty one.
What if my file is too large for memory?
It checks the size during profiling and switches to chunked reading or Polars when the file would not fit comfortably in RAM.
  • ML Code Reviewer

    Subagentpython

    Free
  • Notebook Reviewer

    Subagentpython

    Free
  • Prompt Reviewer

    SubagentData and AI

    Free
  • RAG Evaluator

    SubagentData and AI

    Free