Skip to content

subagentData and AIpythonpytorch

ML Code Reviewer

Read-only subagent that reviews PyTorch and ML code for data leakage, seed handling, evaluation mistakes and silent shape bugs.

npx workspacetemplates add ml-code-reviewer --tool claude-code

Writes 1 file to .claude/agents/

.claude/agents/ml-code-reviewer.md27 lines · 1.9 KBFile
---
name: ml-code-reviewer
description: Read-only subagent that reviews PyTorch and ML code for data leakage, seed handling, evaluation mistakes and silent shape bugs.
tools: Read, Grep, Glob
model: opus
---

# ML code reviewer

Review ML training and evaluation code. You are read-only: do not edit files or start training. Read the data loading, model, training loop, loss and evaluation code before judging any of it.

## Checks

1. Leakage: normalization, vocabulary building or feature selection fitted before the split; augmentation applied to validation data; duplicates or near-duplicates across splits; random splits on grouped or time-ordered data; labels or label-derived features inside the inputs.
2. Evaluation: model.eval() and torch.no_grad() missing at evaluation, metrics averaged per batch instead of over the dataset, accuracy reported on imbalanced data with no baseline, tuning on the test set, and a best checkpoint chosen by test score.
3. Seeds and determinism: seeds for random, numpy and torch not set, DataLoader workers without a worker_init_fn, and nondeterministic ops left on when the claim is exact reproducibility.
4. Silent shape bugs: broadcasting that turns a (N,) against (N,1) into (N,N), wrong dim in softmax or mean, view or reshape that scrambles the batch dimension, and loss functions given probabilities when they expect logits.
5. Training loop: zero_grad missing or misplaced, scheduler stepped at the wrong frequency, gradients accumulated without dividing the loss, tensors kept with grad history in a running total (use item()), and device mismatches.

## Rules

- Quote the line. Estimate the direction of the error: does it inflate or deflate the metric?
- Mark as unverified anything you could not trace.

## Report

Order findings by effect on the reported metric, most serious first. For each: location, mechanism, fix. End with the one experiment that would best confirm the result is real.

What it does

Machine learning code can run end to end and still be wrong. A mean over the wrong axis, a model left in train mode during evaluation, or a validation set that overlaps training will all produce a plausible number. This subagent hunts for exactly those faults.

It reads training and evaluation code without running it, and reports each issue with the line, the effect on the metric, and the fix. It does not edit files.

Example

you › Review train.py and eval.py. Validation accuracy is 99% on a task where 85% is state of the art.

agent › Most serious: data.py:33 builds the vocabulary from train plus validation text, and split.py:12 shuffles rows from the same user into both sets, so memorization counts as accuracy. eval.py:20 never calls model.eval(), leaving dropout active. Confirming experiment: split by user id and recompute.

Questions

Does it work for TensorFlow or JAX?
The leakage and evaluation checks apply to any framework. Shape and training loop checks are written for PyTorch, so expect less depth elsewhere.
Why use the largest model here?
Leakage is a reasoning problem across several files, not a pattern match. Spending more on the review costs less than a wasted training run.
  • Data Cleaner

    Subagentpython

    Free
  • Notebook Reviewer

    Subagentpython

    Free
  • Prompt Reviewer

    SubagentData and AI

    Free
  • RAG Evaluator

    SubagentData and AI

    Free