Skip to content

project rulesData and AIpythonpytorch

PyTorch Rules

Project rules for PyTorch projects: config-driven training, deterministic seeding, device-agnostic code, safe checkpoints and strict evaluation.

npx workspacetemplates add pytorch-rules --tool claude-code

Writes 1 file to your project root

CLAUDE.mdadd to your file21 lines · 1.4 KBFile
# Project rules: PyTorch

## Structure
- Split code into `models/`, `data/`, `train.py` and `eval.py`. Models are plain `nn.Module` classes that know nothing about paths or logging.
- All hyperparameters, paths and sizes come from a config file (YAML with a dataclass or Hydra). Log the resolved config with every run.

## Devices and tensors
- Pick the device once: `torch.device("cuda" if torch.cuda.is_available() else "cpu")`. Move models and batches with `.to(device)`; never call `.cuda()` directly.
- Create new tensors with `device=` and `dtype=` taken from existing ones. Avoid `.item()` and `.cpu()` inside hot loops.
- Use `torch.amp` autocast for mixed precision, with a gradient scaler when using fp16.

## Reproducibility
- Seed Python, NumPy and torch from one function; seed `DataLoader` workers with a generator and `worker_init_fn`. State clearly when deterministic algorithms are off.

## Training loop
- Call `model.train()` and `model.eval()` explicitly. Evaluate under `torch.inference_mode()`. Zero gradients each step, clip when needed and step the scheduler in one known place.
- Save the model, optimizer, scheduler, epoch and config in each checkpoint, and load with `weights_only=True`.

## Evaluation and checks
- Hold out validation and test sets, never tune on test. Report metrics with the seed and the commit.
- Run `ruff check` and `pytest`, including a tiny-batch smoke test that overfits eight samples on CPU.

What it does

Training scripts tend to become one long file with hard-coded paths and a GPU assumption. These rules separate model, data and loop, and move every tunable value into a config.

They also cover checkpoints, seeding and evaluation, so a result can be reproduced and a run can be resumed without surprises.

Example

you › Add early stopping and resume support to the training script

agent › The agent adds a patience value to the config, saves full checkpoints with optimizer and epoch state, loads them through a resume flag, and adds a CPU smoke test that stops early on a toy dataset.

Questions

Does it cover Lightning or Accelerate?
The conventions still apply, but those libraries own the loop. Keep the config, seeding and checkpoint rules and drop the manual loop advice.
Why load with weights_only?
It avoids executing arbitrary pickled code when reading a checkpoint from an unknown source.
  • LLM App Rules

    Project rulespython

    Free
  • Python Data Science Rules

    Project rulespython

    Free
  • Data Pipeline Design

    SkillData and AI

    Free