Skip to content

project rulesData and AIpythontypescript

LLM App Rules

Project rules for LLM applications: prompts stored as files, validated structured output, bounded retries, evals, redacted logs and cost caps.

npx workspacetemplates add llm-app-rules --tool claude-code

Writes 1 file to your project root

CLAUDE.mdadd to your file22 lines · 1.4 KBFile
# Project rules: LLM applications

## Prompts
- Store prompts as files (`prompts/*.md` or templates), not inline strings. Version them with the code and name the variables explicitly. Never build a prompt by concatenating raw user text into instructions; put user content in a clearly delimited section.
- Keep model names, temperature and token limits in config, not scattered through call sites.

## Output
- Ask for structured output through the provider's schema or tool-calling feature and validate it with Pydantic or Zod. On a validation failure, retry once with the error, then fail with a typed error.
- Treat model text as untrusted. Never run it as code, shell commands or SQL, and escape it before rendering.

## Reliability and cost
- Wrap every call with a timeout, a bounded retry with exponential backoff and jitter for rate limits and 5xx errors, and a per-request and per-user token budget. Stream long answers.
- Cache deterministic calls. Use a smaller model for classification and routing.

## Evals
- Keep a golden set of real inputs with expected properties. Run it in CI and on every prompt or model change, and compare scores to the last run.

## Logging and secrets
- Log model, prompt version, token counts, latency and request id. Redact personal data and never log keys. Load keys from the environment.

## Done checks
- Run the type checker, linter, unit tests with a mocked client, and the eval suite.

What it does

An LLM call is an unreliable network service with a variable price. These rules make an agent treat it that way: prompts live in versioned files, outputs are validated, and every call has limits.

They add evals and logging so quality changes are measured instead of guessed, whether the app is written in Python or TypeScript.

Example

you › Add a feature that summarizes support tickets into three bullet points

agent › The agent adds a prompt file, a schema with a bullets array of fixed length, a client wrapper with timeout and one retry, a mocked unit test, and five golden tickets in the eval set.

Questions

Which provider does this assume?
None. The rules describe habits that apply to any hosted or local model API.
Why validate output when the model supports JSON mode?
JSON mode guarantees syntax, not meaning. Validation catches missing fields, wrong types and out-of-range values.
  • Python Data Science Rules

    Project rulespython

    Free
  • PyTorch Rules

    Project rulespython

    Free
  • Data Pipeline Design

    SkillData and AI

    Free