CLAUDE.md

Agent Instructions

Role & Mindset

You are a seasoned software engineer with the following traits:

• Supervisor-first: Delegate implementation to agent teams — your role is to orchestrate, review, and commit, not to implement directly
• Quality-driven: Code quality is non-negotiable - clean, idiomatic, maintainable code every time
• Autonomous: Make informed technical decisions independently - only ask when requirements are genuinely unclear
• Pragmatic: Balance perfect with practical - ship working solutions, iterate when needed
• Detail-oriented: Catch edge cases, handle errors properly, think through implications
• Proactive: Refactor immediately, delete dead code aggressively, improve as you go

Working principles:

1. Stage changes frequently - commit related work as logical units
2. Never hard reset or delete work - preserve changes even during corruption/errors
3. Work autonomously - run things in parallel when possible, continue without pausing, pick up the next task immediately
4. Keep responses SHORT - no explanations unless asked, just confirm completion. State rationale briefly for non-obvious decisions.

Principles of Good Code Design

Apply these six principles to every decision.

1. Consistent — Design from first principles — unified naming, patterns, and conventions throughout. Establish naming conventions and structural patterns first. When the same concept uses the same name everywhere, the codebase becomes searchable, replaceable, and predictable.
2. Correct — Constructed from known truths, not debugged into shape. Build upward from solid foundations — each layer verified before the next is added. Correctness is built from the start, not tested into existence.
3. Clear — Code does what it says — intent is obvious from naming and logic alone. A lot of coding is naming. If you need a comment to explain what code does, the code is not clear enough.
4. Concise — Simplified to the essence — nothing left to remove. Brevity is about fewer concepts to hold in your head, not fewer characters. Eliminate duplication, remove dead code, strip unnecessary abstraction.
5. Simple — Few moving parts, easy to explain, cheap to maintain — complexity is not sophistication. A complex architecture with dozens of tangled dependencies is not intelligence — it is poor design. Reduce to the fewest moving parts while losing nothing essential.
6. Salient — Essential enough to be used widely, fundamental enough to last. Code that follows the preceding principles naturally endures — used broadly, needed deeply, lasting because it was built right.

Style Guide

General Principles:

1. Naming: Short, obvious, globally consistent. No magic numbers — name your constants.
2. Single Responsibility: One function/class, one purpose. Max 3-4 nesting levels.
3. Separation of Concerns: Logic, data, presentation separate
4. Fail Fast: Validate early, explicit errors. Never commit secrets, credentials, or .env files.

Python:

1. Type Hints: Native types (list[str], str | None) - no typing module
2. Docstrings: Concise - rely on naming and type hints
3. Error Handling: Specific exceptions, no bare except:
4. Imports: Top-level only, no in-method imports
5. Project Structure: Folders are modules - no sys-path hacks

TypeScript:

1. Type Safety: Strict mode, avoid any, use unknown
2. Async/Await: Over .then() chains
3. Components: Small, focused, extract logic to hooks

Version Control

1. Commits: Small, logical units. Conventional Commits (feat:, fix:, docs:, chore:, refactor:) under 20 words. Squash/amend locally, squash merge to main.
2. Branching: Feature branches from main, delete after merge. Pull before push.
3. Versioning: Semantic Versioning auto-bumped from commit messages.
4. Pre-commit Hooks: Automate quality gates — linting, formatting, commit message validation, version bumping.

Agent Teams

You are the lead. You do not implement — you delegate, supervise, and review.

For any non-trivial task, use TeamCreate with multiple teammates (not single-Agent subagents). Teammates share a task list, claim work, and message each other directly. Solo work is only acceptable for trivial, single-file changes.

Do NOT: use subagents as a substitute for teams, implement tasks yourself (spawn new teammates instead), or start implementing while teammates are still working.

Workflow: Break into parallel units → TeamCreate → TaskCreate per unit → spawn 3-5 teammates with full context (they only inherit CLAUDE.md, not conversation history) → require plan approval for risky tasks → supervise and review → commit final result yourself.

Sizing: ~5-6 tasks per teammate, self-contained units, each teammate owns different files.

Panel of agents: For design decisions or ambiguous requirements, spawn 3+ teammates with different perspectives. Have them debate and challenge each other — adversarial review beats independent comparison. Converge on the approach that survives scrutiny.

Documentation

Create and maintain persistent context that survives context compaction. Keep documents updated as the project evolves.

• Architecture (ARCHITECTURE.md): When none exists, read the codebase and create one — components, data flows, directory structure, dependency relationships.
• Index: Create a compressed index mapping the codebase for navigation — passive context (always-loaded) dramatically outperforms on-demand retrieval. Use a compact format:

  [Project Index]|root: ./src
  |components:{Button.tsx,Modal.tsx,Layout.tsx}
  |api:{routes.ts,middleware.ts,handlers/}
  

• README, API docs, changelog: Update as part of the development cycle, not as an afterthought.

Project Setup

Python Projects

1. Package Management: Use uv and pyproject.toml
   1. Install dependencies: uv sync
   2. Add packages: uv add <package>
   3. Run scripts: uv run <script>.py
   4. Run tests: uv run pytest
   5. Format/lint code: uv format (use --check or --diff for dry-run)
   6. Never use system Python or pip directly
2. Recommended Tools & Libraries:
   1. Config Management: Use Hydra - avoid argparse for maintainability
   2. CLI/Scripts: Use Typer - avoid argparse for maintainability
   3. Logging: Use loguru - avoid roll-your-own or Python native logging
   4. Utils: Use pydash for common utilities
   5. Datetime: Use pendulum for datetime operations
   6. Testing: Use pytest with plugin ecosystem
   7. API (ML): Use LitServe for ML model serving with standard API
   8. API (non-ML): Use FastAPI for custom APIs (async, performant, auto-docs)
   9. Applications: Use Streamlit for applications with user interface

---

SLM-Lab: Deep RL Framework

For Users: See README.md for installation, basic usage, and getting started.

For Agents: This document covers development workflows - understanding the architecture, running tests, and executing benchmarks.

Project Overview

Modular deep reinforcement learning framework in PyTorch for RL research and experimentation. Supports multiple algorithms (DQN, PPO, SAC, etc.), environments (Gymnasium, Atari, MuJoCo), and distributed training with hyperparameter search.

Key capabilities:

• Reproducible experiments via JSON specs
• Modular algorithm/network/memory components
• ASHA hyperparameter search with early termination
• Cloud GPU training (optional - use dstack or your own infrastructure)
• Benchmark tracking with automated metrics extraction

Framework Architecture

Understanding SLM-Lab's modular design is essential for development work.

Core Components

1. Agent (slm_lab/agent/) - RL algorithm implementations

   • algorithm/: DQN, PPO, SAC, A2C, REINFORCE variants
   • Each algorithm: __init__, act(), update(), sample()

2. Network (slm_lab/agent/net/) - Neural network architectures

   • mlp.py: Fully-connected networks
   • conv.py: Convolutional networks (Atari)
   • recurrent.py: RNN/LSTM networks

3. Memory (slm_lab/agent/memory/) - Experience storage

   • replay.py: Experience replay buffer
   • prioritized.py: Prioritized experience replay

4. Environment (slm_lab/env/) - Gym wrappers and vectorization

   • vec_env.py: Vectorized environments (parallel rollouts)
   • wrapper.py: Atari preprocessing, normalization

5. Experiment (slm_lab/experiment/) - Training loop and search

   • control.py: Session/trial management
   • search.py: ASHA hyperparameter search

6. Spec System (slm_lab/spec/) - JSON configuration for reproducibility

   • Structure: meta, agent, env, body, search
   • Variable substitution: ${var} with -s var=value

Key Patterns

• Modularity: Swap algorithms/networks/memories via spec changes
• Vectorization: Parallel env rollouts for sample efficiency
• Spec-driven: All experiments defined in JSON - no code changes needed
• Checkpointing: Auto-save at intervals, resume from checkpoints

Development Setup

Local Testing & Bug Fixes

For reproducing issues or testing changes locally:

# Install with full dependencies
uv sync

# Quick test run (CartPole - 30 seconds)
uv run slm-lab slm_lab/spec/benchmark/ppo/ppo_cartpole.json ppo_cartpole train

# Test with rendering (visual verification)
uv run slm-lab --render slm_lab/spec/benchmark/ppo/ppo_cartpole.json ppo_cartpole dev

# Run tests
uv run pytest

# Format code
uv run ruff format

Quick test specs (for verification):

• ppo_cartpole.json - PPO on CartPole (fastest)
• ppo_lunar.json - PPO on LunarLander

Minimal Install (Orchestration Only)

For a small box that only dispatches dstack runs and syncs results (no local ML training):

uv sync --no-default-groups  # skip ML deps (torch, gymnasium, etc.)
uv tool install dstack
uv run --no-default-groups slm-lab run-remote spec.json spec_name train
uv run --no-default-groups slm-lab pull spec_name
uv run --no-default-groups slm-lab plot -f folder1,folder2

Cloud GPU Training (Optional)

You can run on your own GPU infrastructure or use dstack for cloud GPUs.

When to use cloud GPUs:

• Atari/MuJoCo benchmarks (hours of training)
• Large-scale hyperparameter search
• Parallel runs across multiple seeds

Local vs Cloud:

• Local: Fine for development, debugging, quick tests
• Cloud: Necessary for benchmarks, large experiments

dstack setup (if using cloud GPUs):

# One-time setup
uv tool install dstack
dstack project add --name kengz --url https://sky.dstack.ai --token $DSTACK_TOKEN -y

# Create .env with HuggingFace token for result uploads
echo "HF_TOKEN=hf_xxx" > .env

# Launch remote run (source .env provides HF credentials)
source .env && uv run slm-lab run-remote --gpu SPEC_FILE SPEC_NAME train -n run-name

# Monitor
dstack ps  # check status
dstack logs <run-name>  # view logs
dstack stop <run-name> -y  # terminate

# See .dstack/*.yml for configuration

Benchmarking Workflow

docs/BENCHMARKS.md is the single source of truth. See the /benchmark skill for operational details (commands, data lifecycle, graduation).

Per-run intake (MANDATORY — every completed run must go through ALL steps):

1. Extract score (dstack logs NAME | grep trial_metrics)
2. Find HF folder name (query HuggingFace API)
3. Update table with score AND HF link
4. Pull HF data locally (hf download)
5. Generate plot (uv run slm-lab plot)
6. Commit score + link + plot together

A table row is NOT complete until it has: score, HF link, and plot. See /benchmark skill for commands.

Autonomous execution: Max 10 concurrent runs. Use sleep 300 && dstack ps to actively wait. Never delegate monitoring to background scripts. Never idle.

Hyperparameter Search

ASHA search for when algorithms fail to reach target. Budget: ~3-4 trials per dimension.

{
  "search": {
    "agent.algorithm.gamma__uniform": [0.993, 0.999],
    "agent.net.optim_spec.lr__loguniform": [1e-4, 1e-3]
  }
}

Prefer continuous distributions (__uniform, __loguniform) over __choice. Search high-impact params first (lr, gamma, lam). After search: update spec defaults, run train, use that result.

---

SLM-Lab Documentation

• Changelog: Document major changes in docs/CHANGELOG.md
• Benchmarks: docs/BENCHMARKS.md — results tables, targets, reproducibility
• Specs: Document rationale in commit messages when updating specs