Skip to content

feat: Add optional setup command to helix.yaml #11

@amrit110

Description

@amrit110

Summary

Add support for an optional setup command in helix.yaml that runs once before the first experiment in a session. This allows helices to declare one-time setup steps (e.g. downloading model weights, caching datasets) as part of their configuration, making helix run fully self-contained without requiring manual pre-run steps.

Motivation

Currently, some helices require a one-time setup step to be run manually before helix run can work — for example, uv run prepare.py in helix-inference-opt downloads model weights and caches datasets. There is no way to declare this in helix.yaml, so users must remember to run it themselves on each new machine. This is a poor experience and easy to forget.

Proposed Change

Add an optional setup section to helix.yaml:

setup:
  command: uv run prepare.py

Behaviour

  • If setup.command is provided, the runner executes it once at the start of helix run, before the agent is invoked or any experiments are run.
  • If setup.command is absent, the runner proceeds as today (no change in behaviour).
  • The command runs in the repo root, inheriting the same environment as the evaluate command.
  • If the setup command exits non-zero, helix run should abort with a clear error message.

Schema Changes

In config.py, add a new optional SetupConfig model and field on HelixConfig:

class SetupConfig(BaseModel):
    command: str

class HelixConfig(BaseModel):
    ...
    setup: Optional[SetupConfig] = None

Runner Changes

In runner.py, after _preflight() and before launching the agent, add a setup step:

if self.config.setup:
    # run setup.command, stream output, abort on non-zero exit

Example helix.yaml

name: inference-opt
setup:
  command: uv run prepare.py
metrics:
  primary:
    name: tokens_per_sec
    optimize: maximize
  evaluate:
    command: uv run infer.py

Acceptance Criteria

  • SetupConfig added to config.py with command field
  • HelixConfig.setup is optional (existing helices without it are unaffected)
  • Runner executes setup command once per helix run session before the agent starts
  • Setup output is streamed to the terminal so the user can see progress
  • Non-zero exit aborts the session with a clear error
  • Docs/template updated to show the new field

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions