Skip to content

emperorhan/codex-ralph

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

189 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Ralph hero

codex-ralph

manager / planner / developer / qa ์ž์œจ ์—์ด์ „ํŠธ ๋ฃจํ”„ CLI

์„ธํŒ…

1) Codex ์—ฐ๊ฒฐ

ralph ๋ฃจํ”„๋Š” ๋‚ด๋ถ€์ ์œผ๋กœ codex exec๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

codex --version
codex login status

๋กœ๊ทธ์ธ์ด ์•ˆ ๋˜์–ด ์žˆ์œผ๋ฉด:

codex login

API ํ‚ค ๋กœ๊ทธ์ธ:

printenv OPENAI_API_KEY | codex login --with-api-key

2) ralphctl ์„ค์น˜

macOS / Ubuntu:

curl -fsSL https://raw.githubusercontent.com/emperorhan/codex-ralph/main/scripts/install-ralphctl.sh | bash

์„ค์น˜ ํ™•์ธ:

ralphctl --help
  • ๊ธฐ๋ณธ control dir: ~/.ralph-control
  • ์ฒซ ์‹คํ–‰ ์‹œ ํ•„์š”ํ•œ ๊ธฐ๋ณธ plugin/registry๊ฐ€ ์ž๋™ ์ค€๋น„๋ฉ๋‹ˆ๋‹ค.

3) ํ”„๋กœ์ ํŠธ ์—ฐ๊ฒฐ (๊ถŒ์žฅ)

ํ”„๋กœ์ ํŠธ ๋ฃจํŠธ์—์„œ:

ralphctl --project-dir "$PWD" bootstrap

์ดํ›„:

  • ./ralph ์‹คํ–‰ ํ—ฌํผ๊ฐ€ ์ƒ์„ฑ๋ฉ๋‹ˆ๋‹ค.
  • ๋กœ์ปฌ Git ์ €์žฅ์†Œ๊ฐ€ ์—†์œผ๋ฉด ์ž๋™ ์ดˆ๊ธฐํ™”๋ฉ๋‹ˆ๋‹ค.
  • ๊ธฐ๋ณธ ์„ค์ • ํŒŒ์ผ:
    • .ralph/profile.yaml
    • .ralph/profile.local.yaml
  • ์•ˆ์ •์„ฑ ๊ธฐ๋ณธ๊ฐ’(timeout/retry/watchdog/primary daemon restart)์ด ์ ์šฉ๋ฉ๋‹ˆ๋‹ค.
  • ์™„๋ฃŒ(done)๋œ ์ด์Šˆ ๋‹จ์œ„๋กœ ์ž๋™ ์ปค๋ฐ‹์ด ๋ˆ„์ ๋ฉ๋‹ˆ๋‹ค(์ž„์‹œ/๋Ÿฐํƒ€์ž„ ํŒŒ์ผ ์ œ์™ธ).
  • canonical runtime์€ Ralph Runtime (.ralph-runtime) ์ž…๋‹ˆ๋‹ค.

4) ์ฒซ ๋™์ž‘ ํ™•์ธ

ralphctl --project-dir "$PWD" doctor
ralphctl --project-dir "$PWD" status
ralphctl --project-dir "$PWD" work
ralphctl --project-dir "$PWD" eval

๋ฌธ์ œ๊ฐ€ ์žˆ์œผ๋ฉด:

ralphctl --project-dir "$PWD" doctor --repair
ralphctl --project-dir "$PWD" cp recover
ralphctl --project-dir "$PWD" retry-blocked --reason codex_failed_after

Public Quickstart

public ์‚ฌ์šฉ์ž๋Š” ์•„๋ž˜ happy path๋ฅผ ๊ธฐ์ค€์œผ๋กœ ์‹œ์ž‘ํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค.

ralphctl --project-dir "$PWD" bootstrap
ralphctl --project-dir "$PWD" intake "ํ—ฌ์Šค์ฒดํฌ ์—”๋“œํฌ์ธํŠธ ์ถ”๊ฐ€ํ•˜๊ณ  ํ…Œ์ŠคํŠธ๊นŒ์ง€ ๋งˆ๋ฌด๋ฆฌํ•ด์ค˜"
ralphctl --project-dir "$PWD" intake status
ralphctl --project-dir "$PWD" intake revise "๋ฐ˜๋“œ์‹œ ๋๋‚˜์•ผ ํ•˜๋Š” ๊ฒฐ๊ณผ๋Š” /health ์—”๋“œํฌ์ธํŠธ์™€ ๊ด€๋ จ ํ…Œ์ŠคํŠธ ํ†ต๊ณผ์ž…๋‹ˆ๋‹ค."
ralphctl --project-dir "$PWD" intake status
ralphctl --project-dir "$PWD" intake approve
ralphctl --project-dir "$PWD" status

์ง€์› ํ™˜๊ฒฝ:

  • macOS / Linux
  • ๋กœ์ปฌ Git ํ”„๋กœ์ ํŠธ
  • codex exec ์‚ฌ์šฉ ๊ฐ€๋Šฅ ํ™˜๊ฒฝ

์ฐธ๊ณ  ๋ฌธ์„œ:

ํ™œ์šฉ๋ฐฉ๋ฒ•

1) ์ž‘์—… ํˆฌ์ž…

์ˆ˜๋™ ์ด์Šˆ ์ƒ์„ฑ:

ralphctl --project-dir "$PWD" intake issue developer "health endpoint ๊ตฌํ˜„"
ralphctl --project-dir "$PWD" intake issue --priority 10 --story-id US-001 developer "๊ฒฐ์ œ API ์—๋Ÿฌ ์ฒ˜๋ฆฌ ๊ฐœ์„ "

PRD JSON ์ผ๊ด„ ์ƒ์„ฑ:

ralphctl --project-dir "$PWD" intake prd --file prd.json --default-role developer

์ž์—ฐ์–ด ์š”์ฒญ์—์„œ ์ด์Šˆ ์ƒ์„ฑ:

ralphctl --project-dir "$PWD" intake request "๊ฒฐ์ œ API ์—๋Ÿฌ ์ฒ˜๋ฆฌ๋ฅผ ์ •๋ฆฌํ•˜๊ณ  ํ…Œ์ŠคํŠธ๊นŒ์ง€ ์ถ”๊ฐ€ํ•ด์ค˜"
ralphctl --project-dir "$PWD" intake "ํ—ฌ์Šค์ฒดํฌ ์—”๋“œํฌ์ธํŠธ ์ถ”๊ฐ€ํ•˜๊ณ  ๋ฌธ์„œ๋„ ์—…๋ฐ์ดํŠธํ•ด์ค˜"

intent session ์ ๊ฒ€/์Šน์ธ:

ralphctl --project-dir "$PWD" intake status
ralphctl --project-dir "$PWD" intake brief
ralphctl --project-dir "$PWD" intake approve
ralphctl --project-dir "$PWD" intake revise "์„ฑ๊ณต ๊ธฐ์ค€์€ smoke test ํ†ต๊ณผ"
ralphctl --project-dir "$PWD" intake abort

intake approve ์ดํ›„์—๋Š” workload๋ณ„ auto-run ์ •์ฑ…์ด ์ ์šฉ๋ฉ๋‹ˆ๋‹ค. ๊ธฐ๋ณธ๊ฐ’์—์„œ๋Š” terminal verify ์ „๊นŒ์ง€๋งŒ ์ž๋™ ์‹คํ–‰ํ•˜๊ณ , status์™€ Telegram /task status๋Š” summary / situation / next_action ํ˜•์‹์˜ operator cockpit ๋ฌธ๊ตฌ๋กœ ํ˜„์žฌ ๊ฒฝ๊ณ„์™€ ์žฌ๊ฐœ ํฌ์ธํŠธ๋ฅผ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค. public-safe ๊ธฐ๋ณธ๊ฐ’์„ ๋ฒ—์–ด๋‚˜๋ฉด public_guardrail ๊ฒฝ๊ณ ์™€ emergency_action ์•ˆ๋‚ด๊ฐ€ ๊ฐ™์ด ์ถœ๋ ฅ๋ฉ๋‹ˆ๋‹ค.

2) ๋ฃจํ”„ ์šด์˜

์ด์ „ ./ralph start, ./ralph run, ralphctl supervise ๊ธฐ๋ฐ˜ loop ๊ฒฝ๋กœ๋Š” ์ œ๊ฑฐ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ํ˜„์žฌ ๊ถŒ์žฅ ์‹คํ–‰ ๊ฒฝ๋กœ๋Š” Ralph Runtime + session orchestration ์ž…๋‹ˆ๋‹ค.

ralphctl --project-dir "$PWD" work
ralphctl --project-dir "$PWD" status
ralphctl --project-dir "$PWD" eval
ralphctl --project-dir "$PWD" cp doctor --repair --strict

๋Œ€๋ถ€๋ถ„์˜ ์‚ฌ์šฉ์ž๋Š” bootstrap, intake, work, status, eval, doctor๋งŒ ์“ฐ๋ฉด ๋ฉ๋‹ˆ๋‹ค. cp๋Š” low-level control-plane ops/debug namespace๋กœ ๋‚จ๊ฒจ๋‘” advanced ๋ช…๋ น๊ตฐ์ž…๋‹ˆ๋‹ค.

3) ๊ฒฐ๊ณผ ํ™•์ธ

์ฃผ์š” ์‚ฐ์ถœ๋ฌผ:

  • .ralph/done/: ์™„๋ฃŒ ์ด์Šˆ
  • .ralph/blocked/: ์‹คํŒจ/๊ฒ€ํ†  ํ•„์š” ์ด์Šˆ
  • .ralph/logs/: ์‹คํ–‰ ๋กœ๊ทธ

legacy blocked ์ด์Šˆ ์žฌ์‹œ๋„:

ralphctl --project-dir "$PWD" retry-blocked
ralphctl --project-dir "$PWD" retry-blocked --reason codex_failed_after
ralphctl --project-dir "$PWD" retry-blocked --reason codex_permission_denied --limit 1

Advanced: Ralph Runtime (Intent -> Graph -> Execution)

์ด ์„น์…˜์€ ์šด์˜/๋””๋ฒ„๊น…์ด ํ•„์š”ํ•œ ๊ฒฝ์šฐ์—๋งŒ ์ง์ ‘ ์“ฐ๋Š” advanced cp ๋ช…๋ น๋“ค์ž…๋‹ˆ๋‹ค.

1) ์ดˆ๊ธฐํ™” ๋ฐ Intent ์ž…๋ ฅ

ralphctl --project-dir "$PWD" cp init
ralphctl --project-dir "$PWD" cp import-intent --file intent.json
ralphctl --project-dir "$PWD" cp plan --intent-id <intent-id>

2) ์‹คํ–‰/๊ฒ€์ฆ

# ๊ธฐ๋ณธ ์‹คํ–‰ (verify ์ค‘์‹ฌ)
ralphctl --project-dir "$PWD" cp run --max-workers 1

# Codex ์‹คํ–‰ + verify
ralphctl --project-dir "$PWD" cp run --max-workers 1 --execute-with-codex

# blocked ๋ณต๊ตฌ (policy ์กด์ค‘: retry budget/backoff/circuit)
ralphctl --project-dir "$PWD" cp recover

# ์ฆ‰์‹œ ๊ฐ•์ œ ๋ณต๊ตฌ (cooldown/circuit ์šฐํšŒ)
ralphctl --project-dir "$PWD" cp recover --force

ralphctl --project-dir "$PWD" cp status --json
ralphctl --project-dir "$PWD" cp metrics --json
ralphctl --project-dir "$PWD" cp metrics --with-baseline --json
ralphctl --project-dir "$PWD" cp doctor --repair --repair-recover-limit 10 --strict
# cooldown/circuit ์šฐํšŒ๊ฐ€ ํ•„์š”ํ•  ๋•Œ
ralphctl --project-dir "$PWD" cp doctor --repair --repair-force-recover --strict
# blocked circuit/์žฌ์‹œ๋„ ์˜ˆ์‚ฐ์„ ์ˆ˜๋™ ์ดˆ๊ธฐํ™”ํ•  ๋•Œ
ralphctl --project-dir "$PWD" cp doctor --repair --repair-reset-circuit --repair-reset-retry-budget --strict
# fault injection (์‹ ๋ขฐ์„ฑ ํ…Œ์ŠคํŠธ)
ralphctl --project-dir "$PWD" cp fault-inject --task-id task-1 --mode lease-expire
ralphctl --project-dir "$PWD" cp fault-inject --task-id task-1 --mode verify-fail
ralphctl --project-dir "$PWD" cp fault-inject --task-id task-1 --mode execute-fail
ralphctl --project-dir "$PWD" cp fault-inject --task-id task-1 --mode permission-denied

advanced cp doctor ํ•ต์‹ฌ data integrity ์ฒดํฌ:

  • task_json_consistency
  • event_journal_consistency
  • learning_journal_consistency
  • event_ledger_consistency
  • learning_ledger_consistency

๊ฒ€์ฆ ์ •์ฑ… ์—”์ง„(unit|integration|lint|custom) ์‚ฌ์šฉ ์˜ˆ์‹œ:

{
  "verify_cmd": "lint: golangci-lint run\nunit: go test ./...\ncustom: ./scripts/verify_extra.sh"
}

์ ‘๋‘์–ด๊ฐ€ ์—†์œผ๋ฉด custom์œผ๋กœ ์ฒ˜๋ฆฌ๋ฉ๋‹ˆ๋‹ค.

readiness eval

readiness eval์€ Ralph Runtime + session harness์˜ ๋ณ‘ํ•ฉ ๊ฐ€๋Šฅ์„ฑ์„ ์ ์ˆ˜ํ‘œ์™€ ์‹œ๋‚˜๋ฆฌ์˜ค ๊ฒฐ๊ณผ๋กœ ํ™•์ธํ•˜๋Š” ๋ช…๋ น์ž…๋‹ˆ๋‹ค.

ralphctl --project-dir "$PWD" eval

์‹คํ–‰ํ•˜๋ฉด ํ…์ŠคํŠธ ์š”์•ฝ๊ณผ ํ•จ๊ป˜ ์•„๋ž˜ ์•„ํ‹ฐํŒฉํŠธ๊ฐ€ ์ƒ์„ฑ๋ฉ๋‹ˆ๋‹ค.

  • .ralph-runtime/reports/eval/latest.json
  • .ralph-runtime/reports/eval/latest.md
  • .ralph-runtime/reports/eval/runs/<timestamp>/scorecard.json
  • .ralph-runtime/reports/eval/runs/<timestamp>/scenarios.json
  • .ralph-runtime/reports/eval/runs/<timestamp>/summary.md

์ถœ๋ ฅ์—๋Š” ready, hard_gate_passed, soft_gate_passed, total_score์™€ ์‹œ๋‚˜๋ฆฌ์˜ค๋ณ„ pass/fail ์š”์•ฝ์ด ํฌํ•จ๋ฉ๋‹ˆ๋‹ค.

  • Ready: hard gate๋ฅผ ํ†ต๊ณผํ–ˆ๊ณ  ์ด์  ๊ธฐ์ค€๋„ ์ถฉ์กฑํ•œ ์ƒํƒœ
  • Not Ready: hard gate๊ฐ€ ํ•˜๋‚˜๋ผ๋„ ์‹คํŒจํ•œ ์ƒํƒœ
  • hard gate: task execution, memory safety, operational resilience ๊ฐ™์€ ํ•„์ˆ˜ ์ถ•์˜ ์ตœ์†Œ ํ†ต๊ณผ ์กฐ๊ฑด
  • soft gate: ๋ณ‘ํ•ฉ์€ ๊ฐ€๋Šฅํ•˜์ง€๋งŒ ํ›„์† ๊ฐœ์„ ์ด ํ•„์š”ํ•œ ๋ณด์กฐ ์กฐ๊ฑด

intent eval

intent eval์€ ์ตœ์‹  intent session๊ณผ execution brief์˜ ๋ช…ํ™•๋„๋ฅผ ๋น ๋ฅด๊ฒŒ ์ ๊ฒ€ํ•˜๋Š” ๋ช…๋ น์ž…๋‹ˆ๋‹ค.

ralphctl --project-dir "$PWD" cp intent-eval

์‹คํ–‰ํ•˜๋ฉด ์•„๋ž˜ ์•„ํ‹ฐํŒฉํŠธ๊ฐ€ ์ƒ์„ฑ๋ฉ๋‹ˆ๋‹ค.

  • .ralph-runtime/reports/intent-eval/latest.json
  • .ralph-runtime/reports/intent-eval/latest.md
  • .ralph-runtime/reports/intent-eval/runs/<timestamp>/report.json
  • .ralph-runtime/reports/intent-eval/runs/<timestamp>/summary.md

competitive eval

competitive eval์€ internal benchmark/experiment surface์ž…๋‹ˆ๋‹ค. ์ผ๋ฐ˜ ์‚ฌ์šฉ์ž ๊ธฐ๋Šฅ์ด ์•„๋‹ˆ๋ผ, ํ•˜๋„ค์Šค ํ’ˆ์งˆ๊ณผ operator ๊ฒฝํ—˜์„ ๋‚ด๋ถ€์ ์œผ๋กœ ๋น„๊ต ์ ๊ฒ€ํ•˜๊ธฐ ์œ„ํ•œ ํ‰๊ฐ€ ํ‘œ๋ฉด์ž…๋‹ˆ๋‹ค.

ralphctl --project-dir "$PWD" cp competitive-eval

baseline mode๋Š” ํ˜„์žฌ์ฒ˜๋Ÿผ codex-ralph๋ฅผ ์ง์ ‘ ์‹คํ–‰ํ•ด ์ ์ˆ˜ํ™”ํ•ฉ๋‹ˆ๋‹ค.

ralphctl --project-dir "$PWD" cp competitive-eval

manual adapter path๋Š” ์™ธ๋ถ€ ํ•˜๋„ค์Šค๋ฅผ ์ง์ ‘ ์‹คํ–‰ํ•˜๋Š” ๊ธฐ๋Šฅ์ด ์•„๋‹ˆ๋ผ, internal benchmark ์‹คํ—˜์„ ์œ„ํ•œ guide/evidence ํ‘œ๋ฉด์ž…๋‹ˆ๋‹ค.

bundle emit:

ralphctl --project-dir "$PWD" cp competitive-eval \
  --emit-manual-bundle \
  --harness <internal-benchmark-harness> \
  --scenario <scenario>

manual parse:

ralphctl --project-dir "$PWD" cp competitive-eval \
  --parse-manual-bundle .ralph-runtime/reports/competitive-eval/manual-bundles/<internal-benchmark-harness>/<scenario>

์‹คํ–‰ํ•˜๋ฉด ์•„๋ž˜ ์•„ํ‹ฐํŒฉํŠธ๊ฐ€ ์ƒ์„ฑ๋ฉ๋‹ˆ๋‹ค.

  • .ralph-runtime/reports/competitive-eval/latest.json
  • .ralph-runtime/reports/competitive-eval/latest.md
  • .ralph-runtime/reports/competitive-eval/runs/<timestamp>/result.json
  • .ralph-runtime/reports/competitive-eval/runs/<timestamp>/summary.md
  • .ralph-runtime/reports/competitive-eval/manual-bundles/<harness>/<scenario>/scenario.json
  • .ralph-runtime/reports/competitive-eval/manual-bundles/<harness>/<scenario>/instructions.md
  • .ralph-runtime/reports/competitive-eval/manual-bundles/<harness>/<scenario>/evidence-template.json

manual bundle๊ณผ parse ๊ฒฝ๋กœ๋Š” internal benchmark harness๋ฅผ ์ƒ์ •ํ•˜๋ฉฐ, ์ผ๋ฐ˜ ์ œํ’ˆ ๊ธฐ๋Šฅ์œผ๋กœ ๋ฌธ์„œํ™”ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

ํ˜„์žฌ semantics๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

  • baseline mode
    • run_mode=baseline
    • harness_kind=command-driven
  • manual parse mode
    • run_mode=manual-parse
    • harness_kind=manual
  • manual interpreted mode
    • run_mode=manual-parse
    • measurement_mode=manual-interpreted
    • harness-specific parser๊ฐ€ competitor ๊ฒฐ๊ณผ๋ฅผ ํ•ด์„ํ•œ ์ƒํƒœ

manual parse๋Š” generic parser ๋‹จ๊ณ„์—์„œ๋Š” competitor ์„ฑ๊ณต/์‹คํŒจ๋ฅผ ์ตœ์ข… ํŒ์ •ํ•˜์ง€ ์•Š๊ณ , evidence bundle์„ neutral provenance๋กœ ์ •๋ฆฌํ•ฉ๋‹ˆ๋‹ค. ์ดํ›„ harness-specific parser/adapter๊ฐ€ ๋ถ™์œผ๋ฉด ๊ฐ™์€ report ํŒŒ์ดํ”„๋ผ์ธ ์•ˆ์—์„œ scorecard ํ•ด์„์„ ๋” ์ •๊ตํ•˜๊ฒŒ ์˜ฌ๋ฆด ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

1์ฐจ scenario pack:

  • one_line_feature
  • blocked_then_resume
  • telegram_control_roundtrip
  • integration_boundary_change

operator playbook:

  • pre-run
    • clean workspace ํ™•์ธ
    • ๊ฐ™์€ scenario contract์™€ success ๊ธฐ์ค€ ํ™•์ธ
  • during-run
    • clarification, blocked, remote control ๊ฐœ์ž… ์ง€์ ์„ notes.txt์— ๊ธฐ๋ก
    • competitor-specific deviation์ด ์žˆ์œผ๋ฉด ํ•จ๊ป˜ ๋‚จ๊น€
  • post-run
    • stdout.log, stderr.log, notes.txt ์ •๋ฆฌ
    • ๊ฐ€๋Šฅํ•˜๋ฉด summary.json์— timing, operator_count๋ฅผ ํ•จ๊ป˜ ๋‚จ๊น€
    • listed artifact๋ฅผ bundle์— ๊ฐ™์ด ๋ณด๊ด€

evidence discipline:

  • ํ•„์ˆ˜
    • notes.txt
    • transcript์— ์ค€ํ•˜๋Š” text evidence
  • ๊ถŒ์žฅ
    • summary.json
    • harness-specific artifact
  • ์›์น™
    • generic parser๋Š” evidence๋ฅผ neutral provenance๋กœ๋งŒ ์ฝ์Œ
    • scorecard interpretation์€ harness-specific parser๊ฐ€ ๋ถ™์„ ๋•Œ ์˜ฌ๋ฆผ

completion regression suite

intent-driven autonomous loop์˜ ์™„์„ฑ๋„๋Š” ์•„๋ž˜ targeted regression ๋ฌถ์Œ์œผ๋กœ ์ ๊ฒ€ํ•ฉ๋‹ˆ๋‹ค.

env GOCACHE=/tmp/ralph-gocache CGO_ENABLED=0 go test ./internal/ralph -run 'Test(Intent(FeatureFlowAutoRunsUntilValidateBoundary|IntegrationFlowAutoRunsUntilFinalVerifyBoundary|OpsFlowAutoRunsUntilHealthVerifyBoundary)|RunCompetitiveEvalBaselineProducesStableMetrics|WriteCompetitiveEvalReportsWritesLatestAndTimestampedRuns)' -count=1

env GOCACHE=/tmp/ralph-gocache CGO_ENABLED=0 go test ./cmd/ralphctl -run 'Test(TelegramBlocked(ReplyRefreshesGraphAndAutoRuns|SummaryExplainsResumePoint)|RunControlPlaneCompetitiveEval(ShowsBaselineSummary|JSONMatchesReport)|PrintIntakeSessionStatusShowsCockpitSummary|TelegramTaskStatusIncludesCockpitNarrative)' -count=1

์œ„ ๋‘ ๋ช…๋ น์€ ๋‹ค์Œ์„ ํ•จ๊ป˜ ๊ฒ€์ฆํ•ฉ๋‹ˆ๋‹ค.

  • workload๋ณ„ autonomous halt semantics
  • Telegram blocked -> reply -> refreshed graph -> auto-run resume
  • competitive baseline metrics/report consistency
  • operator cockpit summary์™€ CLI/Telegram surface

2-1) --execute-with-codex์˜ ํ˜„์žฌ ์‹คํ–‰ ๋ชจ๋ธ

advanced cp run --execute-with-codex๋Š” ์ด์ œ ๋‹จ์ˆœํ•œ one-shot codex exec ํ˜ธ์ถœ์ด ์•„๋‹ˆ๋ผ, Ralph Runtime ์œ„์— ์–นํžŒ session harness๋ฅผ ํ†ตํ•ด ์‹คํ–‰๋ฉ๋‹ˆ๋‹ค.

ํ•ต์‹ฌ ๊ตฌ์กฐ:

  • control plane: lease, heartbeat, recover, metrics, retry budget
  • agent harness: coordinator-owned worker session orchestration
  • worker session: ๊ฐ™์€ task์— ๋Œ€ํ•ด start/continue turn์„ ๊ฐ€์งˆ ์ˆ˜ ์žˆ๋Š” ์‹คํ–‰ ๋‹จ์œ„
  • verification: worker ์™„๋ฃŒ ์„ ์–ธ๊ณผ ๋ถ„๋ฆฌ๋œ ๋…๋ฆฝ ๊ฒ€์ฆ ๋‹จ๊ณ„

ํ˜„์žฌ continuity๋Š” persistent transport๊ฐ€ ์•„๋‹ˆ๋ผ codex exec replay ๋ฐฉ์‹์œผ๋กœ ์œ ์ง€๋ฉ๋‹ˆ๋‹ค.

  • ์™ธ๋ถ€ ๊ณ„์•ฝ์€ sessionful
  • ๋‚ด๋ถ€ transport๋Š” ์—ฌ์ „ํžˆ codex exec
  • turn continuity๋Š” TaskMemory ์žฌ๊ตฌ์„ฑ์œผ๋กœ ์œ ์ง€
  • raw transcript ์ „์ฒด๋ฅผ ์žฌ์ฃผ์ž…ํ•˜์ง€ ์•Š์Œ

2-2) ๋ฉ”๋ชจ๋ฆฌ ๊ณ„์ธต

--execute-with-codex ๊ฒฝ๋กœ๋Š” ์žฅ๊ธฐ ์‹คํ–‰์—์„œ drift๋ฅผ ์ค„์ด๊ธฐ ์œ„ํ•ด ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ๋ถ„๋ฆฌํ•ฉ๋‹ˆ๋‹ค.

  • TaskMemory: ํ˜„์žฌ task์˜ canonical memory
    • accepted_facts
    • open_questions
    • artifacts
    • next_ask
  • ExecutionJournal: recover/audit์šฉ ์‹คํ–‰ ๊ธฐ๋ก
  • ProjectMemory: .ralph/project-memory/MEMORY.md + topic files
  • UserMemory: .ralph/user-memory/MEMORY.md + topic files

์›์น™:

  • memory๋Š” truth store๊ฐ€ ์•„๋‹ˆ๋ผ skeptical hint layer์ž…๋‹ˆ๋‹ค.
  • coordinator๊ฐ€ ์Šน์ธํ•œ ์‚ฌ์‹ค๋งŒ TaskMemory์™€ ์žฅ๊ธฐ memory๋กœ ์Šน๊ฒฉํ•ฉ๋‹ˆ๋‹ค.
  • raw transcript๋Š” ์žฅ๊ธฐ memory๋กœ ๋ณต์ œํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.
  • ExecutionJournal์€ prompt ๊ธฐ๋ณธ payload๊ฐ€ ์•„๋‹ˆ๋ผ ํ•„์š” ์‹œ ์กฐํšŒ์šฉ์ž…๋‹ˆ๋‹ค.

2-3) continuation๊ณผ handoff

ํ˜„์žฌ harness๋Š” coordinator-owned continuation ๋ชจ๋ธ์„ ๋”ฐ๋ฆ…๋‹ˆ๋‹ค.

  • coordinator๋งŒ ๊ฐ™์€ worker session์— ํ›„์† ๋ฉ”์‹œ์ง€๋ฅผ ๋ณด๋ƒ…๋‹ˆ๋‹ค.
  • worker ๊ฒฐ๊ณผ๋Š” structured result๋กœ ์ˆ˜์ง‘๋ฉ๋‹ˆ๋‹ค.
  • structured result๋Š” ๊ธฐ์กด handoff schema๋กœ bridge ๋ฉ๋‹ˆ๋‹ค.
  • handoff validator์™€ role-specific schema๋Š” ์œ ์ง€๋ฉ๋‹ˆ๋‹ค.

์ฆ‰, manager/planner/developer/qa ์—ญํ•  ์ฒด๊ณ„๋Š” ๊ทธ๋Œ€๋กœ ์œ ์ง€ํ•˜๋ฉด์„œ๋„, ์‹ค์ œ Codex ์‹คํ–‰์€ session -> continuation -> verify ํ๋ฆ„์œผ๋กœ ์šด์˜๋ฉ๋‹ˆ๋‹ค.

2-4) ํ˜„์žฌ ๋น„๋ชฉํ‘œ

์ด๋ฒˆ ๋‹จ๊ณ„์˜ ๋ช…์‹œ์  ๋น„๋ชฉํ‘œ:

  • persistent Codex session transport ๋„์ž…
  • ๊ธฐ์กด loop ๊ฒฝ๋กœ ์žฌ๋„์ž…
  • long-term memory๋ฅผ SQLite๋กœ ์ด์ „
  • execute_cmd ์ „์šฉ task ๊ฒฝ๋กœ ์žฌ์ž‘์„ฑ

3) Runtime artifacts and operational checks

ralphctl --project-dir "$PWD" eval
ralphctl --project-dir "$PWD" cp intent-eval
ralphctl --project-dir "$PWD" cp competitive-eval
ralphctl --project-dir "$PWD" cp doctor --repair

์ฃผ์š” ์‚ฐ์ถœ๋ฌผ์€ .ralph-runtime/reports/ ์•„๋ž˜์— ์ €์žฅ๋ฉ๋‹ˆ๋‹ค.

  • .ralph-runtime/reports/eval/latest.json
  • .ralph-runtime/reports/eval/latest.md
  • .ralph-runtime/reports/competitive-eval/latest.json
  • .ralph-runtime/reports/competitive-eval/latest.md
  • .ralph-runtime/autonomous-depth/

๋ ˆ๊ฑฐ์‹œ dual-runtime helper script๋Š” ์ œ๊ฑฐ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. Runtime ์šด์˜์€ bootstrap, intake, work, status, eval, doctor ์ค‘์‹ฌ์œผ๋กœ ์ง„ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

์ฐธ๊ณ :

  • ledger(cp_event_ledger, cp_learning_ledger)๊ฐ€ ๋น„์–ด ์žˆ๊ณ  ๊ธฐ์กด cp_events/cp_learnings ๋ฐ์ดํ„ฐ๊ฐ€ ์žˆ์œผ๋ฉด, ์Šคํ‚ค๋งˆ ๋ณด์žฅ ๋‹จ๊ณ„์—์„œ ์ž๋™ backfill ๋ฉ๋‹ˆ๋‹ค.

์ƒํƒœ ์กฐํšŒ API ์„œ๋ฒ„:

ralphctl --project-dir "$PWD" cp api --listen 127.0.0.1:8787
curl -s http://127.0.0.1:8787/health
curl -s http://127.0.0.1:8787/runtime/status
curl -s http://127.0.0.1:8787/runtime/metrics
curl -s "http://127.0.0.1:8787/runtime/metrics?with_baseline=true"
curl -s http://127.0.0.1:8787/runtime/metrics/summary
curl -s http://127.0.0.1:8787/runtime/doctor
curl -s "http://127.0.0.1:8787/runtime/events?limit=50"
curl -N http://127.0.0.1:8787/runtime/events/stream

advanced

1) ์›๊ฒฉ/์žฅ์‹œ๊ฐ„ ์‹คํ–‰

์„œ๋น„์Šค ๋“ฑ๋ก(systemd/launchd):

ralphctl --project-dir "$PWD" service install --start
ralphctl --project-dir "$PWD" service status
ralphctl --project-dir "$PWD" service uninstall

2) ๋ฉ€ํ‹ฐ ํ”„๋กœ์ ํŠธ ์˜ค์ผ€์ŠคํŠธ๋ ˆ์ด์…˜

๋Œ€ํ™”ํ˜•(๊ถŒ์žฅ):

ralphctl fleet

๋ช…๋ นํ˜• ์˜ˆ์‹œ:

ralphctl fleet register --id wallet --project-dir <wallet-project-dir> --plugin universal-default --prd PRD.md
ralphctl fleet start --all
ralphctl fleet status --all
ralphctl fleet stop --all

3) Telegram ์ฑ„๋„ (์„ ํƒ)

ํ…”๋ ˆ๊ทธ๋žจ์€ ํ•„์ˆ˜ ์„ค์ •์ด ์•„๋‹™๋‹ˆ๋‹ค.

ํ•„์š”ํ•œ ๊ฐ’:

  • bot token
  • chat-ids
  • user-ids (ํŠนํžˆ ๊ทธ๋ฃน ์ œ์–ด ์‹œ ๊ถŒ์žฅ/ํ•„์ˆ˜)

๊ฐ’ ์ˆ˜์ง‘ ๋ฐฉ๋ฒ• (์›น):

  1. @BotFather์—์„œ ๋ด‡ ์ƒ์„ฑ ํ›„ ํ† ํฐ ๋ฐœ๊ธ‰ (/newbot).
  2. ๋ด‡์„ ์‚ฌ์šฉํ•  ๋Œ€ํ™”๋ฐฉ์— ์ถ”๊ฐ€ํ•˜๊ณ  ๋ฉ”์‹œ์ง€ 1๊ฐœ ์ „์†ก.
  3. ๋ธŒ๋ผ์šฐ์ €์—์„œ ์•„๋ž˜ URL ์—ด๊ธฐ:
https://api.telegram.org/bot<bot-token>/getUpdates

JSON์—์„œ ๋‹ค์Œ ๊ฐ’์„ ๋ณต์‚ฌ:

  • message.chat.id -> chat-ids
  • message.from.id -> user-ids
  • ๊ทธ๋ฃน/์Šˆํผ๊ทธ๋ฃน chat.id๋Š” ์Œ์ˆ˜(๋ณดํ†ต -100...)
  • ๊ฒฐ๊ณผ๊ฐ€ ๋น„์–ด ์žˆ์œผ๋ฉด ๋ด‡/๊ทธ๋ฃน์— ๋ฉ”์‹œ์ง€๋ฅผ ํ•œ ๋ฒˆ ๋” ๋ณด๋‚ด๊ณ  ์ƒˆ๋กœ๊ณ ์นจ

์›๊ฒฉ ๋จธ์‹  ์„ธํŒ… ์‹œ์—๋Š” ๋กœ์ปฌ ๋ธŒ๋ผ์šฐ์ €์—์„œ ๊ฐ’์„ ํ™•์ธํ•œ ๋’ค, ์„œ๋ฒ„์—์„œ ์•„๋ž˜์ฒ˜๋Ÿผ ์ž…๋ ฅํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค:

ralphctl --project-dir "$PWD" telegram setup --non-interactive \
  --token "<bot-token>" \
  --chat-ids "<chat-id>" \
  --user-ids "<user-id>" \
  --allow-control=false \
  --notify=true \
  --notify-scope auto \
  --command-timeout-sec 300 \
  --command-concurrency 4

๊ถŒ์žฅ:

ralphctl --project-dir "$PWD" telegram setup
ralphctl --project-dir "$PWD" telegram run
ralphctl --project-dir "$PWD" telegram status
ralphctl --project-dir "$PWD" telegram tail
ralphctl --project-dir "$PWD" telegram stop
  • telegram run์€ ๊ธฐ๋ณธ์ ์œผ๋กœ ๋ฐฑ๊ทธ๋ผ์šด๋“œ daemon์œผ๋กœ ์‹คํ–‰๋ฉ๋‹ˆ๋‹ค.
  • ํ„ฐ๋ฏธ๋„์„ ์ข…๋ฃŒํ•ด๋„ ๊ณ„์† ๋™์ž‘ํ•ฉ๋‹ˆ๋‹ค.
  • ํฌ๊ทธ๋ผ์šด๋“œ๋กœ ์‹คํ–‰ํ•˜๋ ค๋ฉด telegram run --foreground๋ฅผ ์‚ฌ์šฉํ•˜์„ธ์š”.

๋น„๋Œ€ํ™”ํ˜•:

ralphctl --project-dir "$PWD" telegram setup --non-interactive \
  --token "<bot-token>" \
  --chat-ids "<allowed-chat-id-1>,<allowed-chat-id-2>" \
  --user-ids "<allowed-user-id-1>,<allowed-user-id-2>" \
  --allow-control=false \
  --notify=true \
  --notify-scope auto \
  --command-timeout-sec 300 \
  --command-concurrency 4

๋ณด์•ˆ ๊ทœ์น™:

  • allow-control=true + ๊ทธ๋ฃน/์Šˆํผ๊ทธ๋ฃน chat id(์Œ์ˆ˜) ์กฐํ•ฉ์—์„œ๋Š” user-ids๊ฐ€ ํ•„์ˆ˜์ž…๋‹ˆ๋‹ค.
  • ์ œ์–ด ๋ช…๋ น์„ ์—ด ๋•Œ๋Š” user-ids ์„ค์ •์„ ๊ถŒ์žฅํ•ฉ๋‹ˆ๋‹ค.
  • ๊ธฐ๋ณธ ์ •์ฑ…์€ 1 bot = 1 project ์ž…๋‹ˆ๋‹ค. ๊ฐ™์€ bot token์„ ๋‹ค๋ฅธ ํ”„๋กœ์ ํŠธ์—์„œ ์‹คํ–‰ํ•˜๋ฉด ์ฐจ๋‹จ๋ฉ๋‹ˆ๋‹ค.
  • bot token์„ ๋‹ค๋ฅธ ํ”„๋กœ์ ํŠธ๋กœ ์ด๋™ํ•˜๋ ค๋ฉด: telegram run --rebind-bot
  • telegram offset์€ ํ”„๋กœ์ ํŠธ๋ณ„๋กœ ์ž๋™ ๋ถ„๋ฆฌ๋˜์–ด ~/.ralph-control/telegram-offsets/*.offset์— ์ €์žฅ๋ฉ๋‹ˆ๋‹ค.

์ฃผ์š” ๋ช…๋ น:

  • /status [all|<project_id>]
  • /doctor [all|<project_id>]
  • /fleet [all|<project_id>]
  • ํ‰๋ฌธ ๋ฉ”์‹œ์ง€: ํ”„๋กœ์ ํŠธ ์ปจํ…์ŠคํŠธ Codex ๋Œ€ํ™” (์˜ˆ: ๊ฒฐ์ œ PRD ์ดˆ์•ˆ ๋งŒ๋“ค์–ด์ค˜)
  • /chat <message>: Codex ๋Œ€ํ™”๋ฅผ ๋ช…์‹œ์ ์œผ๋กœ ์‹คํ–‰
  • /chat status, /chat reset: Codex ๋Œ€ํ™” ์ปจํ…์ŠคํŠธ ํ™•์ธ/์ดˆ๊ธฐํ™”
  • (--allow-control์ผ ๋•Œ) /start|/stop|/restart|/doctor_repair|/recover|/retry_blocked [all|<project_id>]
  • /doctor_repair๋Š” ํ˜„์žฌ ํ”„๋กœ์ ํŠธ ๊ธฐ์ค€์œผ๋กœ repair + recover + codex blocked ์žฌํ์ž‰ + ํ•„์š” ์‹œ circuit reset + daemon ์ž๋™ ์‹œ์ž‘๊นŒ์ง€ ํ•œ ๋ฒˆ์— ์ˆ˜ํ–‰ํ•ฉ๋‹ˆ๋‹ค.
  • (--allow-control์ผ ๋•Œ) /new [manager|planner|developer|qa] <title> (quick issue create, role ์ƒ๋žต ์‹œ developer)
  • (--allow-control์ผ ๋•Œ) /task <์ž์—ฐ์–ด ์š”์ฒญ> (canonical intake request; Codex๊ฐ€ role/title/objective/acceptance๋ฅผ ๊ตฌ์กฐํ™”ํ•ด ์ด์Šˆ ์ƒ์„ฑ)
  • (--allow-control์ผ ๋•Œ) /prd help (๋Œ€ํ™”ํ˜• PRD wizard + clarity refine)

PRD wizard ๋น ๋ฅธ ํ๋ฆ„:

  1. /prd start
  2. /prd refine (๋ถ€์กฑํ•œ ์ปจํ…์ŠคํŠธ๋ฅผ ์งˆ๋ฌธ์œผ๋กœ ์ฑ„์›€)
  3. ์Šคํ† ๋ฆฌ ์ž…๋ ฅ: title -> description -> role -> priority
  4. /prd score ๋˜๋Š” /prd preview
  5. /prd apply (์ ์ˆ˜ ๋ฏธ๋‹ฌ์ด๋ฉด /prd refine ์œ ๋„)

๋Œ€ํ™”ํ˜• ์ž…๋ ฅ ํŒ:

  • refine ์ค‘์— ์งˆ๋ฌธํ˜• ์ž…๋ ฅ(ํฌํ•จ ๋ฒ”์œ„๊ฐ€ ๋ญ์•ผ?)์„ ๋ณด๋‚ด๋ฉด ๋‹จ๊ณ„๋ฅผ ์œ ์ง€ํ•œ ์ฑ„ ์„ค๋ช…์„ ๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค.
  • ์ถ”์ฒœ ์š”์ฒญ(์ œ์™ธ ๋ฒ”์œ„ ์ถ”์ฒœํ•ด์ค˜)์„ ๋ณด๋‚ด๋ฉด ํ˜„์žฌ ๋‹จ๊ณ„ ๊ธฐ์ค€ ์ถ”์ฒœ์•ˆ์„ ๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค.
  • refine ์ž…๋ ฅ ์˜๋„(๋‹ต๋ณ€/์„ค๋ช…/์ถ”์ฒœ)๋Š” Codex๊ฐ€ ์šฐ์„  ํŒ๋‹จํ•ฉ๋‹ˆ๋‹ค.
  • /prd score, /prd apply์˜ ๊ฒŒ์ดํŠธ ์ ์ˆ˜๋Š” Codex ์ ์ˆ˜๋ฅผ ์šฐ์„  ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค(๋ถˆ๊ฐ€ ์‹œ heuristic ํด๋ฐฑ).
  • PRD ์„ธ์…˜์ด ํ™œ์„ฑํ™”๋œ ์ฑ„ํŒ…์—์„œ๋Š” ํ‰๋ฌธ ์ž…๋ ฅ์ด ์šฐ์„  /prd ์„ธ์…˜ ์ž…๋ ฅ์œผ๋กœ ์ฒ˜๋ฆฌ๋ฉ๋‹ˆ๋‹ค.

4) ์‹คํ–‰ ์ค‘ graceful ์„ค์ • ๋ณ€๊ฒฝ

profile.local.yaml์„ ์ˆ˜์ •ํ•˜๋ฉด ์‹คํ–‰ ์ค‘์ธ ๋ฃจํ”„๊ฐ€ ์ž๋™์œผ๋กœ ์žฌ๋กœ๋”ฉํ•ฉ๋‹ˆ๋‹ค(๋‹ค์Œ loop๋ถ€ํ„ฐ ๋ฐ˜์˜).

์ž์ฃผ ์“ฐ๋Š” ๊ฐ’:

codex_model: auto
codex_home: .codex-home
codex_exec_timeout_sec: 900
codex_retry_max_attempts: 3
codex_retry_backoff_sec: 10
codex_require_exit_signal: true
codex_exit_signal: "EXIT_SIGNAL: DONE"
codex_context_summary_enabled: true
codex_context_summary_lines: 8
codex_circuit_breaker_enabled: true
codex_circuit_breaker_failures: 3
codex_circuit_breaker_cooldown_sec: 120
idle_sleep_sec: 20
inprogress_watchdog_enabled: true
inprogress_watchdog_stale_sec: 1800
inprogress_watchdog_scan_loops: 1

codex_home ๊ธฐ๋ณธ๊ฐ’์€ ํ”„๋กœ์ ํŠธ ๋กœ์ปฌ ./.codex-home์ž…๋‹ˆ๋‹ค. ๋กœ๊ทธ์ธ/์„ค์ • ํŒŒ์ผ(auth.json, config.toml)์€ ํ•„์š” ์‹œ ์ž๋™ ์‹œ๋“œ๋ฉ๋‹ˆ๋‹ค.

๋ฐ˜์˜ ํ™•์ธ:

./ralph status

last_profile_reload_at, profile_reload_count๊ฐ€ ์—…๋ฐ์ดํŠธ๋ฉ๋‹ˆ๋‹ค.

Codex ๋„คํŠธ์›Œํฌ ๋ฌธ์ œ๊ฐ€ ์˜์‹ฌ๋˜๋ฉด:

./ralph doctor

network:dns:chatgpt.com, network:codex-api, codex-home ์ฒดํฌ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

์ฃผ์˜:

  • supervisor_enabled, supervisor_restart_delay_sec๋Š” legacy compatibility ์„ค์ •์ด๋ฉฐ, ๋ณ€๊ฒฝ ์‹œ primary daemon ์žฌ์‹œ์ž‘ ํ›„ ๋ฐ˜์˜๋ฉ๋‹ˆ๋‹ค.

5) ๋ฐ”์ด๋„ˆ๋ฆฌ ์—…๋ฐ์ดํŠธ ํ›„ ์ผ๊ด„ ๋ฐ˜์˜

์ƒˆ ralphctl ๋ฐ”์ด๋„ˆ๋ฆฌ ์„ค์น˜ ํ›„:

ralphctl reload
  • ์—ฐ๊ฒฐ๋œ ํ”„๋กœ์ ํŠธ(ํ˜„์žฌ ํ”„๋กœ์ ํŠธ + fleet ๋“ฑ๋ก ํ”„๋กœ์ ํŠธ)์˜ ./ralph wrapper๋ฅผ ์ƒˆ ๋ฐ”์ด๋„ˆ๋ฆฌ๋กœ ๊ฐฑ์‹ 
  • ๊ธฐ์กด์— ์‹คํ–‰ ์ค‘์ด๋˜ primary daemon/telegram daemon๋งŒ ์ž๋™ ์žฌ์‹œ์ž‘
  • ํ˜„์žฌ ํ”„๋กœ์ ํŠธ๋งŒ ๋ฐ˜์˜ํ•˜๋ ค๋ฉด: ralphctl reload --current-only

License

MIT (LICENSE)

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors