feat: add Evaluation + Optimization closed-loop pipeline (#91)#194

Open

DNKYr wants to merge 16 commits into

trpc-group:mainfrom

DNKYr:feature/eval-optimize-loop

Commits on Jul 16, 2026

feat(eval_optimize_loop): add project scaffolding and data models
DNKYr
committed
feat(eval_optimize_loop): add rule-based failure attribution
DNKYr
committed
fix(eval_optimize_loop): add 2 tests for 100% coverage + remove unused imports
DNKYr
committed
feat(eval_optimize_loop): add per-case delta computation
DNKYr
committed
feat(eval_optimize_loop): add configurable acceptance gate
DNKYr
committed
feat(eval_optimize_loop): add JSON + Markdown report generation
DNKYr
committed
fix(eval_optimize_loop): improve test coverage to 100% for reporting, remove unused import
DNKYr
committed
feat(eval_optimize_loop): add EvalOptimizePipeline orchestrator
DNKYr
committed
fix(eval_optimize_loop): address review feedback for pipeline orchestrator
DNKYr
committed
feat(eval_optimize_loop): add sample evalsets, prompts, and pipeline configs
DNKYr
committed
feat(eval_optimize_loop): add CLI entry point and README
DNKYr
committed
test(eval_optimize_loop): add integration tests for trace mode
DNKYr
committed
feat(eval_optimize_loop): finalize pipeline with path resolution, example output, gitignore
DNKYr
committed
fix(eval_optimize_loop): address code review feedback
DNKYr
committed
fix(eval_optimize_loop): address second code review round
DNKYr
committed
fix(eval_optimize_loop): address adversarial review findings
DNKYr
committed