v3 Stage A — route per model, so a second one can be real - #14
Merged
Conversation
v1's KimodoGenerator answered three questions at once — where a model lives, which model a request goes to, and whether that model is real — and answered the last one twice: a literal list of three dicts in capabilities(), and `if model != "kimodo"` as the first statement of generate(). Both were correct exactly once, for exactly one model. Neither could express "SnapMoGen is real and Language of Motion is not". providers.py owns where a model lives; generators.py owns which provider a request goes to. - WorkerProvider — one model behind HTTP. Local and remote are the SAME class: a Compose container here and a GPU in another building differ by a URL and an optional bearer token, and by nothing else. The memory ceiling on one laptop is a fact about a laptop, not something the architecture should be shaped around. - FixtureProvider — a model that is not real yet, so "not real yet" is configuration rather than a branch in the router. - RouterGenerator — a dict of providers. capabilities() is a loop over the registry, so it is truthful by construction rather than by remembering to edit it. BODYPROMPT_MODEL_<NAME> is a URL or `fixture`, with optional _TOKEN, _HOSTING and _CONCURRENCY. Every known model always gets a provider, so /health never has a silent hole, and a fourth model needs no code change. BODYPROMPT_BACKEND is translated into the new form, so no existing doc, .env or command breaks; being explicit wins over it. model_version now comes from the worker. v1 declared it as a class constant on the service side, which the service cannot actually know across an HTTP boundary and certainly will not know for a remote worker. Same rule as denoising_steps and multi_prompt — record what was done, never what was asked — extended to the one field that had been exempt. Concurrency belongs to hosting, not the UI. The triptych fires all three requests at once, which is right against three remote endpoints and would exhaust one local GPU; the browser has no business knowing where the models are today, so the router holds a semaphore per provider and a local worker gets one. The frontend does not change. An unknown model is now 422, not a hashed fixture: it is a bad request, and 503 would tell the caller to retry something that can never work. 23 service tests pass. The 13 v1 tests keep their assertions verbatim, though five moved class — the provenance rules they pin now live in WorkerProvider. The plan claimed they would be untouched; that was wrong, and forcing it would have meant a worse design. The remote path is proven against a real socket rather than asserted: a fake worker on 127.0.0.1 checks the bearer token, the payload, and that the worker's own reported version reaches provenance. Live on the 5080: the legacy .env still gets real Kimodo, and so does BODYPROMPT_BACKEND=stub with only BODYPROMPT_MODEL_KIMODO set — which is the whole claim of the stage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
.env.example and the README's Kimodo section still taught BODYPROMPT_BACKEND=kimodo as the only way. Not wrong — the legacy translation is tested and works — but someone reading either would never discover per-model routing, which is the whole point of the change. compose.yaml and docs/usage.md had been updated; these two had not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First of five v3 stages. No new model — the point of the stage is that a second one becomes possible.
What v1 hard-coded
KimodoGeneratoranswered three questions at once — where does this model live, which model does this request go to, is this model real — and answered the last one twice:Both were correct exactly once, for exactly one model. Neither could express "SnapMoGen is real and Language of Motion is not."
The reframing behind it
The obvious v3 is "fit three models onto one 16 GB laptop". That is the wrong architecture, and the constraint that suggests it is local. Remote hosting removes the parallelism problem entirely, and most research sessions use one model anyway — the triptych is the exception, not the rule.
So v3 separates three concerns, and this stage does the first two:
What changed
New
service/app/providers.pyowns hosting;generators.pyowns routing.WorkerProvider— one model behind HTTP. Local and remote are the same class. A Compose container on this machine and a GPU in another building differ by a URL and an optional bearer token, and by nothing else.FixtureProvider— a model that is not real yet, so "not real yet" is configuration rather than a branch inside the router.RouterGenerator— adict[str, ModelProvider].capabilities()is a loop over the registry, so it is truthful by construction rather than by remembering to edit it.Optional
_TOKEN,_HOSTING,_CONCURRENCY. Every known model always gets a provider, so/healthnever has a silent hole in it, and a fourth model can be added by environment alone.BODYPROMPT_BACKENDis translated into the new form so no existing document,.envor command breaks — being explicit wins over it.Two changes beyond the plan, both deliberate
model_versionnow comes from the worker. v1 declaredmodel_version = "Kimodo-SOMA-RP-v1.1"as a class constant on the service side. The service cannot know which checkpoint is loaded across an HTTP boundary, and certainly will not know for a remote worker — so it asks, and records what came back. Same rule asdenoising_stepsandmulti_prompt(record what was done, never what was asked), extended to the one field that had been exempt from it.An unknown model is 422, not a hashed fixture. Previously any model string was hashed into a fixture. A model nothing is configured to serve is a bad request; 503 would tell the caller to retry something that can never work.
Concurrency belongs to hosting, not the UI
The triptych fires all three requests at once (
Promise.all), which is right against three remote endpoints and would exhaust one local GPU. The browser has no business knowing where the models happen to live today, so the limit is enforced where the answer is known: a provider declares its concurrency, the router holds a semaphore per provider, a local worker gets 1. The frontend does not change.Verification
23 service tests pass, plus 21 worker and 23 frontend;
tscclean.The 13 v1 tests keep their assertions verbatim — though five of them moved class, since the provenance rules they pin now live in
WorkerProvider. The plan claimed the tests would be untouched. That was wrong, and forcing it would have meant keeping a shim and a worse design.Ten new tests: the registry built from the environment; a model becoming real by configuration alone; hosting reported per model and overridable; an unknown model rejected as a bad request; legacy translation, and explicit-beats-legacy; the router choosing a seed so the motion can name it; and the local gate actually serialising four concurrent generations down to one at a time.
The remote path is proven against a real socket, not asserted — a fake worker on
127.0.0.1checks that the bearer token arrives, that the payload is what was asked for, and that the worker's own reported model version reaches provenance.Live, on the RTX 5080:
BODYPROMPT_BACKEND=kimodo(the legacy form, straight from the existing.env) →kimodoresolves to a local worker,model_versionreadsKimodo-SOMA-RP-v1.1from the worker, and a 3-second prompt returns 90 frames in 3.2 s at 50 steps withsource: kimodo.BODYPROMPT_BACKEND=stubwith onlyBODYPROMPT_MODEL_KIMODOset → the same real Kimodo. A model becomes real by configuration alone, which is the whole claim of the stage.snapmogenstill returnsstub: true; an unknown model returns 422.Still fixtures
SnapMoGen and Language of Motion are unchanged by this PR and remain hand-authored fixtures.
docs/v0-stub.mdkeeps its entries for both, and the README Status section still says so. Nothing here makes the triptych a model comparison — it makes one possible.docs/v3-models.mdis the new v3 log: the verified facts about both models (licences, skeletons, fps, tested environments) and the Blackwell/torch risk that makes Stage B open with a throwaway spike rather than a Dockerfile.🤖 Generated with Claude Code