Skip to content

v3 Stage A — route per model, so a second one can be real - #14

Merged
WWStoryMode merged 2 commits into
mainfrom
feat/v3-providers
Aug 24, 2026
Merged

v3 Stage A — route per model, so a second one can be real#14
WWStoryMode merged 2 commits into
mainfrom
feat/v3-providers

Conversation

@WWStoryMode

Copy link
Copy Markdown
Owner

First of five v3 stages. No new model — the point of the stage is that a second one becomes possible.

What v1 hard-coded

KimodoGenerator answered three questions at once — where does this model live, which model does this request go to, is this model real — and answered the last one twice:

def capabilities(self):
    return [{"model": "kimodo", "source": "kimodo", ...},
            {"model": "snapmogen", "source": "fixture", ...},
            {"model": "language-of-motion", "source": "fixture", ...}]

def generate(self, model, ...):
    if model != "kimodo":
        return self._stub.generate(...)

Both were correct exactly once, for exactly one model. Neither could express "SnapMoGen is real and Language of Motion is not."

The reframing behind it

The obvious v3 is "fit three models onto one 16 GB laptop". That is the wrong architecture, and the constraint that suggests it is local. Remote hosting removes the parallelism problem entirely, and most research sessions use one model anyway — the triptych is the exception, not the rule.

So v3 separates three concerns, and this stage does the first two:

Concern Question
Hosting where does this model live — local container, or remote endpoint
Routing which model does this request go to — the dropdown, the triptych
Remembering what do we keep — Stage C

What changed

New service/app/providers.py owns hosting; generators.py owns routing.

  • WorkerProvider — one model behind HTTP. Local and remote are the same class. A Compose container on this machine and a GPU in another building differ by a URL and an optional bearer token, and by nothing else.
  • FixtureProvider — a model that is not real yet, so "not real yet" is configuration rather than a branch inside the router.
  • RouterGenerator — a dict[str, ModelProvider]. capabilities() is a loop over the registry, so it is truthful by construction rather than by remembering to edit it.
BODYPROMPT_MODEL_KIMODO=http://kimodo-worker:8010          # local worker
BODYPROMPT_MODEL_SNAPMOGEN=https://gpu.example.com/snap    # remote worker, + _TOKEN
BODYPROMPT_MODEL_LANGUAGE_OF_MOTION=fixture                # not real yet

Optional _TOKEN, _HOSTING, _CONCURRENCY. Every known model always gets a provider, so /health never has a silent hole in it, and a fourth model can be added by environment alone. BODYPROMPT_BACKEND is translated into the new form so no existing document, .env or command breaks — being explicit wins over it.

Two changes beyond the plan, both deliberate

model_version now comes from the worker. v1 declared model_version = "Kimodo-SOMA-RP-v1.1" as a class constant on the service side. The service cannot know which checkpoint is loaded across an HTTP boundary, and certainly will not know for a remote worker — so it asks, and records what came back. Same rule as denoising_steps and multi_prompt (record what was done, never what was asked), extended to the one field that had been exempt from it.

An unknown model is 422, not a hashed fixture. Previously any model string was hashed into a fixture. A model nothing is configured to serve is a bad request; 503 would tell the caller to retry something that can never work.

Concurrency belongs to hosting, not the UI

The triptych fires all three requests at once (Promise.all), which is right against three remote endpoints and would exhaust one local GPU. The browser has no business knowing where the models happen to live today, so the limit is enforced where the answer is known: a provider declares its concurrency, the router holds a semaphore per provider, a local worker gets 1. The frontend does not change.

Verification

23 service tests pass, plus 21 worker and 23 frontend; tsc clean.

The 13 v1 tests keep their assertions verbatim — though five of them moved class, since the provenance rules they pin now live in WorkerProvider. The plan claimed the tests would be untouched. That was wrong, and forcing it would have meant keeping a shim and a worse design.

Ten new tests: the registry built from the environment; a model becoming real by configuration alone; hosting reported per model and overridable; an unknown model rejected as a bad request; legacy translation, and explicit-beats-legacy; the router choosing a seed so the motion can name it; and the local gate actually serialising four concurrent generations down to one at a time.

The remote path is proven against a real socket, not asserted — a fake worker on 127.0.0.1 checks that the bearer token arrives, that the payload is what was asked for, and that the worker's own reported model version reaches provenance.

Live, on the RTX 5080:

  • BODYPROMPT_BACKEND=kimodo (the legacy form, straight from the existing .env) → kimodo resolves to a local worker, model_version reads Kimodo-SOMA-RP-v1.1 from the worker, and a 3-second prompt returns 90 frames in 3.2 s at 50 steps with source: kimodo.
  • BODYPROMPT_BACKEND=stub with only BODYPROMPT_MODEL_KIMODO set → the same real Kimodo. A model becomes real by configuration alone, which is the whole claim of the stage.
  • snapmogen still returns stub: true; an unknown model returns 422.

Still fixtures

SnapMoGen and Language of Motion are unchanged by this PR and remain hand-authored fixtures. docs/v0-stub.md keeps its entries for both, and the README Status section still says so. Nothing here makes the triptych a model comparison — it makes one possible.

docs/v3-models.md is the new v3 log: the verified facts about both models (licences, skeletons, fps, tested environments) and the Blackwell/torch risk that makes Stage B open with a throwaway spike rather than a Dockerfile.

🤖 Generated with Claude Code

WWStoryMode and others added 2 commits August 24, 2026 10:22
v1's KimodoGenerator answered three questions at once — where a model
lives, which model a request goes to, and whether that model is real —
and answered the last one twice: a literal list of three dicts in
capabilities(), and `if model != "kimodo"` as the first statement of
generate(). Both were correct exactly once, for exactly one model.
Neither could express "SnapMoGen is real and Language of Motion is not".

providers.py owns where a model lives; generators.py owns which provider
a request goes to.

- WorkerProvider — one model behind HTTP. Local and remote are the SAME
  class: a Compose container here and a GPU in another building differ by
  a URL and an optional bearer token, and by nothing else. The memory
  ceiling on one laptop is a fact about a laptop, not something the
  architecture should be shaped around.
- FixtureProvider — a model that is not real yet, so "not real yet" is
  configuration rather than a branch in the router.
- RouterGenerator — a dict of providers. capabilities() is a loop over
  the registry, so it is truthful by construction rather than by
  remembering to edit it.

BODYPROMPT_MODEL_<NAME> is a URL or `fixture`, with optional _TOKEN,
_HOSTING and _CONCURRENCY. Every known model always gets a provider, so
/health never has a silent hole, and a fourth model needs no code change.
BODYPROMPT_BACKEND is translated into the new form, so no existing doc,
.env or command breaks; being explicit wins over it.

model_version now comes from the worker. v1 declared it as a class
constant on the service side, which the service cannot actually know
across an HTTP boundary and certainly will not know for a remote worker.
Same rule as denoising_steps and multi_prompt — record what was done,
never what was asked — extended to the one field that had been exempt.

Concurrency belongs to hosting, not the UI. The triptych fires all three
requests at once, which is right against three remote endpoints and would
exhaust one local GPU; the browser has no business knowing where the
models are today, so the router holds a semaphore per provider and a
local worker gets one. The frontend does not change.

An unknown model is now 422, not a hashed fixture: it is a bad request,
and 503 would tell the caller to retry something that can never work.

23 service tests pass. The 13 v1 tests keep their assertions verbatim,
though five moved class — the provenance rules they pin now live in
WorkerProvider. The plan claimed they would be untouched; that was wrong,
and forcing it would have meant a worse design. The remote path is proven
against a real socket rather than asserted: a fake worker on 127.0.0.1
checks the bearer token, the payload, and that the worker's own reported
version reaches provenance.

Live on the 5080: the legacy .env still gets real Kimodo, and so does
BODYPROMPT_BACKEND=stub with only BODYPROMPT_MODEL_KIMODO set — which is
the whole claim of the stage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
.env.example and the README's Kimodo section still taught
BODYPROMPT_BACKEND=kimodo as the only way. Not wrong — the legacy
translation is tested and works — but someone reading either would never
discover per-model routing, which is the whole point of the change.
compose.yaml and docs/usage.md had been updated; these two had not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@WWStoryMode
WWStoryMode merged commit bd40fe3 into main Aug 24, 2026
@WWStoryMode
WWStoryMode deleted the feat/v3-providers branch August 24, 2026 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant