Add tensor splitting (row + tensor), lazy loading and autofitting logic#1470

Open

pwilkin wants to merge 9 commits into

leejet:masterfrom

pwilkin:backend-fit

Commits on Apr 30, 2026

feat: auto-fit component placement and per-component backend devices

pwilkin
and
claude
committed

Commits on May 1, 2026

wip: layer-split + lazy load (RAM regression)

pwilkin
and
claude
committed
fix: drop pagecache after each lazy load to bound RAM

pwilkin
and
claude
committed
wip: restrict layer-split to DiT, fall back to single-GPU/CPU for Cond

pwilkin
and
claude
committed
fix: layer-split alloc actually lands on GPU; was a diagnostic artifact

pwilkin
and
claude
committed
feat: add --quiet-unknown-tensors; demote layer-split diagnostics to DEBUG

pwilkin
and
claude
committed
feat: add row-split (cuda_split_buffer_type) alongside layer-split

pwilkin
and
claude
committed
ggml.patch: cuda_split_buffer_type pool allocator

pwilkin
and
claude
committed

Commits on May 2, 2026

review: address PR #1470 #ifdef and old-flag-alias feedback

pwilkin
and
claude
committed