Skip to content

feat(llama-cpp-localai-paged): paged KV cache llama.cpp backend + cross-request prefix sharing + GB10 decode optimization [WIP]#10462

Open
localai-bot wants to merge 321 commits into
masterfrom
worktree-feat+paged-attention
Open

feat(llama-cpp-localai-paged): paged KV cache llama.cpp backend + cross-request prefix sharing + GB10 decode optimization [WIP]#10462
localai-bot wants to merge 321 commits into
masterfrom
worktree-feat+paged-attention

docs(paged): record P6 fp8-KV measured NO-GO - throughput dead end, c…

3159ed0
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar