Single gpu disk offload PTQ for DSR1/Ultra - #2008

Open

Fridah-nv wants to merge 25 commits into

mainfrom

fridah/single-gpu-disk-offload-ptq

Commits on Jul 30, 2026

feat(ptq): single-GPU disk-offload layerwise PTQ (G1/G2)
Fridah-nv
committed
fix(export): materialize non-decoder disk-offloaded tensors before save
Fridah-nv
committed
fix(export): correct offloaded export path — MoE ordering, lm_head handler, memory
Fridah-nv
committed
feat(export): streaming shard writer for 80 GB CPU RAM target

Fridah-nv
and
claude
committed
refactor(export): simplify streaming-export additions per /simplify review

Fridah-nv
and
claude
committed
fix(export): harden streaming export path per code review

Fridah-nv
and
claude
committed
fix(export): avoid save_pretrained shared-tensor crash on MoE models

Fridah-nv
and
claude
committed
refactor(export): remove dead offloaded branch, early-return dispatch, shard regression test

Fridah-nv
and
claude
committed
chore: document new offload recipe and fix pre-commit failures

Fridah-nv
and
claude
committed
fix(export): guard streaming shards against aliases; make compat shims opt-in

Fridah-nv
and
claude
committed
fix(ptq): scope the DeepSeek load-path change and respect --trust_remote_code

Fridah-nv
and
claude
committed
revert(ptq): drop transformers compat shims in favor of a correct environment

Fridah-nv
and
claude
committed
fix(export): scope data_ptr tie identity to resident tensors

Fridah-nv
and
claude
committed
fix(export): stream extra_state_dict tensors in the offload path

Fridah-nv
and
claude
committed
fix(export): remove _tied_cache from fused-expert export to prevent cross-layer aliasing

Fridah-nv
and
claude
committed
fix(export): reconcile tied-cache reset with FSDP2 dedup opt-out

Fridah-nv
and
claude
committed
refactor(export): decide materialization via the shared dispatcher

Fridah-nv
and
claude
committed
fix(export): skip sharded DTensors in sync_tied_input_amax, not just meta

Fridah-nv
and
claude
committed
fix(export): disable tie dedup whenever weights are not resident

Fridah-nv
and
claude
committed
refactor(export): consolidate offload detection into one helper

Fridah-nv
and
claude
committed
test(export): pin the dedup opt-outs, and correct the DTensor rationale

Fridah-nv
and
claude
committed

Commits on Jul 31, 2026

Commits on Aug 1, 2026

test(export): consolidate overlapping offload export tests

Fridah-nv
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Single gpu disk offload PTQ for DSR1/Ultra - #2008

Single gpu disk offload PTQ for DSR1/Ultra#2008
Fridah-nv wants to merge 25 commits into
mainfrom
fridah/single-gpu-disk-offload-ptq

Commits on Jul 30, 2026

feat(ptq): single-GPU disk-offload layerwise PTQ (G1/G2)

fix(export): materialize non-decoder disk-offloaded tensors before save

fix(export): correct offloaded export path — MoE ordering, lm_head handler, memory

feat(export): streaming shard writer for 80 GB CPU RAM target

refactor(export): simplify streaming-export additions per /simplify review

fix(export): harden streaming export path per code review

fix(export): avoid save_pretrained shared-tensor crash on MoE models

refactor(export): remove dead offloaded branch, early-return dispatch, shard regression test

chore: document new offload recipe and fix pre-commit failures

fix(export): guard streaming shards against aliases; make compat shims opt-in

fix(ptq): scope the DeepSeek load-path change and respect --trust_remote_code

revert(ptq): drop transformers compat shims in favor of a correct environment

fix(export): scope data_ptr tie identity to resident tensors

fix(export): stream extra_state_dict tensors in the offload path

fix(export): remove _tied_cache from fused-expert export to prevent cross-layer aliasing

fix(export): reconcile tied-cache reset with FSDP2 dedup opt-out

refactor(export): decide materialization via the shared dispatcher

fix(export): skip sharded DTensors in sync_tied_input_amax, not just meta

fix(export): disable tie dedup whenever weights are not resident

refactor(export): consolidate offload detection into one helper

test(export): pin the dedup opt-outs, and correct the DTensor rationale

Commits on Jul 31, 2026

fix(export): address open review findings on the offload path

refactor(export): narrow the offload changes to the streaming path

refactor(export): share model-level export prep between both exporters

Commits on Aug 1, 2026

test(export): consolidate overlapping offload export tests