Fix ModelOpt integration for the >=0.44 list quant_cfg format - #14536
Open
shengliangxu wants to merge 1 commit into
Open
Fix ModelOpt integration for the >=0.44 list quant_cfg format#14536shengliangxu wants to merge 1 commit into
shengliangxu wants to merge 1 commit into
Conversation
ModelOpt 0.44 changed `quant_cfg` and `mtq.config._default_disabled_quantizer_cfg`
from a `{pattern: cfg}` mapping to a list of `{"quantizer_name": ...}` entries.
`NVIDIAModelOptConfig.get_config_from_quant_type` spread the default-disabled set
into a dict literal, so on ModelOpt >=0.44 building the config raised
`TypeError: 'list' object is not a mapping` (NVIDIA/Model-Optimizer#2001) — hitting
the default `NVIDIAModelOptConfig(quant_type=...)` path.
Emit `quant_cfg` in whichever shape the installed ModelOpt expects, keyed off the
shape ModelOpt exposes for its default-disabled set: a mapping for <0.44 (unchanged
behavior) and the native list of entries for >=0.44 (no deprecation warning). The
`modules_to_not_convert` loop in the quantizer appends a list entry or assigns a
mapping key accordingly.
Add a CPU-only regression test that builds the config across quant types and checks
the installed ModelOpt accepts it — the existing GPU tests are nightly/big-accelerator
gated, so this config-construction crash slipped through CI.
Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes NVIDIA/Model-Optimizer#2001
ModelOpt 0.44 changed
quant_cfgandmtq.config._default_disabled_quantizer_cfgfrom a{pattern: cfg}mapping to a list of{"quantizer_name": ...}entries.NVIDIAModelOptConfig.get_config_from_quant_typespread the default-disabled set into a dict literal, so on ModelOpt >=0.44 building the config raisedTypeError: 'list' object is not a mapping— hitting the defaultNVIDIAModelOptConfig(quant_type=...)path.Emit
quant_cfgin whichever shape the installed ModelOpt expects, keyed off the shape ModelOpt exposes for its default-disabled set: a mapping for <0.44 (unchanged behavior) and the native list of entries for >=0.44 (no deprecation warning). Themodules_to_not_convertloop in the quantizer appends a list entry or assigns a mapping key accordingly.Add a CPU-only regression test that builds the config across quant types and checks the installed ModelOpt accepts it — the existing GPU tests are nightly/big-accelerator gated, so this config-construction crash slipped through CI.
Self-review — Fix ModelOpt integration for the >=0.44 list quant_cfg format
Scope:
src/diffusers/quantizers/quantization_config.py,src/diffusers/quantizers/modelopt/modelopt_quantizer.py,tests/quantization/modelopt/{__init__.py,test_modelopt.py}.Blocking issues
None. The fix is correct on both supported ModelOpt eras — verified by running the new test against real 0.43.0 (dict branch) and 0.45.0 (list branch), 6/6 each. The
<0.44dict path is byte-for-byte identical to the pre-fix code (checked against the original method extracted from git).Non-blocking issues
_weight_quantizer_num_bits(tests/quantization/modelopt/test_modelopt.py:46) has a single caller. Per.ai/AGENTS.md: "If a private helper has only one caller, inlining it at the call site is usually the cleaner choice." It names a real dict-vs-list abstraction, so it's defensible; by the rule it leans toward inlining. Trivial.mtq.config.QuantizeConfig(**modelopt_config)(test_modelopt.py:88) rather than a fullmto.apply_modeon a model — a deliberate CPU-only tradeoff so the regression is caught without a GPU. Worth surfacing to the reviewer, not fixing.Rule checks that passed (not issues)
AGENTS.md): theisinstance(default_disabled_quantizer_cfg, dict)branch and theelif "enable" not in entry: entry["enable"] = Truefallback are both reachable and tested (dict path on 0.43; theenable:Truecase onFP8 weight_only=False), so they're not "just in case."review-rules.md§Common mistakes): all three comments state the reason and stand alone; the doc URL is a stable pointer. None are PR-scoped chatter.testing.md): backend-tier config unit test mirroring the retainedTestTorchAoConfigpattern — pytest-style,@is_quantization/@is_modeloptmarkers (both registered inconftest.py), license header. Pipeline/model tester-mixin rules don't apply. No LoRA/slow/integration added.# Copied fromheader.ruff check+ruff format --checkclean.Dead code (advisory)
Rubric's dead-code pass is scoped to new-model PRs — N/A here.
quantization_config.py:880(dict branch)quantization_config.py:894(enable:True)FP8 weight_only=Falseparam in the testmodelopt_quantizer.py:173(list append)modules_to_not_convertpathDocumentation impact
No public API changed (behavior-restoring fix).
docs/source/en/quantization/modelopt.mddocuments the publicquant_type/modules_to_not_convert/… params — untouched, no staleness introduced. No agent-guide addition warranted (the dict->list gotcha is captured in the code comment + doc link).Summary — READY
Minimal, correct on both ModelOpt eras, lint-clean, and covered by a CPU regression test that would have caught the original
TypeError(which the pre-existing GPU/nightly-gated tests could not)._weight_quantizer_num_bits(Add glide modeling files #1).QuantizeConfig-vs-apply_modeCPU-validation tradeoff (+ cosine schedule and unet config #2) — flag it to the maintainer rather than pre-emptively changing it.Before submitting
self-reviewskill on the diff?documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.