Skip to content

Remove examples/diffusers/eval example#1694

Open
jingyu-ml wants to merge 4 commits into
mainfrom
remove-diffusers-eval-example
Open

Remove examples/diffusers/eval example#1694
jingyu-ml wants to merge 4 commits into
mainfrom
remove-diffusers-eval-example

Conversation

@jingyu-ml

@jingyu-ml jingyu-ml commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Type of change: Documentation / example removal

Removes the unmaintained examples/diffusers/eval/ image-quality evaluation example (ImageReward / CLIP-IQA / CLIP metrics), which is no longer maintained. Also cleans up the now-dangling references to it in examples/diffusers/README.md:

  • the intro sentence clause about evaluation scripts,
  • the table-of-contents "Evaluate Accuracy" row,
  • the per-subsection requirements.txt note (example updated from eval to fastgen, distillation),
  • the entire "Evaluate Accuracy" section.

Scope is intentionally limited to the diffusers example. The unrelated examples/llm_ptq "Evaluate Accuracy" section is not touched.

Usage

N/A — this removes example scripts; there are no library API changes.

Testing

N/A — deletion of example scripts plus documentation cleanup. Verified that examples/diffusers/README.md has no remaining references to the deleted eval/ directory, no broken section anchors, no trailing whitespace, and the final newline is intact.

Before your PR is "Ready for review"

  • Is this change backward compatible?: ✅ — only removes example scripts under examples/diffusers/eval/; no public modelopt API is affected.
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A — no code copied; this removes the example's own requirements.txt.
  • Did you write any new necessary tests?: N/A — removal only.
  • Did you update Changelog?: N/A — example/doc removal, not a library feature or API change.
  • Did you get Claude approval on this PR?: N/A — trivial example/doc removal.

Additional Information

Deletes examples/diffusers/eval/ (5 files) and trims its references from the diffusers README.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • Documentation
    • Updated Diffusers “Model Optimizations” README to remove the Image-Quality evaluation guidance, including the “Evaluate Accuracy” section and related links.
    • Adjusted notes about per-subsection dependency files.
  • Deprecations
    • Marked the Diffusers image-quality evaluation example as no longer maintained in the 0.46 changelog.
  • Chores
    • Removed the Diffusers evaluation scripts and associated metric computation utilities from the examples.

The diffusers image-quality eval scripts (ImageReward/CLIP-IQA/CLIP) are no longer maintained. Delete examples/diffusers/eval/ and drop the now-dangling "Evaluate Accuracy" references from examples/diffusers/README.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jingyu Xin <jingyux@nvidia.com>
@jingyu-ml jingyu-ml requested review from a team as code owners June 11, 2026 23:01
@jingyu-ml jingyu-ml requested a review from kevalmorabia97 June 11, 2026 23:01
@copy-pr-bot

copy-pr-bot Bot commented Jun 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9619bbae-1e8d-4a40-907d-ab175b886568

📥 Commits

Reviewing files that changed from the base of the PR and between f352498 and e50422a.

📒 Files selected for processing (1)
  • CHANGELOG.rst
✅ Files skipped from review due to trivial changes (1)
  • CHANGELOG.rst

📝 Walkthrough

Walkthrough

The Diffusers example documentation is updated to remove evaluation feature references. The README intro, navigation table, and installation guidance are modified to eliminate mentions of evaluation. The complete "Evaluate Accuracy" section is deleted from the document. A deprecation entry is added to the changelog.

Changes

Diffusers evaluation documentation removal

Layer / File(s) Summary
Documentation navigation and guidance updates
examples/diffusers/README.md
Top-level description removes the clause mentioning evaluation scripts; the "Evaluate Accuracy" link is removed from the centered navigation table; the Local Installation note is updated to reference subsections like fastgen and distillation instead of eval.
Evaluate Accuracy section deletion
examples/diffusers/README.md
The entire "## Evaluate Accuracy" section—including requirements installation, JSON input format specification, evaluation command usage, and sample results table—is deleted from the document.
Deprecation changelog entry
CHANGELOG.rst
Version 0.46 Deprecations section now records the removal of the evaluation example and README references due to lack of maintenance.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Remove examples/diffusers/eval example' directly and accurately describes the primary change: removing the examples/diffusers/eval directory and its associated documentation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns ✅ Passed PR removes unmaintained example code and cleans up documentation. No new files, dependencies, or security-sensitive patterns are introduced; no CRITICAL security issues found.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch remove-diffusers-eval-example

Comment @coderabbitai help to get the list of available commands and usage tips.

@jingyu-ml jingyu-ml self-assigned this Jun 11, 2026
@jingyu-ml jingyu-ml added the cherry-pick-0.45.0 After code freeze, cherry-pick to release branch for next rc (bulk update). Only for bug fixes / doc label Jun 11, 2026
@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.10%. Comparing base (c4f39bd) to head (e50422a).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1694      +/-   ##
==========================================
- Coverage   77.12%   77.10%   -0.02%     
==========================================
  Files         511      511              
  Lines       56236    56236              
==========================================
- Hits        43370    43359      -11     
- Misses      12866    12877      +11     
Flag Coverage Δ
examples 41.81% <ø> (-0.15%) ⬇️
unit 54.38% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jingyu-ml

Copy link
Copy Markdown
Contributor Author

/ok to test f352498

@kevalmorabia97

Copy link
Copy Markdown
Collaborator

@jingyu-ml please update Changelog to mention removal of this example

jingyu-ml and others added 3 commits June 13, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cherry-pick-0.45.0 After code freeze, cherry-pick to release branch for next rc (bulk update). Only for bug fixes / doc

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants