Skip to content

Tighten screenshot comparison coverage#1730

Open
matthargett wants to merge 1 commit into
BabylonJS:masterfrom
rebeckerspecialties:tighten-screenshot-thresholds
Open

Tighten screenshot comparison coverage#1730
matthargett wants to merge 1 commit into
BabylonJS:masterfrom
rebeckerspecialties:tighten-screenshot-thresholds

Conversation

@matthargett
Copy link
Copy Markdown

@matthargett matthargett commented Jun 3, 2026

Summary

  • enable tightened visual coverage for GUI3D SpherePanel, GUI Slate, GUI Near Menu, and the GUI gradient tests in automatic Playground validation
  • add explicit pixel comparison budgets for compact GUI/GUI3D cases where the default 2.5% error ratio is too permissive
  • import the Babylon.js texture repetition Standard/PBR visual cases and references, using the Standard case as a near-exact control and a 1% diff budget for the PBR variant

Why

The default screenshot budget allows up to 2.5% of pixels to differ. On the current 600x400 references that is roughly 6,000 pixels, which is large enough for visible but spatially small failures to pass. In recent native validation on my wgpu fork, this hid or nearly hid issues such as dim/underdrawn GUI Slate header controls, compact GUI3D/Near Menu differences, GUI gradient color/alpha errors, and material-specific texture repetition mismatches.

The new budgets keep broad per-channel anti-aliasing/color tolerance where needed, but lower the allowed differing-pixel count for tests whose important content occupies a small part of the screenshot. The texture repetition pair also adds a useful control: Standard Material was effectively exact in native validation, while the PBR variant exposed a real material-path mismatch that should not be treated as acceptable drift.

Validation

  • parsed Apps/Playground/Scripts/config.json with Node to verify valid JSON
  • ran git diff --check
  • verified the new texture repetition references are 600x400 PNGs

Note

I didn't run the entire screenshot test suite on my fork, just first first 50-60 and then bug deep on GUI 3D and Canvas-related things. if doing an audit of the full suite and tightening/updating things would be helpful/welcome, I'm happy to do it.

Copilot AI review requested due to automatic review settings June 3, 2026 20:42
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates the Playground visual regression configuration to better handle pixel-diff variability and to add new texture repetition baseline tests.

Changes:

  • Added per-test pixel comparison tuning via threshold and/or errorRatio for several existing tests.
  • Added two new “Texture Repetition” playground entries with reference images and engine exclusions.

Comment thread Apps/Playground/Scripts/config.json
Comment thread Apps/Playground/Scripts/config.json
Comment thread Apps/Playground/Scripts/config.json
@matthargett matthargett force-pushed the tighten-screenshot-thresholds branch from 5b1c447 to b85623e Compare June 3, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants