Reinstate image comparison tests#27
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #27 +/- ##
===========================================
- Coverage 100.00% 94.50% -5.50%
===========================================
Files 17 17
Lines 844 855 +11
===========================================
- Hits 844 808 -36
- Misses 0 47 +47 ☔ View full report in Codecov by Sentry. |
|
I'm confused why the tests are now showing as uncovered |
|
@williamjameshandley rolling back codecov has fixed it 🥲 |
williamjameshandley
left a comment
williamjameshandley
left a comment
There was a problem hiding this comment.
Many thanks @AdamOrmondroyd
|
you're happy to roll back codecov? |
|
Yes. |
|
Ah - the tests are skipping in the CI! Codecov V4 was doing its job |
|
Disappointed, but not surprised |
The decorator pointed at plot_transparent instead of plot_fineness, and the plot_fineness.pdf baseline was the wrong page size (6.4x4.8" vs the 8x6" all other baselines use), so even fixing the decorator would have failed an image-size check. Regenerated from a fresh local run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…_figures_equal for no_ax tests
Set tol=5 on image_comparison decorators: macOS-generated PDF baselines
rasterise slightly differently under Linux ghostscript (observed RMS
~1.3-1.5 on Ubuntu CI). tol=5 absorbs the platform jitter while staying
well below real visual regressions.
Swap test_plot_no_ax and test_plot_lines_no_ax to check_figures_equal:
they're equivalence assertions ("default ax behaves like explicit ax"),
not visual-baseline checks, so they don't need stored baselines and
sidestep platform drift entirely.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Had to go gentle on the tolerances to get the tests to pass 😢 |
|
I think that's as good as we're going to get |
2 participants
Description
Spotted #6, so have reinstated the tests. Since then, it seems the$3.5\sigma$ contour doesn't plot any more, and the style of the line plots is a bit different. Comparison below:
Old
New
I realised that the "
no_ax" tests should output the same as their "fig, ax = plt.subplots()" counterparts, so they use the same images for comparisonFixes #6
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Test Configuration:
Checklist: