Skip to content

Striation speedup#214

Open
SimoneAriens wants to merge 4 commits into
mainfrom
striation_speedup
Open

Striation speedup#214
SimoneAriens wants to merge 4 commits into
mainfrom
striation_speedup

Conversation

@SimoneAriens
Copy link
Copy Markdown
Collaborator

@SimoneAriens SimoneAriens commented Apr 3, 2026

especially important when running the bulk conversion

Replaced the brute-force shift loop in _find_best_alignment with FFT-based normalized cross-correlation (scipy.signal.fftconvolve). Mathematically equivalent — computes Pearson correlation at all shifts simultaneously instead of looping over ~8,500 individual calls per comparison.
42× faster: 11.75s → 0.28s per comparison on production data. Validated against the old implementation on synthetic and production pairs (correlation, shift, and scale all match within 1e-6).

@SimoneAriens SimoneAriens requested review from cfs-data and vergep April 3, 2026 11:25
@SimoneAriens SimoneAriens marked this pull request as draft April 3, 2026 11:26
@SimoneAriens SimoneAriens marked this pull request as ready for review May 11, 2026 12:35
Comment thread packages/scratch-core/src/conversion/profile_correlator/profile_correlator.py Outdated
Comment thread packages/scratch-core/src/conversion/profile_correlator/profile_correlator.py Outdated
Comment thread packages/scratch-core/src/conversion/profile_correlator/profile_correlator.py Outdated
Copy link
Copy Markdown
Collaborator

@cfs-data cfs-data left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with suggestions

@github-actions
Copy link
Copy Markdown

Diff Coverage

Diff: origin/main..HEAD, staged and unstaged changes

  • packages/scratch-core/src/conversion/profile_correlator/profile_correlator.py (100%)

Summary

  • Total: 27 lines
  • Missing: 0 lines
  • Coverage: 100%

@github-actions
Copy link
Copy Markdown

Code Coverage

Package Line Rate Branch Rate Health
. 96% 92%
computations 94% 67%
container_models 99% 100%
conversion 96% 89%
conversion.export 99% 93%
conversion.filter 97% 89%
conversion.leveling 100% 100%
conversion.leveling.solver 100% 75%
conversion.plots 99% 88%
conversion.preprocess_impression 99% 91%
conversion.preprocess_striation 90% 62%
conversion.profile_correlator 97% 81%
conversion.surface_comparison 99% 89%
conversion.surface_comparison.cell_registration 100% 90%
conversion.surface_comparison.cmc_consensus 91% 65%
extractors 97% 75%
mutations 100% 100%
parsers 97% 50%
parsers.patches 89% 60%
preprocessors 100% 100%
processors 100% 83%
renders 99% 50%
utils 71% 100%
Summary 98% (3389 / 3470) 86% (346 / 404)

Minimum allowed line rate is 50%

Copy link
Copy Markdown
Collaborator

@vergep vergep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wellicht nog wat uitleg bij de kruis correlatie formule met 6 termen. Zullen we daar even samen voor gaan zitten?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants