Skip to content

perf: optimize spark soundex function#23909

Open
kazantsev-maksim wants to merge 3 commits into
apache:mainfrom
kazantsev-maksim:perf_soundex
Open

perf: optimize spark soundex function#23909
kazantsev-maksim wants to merge 3 commits into
apache:mainfrom
kazantsev-maksim:perf_soundex

Conversation

@kazantsev-maksim

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

N/A

Rationale for this change

Optimize existing expression.

What changes are included in this PR?

  • Replaced dynamic string building with a fixed-size stack array [b'0'; 4] initialized with zeros. The code is filled in place and converted instantly using unsafe { std::str::from_utf8_unchecked(&buf).to_string() }, resulting in zero heap allocations for code formatting.
  • Added an ASCII Fast-Path (s.is_ascii()). When inputs consist entirely of ASCII characters, the function operates directly on raw UTF-8 byte slices (&[u8]) with optimized byte classification (classify_byte), falling back to the Unicode char path only for non-ASCII text.
  • Replaced .collect() with GenericStringBuilder::::with_capacity(rows, rows * 4) and StringBuilder::with_capacity for Utf8View. Since every Soundex result is guaranteed to be exactly 4 bytes, memory for the entire output column is pre-allocated in a single block, eliminating column-level heap re-allocations entirely.

Are these changes tested?

Existing tests.

Benchmark (criterion):

image

Are there any user-facing changes?

No

@github-actions github-actions Bot added the spark label Jul 26, 2026
Kazantsev Maksim added 2 commits July 26, 2026 20:33
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.28571% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.65%. Comparing base (88365dd) to head (0cb31d5).

Files with missing lines Patch % Lines
datafusion/spark/src/function/string/soundex.rs 84.28% 10 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23909      +/-   ##
==========================================
- Coverage   80.65%   80.65%   -0.01%     
==========================================
  Files        1093     1093              
  Lines      371619   371671      +52     
  Branches   371619   371671      +52     
==========================================
+ Hits       299730   299756      +26     
- Misses      53993    54017      +24     
- Partials    17896    17898       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@andygrove

Copy link
Copy Markdown
Member

There is another soundex optimization PR open. #23880

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants