Encode string before xxhash in sagemaker boto3 mixin (fixes xxhash 4.0 TypeError) - #3462
Merged
Conversation
xxhash 4.0 dropped implicit str encoding, so xxh64() on a str now raises TypeError: Strings must be encoded before hashing. Encode the config string as utf-8, which is what xxhash <4 did internally, so the idempotence tokens are unchanged. Signed-off-by: Kevin Su <pingsutw@apache.org>
pingsutw
requested review from
cosmicBboy,
davidmirror-ops,
kumare3,
machichima,
samhita-alla and
wild-endeavor
as code owners
August 18, 2026 06:08
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## master #3462 +/- ##
===========================================
- Coverage 76.27% 52.00% -24.27%
===========================================
Files 217 329 +112
Lines 22943 29585 +6642
Branches 3036 3036
===========================================
- Hits 17499 15387 -2112
- Misses 4556 14069 +9513
+ Partials 888 129 -759 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
popojk
approved these changes
Aug 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
build-plugins (flytekit-aws-sagemaker)is failing on master:xxhash 4.0 removed the implicit encoding of
strinputs, soxxh64()on astring now raises. The plugin pins
xxhashunbounded (plugins/flytekit-aws-sagemaker/setup.py),so CI picked up 4.x and every code path that computes an idempotence token broke.
Failing tests:
test_call_with_idempotence_token,test_call_with_truncated_idempotence_token,test_call_with_truncated_idempotence_token_as_input,test_call_hashes_normalised_integer_values_consistently.What changes were proposed in this pull request?
Encode the config string as utf-8 before hashing:
utf-8 is exactly what xxhash <4 did internally, so the digests are unchanged —
existing idempotence tokens (and the tokens asserted in the tests) stay the
same, and this works on both xxhash 3.x and 4.x. Pinning
xxhash<4in setup.pywould also make CI green, but it leaves the latent bug in place, so I fixed the
call instead.
Verified the digest is stable across versions:
How was this patch tested?
No new tests — this restores 4 existing tests that fail on master. Ran the
suite locally against xxhash 4.0.1:
The token assertions (
2de338ec95ee0da6,1a25e4bb988697a7,ce735d6a183643f1)pass unchanged, confirming the fix does not alter existing hashes.
Check all the applicable boxes