S3UTILS-238: regression test for crrExistingObjects with non-ASCII keys by tcarmet · Pull Request #391 · scality/s3utils

tcarmet · 2026-05-19T22:41:34Z

Intent: why does this change exist?

S3C-11235 / RD-1751 reported that crrExistingObjects fails with SignatureDoesNotMatch (HTTP 403) for any object whose key contains non-ASCII characters such as Polish diacritics, blocking the customer from replicating ~131k existing objects. The underlying bug was fixed in @scality/cloudserverclient 1.0.8, which s3utils picked up in S3UTILS-232, but no automated test exercises the path. This change adds the regression test so any future encoding regression in the dependency stack is caught in CI.

References: S3UTILS-238 · S3C-11235 · RD-1751 · CLDSRVCLT-13 · S3UTILS-232

bert-e · 2026-05-19T22:41:38Z

Hello tcarmet,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-05-19T22:41:42Z

Incorrect fix version

The Fix Version/s in issue S3UTILS-238 contains:

None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

1.17.10
1.18.0

Please check the Fix Version/s of S3UTILS-238, or the target
branch of this pull request.

claude · 2026-05-19T22:44:27Z

LGTM

Review by Claude Code

codecov · 2026-05-19T22:49:16Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 45.02%. Comparing base (949dd51) to head (bcc19cc).
⚠️ Report is 5 commits behind head on development/1.17.

Additional details and impacted files

@@                 Coverage Diff                  @@
##           development/1.17     #391      +/-   ##
====================================================
+ Coverage             44.90%   45.02%   +0.12%     
====================================================
  Files                    88       88              
  Lines                  6456     6456              
  Branches               1352     1352              
====================================================
+ Hits                   2899     2907       +8     
+ Misses                 3511     3503       -8     
  Partials                 46       46

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

claude · 2026-05-19T22:56:45Z

LGTM

Review by Claude Code

bert-e · 2026-05-26T21:46:40Z

Incorrect fix version

The Fix Version/s in issue S3UTILS-238 contains:

None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

1.17.12
1.18.0

Please check the Fix Version/s of S3UTILS-238, or the target
branch of this pull request.

Lift the CRR setup and teardown helpers out of the listObjectsByReplicationStatus functional test and into tests/utils/S3Setup.js so they can be reused by other functional tests. configureCrr gains an optional `storageClass` option for callers whose flow matches by destination storage class.

…SCII keys End-to-end regression test that drives ReplicationStatusUpdater against the workbench cloudserver with objects whose keys contain Polish diacritics (BŚ-test.txt, ąęóćśźżł-all-diacritics.txt, mixed/żółć/Łódź.dat) and an ASCII control. Asserts no "error updating object" log was emitted and that each source object's ReplicationStatus is updated to PENDING. Guards against future regressions in the path-encoding behavior of the SigV4 signer used by @scality/cloudserverclient to talk to cloudserver's /_/backbeat/metadata route - the failure mode reported in S3C-11235 / RD-1751.

claude · 2026-05-27T23:47:16Z

LGTM

Review by Claude Code

bert-e · 2026-05-27T23:47:24Z

Incorrect fix version

The Fix Version/s in issue S3UTILS-238 contains:

1.17.2
1.18.0

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

1.17.12
1.18.0

Please check the Fix Version/s of S3UTILS-238, or the target
branch of this pull request.

…-8 sequences Add test cases for CJK/Arabic characters (3-byte UTF-8, U+0800-U+FFFF) and emoji/supplementary scripts (4-byte UTF-8, U+10000-U+10FFFF) alongside the existing Polish diacritics coverage (2-byte UTF-8).

bert-e · 2026-06-11T21:18:09Z

Incorrect fix version

The Fix Version/s in issue S3UTILS-238 contains:

1.17.2
1.18.0

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

1.17.13
1.18.0

Please check the Fix Version/s of S3UTILS-238, or the target
branch of this pull request.

claude · 2026-06-11T21:20:52Z

LGTM

Review by Claude Code

bert-e · 2026-06-11T21:21:00Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

tcarmet · 2026-06-11T22:04:49Z

/create_integration_branches

bert-e · 2026-06-11T22:04:59Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge bugfix/S3UTILS-238/crr-non-ascii-keys into
development/1.17
w/1/bugfix/S3UTILS-238/crr-non-ascii-keys will be merged into development/1

The following branches will NOT be impacted:

development/1.13
development/1.14
development/1.15
development/1.16
development/1.4
development/1.7

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: create_integration_branches

bert-e · 2026-06-11T22:05:00Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following options are set: create_integration_branches

jonathan-gramain · 2026-06-18T00:23:43Z

+        Statement: [
+            {
+                Effect: 'Allow',
+                Action: ['s3:GetObjectVersion', 's3:GetObjectVersionAcl', 's3:ReplicateObject'],


style nitpick

Suggested change

Action: ['s3:GetObjectVersion', 's3:GetObjectVersionAcl', 's3:ReplicateObject'],

Action: [

's3:GetObjectVersion',

's3:GetObjectVersionAcl',

's3:ReplicateObject',

],

jonathan-gramain · 2026-06-18T00:26:43Z

+    };
+}
+
+function runUpdater(updater) {


I think this can be replaced at the call site by await promisify(updater.run)

jonathan-gramain · 2026-06-18T00:30:29Z

+    const wrap = level => (msg, data) => {
+        captured.push({ level, message: msg, ...(data || {}) });
+        return inner[level](msg, data);
+    };


Adding parenthesis would show more clearly what is returned by wrap

Suggested change

const wrap = level => (msg, data) => {

captured.push({ level, message: msg, ...(data || {}) });

return inner[level](msg, data);

};

const wrap = level => (

(msg, data) => {

captured.push({ level, message: msg, ...(data || {}) });

return inner[level](msg, data);

}

);

jonathan-gramain · 2026-06-18T00:34:36Z

+        log.info('Test accounts deleted');
+    });
+
+    async function runAndAssert(testObjects) {


Nitpick: the function is called runAndAssert but there is no call to assert, just expect, what about simply naming it runTest

tcarmet force-pushed the bugfix/S3UTILS-238/crr-non-ascii-keys branch from 4647b55 to df67850 Compare May 19, 2026 22:53

tcarmet requested a review from a team May 26, 2026 21:46

tcarmet marked this pull request as ready for review May 26, 2026 21:46

tcarmet added 2 commits May 27, 2026 16:44

tcarmet force-pushed the bugfix/S3UTILS-238/crr-non-ascii-keys branch from df67850 to 85b77e9 Compare May 27, 2026 23:44

BourgoisMickael reviewed May 28, 2026

View reviewed changes

Comment thread tests/functional/crrExistingObjects.js

BourgoisMickael approved these changes May 28, 2026

View reviewed changes

tcarmet requested a review from a team June 11, 2026 21:18

BourgoisMickael approved these changes Jun 12, 2026

View reviewed changes

tcarmet requested a review from a team June 17, 2026 23:26

jonathan-gramain approved these changes Jun 18, 2026

View reviewed changes

Conversation

tcarmet commented May 19, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Intent: why does this change exist?

Uh oh!

bert-e commented May 19, 2026

Hello tcarmet,

Uh oh!

bert-e commented May 19, 2026

Incorrect fix version

Uh oh!

claude Bot commented May 19, 2026

Uh oh!

codecov Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

claude Bot commented May 19, 2026

Uh oh!

bert-e commented May 26, 2026

Incorrect fix version

Uh oh!

claude Bot commented May 27, 2026

Uh oh!

bert-e commented May 27, 2026

Incorrect fix version

Uh oh!

Uh oh!

bert-e commented Jun 11, 2026

Incorrect fix version

Uh oh!

claude Bot commented Jun 11, 2026

Uh oh!

bert-e commented Jun 11, 2026

Request integration branches

Uh oh!

tcarmet commented Jun 11, 2026

Uh oh!

bert-e commented Jun 11, 2026

Integration data created

Uh oh!

bert-e commented Jun 11, 2026

Waiting for approval

Uh oh!

jonathan-gramain Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

jonathan-gramain Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

jonathan-gramain Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

jonathan-gramain Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tcarmet commented May 19, 2026 •

edited by atlassian Bot

Loading

codecov Bot commented May 19, 2026 •

edited

Loading