Add tagging support for the s3 cp, mv, and sync commands#10370
Open
sveniu-kahoot wants to merge 2 commits into
Open
Add tagging support for the s3 cp, mv, and sync commands#10370sveniu-kahoot wants to merge 2 commits into
sveniu-kahoot wants to merge 2 commits into
Conversation
The aws s3 cp/mv/sync commands already let users set most write-time object properties (storage class, metadata, SSE, ...) but not tags, forcing a separate PutObjectTagging pass after the transfer. That pass doubles the request count, and any object whose tagging step is skipped or fails stays untagged, so tag-based lifecycle rules never match it. Teach RequestParamsMapper to build the Tagging member of PutObject and CreateMultipartUpload from the repeatable --tags key/value pairs, URL-encoding them into the x-amz-tagging query-string form. s3transfer forwards these extra args to CreateMultipartUpload and drops them from the per-part requests, so the tag set is applied to the object exactly once regardless of size. This commit only adds the mapping; no CLI parameter feeds it yet, so it is inert on its own.
Expose the object tagging support wired into RequestParamsMapper as a --tags parameter on the transfer commands. It takes repeatable --tags <key> <value> pairs, matching the s3 mb --tags syntax, and applies them to each object as it is created. Like every other write-time property, it only affects objects that are actually uploaded; in a sync, files that have not changed are not retagged. This matches the long-standing behaviour of --metadata, --storage-class and --acl and is called out in the parameter's help text. S3-to-S3 copies remain governed by --copy-props and are intentionally out of scope for this change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
#2458
Description of changes:
This adds object tagging to the s3 transfer commands, scoped to uploads. It reuses the existing RequestParamsMapper/extra_args path that
--metadataand the recently mergeds3 mb --tags(#10119) already use, so it applies on both the classic and CRT transfer clients without changing how the transfer client or config are obtained. An earlier, broader attempt (#6079) was closed as not-planned; this version is intentionally narrower, leaving S3-to-S3 copies to--copy-props.--tagsparameter toaws s3 cp,aws s3 mv, andaws s3 syncto set object tags at upload time, avoiding a separateput-object-taggingrequest. Mirrors theaws s3 mb --tagssyntax.Taggingparameter, which s3transfer applies toPutObject(single-shot) andCreateMultipartUpload(multipart) and excludes from the part uploads.--copy-props.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.