Support nonseekable streams for UploadPartRequest.InputStream

[StephenCleary]

Describe the feature

In the scenario where DisablePayloadSigning is true, the UploadPartRequest.InputStream should support nonseekable streams.

Use Case

Currently, the InputStream is always wrapped in a PartialWrapperStream just in case the user passes in a stream that's too big. This validation requires the underlying stream to be seekable.

Please provide an option that avoids the PartialWrapperStream for users that provide non-seekable streams guaranteed to be equal to the part size (or smaller for the last part).

Proposed Solution

Add a flag/option to UploadPartRequest to skip creating the PartialWrapperStream.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

AWS .NET SDK and/or Package version used

Latest

Targeted .NET Platform

.NET 8

Operating System and version

Windows 11

[normj]

Hi @StephenCleary. S3 always requires content length as well as the SDK does compute checksums as headers so the put object and upload part APIs do not support non seekable streams.

In the SDK we have the Amazon.S3.Transfer.TransferUtility class and it's UploadAsync methods will simulate the upload of a non-seekable stream by buffering up to a part size, default 5MB, in memory then uploading that buffer using the upload part API.

[StephenCleary]

the SDK does compute checksums as headers

Not if DisablePayloadSigning is true and the S3 additional checksums are already calculated by my code.

[normj]

@StephenCleary I found some time to do some more investigation. There are some discrepancies with what is possible with PutObject and UploadPart when working nonseekable streams. I'm attempting to normalize the code which I think will unblock you.