Add wslc container cp command for tar archive upload#40835
Conversation
Implements 'wslc container cp - CONTAINER:PATH' to copy a tar archive
from stdin into a running container via Docker's PUT /containers/{id}/archive API.
Usage: tar.exe -cf - files | wslc container cp - my_container:/dest
Changes across all layers:
- IDL: Added UploadArchive to IWSLCContainer
- DockerHTTPClient: Added PutArchive method (omits Content-Length for pipes)
- WSLCContainerImpl: Relay stdin to Docker socket with SD_SEND on EOF
- ContainerService: Added CopyToContainer static method
- ContainerTasks: Added CopyToContainer task with CONTAINER:PATH parsing
- ContainerCpCommand: New command registered under 'container cp'
- Localization: Added all user-facing strings
- Tests: Added CLI parsing test cases and updated e2e command list
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a new wslc container cp subcommand that uploads a tar archive from stdin into a running container by relaying stdin to Docker’s PUT /containers/{id}/archive endpoint, with supporting wiring through the CLI task/service layers and new localized help/error strings.
Changes:
- Adds a new
container cpcommand and CLI task/service method to parseCONTAINER:PATHand stream stdin into the container. - Extends the Docker HTTP client with a
PutArchiverequest helper used by the session/container implementation. - Updates tests (CLI parsing + e2e container command list) and adds localized user-facing strings for the new command.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/windows/wslc/e2e/WSLCE2EContainerTests.cpp | Adds cp to the container subcommand help list assertions. |
| test/windows/wslc/CommandLineTestCases.h | Adds command-line parsing coverage for container cp. |
| src/windows/wslcsession/WSLCContainer.h | Declares new UploadArchive APIs on the container implementation and COM wrapper. |
| src/windows/wslcsession/WSLCContainer.cpp | Implements archive upload by relaying stdin to Docker and handling HTTP responses/errors. |
| src/windows/wslcsession/DockerHTTPClient.h | Declares PutArchive for Docker archive upload. |
| src/windows/wslcsession/DockerHTTPClient.cpp | Implements PutArchive request creation (optionally omitting Content-Length). |
| src/windows/wslc/tasks/ContainerTasks.h | Declares CopyToContainer task entrypoint. |
| src/windows/wslc/tasks/ContainerTasks.cpp | Implements CONTAINER:PATH parsing and stdin validation/size detection. |
| src/windows/wslc/services/ContainerService.h | Adds CopyToContainer service API. |
| src/windows/wslc/services/ContainerService.cpp | Wires CLI service call to COM IWSLCContainer::UploadArchive. |
| src/windows/wslc/commands/ContainerCpCommand.cpp | Adds the new container cp command and argument definitions. |
| src/windows/wslc/commands/ContainerCommand.h | Declares ContainerCpCommand. |
| src/windows/wslc/commands/ContainerCommand.cpp | Registers container cp under the container root command. |
| src/windows/service/inc/wslc.idl | Adds UploadArchive to IWSLCContainer. |
| localization/strings/en-US/Resources.resw | Adds help text and user-facing error messages for container cp. |
dkbennett
left a comment
There was a problem hiding this comment.
Implementation looks good, but I think this should really have at least 1 E2E test verifying the copy actually works as expected.
Can put it in with one of the other container tests so we dont create an entirely new file for it if necessary, but should have something validating we do not regress this command's functionality in the future and have confidence that it is working correctly.
|
Just adding a note that this PR will have a conflict with #40832 due to the change from CreateSession (which is poorly named since it does an open or a create default only) to ResolveSession, depending on which one goes in first. If this one goes in first I can fix it in 40832. |
- Add 5 new CLI parsing test cases for cp in CommandLineTestCases.h - Add RunWslcWithStdinFile helper to pipe file contents to wslc stdin - Add WSLCE2EContainerCpTests with 11 e2e test methods covering: - Help output, missing arguments, invalid target formats - Stdin terminal detection, source validation - Container not found error handling - Successful tar upload to running container with exec verification - Copy to stopped container (Docker PUT /archive filesystem operation) - CreateTestTarFile builds minimal POSIX tar at runtime for tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix ContainerCpDesc to accurately describe stdin-only tar upload (was misleadingly implying bidirectional file copy) - Change PutArchive ContentLength parameter to std::optional<uint64_t> to distinguish 'unknown size' (pipe) from 'known zero size' (empty file) - WSLCContainerImpl::UploadArchive passes std::nullopt when ContentSize is 0 Note: Kept UploadArchive on IWSLCContainer (not a separate IWSLCContainer2) because wslc interfaces are internal — client and server are always deployed together from the same build, so there is no ABI compatibility concern. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Addressed code review feedback:
|
| // Running without piped stdin should fail with a terminal error. | ||
| // RunWslcAndRedirectToFile gives the child a real console stdout handle, | ||
| // and since RunWslc pipes NUL to stdin, we use RunWslcAndRedirectToFile | ||
| // with no output path to get a real console for the child. | ||
| const auto result = RunWslcAndRedirectToFile(L"container cp - fakecontainer:/path"); |
- Add Archive argument type in ArgumentDefinitions.h - Add --archive/-a flag to ContainerCpCommand arguments - Add WSLCCLI_ArchiveArgDescription localization string - Add 2 CLI parsing test cases for -a and --archive - Add 2 e2e tests: ArchiveFlag (-a) and ArchiveFlagLongForm (--archive) The flag is accepted for docker cp compatibility. Since the tar stream is relayed directly to Docker's PUT /archive API, uid/gid information from the source tar is always preserved (equivalent to -a behavior). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rue/false) Add ParseBoolValue() helper to ArgumentParser that accepts true/false/1/0 (case-insensitive). Modify ProcessNamedArgument() and ProcessAliasArgument() to parse adjoined boolean values for flag-type arguments. - --flag=true and -f=true set the flag (equivalent to --flag / -f) - --flag=false and -f=false leave the flag unset - Invalid values produce FlagInvalidBooleanError - Add 8 CLI parsing unit test cases for boolean flag values - Add 5 e2e tests for -a/--archive boolean value variants - All 143 unit tests and 18 e2e tests pass Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| contentSize = static_cast<ULONGLONG>(fileSize.QuadPart); | ||
| } | ||
|
|
||
| ContainerService::CopyToContainer(session, containerId, destPath, stdinHandle, contentSize); |
- Validate DestPath is non-empty in UploadArchive (E_INVALIDARG) - Unify CONTAINER:DEST_PATH to CONTAINER:PATH in long description - Guard FromJson with try-catch for non-JSON error responses - Fix RunWslcWithStdinFile declaration line length (>130 col) - Add comment explaining archive flag is intentional no-op for stdin tar Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| <value>Copy a tar archive from stdin into a container. Use '-' as the source to read from stdin. | ||
| Usage: wslc container cp - CONTAINER:PATH</value> | ||
| <comment>{Locked="wslc"}{Locked="container cp"}{Locked="CONTAINER:PATH"}</comment> |
| WSLC_TEST_METHOD(WSLCE2E_Container_Cp_StdinIsTerminal) | ||
| { | ||
| // Running without piped stdin should fail with a terminal error. | ||
| // RunWslcAndRedirectToFile gives the child a real console stdout handle, | ||
| // and since RunWslc pipes NUL to stdin, we use RunWslcAndRedirectToFile | ||
| // with no output path to get a real console for the child. | ||
| const auto result = RunWslcAndRedirectToFile(L"container cp - fakecontainer:/path"); | ||
| VERIFY_IS_TRUE(result.ExitCode.has_value()); | ||
| VERIFY_ARE_EQUAL(1u, result.ExitCode.value()); | ||
| VERIFY_IS_TRUE(result.Stderr.has_value()); | ||
| VERIFY_ARE_NOT_EQUAL(0u, result.Stderr.value().size()); | ||
| } |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mirrors DownloadArchive's pattern: if the error body is empty or truncated (e.g. timeout), fall back to surfacing the raw body text instead of letting FromJson throw an unhelpful exception. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use size() > 1 guard to preserve root paths like C:\ (matching the upload path logic), preventing tar.exe from receiving invalid 'C:'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| }; | ||
|
|
||
| void CreateContainer(CLIExecutionContext& context); | ||
| void CopyToContainer(CLIExecutionContext& context); |
The task handles both directions (local→container and container→local), so the old name was misleading. ContainerService::CopyToContainer (upload-only) keeps its name since it genuinely only copies to. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| auto wideTarget = MultiByteToWide(target); | ||
| auto absTarget = std::filesystem::absolute(wideTarget); | ||
|
|
||
| // Ensure target directory exists | ||
| std::error_code dirError; | ||
| std::filesystem::create_directories(absTarget, dirError); | ||
| THROW_HR_IF_MSG(HRESULT_FROM_WIN32(dirError.value()), !!dirError, "Failed to create directory: %ls", absTarget.c_str()); | ||
|
|
| wil::unique_hfile nul; | ||
| HANDLE effectiveStdin = nullptr; | ||
| if (stdinHandle.has_value()) | ||
| { | ||
| effectiveStdin = stdinHandle.value(); | ||
| THROW_IF_WIN32_BOOL_FALSE(SetHandleInformation(effectiveStdin, HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT)); | ||
| } | ||
| else | ||
| { | ||
| nul = wsl::windows::common::filesystem::OpenNulDevice(GENERIC_READ); | ||
| THROW_IF_WIN32_BOOL_FALSE(SetHandleInformation(nul.get(), HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT)); | ||
| effectiveStdin = nul.get(); | ||
| } |
| auto isContainerPath = [](const std::string& path) -> bool { | ||
| auto colonPos = path.find(':'); | ||
| if (colonPos == std::string::npos || colonPos == 0) | ||
| { | ||
| return false; | ||
| } | ||
|
|
||
| // Single letter before colon is a Windows drive path | ||
| if (colonPos == 1 && std::isalpha(static_cast<unsigned char>(path[0]))) | ||
| { | ||
| return false; | ||
| } | ||
|
|
||
| return true; | ||
| }; |
| auto wideTarget = MultiByteToWide(target); | ||
| auto absTarget = std::filesystem::absolute(wideTarget); | ||
|
|
||
| // Ensure target directory exists | ||
| std::error_code dirError; | ||
| std::filesystem::create_directories(absTarget, dirError); | ||
| THROW_HR_IF_MSG(HRESULT_FROM_WIN32(dirError.value()), !!dirError, "Failed to create directory: %ls", absTarget.c_str()); |
When the target path does not end with a separator and is not an existing directory, treat it as a file destination instead of a directory. This matches docker cp behavior where 'cp CONTAINER:/file.txt C:\local\out.txt' creates out.txt as a file rather than extracting into a directory named out.txt. Implementation: extract to temp dir, then rename/copy the extracted file to the target path. Falls back to copy+delete if rename fails across volumes. Also adds an e2e test verifying this behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
|
||
| // Parses a boolean string value (true/false/1/0, case-insensitive). | ||
| // Returns the parsed value, or std::nullopt if the string is not a valid boolean. | ||
| static std::optional<bool> ParseBoolValue(std::wstring_view value) |
There was a problem hiding this comment.
Not necessarily a blocker, but in the future I think we should separate parser changes from new commands being introduced. This will make changes smaller and easier to review
| // Returns the parsed value, or std::nullopt if the string is not a valid boolean. | ||
| static std::optional<bool> ParseBoolValue(std::wstring_view value) | ||
| { | ||
| if (string::IsEqual(value, L"true") || value == L"1") |
There was a problem hiding this comment.
I recommend using wsl::shared::string::ParseBool() instead of reimplementing it here
|
|
||
| LARGE_INTEGER fileSize{}; | ||
| ULONGLONG contentSize = 0; | ||
| if (GetFileSizeEx(inputHandle, &fileSize)) |
There was a problem hiding this comment.
Long term, we'll probably need to read the entire handle into a temporary file to resolve this
| else | ||
| { | ||
| // Local path → container: create tar from local path using tar.exe | ||
| auto widePath = MultiByteToWide(source); |
There was a problem hiding this comment.
I don't think we need to explicitly convert this path. std::filesystem and our localization methods can handle narrow and wide strings
| parentDirStr.pop_back(); | ||
| } | ||
|
|
||
| auto tarCmd = std::format(L"tar.exe -cf \"{}\" -C \"{}\" \"{}\"", tempPath, parentDirStr, fileName); |
There was a problem hiding this comment.
Instead of passing the file path to tar, I recommend opening the file handle ourselves with FILE_FLAG_DELETE_ON_CLOSE and passing it as tar's stdout. Then we can reset the handle's file pointer and pass the same handle to the service.
Otherwise there's a race between tar closing the file and us opening it
| } | ||
| else | ||
| { | ||
| if (isChunked) |
There was a problem hiding this comment.
I recommend using DockerHttpResponseHandle, which already handles this
| { | ||
| auto error = wsl::shared::FromJson<ErrorResponse>(errorJson.c_str()); | ||
|
|
||
| THROW_HR_WITH_USER_ERROR_IF(WSLC_E_CONTAINER_NOT_FOUND, error.message, statusCode == 404); |
There was a problem hiding this comment.
Same comments as above: 404 shouldn't be possible here, and the below catch block can be removed
| VERIFY_IS_TRUE(result.ExitCode.has_value()); | ||
| VERIFY_ARE_EQUAL(1u, result.ExitCode.value()); | ||
| VERIFY_IS_TRUE(result.Stderr.has_value()); | ||
| VERIFY_IS_TRUE(result.Stderr->find(L"tar") != std::wstring::npos); |
There was a problem hiding this comment.
I recommend validating the exact error message (same for below)
| auto tarCmd = std::format(L"tar.exe -cf \"{}\" -C \"{}\" testfile.txt", TarPath.wstring(), tarSrcDir.wstring()); | ||
| STARTUPINFOW si{sizeof(si)}; | ||
| PROCESS_INFORMATION pi{}; | ||
| THROW_LAST_ERROR_IF(!CreateProcessW(nullptr, tarCmd.data(), nullptr, nullptr, FALSE, 0, nullptr, nullptr, &si, &pi)); |
There was a problem hiding this comment.
I recommend using the subprocess helper here as well
| } | ||
|
|
||
| WSLCExecutionResult RunWslc(const std::wstring& commandLine, ElevationType elevationType) | ||
| WSLCExecutionResult RunWslc(const std::wstring& commandLine, ElevationType elevationType, std::optional<HANDLE> stdinHandle) |
There was a problem hiding this comment.
I recommend using a regular HANDLE here with NULL as default (that way we can reassign easily without needing an additional variable)
| if (boolVal.value()) | ||
| { | ||
| AddFlag(nextArg->Type()); | ||
| } |
| auto extractDir = std::filesystem::path(tempDir) / L"wslc-cp-extract"; | ||
| std::filesystem::create_directories(extractDir); | ||
| auto cleanupExtract = wil::scope_exit([&] { std::filesystem::remove_all(extractDir); }); |
| std::error_code dirError; | ||
| std::filesystem::create_directories(absTarget.parent_path(), dirError); | ||
|
|
| // Find the extracted file and move it to the target path. | ||
| // Docker's archive API returns a tar with the file at its basename. | ||
| std::filesystem::path extractedFile; | ||
| for (const auto& entry : std::filesystem::directory_iterator(extractDir)) | ||
| { | ||
| extractedFile = entry.path(); | ||
| break; | ||
| } | ||
|
|
||
| THROW_HR_WITH_USER_ERROR_IF(E_FAIL, "No file extracted from container archive", extractedFile.empty()); |
Summary
Implements
wslc container cpfor bidirectional file copy between containers and the local filesystem, plus stdin tar upload into containers, via Docker's archive APIs.Usage
Implementation
Full stack implementation across all layers:
UploadArchiveandDownloadArchivetoIWSLCContainerPutArchiveandGetArchivemethodsSD_SENDon EOF. Download: stream tar from Docker, extract with tar.exeCopyToContainerandCopyFromContainerstatic methodsCopyToContainertask with bidirectional CONTAINER:PATH parsingcontainer cpKey Design Decisions
GetFileSizeExwhen stdin is a file redirect; omits Content-Length header for pipes (relies onSD_SENDshutdown as EOF signal)SO_RCVTIMEOtimeout to avoid HTTP/1.1 keep-alive hang; JSON parse failures fall back to raw body text\"escape bug; root paths preserved withsize() > 1guardTesting
container cp