Add support for user systemd in systemd cgroup driver - #13853
Open
ThatsItForTheOtherOne wants to merge 2 commits into
Open
Add support for user systemd in systemd cgroup driver#13853ThatsItForTheOtherOne wants to merge 2 commits into
ThatsItForTheOtherOne wants to merge 2 commits into
Conversation
runsc always connected to the system service manager to create the transient scope backing a container's cgroup. Unprivileged users cannot create system-wide units, so polkit denies StartTransientUnit and container creation fails with "systemd error: Interactive authentication required". Rootless podman passes --systemd-cgroup, so runsc could only be used with --ignore-cgroups, which discards all resource limits. Connect to the per-user service manager when running unprivileged, and resolve unit cgroup paths relative to that manager's own cgroup. Note that the Slice= property remains relative to the managing instance even though the path is not. This mirrors runc: https://github.com/opencontainers/cgroups/blob/79cbc7c/systemd/user.go https://github.com/opencontainers/cgroups/blob/79cbc7c/systemd/v2.go Rootless podman runs the OCI runtime inside a user namespace with the caller mapped to UID 0, so privilege is determined by translating the effective UID through /proc/self/uid_map rather than by geteuid() alone. The translated UID is also the credential the bus requires for EXTERNAL authentication. Enabling controllers through a parent's cgroup.subtree_control is also made non-fatal, which the above depends on: the path walk starts at the cgroup mount root, which no unprivileged caller can write, so Join() aborted before the delegated scope could be used. Controllers are retried individually; a warning names any mandatory controller that remains unenabled, while optional and unmanaged ones are only logged at debug level. Under delegation these writes are largely redundant, since systemd has already enabled the delegated controllers in the subtree it created. runc does the same, see https://github.com/opencontainers/cgroups/blob/79cbc7c/fs2/create.go This does not make the fs cgroup driver usable for rootless containers. That path fails earlier, when creating the cgroup directory, and is left unchanged. Rootless and ManagerCG are exported so that they survive the CgroupJSON round-trip: podman passes --systemd-cgroup on create but not on delete, so teardown relies on the persisted state. Sandboxes created by an older runsc decode both as zero values and retain the previous behaviour. Finally, correct the systemd cgroup driver documentation. It claimed the driver is backed by the fs driver, as runc's is, but no such fallback has ever existed in runsc: cgroupSystemd overrides Install and Update and never reaches controllerv2.set(). Resources with no systemd equivalent are simply not applied. Fixes google#11543 Assisted-by: Claude Code Signed-off-by: Clair Kirsch <44384688+ThatsItForTheOtherOne@users.noreply.github.com>
runsc generates systemd unit properties from cgroupV2.Controllers, which is read from cgroup.controllers at the mount root. That reports what the kernel supports, not what was delegated to the systemd instance the unit will live under, so runsc can request a limit for a controller that is not available where the container's cgroup is created. systemd records the property and applies what it can; the limit is simply never enforced. For memory, cpu and cpuset this eventually surfaces, since the sandbox reads memory.max, cpu.max and cpuset.cpus.effective at startup and fails if they are absent. For io and pids nothing reads the value back and the limit is dropped silently. The existing controller check cannot catch either case. It also tests against cgroupV2.Controllers, so under delegation every controller appears present and neither the mandatory branch nor controller.skip() is ever reached. Record the controllers that properties were generated for, i.e. those the spec actually requests limits for, and verify after the scope is created that each is present in the leaf cgroup. Container creation fails naming any that are missing. Controllers with no requested limit are not checked, so a delegation that happens to omit a controller nobody is using is still fine. The scope is a transient unit, so systemd releases it when this process exits and failing here does not leak it. Assisted-by: Claude Code Signed-off-by: Clair Kirsch <44384688+ThatsItForTheOtherOne@users.noreply.github.com>
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
runsc supports rootless deployments, but on a host using the systemd cgroup driver it fails for unprivileged users. #11543 has been open since March 2025. The only workaround is
--ignore-cgroups, which disables resource limits.The fix is two commits. One is the proper fix and the second is hardening. It can be dropped without affecting the overall fix.
1.
runsc/cgroup: support rootless containers with the systemd cgroup driverThe driver always talked to the system service manager. This was diagnosed by DemiMarie in Error: OCI runtime error: runsc: creating container: systemd error: Interactive authentication required. #11543. Unprivileged users cannot create system-wide transient units, so polkit rejects the request as requiring interactive authorization. It now connects to the per-user instance when unprivileged, and mirrors runc's implementation.
Enabling controllers was fatal. The cgroupv2 path walk starts at the cgroup mount root, whose
cgroup.subtree_controlno unprivileged caller can write, soJoin()aborted before the delegated scope could be used. This contrasted with runc's implementation. Controllers are now retried individually, and failures warn rather than abort.Privilege is determined by translating the effective UID through
/proc/self/uid_map, notgeteuid(): rootless podman runs the OCI runtime inside a user namespace with the caller mapped to UID 0, sogeteuid()returns 0 in exactly the case being fixed. The translated UID is also the credential the session bus requires for EXTERNAL authentication.This commit also corrects
g3doc/user_guide/systemd.md, which claimed the systemd driver is backed by the fs driver as runc's is. The text appears to have been reproduced from runc but was left unimplemented.2.
runsc/cgroup: fail when a requested limit's controller is unavailablerunsc generates unit properties from what the kernel supports, but not what is explicitly delegated to the user. runsc can request a limit for a controller that is not available in the container's cgroup. This can cause cases where limits are left silently unenforced.
Every controller that properties were generated for, that a spec actually requests a limit for, is recorded and verified against the leaf once the scope exists. Container creation fails, naming any that are missing. Controllers with no requested limit are not checked, so a delegation omitting something nobody is using still works.
Fixes #11543
Testing
Ubuntu 24.04, aarch64, kernel 6.14, cgroup v2, systemd 255, podman 4.9.3.
Requires CPU/cpuset delegation to user sessions, a standard rootless prerequisite (runc
docs/cgroup-v2.md, "Configuring delegation"):Automated
//runsc/container:container_testfails identically on master and on thisbranch in my environment, so it could not serve as a signal.
Rootless, end to end
Checks against three builds —
master, first commit only (c1), both(
c1+2):memory.max/cpu.maxappliedBefore, on master:
After:
runsc runs inside a user namespace here, confirming the uid_map translation is
load-bearing rather than merely unit-tested:
Limits reach the kernel:
And they are applied by the user manager, not the system one.
LoadStateisthe property that distinguishes a unit a manager has from one it does not;
systemctl showexits 0 and reports defaults for unknown units:Enforced, and cleaned up — the
deletepath runs withoutDBUS_SESSION_BUS_ADDRESS, exercising the$XDG_RUNTIME_DIR/busfallback andthe persisted state:
podman updateexercises the second D-Bus connection, made in a separateprocess using
Rootlessrestored from the state file:A container created by a master build and torn down after swapping in the
patched binary exits cleanly, confirming old state files still decode:
Rootful — unchanged
Both builds, both cgroup drivers, all exit 0:
Limits still land on the system manager, with the cgroup under
machine.slicerather thanuser@1000.service, confirmingRootless=falsetakes the original path with an empty
ManagerCG:Second commit: a requested limit that cannot be enforced is now fatal
The check applies to any controller the spec requests a limit for.
iois usedto demonstrate it.
With
ioremoved fromDelegate=and an io limit requested, the first commitalone warns and runs anyway:
With both commits the same command fails and names the controller. Containers requesting no io limit are unaffected, every other check still passes with io undelegated. No systemd unit or cgroup is left behind by the failure, since the scope is transient.
Not covered
runsc updatefor an undelegated controller is not re-checked.iowas tested un-delegated; the other controllers are covered by the same code path but were not exercised. On Ubuntu, I could not easily un-delegate the other controllers besides cpuset, which caused a crash anyway without commit 2.hugeTLB2.generatePropertiesreturns nothing, so hugepage limits are never sent to systemd and the new check does not cover them. Pre-existing and unchanged.