Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1747933
Summary: | systemd does not work with podman and cgroupsV2 | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Lukas Slebodnik <lslebodn> |
Component: | crun | Assignee: | Giuseppe Scrivano <gscrivan> |
Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | unspecified | Docs Contact: | |
Priority: | unspecified | ||
Version: | 31 | CC: | bbaude, dwalsh, frantisek.kluknavsky, gscrivan, jnovy, lsm5, mheon, santiago, splinux25 |
Target Milestone: | --- | Keywords: | Regression |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | crun-0.9.1-1.fc31 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2019-09-19 14:30:14 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Lukas Slebodnik
2019-09-02 09:50:15 UTC
Lukas, any idea what is beling blocked? Could you try with a --privileged container, to see if it is security blocking the creation? It requires support from systemd as well. I don't think the version shipped with rhel7 has cgroups v2 support. Could you try with a rhel8 image? Also, exec with systemd containers is known to be broken on cgroups v2. On cgroups v2 it is not possible to join a parent node, since systemd modifies the cgroup hierarchy, the exec will fail with "Device or resource busy". I am not sure yet how to solve this issue (In reply to Giuseppe Scrivano from comment #2) > It requires support from systemd as well. I don't think the version shipped > with rhel7 has cgroups v2 support. > > Could you try with a rhel8 image? > > Also, exec with systemd containers is known to be broken on cgroups v2. On > cgroups v2 it is not possible to join a parent node, since systemd modifies > the cgroup hierarchy, the exec will fail with "Device or resource busy". I > am not sure yet how to solve this issue yep, sh# podman run --name test -d registry.access.redhat.com/ubi8-init:latest && sleep 10 && podman exec test systemctl status e01001c8e5513b603dc8d752a22789f8d945f27367ed336f4e1b151eec0e5253 Error: writing file '/sys/fs/cgroup//machine.slice/libpod-e01001c8e5513b603dc8d752a22789f8d945f27367ed336f4e1b151eec0e5253.scope/cgroup.procs': Device or resource busy: OCI runtime error But that's quite problematic if new podman cannot run some older (rhel7/fedora/ random image from net) with systemd. People will either disable cgroupsV2 or even will not use podman at all. (In reply to Daniel Walsh from comment #1) > Lukas, any idea what is beling blocked? > > Could you try with a --privileged container, to see if it is security > blocking the creation? I think Giuseppe already provided an explanation but just for the record. There is not any difference with --privileged. > But that's quite problematic if new podman cannot run some older
> (rhel7/fedora/ random image from net)
> with systemd. People will either disable cgroupsV2 or even will not use
> podman at all.
the issue only happens when the container payload tries to access cgroups v1. It is a known issue, for example cgroups v2 adoption was/is also blocked by the Java VM that reads cgroups stats.
There is not really much Libpod can do. Cgroups are a kernel interface, so either the container payload supports cgroups v2 or you'll need to use cgroups v1.
Please enhance documentation (details about systemd would be good as well. Moreover I tried with rawhide container which definitely has right version of systemd and it did not help either sh-5.0# mkdir temp sh-5.0# cat >temp/Dockerfile <<EOF FROM fedora:rawhide CMD ["/sbin/init"] STOPSIGNAL SIGRTMIN+3 RUN dnf update -y --best && dnf clean all #mask systemd-machine-id-commit.service - partial fix for https://bugzilla.redhat.com/show_bug.cgi?id=1472439 RUN systemctl mask systemd-remount-fs.service dev-hugepages.mount sys-fs-fuse-connections.mount systemd-logind.service getty.target console-getty.service systemd-udev-trigger.service systemd-udevd.service systemd-random-seed.service systemd-machine-id-commit.service RUN dnf -y install procps-ng && dnf clean all EOF sh-5.0# podman build -t fedora-init-cgroupsv2 temp/ //snip sh-5.0# podman run --name test -d fedora-init-cgroupsv2 && sleep 10 && podman exec test systemctl status 0eefd01dfaa8d9cc5b9abe4c46f60dbc7301eb0916e2c65cac074064310763f6 Error: writing file '/sys/fs/cgroup//machine.slice/libpod-0eefd01dfaa8d9cc5b9abe4c46f60dbc7301eb0916e2c65cac074064310763f6.scope/cgroup.procs': Device or resource busy: OCI runtime error opened a PR here: https://github.com/containers/libpod/pull/3922 The error you are seeing is coming from exec. It is a known issue with joining an existing cgroups v2, and I am still unsure how to fix it correctly. Basically, we cannot join the initial cgroup path as it will have sub directories, so we will need to join a subdirectory. also opened a PR for crun to address the exec issue: https://github.com/containers/crun/pull/81 FEDORA-2019-e53d9e7494 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-e53d9e7494 crun-0.9-1.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-e53d9e7494 FEDORA-2019-f73801f1f2 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-f73801f1f2 crun-0.9.1-1.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-f73801f1f2 crun-0.9.1-1.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report. |