Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1955585
| Summary: | selinux prevents tracing containers with BPF | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Jakub Hrozek <jhrozek> |
| Component: | kernel | Assignee: | Ondrej Mosnacek <omosnace> |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 34 | CC: | acaringi, adscvr, airlied, alciregi, bskeggs, bugzilla, dwalsh, grepl.miroslav, gwhite, hdegoede, jarodwilson, jeremy, jforbes, jglisse, jloscar, jonathan, josef, kernel-maint, lgoncalv, linville, lvrabec, masami256, mchehab, mmalik, omosnace, plautrba, ptalbert, steved, vmojzis, zpytela |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | kernel-5.12.10-300.fc34 kernel-5.12.10-200.fc33 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-06-15 01:05:17 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Actually, this seems to be a kernel bug. Apparently there is a security_locked_down() call in a few BPF helper functions, so it gets called when the BPF program is run, leading to SELinux looking up the permission for the domain of the process that performs the given syscall and not the one that loaded the program. I'm currently working on a patch to fix this and a few related issues, so reassigning this to myself. Anyway, I don't understand why this particular program is triggering the calls, since it doesn't call any of the problematic helpers explicitly... There might be some BPF compiler weirdness behind this, too... *** Bug 1958025 has been marked as a duplicate of this bug. *** Also with 5.12.2-300.fc34.x86_64+debug. And it's rampant enough that the audit.log is being rotated every 5 seconds. FYI, in the meantime I posted a patch to fix this issue upstream: [v1] https://lore.kernel.org/selinux/20210507114048.138933-1-omosnace@redhat.com/T/ [v2] https://lore.kernel.org/selinux/20210517092006.803332-1-omosnace@redhat.com/T/ @Ondrej do you mind updating here when this patch has been accepted upstream. It doesn't have to be in linus tree yet, just accepted upstream. I can pull it back to 5.12 and newer releases at that point. So, this particular issue has now been addressed on the BPF side with this commit: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/kernel/bpf?id=ff40e51043af63715ab413995ff46996ecf9583f There is still an an ongoing discussion on how to address the problem better from security perspective, but that's beyond the scope of this bug. @Justin, it would be good to know which kernel release this lands in for Fedora. If you're able to update here when you've pulled it into 5.12 or other releases. Thanks! BTW, the patch has been queued for the 5.12.10 stable release yesterday [1], so it may be best to just let Fedora inherit it naturally from upstream at this point. [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/commit/?h=linux-5.12.y&id=de9dc6a1cc8d21cae0efa923dc370683ac842308 OK, great. Thanks very much. Yes, 5.12.10 is in rc phase now, expected to release Thursday sometime. Depending on that time, the build will be done for Fedora either Thursday or Friday. This bug will be included in the update, so bodhi should auto post here as it is filed and works through to stable. FEDORA-2021-bc2a819bc5 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-bc2a819bc5 FEDORA-2021-db2bb87f35 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-db2bb87f35 FEDORA-2021-db2bb87f35 has been pushed to the Fedora 33 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-db2bb87f35` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-db2bb87f35 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates. FEDORA-2021-bc2a819bc5 has been pushed to the Fedora 34 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-bc2a819bc5` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-bc2a819bc5 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates. FEDORA-2021-bc2a819bc5 has been pushed to the Fedora 34 stable repository. If problem still persists, please make note of it in this bug report. FEDORA-2021-db2bb87f35 has been pushed to the Fedora 33 stable repository. If problem still persists, please make note of it in this bug report. |
Description of problem: I starting seeing this with F-34. When I run a container that is traced with eBPF to record the syscalls it is doing, auditd is flooded with messages like: type=AVC msg=audit(1619784520.593:282387): avc: denied { confidentiality } for pid=476 comm="auditd" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:auditd_t:s0 tcontext=system_u:system_r:auditd_t:s0 tclass=lockdown permissive=0 This seems to be leading to auditd running out of space in the backlog buffer and eventually OOMs the machine. Version-Release number of selected component (if applicable): podman-3.1.2-1.fc34.x86_64 selinux-policy-34-1.fc34.noarch container-selinux-2.160.0-2.fc34.noarch oci-seccomp-bpf-hook-1.2.2-0.1.git4e42394.fc34.x86_64 kernel-core-5.11.12-300.fc34.x86_64 How reproducible: always Steps to Reproduce: 1. sudo podman run --name=seccomp-test --privileged --annotation io.containers.trace-syscall="of:/tmp/selinuxd-seccomp.json" fedora sh (the container must be started as root for the bpf hook to work) Actual results: auditd running at 99% CPU presumably processing all the messages, eventually I get: Apr 30 12:20:42 fedora kernel: audit: backlog limit exceeded Apr 30 12:20:42 fedora kernel: audit: backlog limit exceeded Apr 30 12:20:42 fedora kernel: audit: audit_backlog=2152579 > audit_backlog_limit=64 Apr 30 12:20:42 fedora kernel: audit: audit_backlog=2152626 > audit_backlog_limit=64 Apr 30 12:20:42 fedora kernel: audit: audit_backlog=2152694 > audit_backlog_limit=64 Apr 30 12:20:42 fedora kernel: audit: audit_lost=6878426 audit_rate_limit=0 audit_backlog_limit=64 Apr 30 12:20:45 fedora kernel: oci-seccomp-bpf invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=-1000 Apr 30 12:20:45 fedora kernel: CPU: 0 PID: 13284 Comm: oci-seccomp-bpf Not tainted 5.11.12-300.fc34.x86_64 #1 Apr 30 12:20:45 fedora kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-2.fc32 04/01/2014 Expected results: the BPF hook should work as it did in F-33 Additional info: I don't know if the issue is in selinux-policy, the kernel, the eBPF hook or somewhere else. Feel free to reassign the bug as appropriate or ask for any other information.