Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1815571

Summary:	Defaulting to lockdown 'confidentiality' breaks all eBPF-based production tooling
Product:	[Fedora] Fedora	Reporter:	Thomas Dullien <thomasdullien>
Component:	kernel	Assignee:	Kernel Maintainer List <kernel-maint>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	31	CC:	airlied, bskeggs, dev, hdegoede, ichavero, itamar, jarodwilson, jcline, jeremy, jglisse, john.j5live, john, jonathan, josef, kernel-maint, linville, masami256, mchehab, mjg59, pbrobinson, steved, wcohen
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	kernel-5.5.11-200.fc31 kernel-5.5.16-100.fc30	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-03-27 13:09:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Thomas Dullien 2020-03-20 15:38:48 UTC

1. Please describe the problem:

Making the default kernel boot into "confidentiality" lockdown breaks a lot of production-observability functionality that relies on eBPF. This means that performance-monitoring tools like BCC will not work (see bug report here: https://github.com/iovisor/bcc/issues/2565) but also that other security tools that rely on eBPF to perform host introspection / monitoring will no longer work by default.

In general, it is unclear to me what the precise threat model is that "confidentiality" lockdown defends against; and why all bpf_probe_read and bpf_probe_read_str is blacklisted etc.

Solution to the problem: Do not default-boot into "lockdown" kernels. IF you absolutely must (and there are few reasons why one would do so), the "integrity" mode is less harmful to production use & inspectability.

The lockdown patchset and "confidentiality" mode are excessive for

2. What is the Version-Release number of the kernel:

The relevant LKML upstream merge is https://lkml.org/lkml/2019/9/10/856
I am not 100% sure when Fedora decided to make "confidentiality" the default.

3. Did it work previously in Fedora? If so, what kernel version did the issue
*first* appear? Old kernels are available for download at
https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :

AFAIK Fedora 30 did not ship a kernel in "confidentiality" lockdown; Fedora 31 does.

4. Can you reproduce this issue? If so, please provide the steps to reproduce
the issue below:

See the bug reports under https://github.com/iovisor/bcc/issues/2565 and https://gehrcke.de/2019/09/running-an-ebpf-program-may-require-lifting-the-kernel-lockdown/

5. Does this problem occur with the latest Rawhide kernel? To install the
Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
``sudo dnf update --enablerepo=rawhide kernel``:

It will; this is not a "bug" per se; rather the decision to enable a highly-controversial and very very intrusive kernel feature by default.

Comment 1 John Viega 2020-03-20 15:51:34 UTC

I 100% agree with Thomas.  Having the default kernel boot into "confidentiality" lockdown will dramatically reduce the security industry's ability to detect real attacks, making this a huge net negative for the industry.  

I'm happy to show you, for instance, a real exploit chain (against published CVEs) that takes a containerized nginx on an SE-Linux enabled system that:

1) Escapes the container
2) Disables SE Linux
3) Does not drop anything that would violate an SELinux policy
4) Puts SE Linux into permissive mode.
5) Does not get detected as an attack by anything stock, including SELinux. 

There are plenty of viable detection techniques that can catch such a chain, all of which will be broken if this patch is the default.

Please, let's learn from the Microsoft PatchGuard debacle.

Comment 2 Jeremy Cline 2020-03-20 20:42:23 UTC

*** Bug 1815663 has been marked as a duplicate of this bug. ***

Comment 3 Jeremy Cline 2020-03-20 22:11:21 UTC

I've adjusted things so if secure boot is on it boots in integrity mode. This is on its way to Rawhide now, and should appear in F31 and F30 with the next stable kernel (5.5.11).

Comment 4 Fedora Update System 2020-03-24 13:12:38 UTC

FEDORA-2020-76966b3419 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-76966b3419

Comment 5 Fedora Update System 2020-03-24 13:12:40 UTC

FEDORA-2020-ded581e74c has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-ded581e74c

Comment 6 Fedora Update System 2020-03-25 09:49:40 UTC

FEDORA-2020-ded581e74c has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-ded581e74c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-ded581e74c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2020-03-25 09:49:52 UTC

FEDORA-2020-76966b3419 has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-76966b3419`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-76966b3419

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Fedora Update System 2020-03-27 13:08:47 UTC

FEDORA-2020-9c05b223cf has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-9c05b223cf`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9c05b223cf

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Fedora Update System 2020-03-27 13:09:41 UTC

FEDORA-2020-76966b3419 has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 10 Fedora Update System 2020-04-04 03:58:01 UTC

FEDORA-2020-cf0857f73a has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-cf0857f73a`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-cf0857f73a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Fedora Update System 2020-04-09 20:14:26 UTC

FEDORA-2020-73c00eda1c has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-73c00eda1c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-73c00eda1c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2020-04-15 21:01:35 UTC

FEDORA-2020-73c00eda1c has been pushed to the Fedora 30 stable repository.
If problem still persists, please make note of it in this bug report.