Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1939485 (CVE-2021-20291)

Summary: CVE-2021-20291 containers/storage: DoS via malicious image
Product: [Other] Security Response Reporter: Mark Cooper <mcooper>
Component: vulnerabilityAssignee: Red Hat Product Security <security-response-team>
Status: CLOSED ERRATA QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: unspecifiedCC: acui, adam.kaplan, alitke, amurdaca, aos-bugs, aos-install, awels, bbaude, bbennett, bdettelb, bmontgom, cnv-qe-bugs, container-sig, dbecker, debarshir, dwalsh, dwhatley, dymurray, eparis, fdeutsch, ibolton, jburrell, jerzhang, jhrozek, jjoyce, jligon, jmatthew, jmontleo, jnovy, jokerman, josorior, jschluet, lhh, lpeer, lsm5, mburns, mheon, mrogers, nalin, nstielau, patrick, pdhamdhe, pehunt, pthomas, rh.container.bot, rhos-maint, rmohr, rphillips, santiago, sclewis, security-response-team, sejug, sgarbour, sgott, sidsharm, slinaber, slucidi, sponnaga, sseago, tomckay, tsweeney, umohnani, xiyuan
Target Milestone: ---Keywords: Security
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: containers/storage 1.28.1 Doc Type: If docs needed, set a value
Doc Text:
A deadlock vulnerability was found in `github.com/containers/storage`. When a container image is processed, each layer is unpacked using `tar`. If one of those layers is not a valid `tar` archive this causes an error leading to an unexpected situation where the code indefinitely waits for the tar unpacked stream, which never finishes. An attacker could use this vulnerability to craft a malicious image, which when downloaded and stored by an application using containers/storage, would then cause a deadlock leading to a Denial of Service (DoS).
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-04-20 22:46:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1949958, 1940479, 1940482, 1942507, 1945639, 1945839, 1945840, 1945841, 1945842, 1945843, 1945844, 1945845, 1947288, 1947289, 2011410    
Bug Blocks: 1945657    

Description Mark Cooper 2021-03-16 13:29:49 UTC
The decompression functionality of github.com/containers/storage in all versions starting from v1.23.8, is susceptible to a Denial of Service (DoS) when used during container image pulls by tools like podman and cri-o.

Comment 9 Mark Cooper 2021-03-24 01:34:25 UTC
Acknowledgments:

Name: Aviv Sasson (Palo Alto Networks)

Comment 20 Mark Cooper 2021-04-01 13:48:20 UTC
Created podman tracking bugs for this issue:

Affects: fedora-all [bug 1945639]

Comment 22 Riccardo Schirone 2021-04-02 08:56:20 UTC
Most versions of podman/buildah/skopeo in Red Hat Enterprise Linux 8 are not affected by this flaw because the version of containers/storage embedded there is old enough to not have the `defer uncompressed.Close()` necessary to trigger this issue.

Comment 23 Mark Cooper 2021-04-08 07:13:15 UTC
The issue was first added/exposed here [1]  and when the defer function close is called regardless if the function is in error or not. For example if an image layer is compressed with something unexpected it creates an unexpected error condition. However since the deferring close function is waiting for the uncompressed stream which will never occur, this creates the DoS.  

Given it really is only exposed since [1], this means it only affects v1.23.8 of containers/storage and greater.

For applications like podman the affect is minimal - podman pull and it seemingly waits to download the image forever, cancel and the affect is negated. For something like crio the affect is more severe, with the malicious image locking the service up, BUT it is still somewhat responsive. The service then must be killed before returning back to normal. 

We've also confirmed when used on OCP, that the node `doesn't` become unresponsive and thus the work load `does not` then gets shifted to another node. 

[1] https://github.com/containers/storage/blob/master/layers.go#L1416

Comment 24 Mark Cooper 2021-04-08 07:25:46 UTC
Created buildah tracking bugs for this issue:

Affects: fedora-all [bug 1947288]


Created skopeo tracking bugs for this issue:

Affects: fedora-all [bug 1947289]

Comment 25 Lokesh Mandvekar 2021-04-08 12:26:11 UTC
Hi Mark, we'll need podman and cri-o bugs as well for this. BTW, cri-o is a fedora module and not regular package.

Comment 26 Lokesh Mandvekar 2021-04-08 12:34:40 UTC
Peter Hunt, the cri-o maintainer said cri-o on fedora should be good to go already. So, we may not need that after all.

Comment 27 Lokesh Mandvekar 2021-04-08 13:39:34 UTC
I heard from podman upstream that podman v3.1.0 also contains the fix already, so we're good to go on the podman side as well.

Comment 31 Mark Cooper 2021-04-16 01:16:03 UTC
External References:

https://unit42.paloaltonetworks.com/cve-2021-20291/

Comment 32 errata-xmlrpc 2021-04-20 18:20:09 UTC
This issue has been addressed in the following products:

  Red Hat OpenShift Container Platform 4.7

Via RHSA-2021:1150 https://access.redhat.com/errata/RHSA-2021:1150

Comment 33 Product Security DevOps Team 2021-04-20 22:46:32 UTC
This bug is now closed. Further updates for individual products will be reflected on the CVE page(s):

https://access.redhat.com/security/cve/cve-2021-20291

Comment 34 Stoyan Nikolov 2021-04-27 11:02:18 UTC
Statement:

The vulnerability was first exposed in v1.23.8 of containers/storage (and subsequently fixed in v1.28.1) with this git commit [1].

* In OpenShift Container Platform (OCP) only 4.7 contains a version of CRI-O (and openshift/ose-docker-builder) which uses a vulnerable version of containers/storage. Subsequently this has been addressed in OCP 4.7.7.

* Red Hat Quay quay-builder-container is not affected because it uses a version of github.com/containers/storage earlier than v1.23.8

* Red Hat Openshift Virtualization is not affected because all containers depending on github.com/containers/storage use a version earlier than v1.23.8

[1] - https://github.com/containers/storage/blob/e9989a949a0d2c32fa82d3ff1076e33cfec58cb9/layers.go#L1332

Comment 36 Fedora Update System 2021-05-04 01:00:38 UTC
FEDORA-2021-a3703b9dc8 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 37 Fedora Update System 2021-05-06 00:52:31 UTC
FEDORA-2021-c56a213327 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 38 Mark Cooper 2021-05-07 11:45:11 UTC
With Migration Toolkit for Contains, the openshift-migration-plugin-container only packages an older version of containers/storage v1.15.3 and hence it is not affected.

Upstream reference: https://github.com/konveyor/openshift-migration-plugin/blob/cbc29cae77bd1046165166d9f5c2c55488d24e2f/Gopkg.lock#L209

Comment 40 errata-xmlrpc 2021-07-27 22:32:04 UTC
This issue has been addressed in the following products:

  Red Hat OpenShift Container Platform 4.8

Via RHSA-2021:2438 https://access.redhat.com/errata/RHSA-2021:2438

Comment 41 errata-xmlrpc 2021-11-09 17:25:22 UTC
This issue has been addressed in the following products:

  Red Hat Enterprise Linux 8

Via RHSA-2021:4154 https://access.redhat.com/errata/RHSA-2021:4154

Comment 43 Riccardo Schirone 2022-05-02 08:14:12 UTC
This issue has been addressed in the following products:

  Red Hat Enterprise Linux 8

Via RHBA-2022:0348 https://access.redhat.com/errata/RHBA-2022:0348

Comment 44 errata-xmlrpc 2022-11-15 09:47:44 UTC
This issue has been addressed in the following products:

  Red Hat Enterprise Linux 9

Via RHSA-2022:7954 https://access.redhat.com/errata/RHSA-2022:7954

Comment 45 errata-xmlrpc 2022-11-15 09:47:58 UTC
This issue has been addressed in the following products:

  Red Hat Enterprise Linux 9

Via RHSA-2022:7955 https://access.redhat.com/errata/RHSA-2022:7955

Comment 46 errata-xmlrpc 2022-11-15 09:56:41 UTC
This issue has been addressed in the following products:

  Red Hat Enterprise Linux 9

Via RHSA-2022:8008 https://access.redhat.com/errata/RHSA-2022:8008