Bug 1440875

Created attachment 1270529 [details]
journalctl

This looks like you are mounting and attempting to run fusefs content under a container?

Have you turned on the virt_sandbox_use_fusefs boolean?

No changes were made to the default. This works on other arches (x86, armhfp) without turning on that boolean. 
 
On x86:

sudo docker run -it fedora-docker-base-26_alpha-1.7.x86_64 /bin/bash
[root@0b069afd9787 /]# exit
[pwhalen@i7 ~]$ getenforce 
Enforcing
[pwhalen@i7 ~]$ getsebool virt_sandbox_use_fusefs
virt_sandbox_use_fusefs --> off

On aarch64, when the boolean is turned on it does work as expected:

setsebool virt_sandbox_use_fusefs on
docker run -it --rm fedora-docker-base-26_alpha-1.7.aarch64 /bin/bash
[root@56b0cce69c90 /]# exit

Still seeing this with:

selinux-policy-3.13.1-254.fc26.noarch
docker-1.13.1-13.git51eb16e.fc26.aarch64

Created attachment 1283527 [details]
docker on aarch64

I need avc's.

ausearch -m avc -ts recent

time->Wed May 31 14:52:30 2017
type=AVC msg=audit(1496256750.728:262): avc:  denied  { read execute } for  pid=2078 comm="bash" path="/usr/bin/bash" dev="dm-0" ino=25705300 scontext=system_u:system_r:container_t:s0:c73,c693 tcontext=system_u:object_r:fusefs_t:s0 tclass=file permissive=0

setsebool -P virt_sandbox_use_fusefs 1

Should enable these.

(In reply to Paul Whalen from comment #0)
> Created attachment 1270528 [details]
> Audit log
> 
> Description of problem:
> Docker image fails to run when selinux is running in enforcing on aarch64.
> 
> Version-Release number of selected component (if applicable):
> selinux-policy-3.13.1-249.fc26.noarch
> docker-1.13.1-6.git5be1549.fc26.aarch64
> 
> How reproducible:
> Everytime.
> 
> Steps to Reproduce:
> 1. Download and load F26 Alpha docker image for aarch64.
> 2. docker run -it --rm fedora-docker-base-26_alpha-1.7.aarch64 /bin/bash

Similar problem on ppc64le as well. Have to do setenforce 0 in order to run container. Tried it on F26 beta 1.4 docker image (https://kojipkgs.fedoraproject.org/compose/26/Fedora-26-20170531.0/compose/Docker/ppc64le/images/)

Did turning on this boolean fix the issue?

(In reply to Daniel Walsh from comment #10)
> Did turning on this boolean fix the issue?

Yes Dan, works fine after setting virt_sandbox_use_fusefs on

Before I close this, are you actually using fusefs?

(In reply to Daniel Walsh from comment #12)
> Before I close this, are you actually using fusefs?

I haven't done any modification with base docker image (obtained from https://kojipkgs.fedoraproject.org/compose/26/Fedora-26-20170531.0/compose/Docker/ppc64le/images/ )

After downloading and loading image, I simply ran 
$ sudo docker run -i -t image_name /bin/bash

which didn't succeed.

Ouput of command "ausearch -m avc -ts recent" gives following result though:

time->Fri Jun  2 09:39:06 2017
type=AVC msg=audit(1496389146.567:409): avc:  denied  { read execute } for  pid=2905 comm="bash" path="/usr/bin/bash" dev="dm-0" ino=17010971 scontext=system_u:system_r:container_t:s0:c726,c987 tcontext=system_u:object_r:fusefs_t:s0 tclass=file permissive=0

Later with setting virt_sandbox_use_fusefs ON , docker container runs fine

Inside of the container what does

ls -lZ /usr

Show?  Is everything on a fusefs_t file system?

Created attachment 1284395 [details]
selinux content of /usr on ppc64le

No, everything are of type container_file_t

The question I have is where are these fuse file system files coming from?

I didn't find much useful data to understand from where fuse FS files are coming.

I tried following inside container in case it is useful:
[root@c82177200ad4 /]# ls -lRZ | grep fuse
lrwxrwxrwx. 1 root root system_u:object_r:container_file_t:s0:c153,c932  9 Jun  1 01:16 sys-fs-fuse-connections.mount -> /dev/null
-rw-r--r--. 1 root root system_u:object_r:container_file_t:s0:c153,c932  754 Mar 16 09:25 sys-fs-fuse-connections.mount
lrwxrwxrwx. 1 root root system_u:object_r:container_file_t:s0:c153,c932 32 Mar 16 09:25 sys-fs-fuse-connections.mount -> ../sys-fs-fuse-connections.mount

sys-fs-fuse-connections.mount is coming from systemd package.

Note: Didn't find sys-fs-fuse-connections.mount file inside x86_64 docker container (docker image taken from https://kojipkgs.fedoraproject.org/compose/26/Fedora-26-20170531.0/compose/Docker/x86_64/images/)

Also tried, 
[root@c82177200ad4 /]# mount |grep fuse

which has no matching result

Let me know if I should try something else.

If you execute that command on the host, do you see much?

Created attachment 1284491 [details]
selinux context of /var on ppc64le host vm

On host machine, selinux context of /var seems interesting (rest of directories under / doesn't have any fusefs_t type). Especially files under directory /var/lib/docker/overlay2/ seem to have fusefs_t FS type.

That would seem to be a bug or a setup problem.  Those files should be on the local system...

I think there is a problem with overlayfs on aarch64?

This is working on aarch64 in the F27 Beta. 

[pwhalen@mustang ~]$ uname -r
4.13.3-300.fc27.aarch64
[pwhalen@mustang ~]$ getenforce
Enforcing
[pwhalen@mustang ~]$ getsebool virt_sandbox_use_fusefs
virt_sandbox_use_fusefs --> off
[pwhalen@mustang ~]$ sudo docker run -it --rm fedora-docker-base-27_beta-1.5.aarch64 bash
[root@9bba18573d12 /]#

Problem still exist on ppc64le, F27 Beta.
Was there any changes made to fix this issue?

Sinny please attach AVC messages.

Created attachment 1335349 [details]
Audit log on ppc64le

Hi Dan,

AVC message on F27 Beta, ppc64le:

type=AVC msg=audit(1507186032.058:285): avc:  denied  { map } for  pid=1956 comm="bash" path="/usr/bin/bash" dev="dm-0" ino=1338074 scontext=system_u:system_r:container_t:s0:c20,c706 tcontext=system_u:object_r:fusefs_t:s0 tclass=file permissive=0
type=ANOM_ABEND msg=audit(1507186032.059:286): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:container_t:s0:c20,c706 pid=1956 comm="bash" exe="/usr/bin/bash" sig=11 res=1

Complete audit log is available in attachment.

Stephen, you say that this is only happening when a file is mmap'd?  Do you think bash memory mapps fusefs files?

map permission is only checked during mmap(2) or other calls that internally invoke vm_mmap(), e.g. execve(2) for mapping the executable. bash could easily be invoking mmap(2) directly or via a glibc function for reading a file; the fact that the file happens to live on fusefs is immaterial.
If you enable syscall audit, then we can see the actual system call that triggered the avc denial.

But that means we really need to be fairly loose with the map permissions, if random bash scripts can cause breakage.  I believe we should add it to can_exec.

https://github.com/fedora-selinux/selinux-policy/pull/203

Should fix this along with an updated container-selinux.

Lukas we need this for F27 and F26, and I will rebuild container-selinux off of it.

Dan, 

We have map permission only in F27+. I would say that if you rebuild container-selinux with the latest patches from selinux-policy, it should be fixed.

Fixed in container-selinux-2.28.

Thanks Dan!
It works now on ppc64le as well.

# getenforce
Enforcing
# getsebool virt_sandbox_use_fusefs
virt_sandbox_use_fusefs --> off
# rpm -qa container-selinux
container-selinux-2.28-1.fc27.noarch
# docker run --rm -it fedora-docker-base-27-20171010.n.0.ppc64le bash
[root@bf98e838a90f /]#

Closing as this was fixed in F28. Thanks!