Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1966842 - SELinux is preventing virtlogd from 'read, append' accesses on the file system.token [NEEDINFO]
Summary: SELinux is preventing virtlogd from 'read, append' accesses on the file syste...
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: selinux-policy
Version: 8.5
Hardware: Unspecified
OS: Linux
high
urgent
Target Milestone: beta
: 8.5
Assignee: Zdenek Pytela
QA Contact: Milos Malik
URL:
Whiteboard:
: 1969861 (view as bug list)
Depends On: 1964317
Blocks: 1969209
TreeView+ depends on / blocked
 
Reported: 2021-06-02 02:33 UTC by Fangge Jin
Modified: 2023-07-13 02:44 UTC (History)
29 users (show)

Fixed In Version: selinux-policy-3.14.3-71.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1964317
: 1969209 (view as bug list)
Environment:
Last Closed:
Type: Bug
Target Upstream Version:
Embargoed:
hhan: needinfo? (berrange)

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker AUTOBUG-72 0 None None None 2022-10-18 08:29:52 UTC
Red Hat Issue Tracker REST-562 0 None None None 2022-01-21 08:33:06 UTC

Description Fangge Jin 2021-06-02 02:33:31 UTC 
+++ This bug was initially created as a clone of Bug #1964317 +++

Description of problem:
As subject

Version-Release number of selected component (if applicable):
libvirt-7.4.0-1.module+el8.5.0+11218+83343022.x86_64
selinux-policy-3.14.3-68.el8.noarch

How reproducible:
100%

Steps to Reproduce:
1. Start a vm
# virsh start avocado-vt-vm1
error: Failed to start domain 'avocado-vt-vm1'
error: can't connect to virtlogd: Unable to open system token /run/libvirt/common/system.token: Permission denied

Jun 01 22:09:58 fjin-8-5 setroubleshoot[40499]: SELinux is preventing /usr/sbin/virtlogd from 'read, append' accesses on the file system.token. For complete SELinux messages run: sealert -l f4215403-b8fa-4189-a182-56ff03491a7f
Jun 01 22:09:58 fjin-8-5 setroubleshoot[40499]: SELinux is preventing /usr/sbin/virtlogd from 'read, append' accesses on the file system.token.
                                                
                                                *****  Plugin catchall (100. confidence) suggests   **************************
                                                
                                                If you believe that virtlogd should be allowed read append access on the system.token file by default.
                                                Then you should report this as a bug.
                                                You can generate a local policy module to allow this access.
                                                Do
                                                allow this access for now by executing:
                                                # ausearch -c 'virtlogd' --raw | audit2allow -M my-virtlogd
                                                # semodule -X 300 -i my-virtlogd.pp



Actual results:
As above

Expected results:
No avc deny

Additional info:

--- Additional comment from Daniel Berrangé on 2021-05-26 14:59:26 UTC ---

Note that this system.token file will be created by whichever of the libvirt related daemons is the first to need it. It is intended to be accessible to all libvirt daemons. Essentially it is a way for 1 libvirt daemon to validate that it is talking to another libvirt, as opposed to a 3rd party libvirt client. Libvirtd will acquire fcntl() locks on this file when first creating it to ensure race-free access.
Comment 1 Fangge Jin 2021-06-02 02:37:10 UTC 
type=AVC msg=audit(1622599792.544:1738): avc:  denied  { read append } for  pid=16525 comm="virtlogd" name="system.token" dev="tmpfs" ino=129891 scontext=system_u:system_r:virtlogd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:virt_var_run_t:s0 tclass=file permissive=0
Comment 2 Fangge Jin 2021-06-02 02:50:50 UTC 
Additional info:
1. Can't be reproduced with libvirt-7.3.0

2. I have confirmed that libvirtd and virtlogd are running

3. After the first failure, try to start vm again, it failed as first time, but there is NO new avc denied info produced in audit.log

4. Restart virtlogd, then try to start vm again, it failed as before, and there is new avc denied info produced in audit.log

5. "setenforce 0" won't work around this issue
Comment 3 Fangge Jin 2021-06-02 06:11:29 UTC 
One workaround is:
set stdio_handler = "file" in /etc/libvirt/qemu.conf, and restart libvirtd
Comment 4 Michal Privoznik 2021-06-02 11:16:39 UTC 
@fjin should this be filed against selinux-policy instead of libvirt (just like the fedora bug)?
Comment 7 Fangge Jin 2021-06-03 02:27:47 UTC 
Another workaround:
Start libvirtd and virtlogd by shell command:
1) # systemctl stop libvirtd; systesmctl stop virtlogd; systemctl stop libvirtd.socket; systemctl stop virtlogd.socket
2) # libvirtd -d; virtlogd -d
Comment 8 Daniel Berrangé 2021-06-04 11:27:32 UTC 
(In reply to Fangge Jin from comment #2)
> Additional info:
> 1. Can't be reproduced with libvirt-7.3.0
> 
> 2. I have confirmed that libvirtd and virtlogd are running
> 
> 3. After the first failure, try to start vm again, it failed as first time,
> but there is NO new avc denied info produced in audit.log
> 
> 4. Restart virtlogd, then try to start vm again, it failed as before, and
> there is new avc denied info produced in audit.log
> 
> 5. "setenforce 0" won't work around this issue

'setenforce 0' should work, but the error message is cached by libvirtd as it is hit in a one-time initializer. So after making selinux permissive, you need to also restart libvirtd I believe. There's no need to restart virtlogd.
Comment 9 Fangge Jin 2021-06-04 13:13:03 UTC 
Hi Daniel

I just tried, it seems that restart virtlogd after 'setenforce 0' can work around the issue.

1. Start a vm, it failed
[root@fjin-8-5-1 ~]# virsh start avocado-vt-vm1
error: Failed to start domain 'avocado-vt-vm1'
error: can't connect to virtlogd: Unable to open system token /run/libvirt/common/system.token: Permission denied


2. Set enforce to 0, restart libvirtd, then try to start vm, still failed
[root@fjin-8-5-1 ~]# setenforce 0

[root@fjin-8-5-1 ~]# systemctl restart libvirtd

[root@fjin-8-5-1 ~]# virsh start avocado-vt-vm1
error: Failed to start domain 'avocado-vt-vm1'
error: can't connect to virtlogd: Unable to open system token /run/libvirt/common/system.token: Permission denied


3. Restart virtlogd, then try to start vm, can succeed
[root@fjin-8-5-1 ~]# systemctl restart virtlogd
[root@fjin-8-5-1 ~]# virsh start avocado-vt-vm1
Domain 'avocado-vt-vm1' started

[root@fjin-8-5-1 ~]# virsh destroy avocado-vt-vm1
Domain 'avocado-vt-vm1' destroyed


4. Now, set enforce to 1, try to start vm, can succeed
[root@fjin-8-5-1 ~]# setenforce 1

[root@fjin-8-5-1 ~]# virsh start avocado-vt-vm1
Domain 'avocado-vt-vm1' started

[root@fjin-8-5-1 ~]# virsh destroy !$
virsh destroy avocado-vt-vm1


5. Then restart virtlogd, and try to start vm, it failed.
[root@fjin-8-5-1 ~]# systemctl restart virtlogd

[root@fjin-8-5-1 ~]# virsh start avocado-vt-vm1
error: Failed to start domain 'avocado-vt-vm1'
error: can't connect to virtlogd: Unable to open system token /run/libvirt/common/system.token: Permission denied
Comment 10 nknazeko 2021-06-07 07:52:52 UTC 
Hi Fangge, do you have any other AVC messages after reproducing this with setenforce 0?
Comment 11 Fangge Jin 2021-06-07 08:14:41 UTC 
(In reply to nknazeko from comment #10)
> Hi Fangge, do you have any other AVC messages after reproducing this with
> setenforce 0?

There is no new AVC denied info after setenfore 0.
And if restart virtlogd after setenforce 0, it can't be reproduced.
Comment 12 Milos Malik 2021-06-07 10:06:44 UTC 
Could you run the following commands on a machine where the problems appears?

# ls -lZ /run/libvirt/common/system.token
# matchpathcon /run/libvirt/common/system.token

Thank you.
Comment 13 Fangge Jin 2021-06-07 11:06:10 UTC 
[root@fjin-8-5-1 ~]# ll /run/libvirt/common/system.token -Z
-rw-------. 1 root root system_u:object_r:virt_var_run_t:s0 32 Jun  2 02:08 /run/libvirt/common/system.token

[root@fjin-8-5-1 ~]# matchpathcon /run/libvirt/common/system.token
/run/libvirt/common/system.token	system_u:object_r:virt_var_run_t:s0
Comment 15 Nir Soffer 2021-06-08 22:01:10 UTC 
This breaks oVirt users using Cantos Stream.
Comment 20 Zdenek Pytela 2021-06-14 11:30:10 UTC 
Hello,

There is a selinux-policy PR available, I verified the token file is created with expected label, but I cannot verify the fix is complete and sufficient to back the libvirt feature. Should there be somebody able to test it out, please either use the PR builds for Fedora 34 and 35:
https://github.com/fedora-selinux/selinux-policy/pull/777
Show all checks -> build-rpm -> Details -> Artifacts -> rpms

or apply the following steps on any system:

1. Install devel package
dnf install setools-console selinux-policy-devel -y

2. Create a local selinux policy module

  # cat local_virt.fc
/var/run/libvirt/common(/.*)?          gen_context(system_u:object_r:virt_common_var_run_t,s0)
  # cat local_virt.te
policy_module(local_virt, 1.0)

gen_require(`
        type virt_var_run_t;
        type virtd_t;
        type virtlogd_t;
')

type virt_common_var_run_t;
files_pid_file(virt_common_var_run_t)

allow virtd_t virt_common_var_run_t:file { append_file_perms read_file_perms };
manage_dirs_pattern(virtd_t, virt_common_var_run_t, virt_common_var_run_t)
manage_files_pattern(virtd_t, virt_common_var_run_t, virt_common_var_run_t)
filetrans_pattern(virtd_t, virt_var_run_t, virt_common_var_run_t, dir, "common")

allow virtlogd_t virt_common_var_run_t:file { append_file_perms read_file_perms };
manage_files_pattern(virtlogd_t, virt_common_var_run_t, virt_common_var_run_t)

  # make -f /usr/share/selinux/devel/Makefile local_virt.pp
  # semodule -i local_virt.pp

3. Reproduce the scenario
4. Look for avc denials
ausearch -i -m avc,user_avc,selinux_err,user_selinux_err -ts recent
Comment 21 Daniel Berrangé 2021-06-14 11:51:26 UTC 
(In reply to Zdenek Pytela from comment #20)
> Hello,
> 
> There is a selinux-policy PR available, I verified the token file is created
> with expected label, but I cannot verify the fix is complete and sufficient
> to back the libvirt feature. Should there be somebody able to test it out,
> please either use the PR builds for Fedora 34 and 35:
> https://github.com/fedora-selinux/selinux-policy/pull/777
> Show all checks -> build-rpm -> Details -> Artifacts -> rpms

I tested libvirt on a Fedora 34 host with this build of the policy and it solved the problems I first hit.
Comment 22 Zdenek Pytela 2021-06-14 12:04:21 UTC 
(In reply to Daniel Berrangé from comment #21)
> I tested libvirt on a Fedora 34 host with this build of the policy and it
> solved the problems I first hit.
Thank you, merging the PR to move it to further testing then.
Comment 24 Han Han 2021-06-15 07:26:52 UTC 
What's more, an dependency of bug-fixed selinux-policy should be added for the libvirt>=7.4(the version with /run/libvirt/common).

Otherwise, when the user update libvirt only to the latest. They will encounter the bug, either. The bug is even there unless they update selinux-policy and then cleanup /run/libvirt/common and restart libvirtd+virtlogd: https://bugzilla.redhat.com/show_bug.cgi?id=1964317#c16

What's your idea?
Comment 25 Zdenek Pytela 2021-06-15 08:40:50 UTC 
(In reply to Han Han from comment #24)
> What's more, an dependency of bug-fixed selinux-policy should be added for
> the libvirt>=7.4(the version with /run/libvirt/common).
> 
> Otherwise, when the user update libvirt only to the latest. They will
> encounter the bug, either. The bug is even there unless they update
> selinux-policy and then cleanup /run/libvirt/common and restart
> libvirtd+virtlogd: https://bugzilla.redhat.com/show_bug.cgi?id=1964317#c16
A part of selinux-policy rpm scriptlets is relabeling of files were the label changed, but it does not apply to filesystems like /run.

So if the update order is 1. selinux-policy 2. libvirtd, the service restart with cleanup would help.
In the inverse order the labels are kept in the previous state until libvirtd restart, but I suppose this part was not working anyway before.
Comment 26 Han Han 2021-06-15 09:08:36 UTC 
(In reply to Zdenek Pytela from comment #25)
> (In reply to Han Han from comment #24)
> > What's more, an dependency of bug-fixed selinux-policy should be added for
> > the libvirt>=7.4(the version with /run/libvirt/common).
> > 
> > Otherwise, when the user update libvirt only to the latest. They will
> > encounter the bug, either. The bug is even there unless they update
> > selinux-policy and then cleanup /run/libvirt/common and restart
> > libvirtd+virtlogd: https://bugzilla.redhat.com/show_bug.cgi?id=1964317#c16
> A part of selinux-policy rpm scriptlets is relabeling of files were the
> label changed, but it does not apply to filesystems like /run.
> 
> So if the update order is 1. selinux-policy 2. libvirtd, the service restart
> with cleanup would help.
> In the inverse order the labels are kept in the previous state until
> libvirtd restart, but I suppose this part was not working anyway before.

Yes. From the customer perspective, a user may update 2 first and 1 then. We should make sure there is always the 1->2 order.
Comment 27 Zdenek Pytela 2021-06-16 07:16:45 UTC 
Hi team,

The selinux-policy-3.14.3-71.el8 build is now available with support for the /run/libvirt/common/system.token feature. Please remove the local policy and use this package to test if everything works as expected.
Comment 32 Han Han 2021-06-17 09:59:06 UTC 
Any idea on the comment24 to coment26? Do we need to file a bug against libvirt?
Comment 33 Zdenek Pytela 2021-06-17 19:59:36 UTC 
*** Bug 1969861 has been marked as a duplicate of this bug. ***
Comment 34 Fangge Jin 2021-06-18 03:01:16 UTC 
(In reply to Han Han from comment #32)
> Any idea on the comment24 to coment26? Do we need to file a bug against
> libvirt?

I assume selinux-policy and libvirt will be updated together when customer upgrade from RHEL8.4 to RHEL8.5, but I'm not so sure about this. 

And considering if customer deploy libvirtd and virtlogd by shell command "libvirtd -d; virtlogd -d", the issue in this bug won't happen. Is it suitable to add dependency on selinux-policy in this condition?
Comment 35 Han Han 2021-06-18 03:23:23 UTC 
(In reply to Fangge Jin from comment #34)
> (In reply to Han Han from comment #32)
> > Any idea on the comment24 to coment26? Do we need to file a bug against
> > libvirt?
> 
> I assume selinux-policy and libvirt will be updated together when customer
> upgrade from RHEL8.4 to RHEL8.5, but I'm not so sure about this. 
From the former z-stream bug https://bugzilla.redhat.com/show_bug.cgi?id=1679569 , this assumption is not correct.
> 
> And considering if customer deploy libvirtd and virtlogd by shell command
> "libvirtd -d; virtlogd -d", the issue in this bug won't happen. Is it
> suitable to add dependency on selinux-policy in this condition?
Comment 36 Han Han 2021-06-18 03:44:27 UTC 
New a libvirt bug on the dependency: https://bugzilla.redhat.com/show_bug.cgi?id=1973510
Note You need to log in before you can comment on or make changes to this bug.