Hans,

Without the denials it will be quite hard to troubleshoot. I understand auditd is not running, but is journal still accessible? With persistent journal files, the AVC denials should be visible after reboot.

We can try small steps with inserting local policy modules.
Can you explain the new functionality?
- how the plymouthd-drm-escrow is executed, is is a service?
- which files are accessed and which resources are used by plymouthd or a child processes during shutdown that were not used before?

> But I guess that plymouthd is simply forbidden from execing anything by its current selinux policy and that this is the problem.
The same policy applies as during the system lifetime.

> It would be nice if we can get this fixed before Fedora 34 final.
I am afraid this is hardly possible given the deadline next Tuesday.

(In reply to Zdenek Pytela from comment #1)
> Hans,
> 
> Without the denials it will be quite hard to troubleshoot. I understand
> auditd is not running, but is journal still accessible? With persistent
> journal files, the AVC denials should be visible after reboot.

Yes the journal is still visible, but I could not find anything
in there. I do see these:

Apr 01 11:53:34 x1.localdomain audit[3638]: AVC avc:  denied  { read } for  pid=3638 comm="plymouthd" name="cmdline" dev="proc" ino=43089 scontext=system_u:system_r:plymouthd_t:s0 tcontext=system_u:system_r:init_t:s0 tclass=file permissive=1
Apr 01 11:53:34 x1.localdomain audit[3638]: AVC avc:  denied  { open } for  pid=3638 comm="plymouthd" path="/proc/3640/cmdline" dev="proc" ino=43089 scontext=system_u:system_r:plymouthd_t:s0 tcontext=system_u:system_r:init_t:s0 tclass=file permissive=1
Apr 01 11:53:34 x1.localdomain audit[3638]: AVC avc:  denied  { getattr } for  pid=3638 comm="plymouthd" path="/proc/3640/stat" dev="proc" ino=43090 scontext=system_u:system_r:plymouthd_t:s0 tcontext=system_u:system_r:init_t:s0 tclass=file permissive=1

In the journal which happen shortly before the troublesome exec. Notice that these are still logged by the audit daemon, I think that the audit team turns of direct logging of AVC-s by the kernel when it starts; and maybe it does not re-enable this when it quits?

BTW these are harmless, they are plymouthd reading the cmdline from a request coming from the "plymouth" client which controls it to log some debug info. Things work fine without these. Regardless this is a different issue. I guess I should also file a separate bug for this, because in the end we don't want to have any denials in the journal / audit logs.

> We can try small steps with inserting local policy modules.

That sounds like a good idea, thank you for your help with this.

> Can you explain the new functionality?
> - how the plymouthd-drm-escrow is executed, is is a service?
> - which files are accessed and which resources are used by plymouthd or a
> child processes during shutdown that were not used before?

The new functionality is this:

pid = fork ();
if (pid == 0) {
  const char *argv[] = { "/usr/libexec/plymouthd-drm-escrow", NULL };
  execve (argv[0], (char * const *) argv, NULL);
}

Where /usr/libexec/plymouthd-drm-escrow is:

int
main(int argc, char **argv)
{
        signal (SIGTERM, SIG_IGN);

        /* Make the first byte in argv be '@' so that we can survive systemd's killing
         * spree until the power is killed at shutdown.
         * http://www.freedesktop.org/wiki/Software/systemd/RootStorageDaemons
         */
        argv[0][0] = '@';

        while (pause());

        return 0;
}

plymouthd does already use fork, so AFAICT the only new permission which it needs is executing "/usr/libexec/plymouthd-drm-escrow" and "/usr/libexec/plymouthd-drm-escrow" can stay in the same process-context and it does not need any special rights.

> > It would be nice if we can get this fixed before Fedora 34 final.
> I am afraid this is hardly possible given the deadline next Tuesday.

Sorry I was not clear, what I meant was that it would be nice if we can get this fixed with a 0-day update. I understand that we are way too close to the freeze to get a fix included in final.

(In reply to Hans de Goede from comment #2)
> (In reply to Zdenek Pytela from comment #1)
> > Hans,
> > 
> > Without the denials it will be quite hard to troubleshoot. I understand
> > auditd is not running, but is journal still accessible? With persistent
> > journal files, the AVC denials should be visible after reboot.
> 
> Yes the journal is still visible, but I could not find anything
> in there. I do see these:
> 
> Apr 01 11:53:34 x1.localdomain audit[3638]: AVC avc:  denied  { read } for 
> pid=3638 comm="plymouthd" name="cmdline" dev="proc" ino=43089
> scontext=system_u:system_r:plymouthd_t:s0
> tcontext=system_u:system_r:init_t:s0 tclass=file permissive=1
> Apr 01 11:53:34 x1.localdomain audit[3638]: AVC avc:  denied  { open } for 
> pid=3638 comm="plymouthd" path="/proc/3640/cmdline" dev="proc" ino=43089
> scontext=system_u:system_r:plymouthd_t:s0
> tcontext=system_u:system_r:init_t:s0 tclass=file permissive=1
> Apr 01 11:53:34 x1.localdomain audit[3638]: AVC avc:  denied  { getattr }
> for  pid=3638 comm="plymouthd" path="/proc/3640/stat" dev="proc" ino=43090
> scontext=system_u:system_r:plymouthd_t:s0
> tcontext=system_u:system_r:init_t:s0 tclass=file permissive=1
> 
> In the journal which happen shortly before the troublesome exec. Notice that
> these are still logged by the audit daemon, I think that the audit team
> turns of direct logging of AVC-s by the kernel when it starts; and maybe it
> does not re-enable this when it quits?
> 
> BTW these are harmless, they are plymouthd reading the cmdline from a
> request coming from the "plymouth" client which controls it to log some
> debug info. Things work fine without these. Regardless this is a different
> issue. I guess I should also file a separate bug for this, because in the
> end we don't want to have any denials in the journal / audit logs.
You can see those in permissive mode as the access to the directory is dontaudited:

vm-f34# sesearch --dontaudit -s plymouthd_t -t init_t -c dir
dontaudit daemon init_t:dir { getattr open search };

> 
> > We can try small steps with inserting local policy modules.
> 
> That sounds like a good idea, thank you for your help with this.
> 
Let's start with:

vm-f34# cat local_plymouthd.cil
(allow plymouthd_t bin_t (file (getattr ioctl lock open read execute execute_no_trans map)))
vm-f34# semodule -i local_plymouthd.cil

Note you need to have the system in selinux enforcing mode to see the difference. I suppose the process is plymouthd started from one of the plymouth services.

> > Can you explain the new functionality?
> > - how the plymouthd-drm-escrow is executed, is is a service?
> > - which files are accessed and which resources are used by plymouthd or a
> > child processes during shutdown that were not used before?
> 
> The new functionality is this:
> 
> pid = fork ();
> if (pid == 0) {
>   const char *argv[] = { "/usr/libexec/plymouthd-drm-escrow", NULL };
>   execve (argv[0], (char * const *) argv, NULL);
> }
> 
> Where /usr/libexec/plymouthd-drm-escrow is:
> 
> int
> main(int argc, char **argv)
> {
>         signal (SIGTERM, SIG_IGN);
> 
>         /* Make the first byte in argv be '@' so that we can survive
> systemd's killing
>          * spree until the power is killed at shutdown.
>          *
> http://www.freedesktop.org/wiki/Software/systemd/RootStorageDaemons
>          */
>         argv[0][0] = '@';
> 
>         while (pause());
> 
>         return 0;
> }
> 
> plymouthd does already use fork, so AFAICT the only new permission which it
> needs is executing "/usr/libexec/plymouthd-drm-escrow" and
> "/usr/libexec/plymouthd-drm-escrow" can stay in the same process-context and
> it does not need any special rights.
These files have the generic bin_t label. Also note other plymouth executables are in /usr/libexec/plymouth/.

> 
> > > It would be nice if we can get this fixed before Fedora 34 final.
> > I am afraid this is hardly possible given the deadline next Tuesday.
> 
> Sorry I was not clear, what I meant was that it would be nice if we can get
> this fixed with a 0-day update. I understand that we are way too close to
> the freeze to get a fix included in final.
Yep.

> Let's start with:
>
> vm-f34# cat local_plymouthd.cil
> (allow plymouthd_t bin_t (file (getattr ioctl lock open read execute execute_no_trans map)))
> vm-f34# semodule -i local_plymouthd.cil

Thank you, that works. I've run a bunch of tests to figure out the minimal permission set and this also works:

(allow plymouthd_t bin_t (file (execute execute_no_trans map)))

> > plymouthd does already use fork, so AFAICT the only new permission which it
> > needs is executing "/usr/libexec/plymouthd-drm-escrow" and
> > "/usr/libexec/plymouthd-drm-escrow" can stay in the same process-context and
> > it does not need any special rights.
> These files have the generic bin_t label. Also note other plymouth executables are in /usr/libexec/plymouth/.

Oops, my bad, the plymouthd-drm-escrow binary actually also sits under /usr/libexec/plymouth/, so its *correct* full path is: 

/usr/libexec/plymouth/plymouthd-drm-escrow

Note that the other files there are shell-scripts which are used as helpers for initrd generation, they are not executed by plymouthd.

Also during upstream review there were some comments on the plymouthd-drm-escrow name, so it will likely be renamed to plymouthd-fds-escrow . As long as bin_t is used that is fine, but if you plan to add a special plymouth_bin_t to make the new permissions more narrow, please make sure that the labelling will be done on both these paths:

/usr/libexec/plymouth/plymouthd-drm-escrow
/usr/libexec/plymouth/plymouthd-fds-escrow

To make things future proof for the next plymouth version.

(In reply to Hans de Goede from comment #4)
> > Let's start with:
> >
> > vm-f34# cat local_plymouthd.cil
> > (allow plymouthd_t bin_t (file (getattr ioctl lock open read execute execute_no_trans map)))
> > vm-f34# semodule -i local_plymouthd.cil
> 
> Thank you, that works. I've run a bunch of tests to figure out the minimal
> permission set and this also works:
> 
> (allow plymouthd_t bin_t (file (execute execute_no_trans map)))
Does this mean this policy is sufficient to make it working as expected in SELinux enforcing? If yes, it can make it to the last pre-ga build.

> 
> > > plymouthd does already use fork, so AFAICT the only new permission which it
> > > needs is executing "/usr/libexec/plymouthd-drm-escrow" and
> > > "/usr/libexec/plymouthd-drm-escrow" can stay in the same process-context and
> > > it does not need any special rights.
> > These files have the generic bin_t label. Also note other plymouth executables are in /usr/libexec/plymouth/.
> 
> Oops, my bad, the plymouthd-drm-escrow binary actually also sits under
> /usr/libexec/plymouth/, so its *correct* full path is: 
> 
> /usr/libexec/plymouth/plymouthd-drm-escrow
> 
> Note that the other files there are shell-scripts which are used as helpers
> for initrd generation, they are not executed by plymouthd.
> 
> Also during upstream review there were some comments on the
> plymouthd-drm-escrow name, so it will likely be renamed to
> plymouthd-fds-escrow . As long as bin_t is used that is fine, but if you
> plan to add a special plymouth_bin_t to make the new permissions more
> narrow, please make sure that the labelling will be done on both these paths:
> 
> /usr/libexec/plymouth/plymouthd-drm-escrow
> /usr/libexec/plymouth/plymouthd-fds-escrow
> 
> To make things future proof for the next plymouth version.
Discussion for this will continue in bz#1945685, correct?

> > Thank you, that works. I've run a bunch of tests to figure out the minimal
> > permission set and this also works:
> > 
> > (allow plymouthd_t bin_t (file (execute execute_no_trans map)))
> Does this mean this policy is sufficient to make it working as expected in
> SELinux enforcing? If yes, it can make it to the last pre-ga build.

Yes this is sufficient to make thing work with SELinux enforcing.

If this can make the last pre-ga build, then that would be great.

> Discussion for this will continue in bz#1945685, correct?

Yes lets discuss further improvements to plymouth's selinux policy there.

I've submitted a Fedora PR to address the issue:
https://github.com/fedora-selinux/selinux-policy/pull/673

(In reply to Zdenek Pytela from comment #7)
> I've submitted a Fedora PR to address the issue:
> https://github.com/fedora-selinux/selinux-policy/pull/673

Great, thank you.

FEDORA-2021-e221a38cfe has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-e221a38cfe

FEDORA-2021-e221a38cfe has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-e221a38cfe`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-e221a38cfe

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2021-e221a38cfe has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.