Bug 2037047

For completeness:

```
[root@fedora ~]# rpm -q systemd selinux-policy
systemd-250-3.fc36.x86_64
selinux-policy-35.8-1.fc36.noarch
```

Jan  4 16:58:05.278242 systemd-network-generator[888]: Failed to create unit file /run/systemd/network/90-ens5.network: Permission denied

Yeah, systemd-network-generator will create files under /run/systemd/network/.

Note a real simple reproducer would be:


```
sudo su -
systemd-run /usr/lib/systemd/systemd-network-generator -- nameserver=8.8.8.8
journalctl -u $unit
ausearch -m AVC
```

Here's an example run:

```
[dustymabe@media fR]$ vagrant ssh 
[vagrant@vanilla-rawhide ~]$ sudo su -

[root@vanilla-rawhide ~]# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

[root@vanilla-rawhide ~]# systemd-run /usr/lib/systemd/systemd-network-generator -- nameserver=8.8.8.8
Running as unit: run-raf26d43a650c400ca241189406d58075.service

[root@vanilla-rawhide ~]# journalctl -u run-raf26d43a650c400ca241189406d58075.service | cat
Jan 11 15:08:00 vanilla-rawhide systemd[1]: Started run-raf26d43a650c400ca241189406d58075.service - /usr/lib/systemd/systemd-network-generator -- nameserver=8.8.8.8.
Jan 11 15:08:00 vanilla-rawhide systemd-network-generator[4541]: Failed to create unit file /run/systemd/network/91-default.network: Permission denied
Jan 11 15:08:00 vanilla-rawhide systemd[1]: run-raf26d43a650c400ca241189406d58075.service: Main process exited, code=exited, status=1/FAILURE
Jan 11 15:08:00 vanilla-rawhide systemd[1]: run-raf26d43a650c400ca241189406d58075.service: Failed with result 'exit-code'.

[root@vanilla-rawhide ~]# ausearch -m avc
----
time->Tue Jan 11 15:08:00 2022
type=AVC msg=audit(1641913680.338:1145): avc:  denied  { add_name } for  pid=4541 comm="systemd-network" name="91-default.network" scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:net_conf_t:s0 tclass=dir permissive=0
```

I need this fixed in f35 too as we are starting to see this there now with the new systemd:
https://github.com/coreos/fedora-coreos-tracker/issues/1059#issuecomment-1013766710

Caught in enforcing mode:
----
type=PROCTITLE msg=audit(01/18/2022 02:26:36.120:550) : proctitle=/usr/lib/systemd/systemd-network-generator -- nameserver=8.8.8.8 
type=PATH msg=audit(01/18/2022 02:26:36.120:550) : item=0 name=/run/systemd/network/ inode=1301 dev=00:1a mode=dir,755 ouid=root ogid=root rdev=00:00 obj=system_u:object_r:net_conf_t:s0 nametype=PARENT cap_fp=none cap_fi=none cap_fe=0 cap_fver=0 cap_frootid=0 
type=CWD msg=audit(01/18/2022 02:26:36.120:550) : cwd=/ 
type=SYSCALL msg=audit(01/18/2022 02:26:36.120:550) : arch=x86_64 syscall=openat success=no exit=EACCES(Permission denied) a0=0xffffff9c a1=0x7ffdd043c760 a2=O_WRONLY|O_CREAT|O_EXCL|O_TRUNC|O_CLOEXEC a3=0x1b6 items=1 ppid=1 pid=4272 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=unset comm=systemd-network exe=/usr/lib/systemd/systemd-network-generator subj=system_u:system_r:init_t:s0 key=(null) 
type=AVC msg=audit(01/18/2022 02:26:36.120:550) : avc:  denied  { add_name } for  pid=4272 comm=systemd-network name=91-default.network scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:net_conf_t:s0 tclass=dir permissive=0
----

Caught in permissive mode:
----
type=PROCTITLE msg=audit(01/18/2022 02:27:49.905:559) : proctitle=/usr/lib/systemd/systemd-network-generator -- nameserver=8.8.8.8 
type=PATH msg=audit(01/18/2022 02:27:49.905:559) : item=1 name=/run/systemd/network/91-default.network inode=1312 dev=00:1a mode=file,644 ouid=root ogid=root rdev=00:00 obj=system_u:object_r:net_conf_t:s0 nametype=CREATE cap_fp=none cap_fi=none cap_fe=0 cap_fver=0 cap_frootid=0 
type=PATH msg=audit(01/18/2022 02:27:49.905:559) : item=0 name=/run/systemd/network/ inode=1301 dev=00:1a mode=dir,755 ouid=root ogid=root rdev=00:00 obj=system_u:object_r:net_conf_t:s0 nametype=PARENT cap_fp=none cap_fi=none cap_fe=0 cap_fver=0 cap_frootid=0 
type=CWD msg=audit(01/18/2022 02:27:49.905:559) : cwd=/ 
type=SYSCALL msg=audit(01/18/2022 02:27:49.905:559) : arch=x86_64 syscall=openat success=yes exit=3 a0=0xffffff9c a1=0x7ffcf845ed90 a2=O_WRONLY|O_CREAT|O_EXCL|O_TRUNC|O_CLOEXEC a3=0x1b6 items=2 ppid=1 pid=4292 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=unset comm=systemd-network exe=/usr/lib/systemd/systemd-network-generator subj=system_u:system_r:init_t:s0 key=(null) 
type=AVC msg=audit(01/18/2022 02:27:49.905:559) : avc:  denied  { write } for  pid=4292 comm=systemd-network path=/run/systemd/network/91-default.network dev="tmpfs" ino=1312 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:net_conf_t:s0 tclass=file permissive=1 
type=AVC msg=audit(01/18/2022 02:27:49.905:559) : avc:  denied  { create } for  pid=4292 comm=systemd-network name=91-default.network scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:net_conf_t:s0 tclass=file permissive=1 
type=AVC msg=audit(01/18/2022 02:27:49.905:559) : avc:  denied  { add_name } for  pid=4292 comm=systemd-network name=91-default.network scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:net_conf_t:s0 tclass=dir permissive=1
----

# rpm -qa selinux\* systemd\* | sort
selinux-policy-35.8-1.fc35.noarch
selinux-policy-targeted-35.8-1.fc35.noarch
systemd-249.7-2.fc35.x86_64
systemd-libs-249.7-2.fc35.x86_64
systemd-networkd-249.7-2.fc35.x86_64
systemd-oomd-defaults-249.7-2.fc35.noarch
systemd-pam-249.7-2.fc35.x86_64
systemd-resolved-249.7-2.fc35.x86_64
systemd-udev-249.7-2.fc35.x86_64
#

Hey Team. Any updates here?

(In reply to Dusty Mabe from comment #6)
> Hey Team. Any updates here?

Requires a new policy which takes time.

Hey Zdenek,

Thanks for the info. I understand it takes time. Do you mind giving a rough estimate (a week, 2 weeks, a month, multiple months)? Currently we've pinned systemd in Fedora Coreos (based on F35) on an older version. The rough estimate will help us plan accordingly.

This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Moving to Fedora 35 since that's where we're feeling the most pain right now because of the pinned systemd.

In yesterday's meeting, we agreed to accept this as a Prioritized Bug because it affects a common use case for a flagship offering

https://meetbot.fedoraproject.org/teams/fedora_prioritized_bugs_and_issues/fedora_prioritized_bugs_and_issues.2022-02-09-15.00.log.html#l-65

(In reply to Dusty Mabe from comment #8)
> Hey Zdenek,
> 
> Thanks for the info. I understand it takes time. Do you mind giving a rough
> estimate (a week, 2 weeks, a month, multiple months)? Currently we've pinned
> systemd in Fedora Coreos (based on F35) on an older version. The rough
> estimate will help us plan accordingly.

Rough estimate atm (no promises) is 3 weeks to get to F35. Any suggestions or help with testing can effect this in a positive manner.

Does the reproducer in https://bugzilla.redhat.com/show_bug.cgi?id=2037047#c3 help?

We can certainly test this if you come up with a potential fix and provide us a scratch build.

Thank you!

I can reproduce from comment 3 on a clean installed system using Fedora-Server-netinst-x86_64-36-20220308.n.0.iso (today).

type=AVC msg=audit(1646781760.098:273): avc:  denied  { add_name } for  pid=1023 comm="systemd-network" name="91-default.network" scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:net_conf_t:s0 tclass=dir permissive=0

Also fails if I use `nameserver=8.8.8.8` as a kernel boot parameter.
● systemd-network-generator.service                                                        loaded failed failed    Generate network units from Kernel command line

Proposed as a Blocker for 36-beta by Fedora user chrismurphy using the blocker tracking app because:

 https://fedoraproject.org/wiki/Basic_Release_Criteria#Cockpit_management_interface
The default system init daemon (e.g. systemd) must be capable of starting, stopping, enabling and disabling correctly-defined services. 

a. Performing this kind of network service manipulation is pretty basic functionality for any server/VM/cloud, i.e. it is a correctly defined service.
b. systemd isn't capable of enabling it due to the selinux issue

Pretty sure the blocker bug tracker needs to see the bug set to 36 to show it as blocking 36.

>b. systemd isn't capable of enabling it due to the selinux issue

Correction
b. systemd isn't capable of starting it due to the selinux issue

OK per the note in the release criterion about "Correctly-defined services" - I'll argue that systemd is inhibited by SELinux from starting the service, i.e. systemd is functionally prevented from working correctly by this issue.

No. The criterion is about Cockpit functionality. It is definitely *not* meant to be judo'ed into a criterion that makes it a blocker bug if absolutely any systemd service shipped in the distro doesn't work as intended.

Oh I copied the wrong URL it's: https://fedoraproject.org/wiki/Basic_Release_Criteria#System_service_manipulation
Nothing to do with cockpit.

Same deal. That means that systemd must be able to start, stop, enable and disable services. It's about systemd's operation, it is not supposed to be judo'd into this kind of situation.

Ok well the plain language doesn't explain it that way, nor does the example under it. There is nothing wrong with the service unit. Systemd is partially made defective in a narrow scope due to this SELinux problem.

But let's flip this around:
Is it OK to release Server/Cloud edition in a state in which select basic services like this can't be started?

Release blocking desktops have "applications must start successfully and withstand a basic functionality test" and the cited criterion is the closest equivalent to that for Server and Cloud edition. This bug directly inhibits the four basic objectives too.  https://fedoraproject.org/wiki/Basic_Release_Criteria#Basic_Objectives  There is a basic release criterion that requires SELinux enabled, so we can't test if there are subsequent or underlying bugs in this same area while SELinux is enabled due to the bug. It's a catch-22.

Let's move that discussion to the ticket, I guess. https://pagure.io/fedora-qa/blocker-review/issue/651

-5 in https://pagure.io/fedora-qa/blocker-review/issue/651 , marking rejected. Someone voted +1 Beta FE, so proposing as Beta FE for others to vote on.

btw, *is* this a "basic service"? Is this a common or supported way to configure networking on Server or Cloud? I don't recall hearing of it before.

I think systemd-network-generator was enabled by default fairly recently in an update to systemd. Obviously the logical conclusion that people could come to is "We don't use systemd-networkd, so we don't need this generator", but that's not totally true because there are other things (like udev rules) that this generator creates that are needed. Example: https://github.com/systemd/systemd/pull/21766#issuecomment-994411870

Discussed during the 2022-03-14 blocker review meeting: [0]

The decision to classify this bug as an "AcceptedFreezeException (Beta)" was made as it is a noticeable issue that cannot be fixed with an update.

[0] https://meetbot.fedoraproject.org/fedora-blocker-review/2022-03-14/f36-blocker-review.2022-03-14-16.01.txt

hey @zpytela - can we get an updated estimate? Thanks!

(In reply to Dusty Mabe from comment #28)
> hey @zpytela - can we get an updated estimate? Thanks!

Next week if I can reproduce and test it.

There is a PR adding the new domain:
https://github.com/fedora-selinux/selinux-policy/pull/1126

The reproducer from #c3 does not produce any denials, so the fix is expected to be in the next build, but I'd like anybody using this feature in a different way to check the build, rpms for testing can be downloaded from the PR:

Checks -> Details -> Artifacts -> rpms

The CI RPM from the PR (`selinux-policy-36.5-1.20220404_083611.bff1cac.fc37.noarch.rpm`) corrects this issue for me. Thanks!

Thanks for checking.

Sadly, rawhide composes still fail. ;( 

https://pagure.io/releng/failed-composes/issue/3311

I'm not sure if this is the same issue or not however. It looks like anaconda cannot connect to nm-client...

Kevin,

I doubt it's the same issue. This issue has been around since the start of the year.

Zdenek,

Testing of the issue in rawhide for us seems to look good with selinux-policy-36.6-1.fc37. Can we get an update for Fedora 35/36?

Well, untagging systemd-251~rc1-2.fc37 got us a rawhide compose. 

I guess we can put it back and see if it fails again?

(In reply to Dusty Mabe from comment #34)
> Zdenek,
> 
> Testing of the issue in rawhide for us seems to look good with
> selinux-policy-36.6-1.fc37. Can we get an update for Fedora 35/36?

It's on the way since yesterday, it just takes a long time.

FEDORA-2022-690770081a has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-690770081a

(In reply to Kevin Fenzi from comment #35)
> Well, untagging systemd-251~rc1-2.fc37 got us a rawhide compose. 
> 

This bug is against selinux-policy and the fix is in the selinux-policy rpm. I don't think untagging the newest systemd would affect this issue.

FEDORA-2022-690770081a has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-690770081a`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-690770081a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

(In reply to Dusty Mabe from comment #38)
> (In reply to Kevin Fenzi from comment #35)
> > Well, untagging systemd-251~rc1-2.fc37 got us a rawhide compose. 
> > 
> 
> This bug is against selinux-policy and the fix is in the selinux-policy rpm.
> I don't think untagging the newest systemd would affect this issue.

Oops. Wrong bug. ;( 

I was meanting to post that in https://bugzilla.redhat.com/show_bug.cgi?id=2071069
Sorry for the noise.

This was an Accepted Beta FreezeException, re-proposing as Final FE.

Discussed in ticket: https://pagure.io/fedora-qa/blocker-review/issue/698

The decision to classify this bug as an AcceptedFreezeException was made:

"It is a noticeable issue that cannot be fixed with an update."

FEDORA-2022-690770081a has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Hi all.
Not sure if this BZ is a proper one to update, but looks like the issue could be reproduced on RHCOS 4.12:

```
[core@localhost ~]$ cat /etc/os-release 
NAME="Red Hat Enterprise Linux CoreOS"
VERSION_ID="4.12"
RHEL_VERSION="9.0"

[core@localhost ~]$ ip a
2: enc2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic noprefixroute enc2
   
[core@localhost ~]$ sudo rpm-ostree kargs --append="ip=10.0.2.15::10.0.2.2:255.255.255.0:rhcos:enc2:none" && sudo reboot

---- reboot ----

[core@localhost ~]$ systemctl status systemd-network-generator.service
Jan 17 09:58:34 localhost systemd-network-generator[805]: Failed to create unit file /run/systemd/network/90-enc2.network: Permission denied

[core@localhost ~]$ rpm -q selinux-policy
selinux-policy-34.1.29-1.el9_0.2.noarch

[core@localhost ~]$ rpm -q systemd
systemd-250-6.el9_0.1.s390x

[core@localhost ~]$ ls -Z /usr/lib/systemd/systemd-network-generator
system_u:object_r:init_exec_t:s0 /usr/lib/systemd/systemd-network-generator

[core@localhost ~]$ ls -dZ /run/systemd/network/
system_u:object_r:net_conf_t:s0 /run/systemd/network/

```

@ndubrovs I think it'd be better to file a new issue against RHEL 9 instead. You can link to this RHBZ there for more details.