Hmm, on closer inspection, it looks like every failure is a failed attempt to login as root. Graphical desktop login as regular user with password works. Console login as regular user with password may also work; I don't *think* any of the tests we run on updates actually tried this before they failed on a root console login, so I can't say for sure. I'll sacrifice a VM to check shortly.

Yep, on manual testing in a VM, confirmed the issue is that direct root login no longer works. Before updating, I can log into a test VM as root just fine. After updating, attempting to log in as root fails. Logging in as a regular user and then becoming root via `sudo su` or `su` both work OK; it's only direct login as root that fails.

The journal isn't very illuminating, just an 'authentication failure' error.

This looks bad. Unfortunately, I am not able to reproduce it.

[vagrant@fedora ~]$ sudo authselect select sssd --force
Backup stored at /var/lib/authselect/backups/2022-12-02-11-08-18.SGjr3V
Profile "sssd" was selected.

Make sure that SSSD service is configured and enabled. See SSSD documentation for more information.

[vagrant@fedora ~]$ sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-9b9adb4d6d
...
Upgraded:
  authselect-1.4.1-1.fc37.x86_64                                                                                      authselect-libs-1.4.1-1.fc37.x86_64                                                                                     

Complete!
[vagrant@fedora ~]$ su root
Password: 
[root@fedora vagrant]# cat /etc/fedora-release 
Fedora release 37 (Thirty Seven)
[root@fedora vagrant]# authselect current
Profile ID: sssd
Enabled features: None
[root@fedora vagrant]# authselect check
Current configuration is valid.

This is with latest f37 vagrant box.

Is there any missing step?

Ok, I can reproduce it. It works correctly with ssh/sudo/su, it just does not work with login. It correctly tries to authenticate root with pam_unix but fails.

Dec 02 11:36:54 fedora login[2131]: pam_unix(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty1 ruser= rhost=  user=root
Dec 02 11:36:54 fedora audit[2131]: USER_AUTH pid=2131 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:local_login_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=? acct="root" exe="/usr/bin/login" hostname=fedora addr=?>
Dec 02 11:36:57 fedora login[2131]: FAILED LOGIN 1 FROM tty1 FOR root, Authentication failure

This is the breaking change: https://github.com/authselect/authselect/commit/a859e0d3584f478e6f89028211398f11c4766456

Adding systemd to shadow (shadow: files systemd) prevetes log in as root via 'login' command. Interesting is that it prevents only root login and ssh an su as root works fine.

Zbigniew, do you know why? Should I just revert the patch or there a solution or a bug in systemd?

If I'm remembering correctly, this was broken for a long time in F36, but started working after upgrade to F37 (at least in my setup). Then it got broken again with this update.

So, yeah - just root login on console doesn't work for me. Regular logins work.

Bojan: it definitely was not broken in stock F36. This breaks just about every single openQA test; if it was broken in F36 I'd have a flood of red on every F36 update test.

Okay, maybe that was something local then, although I haven't touched any of it after upgrading my two machines with dnf - it just started working again. Anyhow, not really important for the current problem.

Seeing this in F37, downgrading to 1.4.0 fixed things.

FEDORA-2022-3905763e94 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2022-3905763e94

FEDORA-2022-3905763e94 has been pushed to the Fedora 37 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-3905763e94`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-3905763e94

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2022-3905763e94 has been pushed to the Fedora 37 stable repository.
If problem still persists, please make note of it in this bug report.

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days

I did the same change locally, and su/ssh/login all work fine.
But this specific machine doesn't have selinux. So maybe it's something
related to selinux?

I downloaded Fedora-Cloud-Base-37-1.7.x86_64.qcow2, and there the issue
reproduces easily. (systemd-251.7-611.fc37.x86_64, authselect-1.4.0-3.fc37.x86_64)
Also in Fedora-Cloud-Base-38-1.6.x86_64.qcow2.

Sep 19 11:05:41 localhost.localdomain audit[1076]: USER_START pid=1076 uid=0 auid=0 ses=3 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Sep 19 11:05:58 localhost.localdomain login[1073]: pam_unix(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty2 ruser= rhost=  user=root
Sep 19 11:05:58 localhost.localdomain audit[1073]: USER_AUTH pid=1073 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:local_login_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=? acct="root" exe="/usr/bin/login" hostname=localhost.localdomain addr=? terminal=/dev/tty2 res=failed'
Sep 19 11:06:00 localhost.localdomain login[1073]: FAILED LOGIN 1 FROM tty2 FOR root, Authentication failure
Sep 19 11:06:00 localhost.localdomain audit[1073]: USER_LOGIN pid=1073 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:local_login_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/bin/login" hostname=localhost.localdomain addr=? terminal=tty2 res=failed'

Note that it's the 'shadow' line that causes the problem. 'gshadow: files systemd' seems to work OK.

After disabling dontaudit rules:

no systemd in shadow/gshadow:
Sep 19 11:26:09 localhost.localdomain login[1012]: ROOT LOGIN ON tty2
Sep 19 11:26:09 localhost.localdomain audit[1020]: AVC avc:  denied  { noatsecure } for  pid=1020 comm="login" scontext=system_u:system_r:local_login_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=process permissive=0
Sep 19 11:26:09 localhost.localdomain audit[1020]: AVC avc:  denied  { siginh } for  pid=1020 comm="bash" scontext=system_u:system_r:local_login_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=process permissive=0

with systemd:
Sep 19 11:27:28 localhost.localdomain audit[1053]: AVC avc:  denied  { noatsecure } for  pid=1053 comm="agetty" scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:local_login_t:s0-s0:c0.c1023 tclass=process permissive=0
Sep 19 11:27:28 localhost.localdomain audit[1053]: AVC avc:  denied  { read write } for  pid=1053 comm="login" path="socket:[20626]" dev="sockfs" ino=20626 scontext=system_u:system_r:local_login_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=netlink_route_socket permissive=0
Sep 19 11:27:28 localhost.localdomain audit[1053]: AVC avc:  denied  { rlimitinh } for  pid=1053 comm="login" scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:local_login_t:s0-s0:c0.c1023 tclass=process permissive=0
Sep 19 11:27:28 localhost.localdomain audit[1053]: AVC avc:  denied  { siginh } for  pid=1053 comm="login" scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:local_login_t:s0-s0:c0.c1023 tclass=process permissive=0
Sep 19 11:27:28 localhost.localdomain audit[1053]: AVC avc:  denied  { read } for  pid=1053 comm="login" name="shadow" dev="vda5" ino=36939 scontext=system_u:system_r:local_login_t:s0-s0:c0.c1023 tcontext=system_u:object_r:shadow_t:s0 tclass=file permissive=0

Sep 19 11:27:38 localhost.localdomain audit[1053]: AVC avc:  denied  { read } for  pid=1053 comm="login" name="shadow" dev="vda5" ino=36939 scontext=system_u:system_r:local_login_t:s0-s0:c0.c1023 tcontext=system_u:object_r:shadow_t:s0 tclass=file permissive=0
Sep 19 11:27:38 localhost.localdomain audit[1053]: USER_AUTH pid=1053 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:local_login_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=? acct="root" exe="/usr/bin/login" hostname=localhost.localdomain addr=? terminal=/dev/tty2 res=failed'
Sep 19 11:27:38 localhost.localdomain login[1053]: pam_unix(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty2 ruser= rhost=  user=root
Sep 19 11:27:40 localhost.localdomain login[1053]: FAILED LOGIN 1 FROM tty2 FOR root, Authentication failure
Sep 19 11:27:40 localhost.localdomain audit[1053]: USER_LOGIN pid=1053 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:local_login_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/bin/login" hostname=localhost.localdomain addr=? terminal=tty2 res=failed'

I'll attach some strace output.

But looking at this again, I'm not sure why nss_systemd would be involved at all.
With config like 'shadow: files systemd', nss_files should provide the answer.
nss_systemd would provide answer like 'root:!*:::::::', i.e. that the account password
is locked.

After removing PID and memory addresses from a successful and failing strace log,
I don't see any significant difference. There is no failed call or anything like this.

Created attachment 1989532 [details]
strace of login with systemd in shadow and gshadow

Created attachment 1989533 [details]
strace of login with systemd in shadow only

Created attachment 1989534 [details]
strace of login with systemd in gshadow only

Created attachment 1989535 [details]
strace of login with no systemd in shadow/gshadow