Bug 1811935

Can you please attach /etc/nsswitch.conf, /etc/nsswitch.conf.bak and /etc/nsswitch.conf.rpmnew and /etc/authselect/user-nsswitch.conf?

Also what is the output of 'authselect check' and 'authselect current'?

Thank you.

Chris, do you still have those files? Because I got rid of them. If you don't, I'll try to reproduce the issue in a VM.

If you don't have them, I can try to reproduce it first. If I understand it correctly it should be just about installing nss-mdns on F31 and they upgrade, right?

My best guess at reproducing this is to install F31 from Live, make sure machine.local resolving works, fully update, make sure machine.local resolving works, upgrade to F32, see that machine.local resolving is broken. If you can't test it today or Chris can't provide logs, I'll work on reproducing this and providing logs tomorrow.

Created attachment 1668971 [details]
F31 nsswitch.conf

This is the nsswitch.conf file immediately prior to F31->F32 upgrade.

After the upgrade to F32:
- hosts:      files mdns4_minimal [NOTFOUND=return] dns myhostname
+ hosts:      files dns myhostname

Reverting that difference manually fixed the problem.

I cannot reproduce the problem in a VM doing this:

1. Clean install Fedora-Workstation-Live-x86_64-31-1.9.iso (official GA release)
2. Reboot, update only dnf, et al and install system-upgrade plugin
3. dnf system-upgrade download and reboot

Regarding the original report, it's a baremetal system, clean install (not an upgrade from Fedora 39 or 30); but must have been prerelease, which explains why the nsswitch.conf line 1 time stamp does not match between baremetal and VM.

Created attachment 1668974 [details]
F31F32 nsswitch.conf.rpmnew

Following successful F31->F32 upgrade (baremetal, original problem report computer), there is '/etc/nsswitch.conf.rpmnew

Attaching it.

The first file is post-upgrade F31->F32; the second is from a snapshot of this system prior to the upgrade (while it was still F31). This is very suspicious as the source of the problem I experienced.

$ diff /etc/authselect/user-nsswitch.conf root.fc31.20200302/etc/authselect/user-nsswitch.conf
40c40
< hosts:      files dns myhostname
---
> hosts:      files mdns4_minimal [NOTFOUND=return] dns myhostname
$

(In reply to Kamil Páral from comment #4)
> My best guess at reproducing this is to install F31 from Live, make sure
> machine.local resolving works, fully update, make sure machine.local
> resolving works, upgrade to F32, see that machine.local resolving is broken.

I've just reproduced the problem with exactly these steps (in a VM). I'll attach the logs.

In F32:

$ sudo authselect check
Current configuration is valid.

$ sudo authselect current
Profile ID: sssd
Enabled features:
- with-fingerprint
- with-silent-lastlog

Created attachment 1669025 [details]
post-upgrade logs

These are from clean installs from the following media.
media/1 = Fedora-Workstation-Live-x86_64-31-1.9.iso
media/2 = Fedora-Workstation-Live-x86_64-32-20200308.n.0.iso



# diff /media/1/root.clean/etc/authselect/user-nsswitch.conf /media/2/root.clean/etc/authselect/user-nsswitch.conf 
4c4
< # An example Name Service Switch config file. This file should be
---
> # Name Service Switch config file. This file should be
7,11c7,9
< # The entry '[NOTFOUND=return]' means that the search for an
< # entry should stop if the search in the previous entry turned
< # up nothing. Note that if the search failed due to some other reason
< # (like no NIS server responding) then the search continues with the
< # next entry.
---
> # Valid databases are: aliases, ethers, group, gshadow, hosts,
> # initgroups, netgroup, networks, passwd, protocols, publickey,
> # rpc, services, and shadow.
13c11
< # Valid entries include:
---
> # Valid service provider entries include (in alphabetical order):
15,16c13,14
< #	nisplus			Use NIS+ (NIS version 3)
< #	nis			Use NIS (NIS version 2), also called YP
---
> #	compat			Use /etc files plus *_compat pseudo-db
> #	db			Use the pre-processed /var/db files
19,20d16
< #	db			Use the pre-processed /var/db files
< #	compat			Use /etc files plus *_compat pseudo-databases
22,23c18,19
< #	sss			Use sssd (System Security Services Daemon)
< #	[NOTFOUND=return]	Stop searching if not found so far
---
> #	nis			Use NIS (NIS version 2), also called YP
> #	nisplus			Use NIS+ (NIS version 3)
25,26c21
< # 'sssd' performs its own 'files'-based caching, so it should
< # generally come before 'files'.
---
> # See `info libc 'NSS Basics'` for more information.
28,33c23,49
< # WARNING: Running nscd with a secondary caching service like sssd may lead to
< # 	   unexpected behaviour, especially with how long entries are cached.
< 
< # To use 'db', install the nss_db package, and put the 'db' in front
< # of 'files' for entries you want to be looked up first in the
< # databases, like this:
---
> # Commonly used alternative service providers (may need installation):
> #
> #	ldap			Use LDAP directory server
> #	myhostname		Use systemd host names
> #	mymachines		Use systemd machine names
> #	mdns*, mdns*_minimal	Use Avahi mDNS/DNS-SD
> #	resolve			Use systemd resolved resolver
> #	sss			Use System Security Services Daemon (sssd)
> #	systemd			Use systemd for dynamic user option
> #	winbind			Use Samba winbind support
> #	wins			Use Samba wins support
> #	wrapper			Use wrapper module for testing
> #
> # Notes:
> #
> # 'sssd' performs its own 'files'-based caching, so it should generally
> # come before 'files'.
> #
> # WARNING: Running nscd with a secondary caching service like sssd may
> # 	   lead to unexpected behaviour, especially with how long
> # 	   entries are cached.
> #
> # Installation instructions:
> #
> # To use 'db', install the appropriate package(s) (provide 'makedb' and
> # libnss_db.so.*), and place the 'db' in front of 'files' for entries
> # you want to be looked up first in the databases, like this:
38a55
> # In order of likelihood of use to accelerate lookup.
42d58
< 
44,51d59
< 
< bootparams: files
< 
< ethers:     files
< netmasks:   files
< networks:   files
< protocols:  files
< rpc:        files
53d60
< 
55,57d61
< 
< publickey:  files
< 
58a63
> 
59a65,72
> ethers:     files
> gshadow:    files
> # Allow initgroups to default to the setting for group.
> # initgroups: files
> networks:   files dns
> protocols:  files
> publickey:  files
> rpc:        files
# 


A simpler summary:

Both files contain these:
passwd:      sss files systemd
shadow:     files sss
group:       sss files systemd
hosts:      files mdns4_minimal [NOTFOUND=return] dns myhostname
services:   files sss
netgroup:   sss
automount:  files sss
aliases:    files
ethers:     files
protocols:  files
publickey:  files
rpc:        files

F31 contains:

networks:   files
bootparams: files
netmasks:   files

F32 contains:

networks:   files dns
gshadow:    files

I will spin up a custom vm and try to reproduce. I suspect there is an issue in nss-mdns upgrade path.

Here is a little explanations of the different files you see:
- /etc/nsswitch.conf.rpmnew is totally expected - it is a configuration file provided by glibc, upgrade will not change the file but create .rpmnew
- /etc/authselect/user-nsswitch.conf - this is created as a copy of /etc/nsswitch.conf upon authselect installation so the contents might be different for F31-32 upgrade and fresh F32 install
- nss-mdns is used to touch /etc/nsswitch.conf to enable itself on the system, I worked with the maintainer to make this compatible with both non-authselect and authselect configurations. Unfortunately the changes landed in F31 just two months ago so it is possible that it is causing some troubles if it was originally installed without authselect-aware scriptlet.

I will check the upgrade path for both authselect-aware nss-mdns and the older version to see where the problem is.

To work around this issue, please do these steps:
1) Change /etc/authselect/user-nsswitch.conf to contain:
hosts: files mdns4_minimal [NOTFOUND=return] dns myhostname
2) Apply changes
$ sudo authselect apply-changes

Or if you choose to opt out of authselect, you can just edit /etc/nsswitch.conf directly.

I can reproduce it even with latest nss-mdns in F31 installed directly.

1. Boot up fresh F31
2. Install nss-mdns-0.14.1-5.fc31.x86_64.rpm (authselect fix included)
   mdns is correctly enabled in /etc/authselect/user-nsswitch.conf and propagated to /etc/nsswitch.conf
   /etc/authselect/user-nsswitch.conf.bak is created
3. Upgrade to F32
4. mdns is not configured

/etc/authselect/user-nsswitch.conf was copied to /etc/authselect/user-nsswitch.conf.bak and mdns was removed from user-nsswitch.conf. I will check nss-mdns scriptlets to see what is causing it.

The problem is that %preun which removes mdns from nsswitch.conf is run also for upgrade path:
https://src.fedoraproject.org/rpms/nss-mdns/blob/master/f/nss-mdns.spec#_65

Switching to nss-mdns, I will open a pull request against it.

https://src.fedoraproject.org/rpms/nss-mdns/pull-request/2

FEDORA-2020-e8df029f18 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-e8df029f18

FEDORA-2020-5d6d7d1815 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-5d6d7d1815

FEDORA-2020-75a4a1e132 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-75a4a1e132

FEDORA-EPEL-2020-b03097f59d has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-b03097f59d

FEDORA-EPEL-2020-9b0fe95016 has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9b0fe95016

nss-mdns-0.14.1-7.fc32 has been pushed to the Fedora 32 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-e8df029f18

nss-mdns-0.14.1-7.el8 has been pushed to the Fedora EPEL 8 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9b0fe95016

nss-mdns-0.14.1-7.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-b03097f59d

nss-mdns-0.14.1-7.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-75a4a1e132

nss-mdns-0.14.1-7.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-5d6d7d1815

I tested upgrade F31->F32 according to comment 10 and machine.local queries still work even after upgrade. mdns is included in /etc/nsswitch.conf. Looks fixed, thanks.

FEDORA-2020-e8df029f18 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-2020-75a4a1e132 has been pushed to the Fedora 30 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-2020-5d6d7d1815 has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-EPEL-2020-9b0fe95016 has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-EPEL-2020-b03097f59d has been pushed to the Fedora EPEL 7 stable repository.
If problem still persists, please make note of it in this bug report.