Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at
Bug 1409012 - nfs-server runs before network is ready; "failed to resolve" for all hosts; nothing is exported
Summary: nfs-server runs before network is ready; "failed to resolve" for all hosts; n...
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: nfs-utils
Version: 7.3
Hardware: All
OS: Linux
Target Milestone: rc
: ---
Assignee: Steve Dickson
QA Contact: Yongcheng Yang
Depends On:
TreeView+ depends on / blocked
Reported: 2016-12-28 22:47 UTC by Chris Schanzle
Modified: 2017-08-01 19:48 UTC (History)
6 users (show)

Fixed In Version: nfs-utils-1.3.0-0.41.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2017-08-01 19:48:51 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1419351 0 unspecified CLOSED on boot exports fail due to network not up 2022-05-16 11:32:56 UTC
Red Hat Product Errata RHBA-2017:2233 0 normal SHIPPED_LIVE nfs-utils bug fix and enhancement update 2017-08-01 18:19:33 UTC

Internal Links: 1419351

Description Chris Schanzle 2016-12-28 22:47:33 UTC
Greetings, this is coming from a community user running CentOS 7.3.  I hope it is welcomed as the effort to write this is an expression of my appreciation of the work you share.

Description of problem:
systemd may start the nfs-server service before the network is up.  Thus it cannot resolve hostnames in /etc/exports.
This is never retried and the server ends up not serving files as desired.

Version-Release number of selected component (if applicable):

How reproducible:
100% since the upgrade to 7.3 (never noticed this in 7.1 or 7.2).  System is a spinning-rust HD-based Dell T5600 with PERC H710P (LSI Logic / Symbios Logic MegaRAID SAS 2208).
However, bootup can be racey.  Could reproduce 3 out of 7 tries on another ssd-based system, where things where twice things were exported to a *partial* list of hostnames.

Steps to Reproduce:

create /etc/exports exporting to several DNS-only hostnames
TIP: export to one hostname that is not resolvable so you always get an entry in the journal.
I use a wired network connection, configured via DHCP, which can take several seconds to get an IP.
systemctl enable nfs-server
I had enabled NetworkManager-wait-online.service
boot system, check "showmount -e" or "exportfs"

Actual results:
filesystems are not exported; output of "exportfs" is empty.

The below logs show exportfs is run before the network is up.  Other boots with the issue shows about an 8 second delay to get an IP.

journalctl -b -u nfs-server -u NetworkManager
Dec 22 15:51:27 systemd[1]: Starting Network Manager...
Dec 22 15:51:28 NetworkManager[1235]: <info>  [1482439888.2791] NetworkManager (version 1.4.0-13.el7_3) is starting...
Dec 22 15:51:28 NetworkManager[1235]: <info>  [1482439888.2793] Read config: /etc/NetworkManager/NetworkManager.conf
Dec 22 15:51:28 NetworkManager[1235]: <info>  [1482439888.3301] manager[0x7fe9d45a90d0]: monitoring kernel firmware directory '/lib/firmwar
Dec 22 15:51:28 NetworkManager[1235]: <info>  [1482439888.3452] dns-mgr[0x7fe9d45a2040]: init: dns=default, rc-manager=file
Dec 22 15:51:28 systemd[1]: Started Network Manager.
Dec 22 15:51:28 systemd[1]: Starting NFS server and services...
Dec 22 15:51:28 NetworkManager[1235]: <info>  [1482439888.5511] settings: loaded plugin ifcfg-rh: (c) 2007 - 2015 Red Hat, Inc.  To report 
Dec 22 15:51:28 exportfs[1320]: exportfs: Failed to resolve
Dec 22 15:51:28 exportfs[1320]: exportfs: Failed to resolve
Dec 22 15:51:28 exportfs[1320]: exportfs: Failed to resolve
Dec 22 15:51:28 exportfs[1320]: exportfs: Failed to resolve
Dec 22 15:51:28 exportfs[1320]: exportfs: Failed to resolve
Dec 22 15:51:28 exportfs[1320]: exportfs: Failed to resolve
Dec 22 15:51:28 NetworkManager[1235]: <info>  [1482439888.6193] settings: loaded plugin iBFT: (c) 2014 Red Hat, Inc.  To report bugs please
Dec 22 15:51:28 NetworkManager[1235]: <info>  [1482439888.6193] settings: loaded plugin keyfile: (c) 2007 - 2015 Red Hat, Inc.  To report b
Dec 22 15:51:28 NetworkManager[1235]: <info>  [1482439888.6204] ifcfg-rh: new connection /etc/sysconfig/network-scripts/ifcfg-em1 (d924258b
Dec 22 15:51:28 systemd[1]: Started NFS server and services.
Dec 22 15:51:34 dhclient[1426]: DHCPREQUEST on em1 to port 67 (xid=0x5b075789)
Dec 22 15:51:41 dhclient[1426]: DHCPREQUEST on em1 to port 67 (xid=0x5b075789)
Dec 22 15:51:41 dhclient[1426]: DHCPACK from (xid=0x5b075789)
Dec 22 15:51:41 NetworkManager[1235]: <info>  [1482439901.5860] dhcp4 (em1):   address
Dec 22 15:51:41 NetworkManager[1235]: <info>  [1482439901.5860] dhcp4 (em1):   plen 24 (

After creating the following, the system is booting with exported filesystems 4 times out of 4 tries.

$ cat /etc/systemd/system/nfs-server.service.d/wait.conf

Consider also adding Requires (unlikely if IP addresses are used in /etc/exports, don't want nfs-service to fail) or Wants (highly likely, so NMwo will be started even if not enabled already, if I rtfm correctly).

Thank you!

Comment 1 Steve Dickson 2017-01-04 19:08:10 UTC
What I don't understand why only your site is seeing 
this problem... I'm exporting dns hostname is a very
common... Plus Requires= in the 
nfs-server.service should bring DNS up.

Comment 2 Chris Schanzle 2017-01-05 02:57:11 UTC
>Plus Requires= in the nfs-server.service should bring DNS up.

Wrong, per the fine manual! means little per the documentation referenced in the unit file --

If you really want DNS resolution to work, you need to wait for an interface to be up and online, not just have the stack initialized.  Hence my suggestion for wait-online, but per those docs, perhaps I should have used:

Comment 3 Chris Schanzle 2017-01-05 18:35:20 UTC
Not that I understand this, but experimentally speaking, with /etc/systemd/system/nscd.service.d/network-online.conf containing:


and rebooting, host resolution was not working again.  By looking at the journal:

journalctl -b -u nscd -u NetworkManager -u -u

it was clear systemd started nscd before NetworkManager, well-before the network was even configured.  Here are some snippets:

Jan 05 12:17:18 localhost.localdomain systemd[1]: Starting Name Service Cache Daemon...
Jan 05 12:17:20 localhost.localdomain systemd[1]: Starting Network Manager...
Jan 05 12:17:20 localhost.localdomain systemd[1]: Started Name Service Cache Daemon.
Jan 05 12:17:22 localhost.localdomain NetworkManager[1360]: <info>  [1483636642.0293] NetworkManager (version 1.4.2-2.fc25) is starting...
Jan 05 12:17:33 localhost.localdomain dhclient[1540]: bound to -- renewal in 158521 seconds.
Jan 05 12:17:33 NetworkManager[1360]: <info>  [1483636653.4606] device (eno1): Activation: successful, device activated.
Jan 05 12:17:33 NetworkManager[1360]: <info>  [1483636653.4627] connectivity: check for uri '' failed with 'Error resolving '': Name or service not known'

~20 mins passes...I ran "nscd -i hosts" to get resolution working again:

Jan 05 12:37:34 NetworkManager[1360]: <info>  [1483637854.9200] manager: NetworkManager state is now CONNECTED_GLOBAL

So it seems After+Wants=network-online doesn't do what is stated in the docs, at least in Fedora 25.  Using After+Wants=NetworkManager-wait-online.service seems to do what is desired/expected.

But still, it does not explain the issues with nscd.

Perhaps worth noting, in one of these reboots, it appeared IPv6 hostname resolution worked, but not IPv4.  While our network supports IPv6, the system does not have an IPv6 address.  E.g.,

[root@grad ~]# getent hosts

[root@grad ~]# telnet
telnet: Name or service not known

[root@grad ~]# getent hosts
[root@grad ~]# getent hosts
[root@grad ~]# getent hosts

Comment 4 Chris Schanzle 2017-01-05 18:39:38 UTC
My apologies, the above comment was meant for bug #1367565, but some overlaps here as well. [humble shrug]

Comment 7 Steve Dickson 2017-04-26 16:59:35 UTC
This is also needed

commit 09e5c6c2a3f8eac91d5353e6d4ff6aee7757ab08
Author: Steve Dickson <steved>
Date:   Mon Apr 24 11:25:39 2017 -0400

    systemd: Afters are also needed for the

Comment 9 Yongcheng Yang 2017-04-28 02:00:12 UTC
Seems Chunyu has been tracking this issue in upstream.
If possible, please help to generate/update testcase to cover it.

Comment 12 Yongcheng Yang 2017-06-20 08:15:24 UTC
As we have never seen this problem within our test environment, just
verified the systemd scripts have changed to use "network-online".

Moving to VERIFIED and will update testcase to cover it.
[root@ ~]# rpm -q nfs-utils
[root@ ~]# systemctl list-dependencies nfs | grep network-online
● ├─
[root@ ~]# 
[root@ ~]# systemctl cat nfs | grep network-online
[root@ ~]# 
[root@ ~]# systemctl cat nfs-mountd.service | grep network-online
[root@ ~]# 
[root@ ~]# systemctl cat rpc-statd-notify.service | grep network-online
[root@ ~]# 
[root@ ~]# systemctl cat rpc-statd.service | grep network-online rpcbind.socket
[root@ ~]# 

However, there still exits "", don't know whether we
should replace all of them with "".

Will read the doc and test some more to check it.
[root@ ~]# rpm -ql nfs-utils | grep .*service$ | xargs grep ""
/usr/lib/systemd/system/nfs-server.service:Requires= proc-fs-nfsd.mount
/usr/lib/systemd/system/nfs.service:Requires= proc-fs-nfsd.mount
[root@ ~]# systemctl cat nfs | grep ''
Requires= proc-fs-nfsd.mount
[root@ ~]#

Comment 14 errata-xmlrpc 2017-08-01 19:48:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

Note You need to log in before you can comment on or make changes to this bug.