Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1207381 - regression: smbd startup fails after update to 4.2.0-2.fc22
Summary: regression: smbd startup fails after update to 4.2.0-2.fc22
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: samba
Version: 22
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Guenther Deschner
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: RejectedBlocker RejectedFreezeException
: 1209488 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-03-30 19:24 UTC by Cole Robinson
Modified: 2015-04-27 10:11 UTC (History)
17 users (show)

Fixed In Version: samba-4.2.0-3.fc22
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-04-22 22:50:28 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
log with debug level = 10 (393.91 KB, text/plain)
2015-04-01 14:56 UTC, Cole Robinson
no flags Details

Description Cole Robinson 2015-03-30 19:24:55 UTC
Updated today to 4.2.0-2.fc22 from 4.2.0-0.6.rc3.fc22. Later tried to restart smbd, restart hung and systemctl reported failure. Downgraded to the older version, smbd starts up fine again.

$ sudo systemctl status smb -l
● smb.service - Samba SMB Daemon
   Loaded: loaded (/usr/lib/systemd/system/smb.service; enabled; vendor preset: disabled)
   Active: failed (Result: timeout) since Mon 2015-03-30 15:23:04 EDT; 21s ago
  Process: 22123 ExecStart=/usr/sbin/smbd $SMBDOPTIONS (code=killed, signal=TERM)
 Main PID: 22123 (code=killed, signal=TERM)

Mar 30 15:21:34 colepc systemd[1]: Starting Samba SMB Daemon...
Mar 30 15:21:34 colepc smbd[22123]: [2015/03/30 15:21:34.436104,  0] ../lib/util/become_daemon.c:124(daemon_ready)
Mar 30 15:21:34 colepc smbd[22123]: STATUS=daemon 'smbd' finished starting up and ready to serve connections
Mar 30 15:23:04 colepc systemd[1]: smb.service start operation timed out. Terminating.
Mar 30 15:23:04 colepc smbd[22123]: [2015/03/30 15:23:04.400664,  0] ../lib/util/pidfile.c:153(pidfile_unlink)
Mar 30 15:23:04 colepc smbd[22123]: Failed to delete pidfile /run/smbd.pid. Error was No such file or directory
Mar 30 15:23:04 colepc systemd[1]: Failed to start Samba SMB Daemon.
Mar 30 15:23:04 colepc systemd[1]: Unit smb.service entered failed state.
Mar 30 15:23:04 colepc systemd[1]: smb.service failed.

Comment 1 Andreas Schneider 2015-03-31 10:11:10 UTC
Did you check the smbd logfile?

https://www.samba.org/~asn/reporting_samba_bugs.txt

Comment 2 Cole Robinson 2015-03-31 16:01:50 UTC
Cleared all /var/log/samba/*, reproduced, the only file is this:

# cat log.smbd 
[2015/03/31 11:56:48,  0] ../source3/smbd/server.c:1241(main)
  smbd version 4.2.0 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2014
[2015/03/31 11:56:49.067634,  0] ../lib/util/become_daemon.c:124(daemon_ready)
  STATUS=daemon 'smbd' finished starting up and ready to serve connections
[2015/03/31 11:58:18.151431,  0] ../lib/util/pidfile.c:153(pidfile_unlink)
  Failed to delete pidfile /run/smbd.pid. Error was No such file or directory

Comment 3 Andreas Schneider 2015-04-01 08:58:33 UTC
The last line is only an informative message smbd prints. I don't see any error which prevents smbd from starting up. Also the message indicates everything is fine ...

You should increase the log level to find out what is going on. See the link I posted above ...

Comment 4 Cole Robinson 2015-04-01 14:56:10 UTC
Created attachment 1009718 [details]
log with debug level = 10

Comment 5 Anton Guda 2015-04-03 10:15:57 UTC
The same problem. But both smbd and nmbd start ok manualy.
May it be systemd bug?

Comment 6 Andreas Schneider 2015-04-07 09:31:44 UTC
[2015/04/01 10:55:19.152575,  3, pid=5254, effective(0, 0), real(0, 0)] ../source3/smbd/server_exit.c:246(exit_server_common)
  Server exit (termination signal)

The log shows that systemd set SIGTERM to the smbd process. It is the first start of smbd and it updates the databases so it takes some time. I guess that systemd runs into a timeout. It waits for some time smbd didn't create the pid file and then it kills it again.

This did not happen before. Could you edit the smbd.service file and add:

TimeoutStartSec=90s

Comment 7 David Woodhouse 2015-04-07 13:53:49 UTC
*** Bug 1209488 has been marked as a duplicate of this bug. ***

Comment 8 David Woodhouse 2015-04-07 13:56:03 UTC
systemd is waiting for smbd (and winbindd) to *tell* it that it's started up. But they are no longer linked against libsystemd-daemon.

In the build log at https://kojipkgs.fedoraproject.org//packages/samba/4.2.0/2.fc22/data/logs/x86_64/build.log I see:


Checking for libsystemd-daemon                                                                  : yes 
Checking for header systemd/sd-daemon.h                                                         : yes 
Checking for library systemd-daemon                                                             : no

Comment 9 Cole Robinson 2015-04-07 14:11:00 UTC
(In reply to Andreas Schneider from comment #6)
> [2015/04/01 10:55:19.152575,  3, pid=5254, effective(0, 0), real(0, 0)]
> ../source3/smbd/server_exit.c:246(exit_server_common)
>   Server exit (termination signal)
> 
> The log shows that systemd set SIGTERM to the smbd process. It is the first
> start of smbd and it updates the databases so it takes some time. I guess
> that systemd runs into a timeout. It waits for some time smbd didn't create
> the pid file and then it kills it again.
> 
> This did not happen before. Could you edit the smbd.service file and add:
> 
> TimeoutStartSec=90s

Still ends in failure. FWIW this reproduces on a machine with no previous samba config as well, so not sure what database creation should be taking so long.

David's idea in Comment #8 sounds like the culprit

Comment 10 David Woodhouse 2015-04-07 15:03:56 UTC
A simple workaround for now is to add BuildRequires: systemd-compat-libs

I also note that your BuildRequires are broken. You should be requiring 'pkgconfig(systemd-daemon)' rather than requiring 'systemd-devel' by name.

The systemd packaging also looks odd. Although the libsystemd-daemon.so.0.0.12 file is in a separate systemd-compat-libs package, the *development* symlink libsystemd-daemon.so is in systemd-devel, where it's a dangling symlink. And the systemd-daemon.pc file is also there (instructing you to link against -lsystemd).

It probably makes more sense for systemd to move those from systemd-devel to a new systemd-compat-libs-devel package, and then if your BR was actually right in the first place everything would have worked.

Separately, upstream Samba should be moving to using libsystemd in preference to the separate libsystemd-daemon and libsystemd-journal.

Comment 11 David Woodhouse 2015-04-07 16:09:12 UTC
Scratch build with just the systemd-compat-libs BR at http://koji.fedoraproject.org/koji/taskinfo?taskID=9429634

Comment 12 Fedora Update System 2015-04-08 16:08:05 UTC
samba-4.2.0-3.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/samba-4.2.0-3.fc22

Comment 13 Fedora Update System 2015-04-09 16:45:56 UTC
Package samba-4.2.0-3.fc22:
* should fix your issue,
* was pushed to the Fedora 22 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing samba-4.2.0-3.fc22'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-5898/samba-4.2.0-3.fc22
then log in and leave karma (feedback).

Comment 14 Fedora Blocker Bugs Application 2015-04-10 05:28:10 UTC
Proposed as a Blocker and Freeze Exception for 22-beta by Fedora user abbra using the blocker tracking app because:

 Samba 4.2.0-2.fc22 is built without systemd support. As result, it is not possible to use Samba daemons at all in Fedora 22 as systemd expects communication from Samba daemons on startup and kills them after ~90 seconds of inactivity. As result, no file sharing is possible, as well as no domain controller functionality is possible, and no 'Trusts to Active Directory' feature of FreeIPA domain controller is possible.

The fixed version 4.2.0-3.fc22 includes proper detection of a changed systemd development environment and allows Samba to work properly and to be used properly within the context of providing file shares and also being part of FreeIPA 'Trusts to Active Directory' feature.

Comment 15 Anton Guda 2015-04-10 06:53:45 UTC
samba-4.2.0-3 build works for me.

Comment 16 Stephen Gallagher 2015-04-10 13:30:22 UTC
So, this isn't technically a violation of any formal beta or alpha criteria that I can find. That being said, the impact is very high on our users.

I'm -1 blocker, +1 Freeze Exception for Beta.

Comment 17 Adam Williamson 2015-04-10 17:42:41 UTC
yeah, i'm -1 blocker on this, it doesn't really affect any pre-update scenarios I can think of very highly. I'm on the fence about FE, for the reason that it mostly seems to affect post-deployment scenarios: on the one hand that means it's less likely to break anything else important, on the other hand there's little benefit to pulling it in at all.

Comment 18 Alexander Bokovoy 2015-04-10 17:46:21 UTC
Upgrading existing Fedora 21 environment with FreeIPA and AD trusts to Fedora 22 beta repository will break without this fix.

Comment 19 Adam Williamson 2015-04-10 19:12:32 UTC
fedup uses the 'fedora' repo, though, so the fix doesn't need to go into the frozen Beta tree.

Comment 20 Stephen Gallagher 2015-04-13 12:18:01 UTC
(In reply to awilliam from comment #19)
> fedup uses the 'fedora' repo, though, so the fix doesn't need to go into the
> frozen Beta tree.

That's not actually correct. fedup points at the most recent install tree available to mirrormanager. Right now, that still points at the Alpha tree and will until we go live with Beta. So in order to not break people upgrading to Beta, I think we *really* want to allow this as a Freeze Exception.

Comment 21 Kalev Lember 2015-04-13 15:21:00 UTC
-1 blocker, +1 FE

Comment 22 Adam Williamson 2015-04-13 15:42:52 UTC
sgallagh: no. It uses the tree you mention *as the source of upgrade.img*. But for packages, it uses the regular repositories for the target release, with all the same repositories enabled as the release you're upgrading from. So if you're running 21 and have 'fedora' and 'updates' enabled, when you fedup to f22, your f22 packages will come from the f22 'fedora' and 'updates' repositories. It's only the kernel and upgrade.img used for the fedup environment that come from the fedora-install-22 repository.

Comment 23 Stephen Gallagher 2015-04-13 16:24:33 UTC
(In reply to awilliam from comment #22)
> sgallagh: no. It uses the tree you mention *as the source of upgrade.img*.
> But for packages, it uses the regular repositories for the target release,
> with all the same repositories enabled as the release you're upgrading from.
> So if you're running 21 and have 'fedora' and 'updates' enabled, when you
> fedup to f22, your f22 packages will come from the f22 'fedora' and
> 'updates' repositories. It's only the kernel and upgrade.img used for the
> fedup environment that come from the fedora-install-22 repository.

We're both wrong (or at least incomplete). It *does* also consider the tree for packages, which is why BZ #1209140 happened. (The upgrade tree contained packages with higher NVR than the 'fedora' and 'updates' trees).

Comment 24 Petr Schindler 2015-04-13 16:53:38 UTC
Discussed at today's blocker review meeting [1].

This bug was rejected as Beta Blocker and Freeze Exception - this bug has no significant effect on any known scenario related to the frozen package set and can be safely resolved with an update, so does not need to be a blocker or FE issue.

[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2015-04-13/

Comment 25 Fedora Update System 2015-04-22 22:50:28 UTC
samba-4.2.0-3.fc22 has been pushed to the Fedora 22 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 26 Alexander Yurtsev 2015-04-24 17:33:46 UTC
I am still seeing this problem with samba-4.2.1-2.fc22

Comment 27 Alexander Bokovoy 2015-04-24 17:37:02 UTC
You need to use 4.2.1-3.fc22. -2.fc22 was unpushed from updates-testing for a different issue. Use https://admin.fedoraproject.org/updates/samba-4.2.1-3.fc22

Comment 28 Alexander Yurtsev 2015-04-27 10:11:19 UTC
Thanks! There is already 4.2.1-4 and it works as expected.


Note You need to log in before you can comment on or make changes to this bug.