Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1366403 - [abrt] sssd-common: ipa_dyndns_update_send(): sssd_be killed by SIGSEGV
Summary: [abrt] sssd-common: ipa_dyndns_update_send(): sssd_be killed by SIGSEGV
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: sssd
Version: 25
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jakub Hrozek
QA Contact: Fedora Extras Quality Assurance
URL: https://retrace.fedoraproject.org/faf...
Whiteboard: abrt_hash:834ebfd7d84ecc1f38ee1047b14...
Depends On:
Blocks: F25AlphaBlocker F25AlphaFreezeException
TreeView+ depends on / blocked
 
Reported: 2016-08-11 21:35 UTC by Adam Williamson
Modified: 2016-08-19 08:33 UTC (History)
12 users (show)

Fixed In Version: sssd-1.14.0-5.fc25
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-19 02:25:37 UTC
Type: ---
Embargoed:

Attachments (Terms of Use)
File: backtrace (24.69 KB, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: cgroup (223 bytes, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: core_backtrace (3.18 KB, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: dso_list (8.58 KB, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: environ (145 bytes, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: exploitable (82 bytes, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: limits (1.29 KB, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: maps (38.30 KB, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: mountinfo (2.95 KB, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: namespaces (102 bytes, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: open_fds (1.98 KB, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: proc_pid_status (1.07 KB, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
File: var_log_messages (757 bytes, text/plain)
2016-08-11 21:35 UTC, Adam Williamson 		no flags Details
coredump (2.35 MB, application/octet-stream)
2016-08-12 14:59 UTC, Adam Williamson 		no flags Details
View All

Description Adam Williamson 2016-08-11 21:35:01 UTC 
Description of problem:
Happens right after boot finishes - before user logs in - on a Fedora 25 install (from Fedora-Server-dvd-x86_64-25-20160810.n.0.iso ) which was enrolled to a FreeIPA domain during installation via a kickstart. This is an openQA test: https://openqa.stg.fedoraproject.org/tests/31320 . Later on, the test attempts to log in as a FreeIPA user, and the login fails; I believe this crash is why. A system enrolled to the same server post-install does not show the same problem.

Version-Release number of selected component:
sssd-common-1.14.0-4.fc25

Additional info:
reporter:       libreport-2.7.2
backtrace_rating: 4
cmdline:        /usr/libexec/sssd/sssd_be --domain domain.local --uid 0 --gid 0 --debug-to-files
crash_function: ipa_dyndns_update_send
executable:     /usr/libexec/sssd/sssd_be
global_pid:     973
kernel:         4.8.0-0.rc1.git0.1.fc25.x86_64
pkg_vendor:     Fedora Project
runlevel:       N 3
type:           CCpp
uid:            0

Truncated backtrace:
Thread no. 1 (10 frames)
 #0 ipa_dyndns_update_send at src/providers/ipa/ipa_dyndns.c:175
 #1 ipa_dyndns_update at src/providers/ipa/ipa_dyndns.c:122
 #2 be_run_cb_step at src/providers/data_provider_callbacks.c:96
 #3 tevent_common_loop_timer_delay at ../tevent_timed.c:341
 #4 epoll_event_loop at ../tevent_epoll.c:659
 #5 epoll_event_loop_once at ../tevent_epoll.c:926
 #6 std_event_loop_once at ../tevent_standard.c:114
 #7 _tevent_loop_once at ../tevent.c:533
 #8 tevent_common_loop_wait at ../tevent.c:637
 #9 std_event_loop_wait at ../tevent_standard.c:145
Comment 1 Adam Williamson 2016-08-11 21:35:09 UTC 
Created attachment 1190161 [details]
File: backtrace
Comment 2 Adam Williamson 2016-08-11 21:35:10 UTC 
Created attachment 1190162 [details]
File: cgroup
Comment 3 Adam Williamson 2016-08-11 21:35:11 UTC 
Created attachment 1190163 [details]
File: core_backtrace
Comment 4 Adam Williamson 2016-08-11 21:35:13 UTC 
Created attachment 1190164 [details]
File: dso_list
Comment 5 Adam Williamson 2016-08-11 21:35:14 UTC 
Created attachment 1190165 [details]
File: environ
Comment 6 Adam Williamson 2016-08-11 21:35:15 UTC 
Created attachment 1190166 [details]
File: exploitable
Comment 7 Adam Williamson 2016-08-11 21:35:16 UTC 
Created attachment 1190167 [details]
File: limits
Comment 8 Adam Williamson 2016-08-11 21:35:18 UTC 
Created attachment 1190168 [details]
File: maps
Comment 9 Adam Williamson 2016-08-11 21:35:19 UTC 
Created attachment 1190169 [details]
File: mountinfo
Comment 10 Adam Williamson 2016-08-11 21:35:20 UTC 
Created attachment 1190170 [details]
File: namespaces
Comment 11 Adam Williamson 2016-08-11 21:35:21 UTC 
Created attachment 1190171 [details]
File: open_fds
Comment 12 Adam Williamson 2016-08-11 21:35:22 UTC 
Created attachment 1190172 [details]
File: proc_pid_status
Comment 13 Adam Williamson 2016-08-11 21:35:24 UTC 
Created attachment 1190173 [details]
File: var_log_messages
Comment 14 Adam Williamson 2016-08-11 21:44:20 UTC 
I'm gonna at least propose this as an Alpha blocker. Not absolutely sure if it needs to be, but we should at least consider it. Criterion is "It must be possible to join the system to a FreeIPA or Active Directory domain at install time and post-install, and the system must respect the identity, authentication and access control configuration provided by the domain." - post-install works fine with updates-testing packages, but this bug affects install time (kickstart) enrolment; the enrolment works, but the crash seems to result in the installed system not "respect[ing] the identity, authentication and access control configuration provided by the domain."
Comment 15 Lukas Slebodnik 2016-08-12 07:00:01 UTC 
Could you also attach coredump?
Comment 16 Lukas Slebodnik 2016-08-12 07:10:40 UTC 
It is fixed in upstream by commit b5f61f8963300c9ba011436f234e9e10224aff6d
But It was just a band aid because the previous reporter was not able to reproduce the crash.

Is there a simple way how to run the openqa test in local environment?
Comment 17 Adam Williamson 2016-08-12 14:57:46 UTC 
unfortunately not really. The test basically sets up a system - 'ipa001.domain.local' with static IP 10.0.2.100 - as a FreeIPA server using rolectl, adds a couple of users, and sets up a one-time password for enrolment of a client:

ipa host-add client001.domain.local --password=monkeys --force

the client install part of the test runs an install from the Server DVD with this kickstart:

install
cdrom
bootloader --location=mbr
network --device=link --activate --bootproto=static --ip=10.0.2.101 --netmask=255.255.255.0 --gateway=10.0.2.2 --hostname=client001.domain.local --nameserver=10.0.2.100
lang en_US.UTF-8
keyboard us
timezone --utc America/New_York
clearpart --all
autopart
%packages
@^server-product-environment
%end
rootpw anaconda
reboot
realm join --one-time-password=monkeys ipa001.domain.local

then boots the installed system. From the logs the crash happens just at the end of client system boot, before any login or other action is done; trying to log into the client as one of the users created on the server fails.

I'll add the coredump.
Comment 18 Adam Williamson 2016-08-12 14:59:33 UTC 
Created attachment 1190442 [details]
coredump
Comment 19 Adam Williamson 2016-08-15 16:21:34 UTC 
Lukas: can we please get a Fedora build with the band-aid applied? Alpha go/no-go is on Thursday so we are working to a tight deadline here. Thanks!
Comment 20 Fedora Update System 2016-08-15 18:58:54 UTC 
sssd-1.14.0-5.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-97debec731
Comment 21 Geoffrey Marr 2016-08-15 19:02:27 UTC 
Discussed during the 2016-08-15 blocker review meeting: [1]

The decision to classify this bug as an AcceptedBlocker was made as 
this appears to violate "It must be possible to join the system to a FreeIPA or Active Directory domain at install time and post-install, and the system must respect the identity, authentication and access control configuration provided by the domain."

adamw plans to perform some more testing concerning this bug to ensure it is as reported.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2016-08-15/f25-blocker-review.2016-08-15-16.00.txt
Comment 22 Fedora Update System 2016-08-16 16:26:07 UTC 
sssd-1.14.0-5.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-97debec731
Comment 23 Adam Williamson 2016-08-16 22:12:17 UTC 
Good news - the update does stop the crash. Bad news - login still doesn't work :( These are the errors from the journal:

Aug 16 10:00:31 client001.domain.local login[1228]: pam_sss(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=tty3 ruser= rhost= user=test1
Aug 16 10:00:31 client001.domain.local login[1228]: pam_sss(login:auth): received for user test1: 4 (System error)
Aug 16 10:00:31 client001.domain.local login[1228]: FAILED LOGIN 1 FROM tty3 FOR test1, Authentication failure

I'll see if I can enable debugging and get relevant logs from server and client...
Comment 24 Adam Williamson 2016-08-18 00:42:05 UTC 
So I wound up reporting the login issue separately:

https://bugzilla.redhat.com/show_bug.cgi?id=1367604

and then, found out it seems to be caused by issues in anaconda dealing with the system time and timezones:

https://bugzilla.redhat.com/show_bug.cgi?id=1367647

Given that, I'm gonna remove the accepted blocker status from this bug, as we don't know that the crash actually has significant consequences. I will try if I can in the next day or two to test and see what happens if we fix the time issue but do *not* fix the crash.

I'm also gonna nominate it for a freeze exception, because it probably makes sense to fix this crash even if it doesn't *seem* to cause immediately obvious terrible consequences (and thus doesn't count as a blocker).
Comment 25 Adam Williamson 2016-08-18 19:09:27 UTC 
Discussed at 2016-08-18 go/no-go meeting, functioning as a blocker review meeting: https://meetbot-raw.fedoraproject.org/fedora-meeting/2016-08-18/f25-alpha-go_no_go-meeting.2016-08-18-17.00.html . We agreed to delay the decision on blocker status as we are not sure what the practical consequences of the crash are yet, but we agreed a crash in sssd like this is at least serious enough to warrant a freeze exception (especially given the fix is pretty demonstrably safe). If necessary I will test and see what happens when the other bug is fixed, but this one is not.
Comment 26 Lukas Slebodnik 2016-08-18 20:08:16 UTC 
Fixed version should be already in stable.
https://bodhi.fedoraproject.org/updates/FEDORA-2016-97debec731
Comment 27 Adam Williamson 2016-08-18 20:15:37 UTC 
No it isn't, because we're frozen. All pushes during freezes are manual and have to be co-ordinated between QA and releng. I'm taking care of it.
Comment 28 Fedora Update System 2016-08-19 02:25:32 UTC 
sssd-1.14.0-5.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
Comment 29 Lukas Slebodnik 2016-08-19 08:33:36 UTC 
(In reply to Adam Williamson from comment #27)
> No it isn't, because we're frozen. All pushes during freezes are manual and
> have to be co-ordinated between QA and releng. I'm taking care of it.

Thank you for info
Note You need to log in before you can comment on or make changes to this bug.