Bug 1266756

Created attachment 1077675 [details]
File: backtrace

Created attachment 1077676 [details]
File: build_ids

Created attachment 1077677 [details]
File: cgroup

Created attachment 1077678 [details]
File: core_backtrace

Created attachment 1077679 [details]
File: dso_list

Created attachment 1077680 [details]
File: environ

Created attachment 1077681 [details]
File: limits

Created attachment 1077682 [details]
File: maps

Created attachment 1077683 [details]
File: mountinfo

Created attachment 1077684 [details]
File: namespaces

Created attachment 1077685 [details]
File: open_fds

Created attachment 1077686 [details]
File: proc_pid_status

Created attachment 1077687 [details]
File: var_log_messages

Another user experienced a similar problem:

Steps to reproduce:
1. Install Fedora 23 i686 (RC3 compose) on qemu-kvm
2. Boot the installed system.
3. Login to a Plasma session.

Actual result:
1. A window related to kwalletd pops up.
2. abrt catches a crash.

reporter:       libreport-2.6.2
backtrace_rating: 4
cmdline:        /usr/bin/kwalletd5 --pam-login 15 19
crash_function: qt_message_fatal
executable:     /usr/bin/kwalletd5
global_pid:     1374
kernel:         4.2.3-300.fc23.i686
package:        kf5-kwallet-5.14.0-1.fc23
reason:         kwalletd5 killed by SIGABRT
runlevel:       N 5
type:           CCpp
uid:            1000

Created attachment 1085943 [details]
kwalletd's startup window

this window pops up right after the first login

Proposed as a Blocker for 23-final by Fedora user juliuxpigface using the blocker tracking app because:

 I've hit this right after the first login on an installed i686 system (qemu-kvm).

It seems a violation of the "2.4.4 SELinux and crash notifications" final criteria:

"There must be no SELinux denial notifications or crash notifications on boot of or during installation from a release-blocking live image, or at first login after a default install of a release-blocking desktop."

https://fedoraproject.org/wiki/Fedora_23_Final_Release_Criteria#SELinux_and_crash_notifications

Another user experienced a similar problem:

on first boot

reporter:       libreport-2.6.2
backtrace_rating: 4
cmdline:        /usr/bin/kwalletd5 --pam-login 15 19
crash_function: qt_message_fatal
executable:     /usr/bin/kwalletd5
global_pid:     1368
kernel:         4.2.3-300.fc23.i686
package:        kf5-kwallet-5.14.0-1.fc23
reason:         kwalletd5 killed by SIGABRT
runlevel:       N 5
type:           CCpp
uid:            1000

(In reply to satellitgo from comment #17)
> Another user experienced a similar problem:
> 
> on first boot
> 
> reporter:       libreport-2.6.2
> backtrace_rating: 4
> cmdline:        /usr/bin/kwalletd5 --pam-login 15 19
> crash_function: qt_message_fatal
> executable:     /usr/bin/kwalletd5
> global_pid:     1368
> kernel:         4.2.3-300.fc23.i686
> package:        kf5-kwallet-5.14.0-1.fc23
> reason:         kwalletd5 killed by SIGABRT
> runlevel:       N 5
> type:           CCpp
> uid:            1000
on boot: "kwalletd5rc not writiable"

The fatal error says: "Cannot create wallet save location!"

It's this qFatal:
https://quickgit.kde.org/?p=kwallet.git&a=blob&h=fbb014d2b9efc38072634c6178013daabe882587&hb=f1e6b9d168281196010c44af2eae4587c1d2d088&f=src%2Fruntime%2Fkwalletd%2Fbackend%2Fkwalletbackend.cc#l108

I have 2 guesses as to what could be the issue:
1. I don't see HOME being set in environ. If it is really not set in the environment, it might be trying to write to something else as a result, instead of the expected ~/.local/…. (Unfortunately, the backtrace was done without a printer for QString, so I don't know what's in writeLocation and writeDir.)
2. It could also be an SELinux denial. (Try booting with selinux=0, does the error still appear then?)

There's an error message shown on login, as well as the crash:

Configuration file "//.config/kwalletd5rc" not writable. Please contact your system administrator.

That seems to tie in with kk's guess #1 (note the location).

This bug seems to be i686 only - I tested x86_64 and i686 and saw it only on i686.

(In reply to Kevin Kofler from comment #19)
> The fatal error says: "Cannot create wallet save location!"
> 
> It's this qFatal:
> https://quickgit.kde.org/?p=kwallet.
> git&a=blob&h=fbb014d2b9efc38072634c6178013daabe882587&hb=f1e6b9d168281196010c
> 44af2eae4587c1d2d088&f=src%2Fruntime%2Fkwalletd%2Fbackend%2Fkwalletbackend.
> cc#l108
> 
> I have 2 guesses as to what could be the issue:
> 1. I don't see HOME being set in environ. If it is really not set in the
> environment, it might be trying to write to something else as a result,
> instead of the expected ~/.local/…. (Unfortunately, the backtrace was done
> without a printer for QString, so I don't know what's in writeLocation and
> writeDir.)
> 2. It could also be an SELinux denial. (Try booting with selinux=0, does the
> error still appear then?)
selinux=0, in boot line, does not fix it in virtual machine manager install with 2048 memoru and 2 cpu's

So to save anyone else the trouble, we (Kevin and I) think we know what's going on here now: it's very similar to a bug spotted and fixed a few weeks back, https://bugzilla.redhat.com/show_bug.cgi?id=1265813 , more or less just another case of the same thing. The offending code is in src/helper/Backend.cpp:

        pw = getpwnam(qPrintable(qobject_cast<HelperApp*>(parent())->user()));

Kevin's also spotted several other places something similar is happening, so he's just thinking about the best way to go about this. But we don't need more info or diagnosis at present.

Right, we tracked this down to yet another use-after-free in SDDM, similar to the one that was recently fixed upstream, except using qPrintable rather than QByteArray (so the -DQT_NO_CAST_FROM_BYTEARRAY safeguard does not catch it).

Created attachment 1085970 [details]
sddm-0.12.0-fix-use-after-free.patch

The attached patch should fix this use-after-free. There are other uses of qPrintable, some of which are also suspicious (e.g. in the PasswdBackend), but this patch should fix this blocker.

Bad news - it doesn't seem to solve the bug. I built a live image with an SDDM with that patch applied, installed it, booted the installed system, and got the same KWallet error.

I'm out of ideas now. :-( I'm resetting this to the default assignee, but of course, if we can think of something to try, I'm still here to help.

Reassigning back to kf5-kwallet because it looks like that's where the issue lies after all.

My next suspect is https://quickgit.kde.org/?p=kwallet.git&a=commit&h=f1e6b9d168281196010c44af2eae4587c1d2d088 , which refers to a bug that looks  identical to this: https://bugs.kde.org/show_bug.cgi?id=351805 . Testing that theory now.

OK, yup, that's it. Installing from an image built with a patched kf5-kwallet results in no bug.

Possibly of interest, I first tried installing the patched kf5-kwallet to an installed system which had already hit the bug, and KDE session startup broke completely. Not sure if that was just a one-off thing, or something we might have to watch out for, if people have installed from images that suffer this bug.

We have an issue here: there's already a kf5 5.15.0 update in updates-testing, pending stable status.

https://bodhi.fedoraproject.org/updates/FEDORA-2015-084749eee7

I'd very strongly suggest we should *NOT* take a version bump for a couple of dozen packages through freeze at this point. That means we need to 'rewind' kf5-kwallet; we need to do a 5.14.0-2 build and pull that through the freeze instead. Dennis, can you advise on the best way to do that? Do we need to unpush that update, do a 5.14.0-2 build of kf5-kwallet, submit that, and then possibly go and do a 5.15.0-2 build of kf5-kwallet and re-submit the 5.15.0 update (for 0-day / post-release purposes), or what? Thanks!

(In reply to Kevin Kofler from comment #24)
> Created attachment 1085970 [details]
> sddm-0.12.0-fix-use-after-free.patch
> 
> The attached patch should fix this use-after-free. There are other uses of
> qPrintable, some of which are also suspicious (e.g. in the PasswdBackend),
> but this patch should fix this blocker.

It's a problem only if the qPrintable return value is used after the statement where qPrintable is invoked. So as long as that const char* is not referenced later it should be fine.

Fwiw, I pulled the fix into latest kf5-kwallet-5.15.0 builds at least,

%changelog
* Sat Oct 24 2015 Rex Dieter <rdieter> 5.15.0-2
- .spec cosmetics, update URL, backport upstream fixes (#1266756)


I'm personally in favor of pulling kf5-5.15.0 in (it includes several other nice-to-have's, including kf5-kservice related fixes for kbuildsycoca and login delays, https://bugs.kde.org/show_bug.cgi?id=353203), but if folks insist that's not viable, I could try flexing my releng supwercow powers to get an older 5.14 build too

Take your pick from:

kf5-kwallet-5.15.0-2.fc23:
https://bodhi.fedoraproject.org/updates/FEDORA-2015-71c484c99b

(depends on the rest of kf5-5.15.0 too of course,
https://bodhi.fedoraproject.org/updates/FEDORA-2015-084749eee7
)

or

kf5-kwallet-5.14.0-2.fc23:
https://koji.fedoraproject.org/koji/buildinfo?buildID=694085


I'm guessing it would be problematic to try submitting the older one via bodhi, though we could probably try tagging it manually.

+1 blocker. I dont have a strong opinion of whether or not it is best to try a limited fix or do a version upfate.

Thanks for the builds! I would strongly prefer the 5.14.0 build, we have already slipped once, I don't want unnecessary churn in subsequent composes.

I *believe* it's OK to submit older builds to Bodhi, but that's more or less what I was asking for Dennis' advice on. We'll figure it out, anyhow.

Another user experienced a similar problem:

On log-in after rebooting from command line. I've not yet tried to reproduce.

reporter:       libreport-2.6.2
backtrace_rating: 4
cmdline:        /usr/bin/kwalletd5 --pam-login 14 18
crash_function: qt_message_fatal
executable:     /usr/bin/kwalletd5
global_pid:     1379
kernel:         4.2.3-300.fc23.i686
package:        kf5-kwallet-5.14.0-1.fc23
reason:         kwalletd5 killed by SIGABRT
runlevel:       N 5
type:           CCpp
uid:            1000

I'm also +1 blocker here.

Regarding the Bodhi updates: you *can* submit both to Bodhi, but at least back in Bodhi 1.x that could cause issues (because it would cause the second one submitted to supplant the earlier one from the repository, which can result in a stuck upgrade path for anyone who installed the later version). I have no idea if that was fixed in Bodhi 2.x.

Of course, anyone who has installed the 5.15.x builds from u-t won't get this fix until a later 5.15 build arrives with it. And we can't just epoch-bump kf5-wallet because we'd have to do the same to every other package that depends on it... I think the best we can do is unpush the 5.15.x stuff from updates-testing and announce that this happened on announce@ so people can manually downgrade.

This is one of the reasons why I've never been comfortable with KDE's blanket exception to the stable updates policy. Maybe we can at least require for the future that no major/minor updates happen between Beta Freeze and Final Release, even in u-t? It's just too difficult to navigate since all the pieces need to update together.

Discussed at 2015-10-26 blocker review meeting: [1]. 

This bug was accepted as Final blocker: This bug is a clear violation of the following criterion: "There must be no SELinux denial notifications or crash notifications on boot of or during installation from a release-blocking live image, or at first login after a default install of a release-blocking desktop."

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1271993

The right ling to the logs is https://meetbot.fedoraproject.org/fedora-blocker-review/2015-10-26/f23-blocker-review.2015-10-26-16.00.html

kf5-kwallet-5.14.0-2.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-845978d82d

> This is one of the reasons why I've never been comfortable with
> KDE's blanket exception to the stable updates policy.

Especially with upstream's KF5 release policy (no stable branches), there is no other way. And even if it had been a point release (which are commonly pushed for many other packages including GNOME), we would have had the same issue because QA refused to just drag in the new release.

> Maybe we can at least require for the future that no major/minor
> updates happen between Beta Freeze and Final Release, even in u-t?
> It's just too difficult to navigate since all the pieces need to
> update together.

We should just pull those updates, which are mostly (though not entirely) bugfixes, in through the freeze.

kf5-kwallet-5.14.0-2.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update kf5-kwallet'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-845978d82d

kf5-kwallet-5.14.0-2.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

removing needinfo