Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1900878 - The systemd upgrading from 246.6-3.fc34 to 247~rc2-1.fc34 version cause lost gnome and ssh sessions.
Summary: The systemd upgrading from 246.6-3.fc34 to 247~rc2-1.fc34 version cause lost ...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: gnome-shell
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Florian Müllner
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1900735 (view as bug list)
Depends On:
Blocks: F34BetaBlocker F34BetaFreezeException
TreeView+ depends on / blocked
 
Reported: 2020-11-23 21:53 UTC by Mikhail
Modified: 2020-12-18 16:57 UTC (History)
20 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-12-10 17:25:09 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)
dnf-terminal-output (4.18 KB, text/plain)
2020-11-23 21:54 UTC, Mikhail 		no flags Details
dnf.log (802.16 KB, text/plain)
2020-11-23 21:54 UTC, Mikhail 		no flags Details
dnf.rpm.log (740.45 KB, text/plain)
2020-11-23 21:55 UTC, Mikhail 		no flags Details
dnf.librepo.log (176.87 KB, text/plain)
2020-11-23 21:55 UTC, Mikhail 		no flags Details
hawkey.log (12.04 KB, text/plain)
2020-11-23 21:56 UTC, Mikhail 		no flags Details
system-log (11.99 KB, text/plain)
2020-11-23 21:57 UTC, Mikhail 		no flags Details
photo of shut down process (1.91 MB, image/jpeg)
2020-12-13 22:00 UTC, Mikhail 		no flags Details
system log (680.90 KB, text/plain)
2020-12-14 16:52 UTC, Mikhail 		no flags Details
system log (523.93 KB, text/plain)
2020-12-18 09:59 UTC, Mikhail 		no flags Details
View All

Description Mikhail 2020-11-23 21:53:31 UTC 
Description of problem:
The systemd upgrading from 246.6-3.fc34 to 247~rc2-1.fc34 version cause lost gnome and ssh sessions.
This is a very dangerous problem because it turns the system into an inconsistent state.
And also can potentially lead the system to an inoperative state.


How reproducible:


Steps to Reproduce:
1. upgrade systemd from 246.6-3.fc34 to 247~rc2-1.fc34 version

Demonstration: https://youtu.be/rILx1K5j4U4
Comment 1 Mikhail 2020-11-23 21:54:05 UTC 
Created attachment 1732703 [details]
dnf-terminal-output
Comment 2 Mikhail 2020-11-23 21:54:25 UTC 
Created attachment 1732704 [details]
dnf.log
Comment 3 Mikhail 2020-11-23 21:55:02 UTC 
Created attachment 1732705 [details]
dnf.rpm.log
Comment 4 Mikhail 2020-11-23 21:55:27 UTC 
Created attachment 1732706 [details]
dnf.librepo.log
Comment 5 Mikhail 2020-11-23 21:56:05 UTC 
Created attachment 1732719 [details]
hawkey.log
Comment 6 Mikhail 2020-11-23 21:57:02 UTC 
Created attachment 1732720 [details]
system-log
Comment 7 Fedora Blocker Bugs Application 2020-11-23 22:01:43 UTC 
Proposed as a Blocker and Freeze Exception for 34-beta by Fedora user mikhail using the blocker tracking app because:

 This is a very dangerous problem because it turns the system into an inconsistent state.
And also can potentially lead the system to an inoperative state.
Blocker bug criteria: The system should be upgraded without any issues.
Comment 8 Zbigniew Jędrzejewski-Szmek 2020-11-23 22:14:52 UTC 
Maybe https://github.com/systemd/systemd/issues/17605 ?
Comment 9 Zbigniew Jędrzejewski-Szmek 2020-11-27 13:39:20 UTC 
This should be fixed with systemd-247-1. Please check.
Comment 10 Adam Williamson 2020-12-02 17:40:04 UTC 
*** Bug 1900735 has been marked as a duplicate of this bug. ***
Comment 11 Zbigniew Jędrzejewski-Szmek 2020-12-10 17:25:09 UTC 
There have been no further reports, and I expect that systemd-247.1 fixes the issue.
Please reopen if it still occurs.
Comment 12 Mikhail 2020-12-13 22:00:21 UTC 
Created attachment 1738815 [details]
photo of shut down process

> There have been no further reports, and I expect that systemd-247.1 fixes the issue.
Yes the issue was gone but with new systemd the system began to shut down for a long time.
Comment 13 Zbigniew Jędrzejewski-Szmek 2020-12-14 08:17:07 UTC 
(In reply to Mikhail from comment #12)
> Yes the issue was gone but with new systemd the system began to shut down
> for a long time.

It seems that the systemd user instance is not shutting down properly. The reason should
be visible in the logs.
Comment 14 Mikhail 2020-12-14 16:52:16 UTC 
Created attachment 1739006 [details]
system log

> It seems that the systemd user instance is not shutting down properly. The reason should be visible in the logs.


Dec 14 21:28:33 localhost.localdomain systemd[2105]: xdg-permission-store.service: Main process exited, code=exited, status=1/FAILURE
Dec 14 21:28:33 localhost.localdomain systemd[2105]: xdg-permission-store.service: Failed with result 'exit-code'.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: dbus-:1.2-org.gnome.Shell.CalendarServer: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: gvfs-udisks2-volume-monitor.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: dbus-:1.2-org.gnome.Identity: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: dbus-:1.2-ca.desrt.dconf: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: gvfs-afc-volume-monitor.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[1]: run-user-1000-gvfs.mount: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: run-user-1000-gvfs.mount: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[1]: Unmounted /run/user/1000/gvfs.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: evolution-source-registry.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: gvfs-goa-volume-monitor.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: gvfs-gphoto2-volume-monitor.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: evolution-calendar-factory.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: evolution-addressbook-factory.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: dbus-:1.2-org.freedesktop.problems.applet: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: gnome-session-restart-dbus.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: gvfs-daemon.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: app-gnome-org.gnome.SettingsDaemon.DiskUtilityNotify-2649.scope: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Starting D-Bus User Message Bus...
Dec 14 21:28:33 localhost.localdomain systemd[2105]: dbus-:1.2-org.gnome.OnlineAccounts: Succeeded.
Dec 14 21:28:33 localhost.localdomain dbus-broker-launch[3355]: Policy to allow eavesdropping in /usr/share/dbus-1/session.conf +31: Eavesdropping is deprecated and ignored
Dec 14 21:28:33 localhost.localdomain dbus-broker-launch[3355]: Policy to allow eavesdropping in /usr/share/dbus-1/session.conf +33: Eavesdropping is deprecated and ignored
Dec 14 21:28:33 localhost.localdomain dbus-broker-lau[3355]: Ready
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Started D-Bus User Message Bus.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Started dbus-:1.1-ca.desrt.dconf.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: gvfs-metadata.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: org.gnome.Shell-disable-extensions.service: Succeeded.
Dec 14 21:30:09 localhost.localdomain systemd[2105]: Starting Mark boot as successful...
Dec 14 21:30:09 localhost.localdomain systemd[2105]: grub-boot-success.service: Succeeded.
Dec 14 21:30:09 localhost.localdomain systemd[2105]: Finished Mark boot as successful.
Dec 14 21:30:33 localhost.localdomain systemd[1]: user: State 'stop-sigterm' timed out. Killing.
Dec 14 21:30:33 localhost.localdomain systemd[1]: user: Killing process 2105 (systemd) with signal SIGKILL.
Dec 14 21:30:33 localhost.localdomain systemd[1]: user: Killing process 3355 (dbus-broker-lau) with signal SIGKILL.
Dec 14 21:30:33 localhost.localdomain systemd[1]: user: Killing process 3356 (dbus-broker) with signal SIGKILL.
Dec 14 21:30:33 localhost.localdomain systemd[1]: user: Killing process 3358 (dconf-service) with signal SIGKILL.
Dec 14 21:30:33 localhost.localdomain systemd[1]: user: Main process exited, code=killed, status=9/KILL
Dec 14 21:30:33 localhost.localdomain systemd[1]: user: Failed with result 'timeout'.
Dec 14 21:30:33 localhost.localdomain systemd[1]: Stopped User Manager for UID 1000.
Dec 14 21:30:33 localhost.localdomain audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=user@1000 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Dec 14 21:30:33 localhost.localdomain systemd[1]: user: Consumed 14.485s CPU time.
Comment 15 Mikhail 2020-12-18 09:59:40 UTC 
Created attachment 1740195 [details]
system log

Did you see why user: State 'stop-sigterm' is timed out?
I am look at the log and did not seen any suspisious.
Or needed logs with another verbose level?
I just want report that the problem still actual even with yesterday systemd 247.2-1
Comment 16 Zbigniew Jędrzejewski-Szmek 2020-12-18 14:04:31 UTC 
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Stopping GNOME Shell on Wayland...
Dec 14 21:28:33 localhost.localdomain audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=livesys comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Dec 14 21:28:33 localhost.localdomain systemd[1]: fwupd.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[1]: Stopped Firmware update daemon.
Dec 14 21:28:33 localhost.localdomain audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=fwupd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Dec 14 21:28:33 localhost.localdomain uresourced[1442]: Setting resources on user-1000.slice (MemoryMin: 0, MemoryLow: 0, CPUWeight: 100, IOWeight: 100)
Dec 14 21:28:33 localhost.localdomain uresourced[1442]: Setting resources on user (MemoryMin: 0, MemoryLow: 0, CPUWeight: 100, IOWeight: 100)
Dec 14 21:28:33 localhost.localdomain uresourced[1442]: Setting resources on user.slice (MemoryMin: 0, MemoryLow: 0, CPUWeight: -, IOWeight: -)
Dec 14 21:28:33 localhost.localdomain systemd[1]: Started Show Plymouth Reboot Screen.
Dec 14 21:28:33 localhost.localdomain audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=plymouth-reboot comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Dec 14 21:28:33 localhost.localdomain systemd[1]: systemd-logind.service: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[1]: Stopped User Login Management.
Dec 14 21:28:33 localhost.localdomain polkitd[1149]: Unregistered Authentication Agent for unix-session:2 (system bus name :1.81, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Dec 14 21:28:33 localhost.localdomain systemd[2105]: org.gnome.Shell: Main process exited, code=killed, status=11/SEGV
Dec 14 21:28:33 localhost.localdomain systemd[2105]: dbus-:1.2-org.gnome.Shell.Notifications: Succeeded.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: org.gnome.Shell: Failed with result 'signal'.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Stopped GNOME Shell on Wayland.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: org.gnome.Shell: Triggering OnFailure= dependencies.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: org.gnome.Shell: Consumed 6.412s CPU time.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Stopped target GNOME Session Manager is ready.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Stopped target Tasks to be run before GNOME Session starts.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Stopped target Session services which should run early before the graphical session is brought up.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Reached target Shutdown running GNOME Session.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Starting Restart DBus after GNOME Session shutdown...
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Started Disable GNOME Shell extensions after failure.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Stopped target Shutdown running GNOME Session.
Dec 14 21:28:33 localhost.localdomain systemd[2105]: Started Restart DBus after GNOME Session shutdown.

It seems to be a problem with gnome-shell: it crashes and has:
OnFailure=org.gnome.Shell-disable-extensions.service gnome-session-shutdown.target
OnFailureJobMode=replace-irreversibly

This means that the shutdown transaction that systemd was executing gets replaced by the transaction
to start gnome-session-shutdown.target, which does not cause the user@ instance to exit and thus
leads to the observed timeout.

Let's reassign this to gnome-shell for comments.
Comment 17 Jonas Ådahl 2020-12-18 14:33:41 UTC 
Can you attach the backtrace? I suspect it's a crash that should already have been fixed upstream.
Comment 18 Mikhail 2020-12-18 16:55:32 UTC 
> It seems to be a problem with gnome-shell: it crashes and has:

How to find the backtrace of this crash?

# coredumpctl 
TIME                            PID   UID   GID SIG COREFILE  EXE
Wed 2020-12-09 11:57:31 +05    2216  1000  1000  11 missing   /usr/bin/su
Wed 2020-12-09 11:57:34 +05    2257  1000  1000  11 missing   /usr/bin/su
Wed 2020-12-09 13:14:08 +05    8554  1000  1000  11 missing   /usr/bin/su
Wed 2020-12-09 13:14:11 +05    8572  1000  1000  11 missing   /usr/bin/su
Wed 2020-12-09 13:51:30 +05   12066  1000  1000  11 missing   /usr/bin/su
Wed 2020-12-09 13:57:37 +05    2223  1000  1000  11 missing   /usr/bin/su
Wed 2020-12-09 15:21:52 +05    6745  1000  1000  11 missing   /usr/bin/su
Wed 2020-12-09 15:44:39 +05    8390  1000  1000  11 missing   /usr/bin/su
Thu 2020-12-10 14:04:23 +05   56556  1000  1000  11 missing   /usr/lib64/firefox/plugin-container
Comment 19 Jonas Ådahl 2020-12-18 16:57:16 UTC 
If anywhere it should have been visible via coredumpctl, so I don't know where it went.
Note You need to log in before you can comment on or make changes to this bug.