Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1775157
Summary: | lvm fails after glibc upgrade on armhfp | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Paul Whalen <pwhalen> | ||||||||
Component: | glibc | Assignee: | Carlos O'Donell <codonell> | ||||||||
Status: | CLOSED NOTABUG | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||||
Severity: | unspecified | Docs Contact: | |||||||||
Priority: | unspecified | ||||||||||
Version: | rawhide | CC: | aoliva, arjun.is, codonell, dj, fweimer, law, mfabian, pbrobinson, pfrankli, rth, siddhesh | ||||||||
Target Milestone: | --- | ||||||||||
Target Release: | --- | ||||||||||
Hardware: | armhfp | ||||||||||
OS: | Unspecified | ||||||||||
Whiteboard: | |||||||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||
Doc Text: | Story Points: | --- | |||||||||
Clone Of: | Environment: | ||||||||||
Last Closed: | 2019-11-26 17:35:02 UTC | Type: | Bug | ||||||||
Regression: | --- | Mount Type: | --- | ||||||||
Documentation: | --- | CRM: | |||||||||
Verified Versions: | Category: | --- | |||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||
Embargoed: | |||||||||||
Bug Depends On: | |||||||||||
Bug Blocks: | 245418 | ||||||||||
Attachments: |
|
Description
Paul Whalen
2019-11-21 13:53:25 UTC
Is this real hardware, or QEMU? If the latter, which kind? Is there anything in the audit log? On the real hardware (Raspberry Pi 3 and Banana Pi). Not seeing anything in audit.log: cat /var/log/audit/audit.log | grep lvm2 type=SERVICE_START msg=audit(1574307737.071:121): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-monitor comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset" type=SERVICE_START msg=audit(1574309167.423:136): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-pvscan@8:34 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'UID="root" AUID="unset" type=SERVICE_START msg=audit(1574309208.745:145): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-pvscan@8:18 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'UID="root" AUID="unset" type=SERVICE_START msg=audit(1574309386.982:121): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-pvscan@8:18 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset" type=SERVICE_STOP msg=audit(1574309388.820:133): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-pvscan@8:18 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset" type=SERVICE_START msg=audit(1574309673.247:122): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-pvscan@8:18 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset" type=SERVICE_STOP msg=audit(1574309977.200:205): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-pvscan@8:18 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset" [root@rpi3-5 ~]# cat /var/log/audit/audit.log | grep udev type=SERVICE_START msg=audit(1570712185.621:61): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-udevd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset" Created attachment 1638488 [details]
lvs verbose output
Comment on attachment 1638488 [details]
lvs verbose output
Has the command been executed correctly?
09:23:44.236905 lvs[692] cache/lvmcache.c:276 lvmcache has no info for vgname "lvs.txt".
Why lvs.txt?
Created attachment 1638499 [details]
lvs verbose output
Fixed, sorry about that.
Thanks. I would have expected denied system calls in the audit. In the log output, the time stamps align with the waiting, so that is good. I was worried that we broke something in the time-related system call wrappers, considering all the Y2038 work that is currently happening. Sorry, I do not have a way to debug this further. Created attachment 1638552 [details]
Fedora-Rawhide-20191108.n.0 boot log
This also affects installations when glibc-2.30.9000-17.fc32 was added, booting with "debug systemd.log_level=debug udev.log_priority=debug rd.udev.log_priority=debug" doesn't seem to give any useful output and ends with (full logs attached):
[ 90.046073] systemd-udevd (784) used greatest stack depth: 4992 bytes left
[ 90.132279] systemd-udevd (783) used greatest stack depth: 4972 bytes left
[ 90.141240] systemd-udevd (782) used greatest stack depth: 4644 bytes left
[ OK ] Started Create Volatile Files and Directories.[ 90.728507] audit: type=1130 audit(1570712207.728:11): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd'
[ OK ] Reached target System Initialization.
[ OK ] Listening on Open-iSCSI iscsid Socket.
[ OK ] Listening on Open-iSCSI iscsiuio Socket.
[ OK ] Reached target Sockets.
[ OK ] Reached target Basic System.
Starting iSCSI UserSpace I/O driver...
[ 90.862804] systemd-journald[288]: Successfully sent stream file descriptor to service manager.
[ OK ] Started iSCSI UserSpace I/O driver.[ 90.951496] audit: type=1130 audit(1570712207.951:12): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=iscsiuio comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr'
Starting Open-iSCSI...
[ OK ] Started Open-iSCSI.
[ 91.360853] audit: type=1130 audit(1570712208.360:13): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=iscsid comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 169.942795] systemd-journald[288]: Sent WATCHDOG=1 notification.
[ 229.941127] systemd-journald[288]: Sent WATCHDOG=1 notification.
[ 339.940934] systemd-journald[288]: Sent WATCHDOG=1 notification.
[ 409.942431] systemd-journald[288]: Sent WATCHDOG=1 notification.
Does downgrading to 2.30.9000-16 fix the issue? That would at least allow us to track down which *week* the change occurred on and what impact it had (we sync Rawhide weekly). Yes, glibc-2.30.9000-16.fc32 works as expected. Downgrading allows lvm commands to work. To simplify bisection, I have started a scratch build with broken-out patches: https://koji.fedoraproject.org/koji/taskinfo?taskID=39202209 (Some patches for Hurd and locale-only chanhes I dropped, so it's best to verify the .38 build first.) SRPM is also available here: https://people.redhat.com/~fweimer/glibc-2.30.9000-16.bz1775157.38.fc32.src.rpm Paul, Do you know if anyone else is running into this? We're trying to determine if this is some kind of common-mode failure across the systems you are using e.g. corrupted NFS disk etc. This is working again in 'Fedora-Rawhide-20191122.n.0' and onward. The version of glibc did not change (glibc-2.30.9000-19.fc32). Last broken compose was Fedora-Rawhide-20191119.n.2. Limiting updates this appears to be fixed by libseccomp-2.4.2-1.fc32.armv7hl . Once installed lvm commands work as expected. I also confirmed once libseccomp-2.4.2-1.fc32 was included in Rawhide installs worked as expected. Apologies for the noise (and thanks for your help). Thanks for the feedback! |