Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1732612
Summary: | Bootloader install fails on ppc64le since Fedora-Rawhide-20190719.n.1 | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Adam Williamson <awilliam> |
Component: | grub2 | Assignee: | Peter Jones <pjones> |
Status: | CLOSED RAWHIDE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | rawhide | CC: | airlied, bskeggs, dan, fmartine, hannsj_uhl, hdegoede, ichavero, itamar, jarodwilson, jeremy, jglisse, john.j5live, jonathan, josef, kernel-maint, linville, lkundrak, mchehab, menantea, mjg59, normand, pjones, steved |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | ppc64le | ||
OS: | Linux | ||
Whiteboard: | openqa | ||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2019-08-05 16:53:22 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1071880 |
Description
Adam Williamson
2019-07-23 21:36:18 UTC
Note, here's how the same period of logs looks in a successful install from a previous compose: 22:24:09,546 INF program: Running... nvram --print-config=boot-device 22:24:09,572 INF program: nvram: ERROR: cannot open "/dev/nvram": No such file or directory 22:24:09,573 INF program: nvram: ERROR: cannot open "/dev/misc/nvram": No such file or directory 22:24:09,574 DBG program: Return code: 255 22:24:09,575 INF program: Running... ofpathname /dev/vda1 22:24:10,060 INF program: /pci@800000020000000/scsi@7 22:24:10,061 DBG program: Return code: 0 22:24:10,062 INF program: Running... nvram --update-config boot-device=/pci@800000020000000/scsi@7 nvram: ERROR: cannot open /dev/nvram: No such file or directory nvram: ERROR: cannot open /dev/misc/nvram: No such file or directory 22:24:10,076 INF program: nvram: ERROR: cannot open "/dev/nvram": No such file or directory 22:24:10,078 INF program: nvram: ERROR: cannot open "/dev/misc/nvram": No such file or directory 22:24:10,081 DBG program: Return code: 255 22:24:10,083 INF program: Running in chroot '/mnt/sysroot'... grub2-install --no-nvram --no-floppy /dev/vda1 22:24:10,620 INF program: Installing for powerpc-ieee1275 platform. 22:24:10,622 INF program: Installation finished. No error reported. 22:24:10,624 DBG program: Return code: 0 22:24:10,652 INF program: Running... xfs_freeze -f /boot 22:24:10,696 DBG program: Return code: 0 22:24:10,699 INF program: Running... xfs_freeze -u /boot 22:24:10,722 DBG program: Return code: 0 22:24:10,727 INF program: Running in chroot '/mnt/sysroot'... grub2-set-default de505b69040c4b45bc9d5bc9b0f271ec-5.3.0-0.rc0.git4.1.fc31.ppc64le 22:24:10,768 DBG program: Return code: 0 22:24:10,770 INF program: Running in chroot '/mnt/sysroot'... grub2-mkconfig -o /boot/grub2/grub.cfg 22:24:15,908 INF program: Generating grub configuration file ... 22:24:15,909 INF program: Generating boot entries from BLS files... 22:24:15,910 INF program: done 22:24:15,910 DBG program: Return code: 0 so the nvram errors are 'normal' and aren't the problem, probably. It's the grub2-install and grub2-mkconfig failures that are new. FYIO, trying to investigate this problem with last compose 20190724 I raised another failure tracked by It could be a change coming with 5.3 kernel and the configs, seems we have # CONFIG_NVRAM is not set now (https://src.fedoraproject.org/rpms/kernel/blob/master/f/kernel-ppc64le.config#_3611) because NVRAM is disabled by default in https://src.fedoraproject.org/rpms/kernel/blob/master/f/configs/fedora/generic/CONFIG_NVRAM And because PowerNV doesn't need nvram, I haven't noticed it even when running kernel-5.3.0-0.rc1.git0.1.fc31.op.1.ppc64le :-) or maybe it's not nvram's fault ... Better set back to grub, it might be in the rebase/cleanup in grub2-2.02-92.fc31 Note that on an VM installed with Fedora-Rawhide-20190717.n.0 compose, changing /etc/default/grub and running "grub2-mkconfig -o /boot/grub2/grub.cfg" works fine with grub2 tools 2.02-92.fc31.ppc64le.rpm but it fails with on "/usr/sbin/grub2-probe: error: cannot find a GRUB drive for /dev/sda2. Check your device.map." with grub2 tools updated to 2.02-94.fc31.src.rpm on same VM. Yeah, sorry for mentioning the nvram stuff in #c0, but as #c1 says, it's almost certainly not the problem as we get those errors in the logs from a working install too. It's the grub2-install and grub2-mkconfig failures that are new. BTW, one thing I considered is whether the rearrangement in a1dedc8a10fcb7bda743e4dc8470fefb6f6801df meant only grub2-tools-minimal was now included in the installer images but ppc64le needs grub2-tools, or something...but according to the logs, that's not it, both working and broken composes have both grub2-tools and grub2-tools-minimal in the image. (In reply to Adam Williamson from comment #8) > Yeah, sorry for mentioning the nvram stuff in #c0, but as #c1 says, it's > almost certainly not the problem as we get those errors in the logs from a > working install too. It's the grub2-install and grub2-mkconfig failures that > are new. yes, I have focused too much on the nvram issue :-) Anyway, the NVRAM driver should be enabled, patch in https://lists.fedoraproject.org/archives/list/kernel@lists.fedoraproject.org/message/HMZPWGSW5LU6ITH647IGTOELN2XGTIL5/ So I've replicated the failure locally (KVM guest), device.map looks correct [anaconda root@fedora-ppc grub2]# cat device.map # this device map was generated by anaconda (hd0) /dev/vda and a verbose run of grub2-install from the new system's chroot is [anaconda root@fedora-ppc /]# grub2-install -v --no-nvram --no-floppy /dev/vda1 Installing for powerpc-ieee1275 platform. grub2-install: info: adding `hd0' -> `/dev/vda' from device.map. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/minix2_be.mod' -> `/boot/grub2/powerpc-ieee1275/minix2_be.mod'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/sfs.mod' -> `/boot/grub2/powerpc-ieee1275/sfs.mod'. ... grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/div.mod' -> `/boot/grub2/powerpc-ieee1275/div.mod'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/ofnet.mod' -> `/boot/grub2/powerpc-ieee1275/ofnet.mod'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/gcry_seed.mod' -> `/boot/grub2/powerpc-ieee1275/gcry_seed.mod'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/cmp_test.mod' -> `/boot/grub2/powerpc-ieee1275/cmp_test.mod'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/efiemu32.o' -> `/boot/grub2/powerpc-ieee1275/efiemu32.o'. grub2-install: info: cannot open `/usr/lib/grub/powerpc-ieee1275/efiemu32.o': No such file or directory. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/efiemu64.o' -> `/boot/grub2/powerpc-ieee1275/efiemu64.o'. grub2-install: info: cannot open `/usr/lib/grub/powerpc-ieee1275/efiemu64.o': No such file or directory. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/moddep.lst' -> `/boot/grub2/powerpc-ieee1275/moddep.lst'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/command.lst' -> `/boot/grub2/powerpc-ieee1275/command.lst'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/fs.lst' -> `/boot/grub2/powerpc-ieee1275/fs.lst'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/partmap.lst' -> `/boot/grub2/powerpc-ieee1275/partmap.lst'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/parttool.lst' -> `/boot/grub2/powerpc-ieee1275/parttool.lst'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/video.lst' -> `/boot/grub2/powerpc-ieee1275/video.lst'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/crypto.lst' -> `/boot/grub2/powerpc-ieee1275/crypto.lst'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/terminal.lst' -> `/boot/grub2/powerpc-ieee1275/terminal.lst'. grub2-install: info: copying `/usr/lib/grub/powerpc-ieee1275/modinfo.sh' -> `/boot/grub2/powerpc-ieee1275/modinfo.sh'. grub2-install: info: copying `/usr/share/grub/unicode.pf2' -> `/boot/grub2/fonts/unicode.pf2'. grub2-install: info: /dev/vda2 is present. grub2-install: info: Looking for /dev/vda2. grub2-install: info: /dev/vda is a parent of /dev/vda2. grub2-install: info: /dev/vda2 starts from 10240. grub2-install: info: opening the device hd0. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 0. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 0. grub2-install: info: Scanning for DISKFILTER devices on disk hd0. grub2-install: info: Scanning for mdraid1x devices on disk hd0. grub2-install: info: Scanning for mdraid09_be devices on disk hd0. grub2-install: info: Scanning for mdraid09 devices on disk hd0. grub2-install: info: Scanning for dmraid_nv devices on disk hd0. grub2-install: info: Scanning for ldm devices on disk hd0. grub2-install: info: scanning hd0 for LDM. grub2-install: info: no LDM signature found. grub2-install: info: Scanning for lvm devices on disk hd0. grub2-install: info: cannot find the partition of `/dev/vda2'. grub2-install: info: /dev/vda2 is present. grub2-install: info: Looking for /dev/vda2. grub2-install: info: /dev/vda is a parent of /dev/vda2. grub2-install: info: /dev/vda2 is present. grub2-install: info: Looking for /dev/vda2. grub2-install: info: /dev/vda is a parent of /dev/vda2. grub2-install: error: cannot find a GRUB drive for /dev/vda2. Check your device.map. and it's a change between grub2-2.02-92.fc31 (works) and grub2-2.02-93.fc31 (fails) to whose I've downgraded grub2 from Fedora-Rawhide-20190726.n.0 installation and grub2-2.02-92.fc31 rebuilt in current buildroot works too The difference between good and bad is in ... grub2-install: info: /dev/vda2 is present. grub2-install: info: Looking for /dev/vda2. grub2-install: info: /dev/vda is a parent of /dev/vda2. grub2-install: info: /dev/vda2 starts from 10240. grub2-install: info: opening the device hd0. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 0. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 0. ... while the good run has ... grub2-install: info: /dev/vda2 is present. grub2-install: info: Looking for /dev/vda2. grub2-install: info: /dev/vda is a parent of /dev/vda2. grub2-install: info: /dev/vda2 starts from 10240. grub2-install: info: opening the device hd0. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 20971520. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 20971520. ... See the hd0 site info. Hello Dan, (In reply to Dan Horák from comment #13) > and grub2-2.02-92.fc31 rebuilt in current buildroot works too > > The difference between good and bad is in > > ... > grub2-install: info: /dev/vda2 is present. > grub2-install: info: Looking for /dev/vda2. > grub2-install: info: /dev/vda is a parent of /dev/vda2. > grub2-install: info: /dev/vda2 starts from 10240. > grub2-install: info: opening the device hd0. > grub2-install: info: drive = 0. > grub2-install: info: the size of hd0 is 0. > grub2-install: info: drive = 0. > grub2-install: info: the size of hd0 is 0. > ... > > while the good run has > > ... > grub2-install: info: /dev/vda2 is present. > grub2-install: info: Looking for /dev/vda2. > grub2-install: info: /dev/vda is a parent of /dev/vda2. > grub2-install: info: /dev/vda2 starts from 10240. > grub2-install: info: opening the device hd0. > grub2-install: info: drive = 0. > grub2-install: info: the size of hd0 is 20971520. > grub2-install: info: drive = 0. > grub2-install: info: the size of hd0 is 20971520. > ... > > See the hd0 site info. Indeed. That's strange since the changes in the package between the working (grub2-2.02-92.fc31) and failing (grub2-2.02-93.fc31) versions are not related to grub2-install at all. Maybe this is an incompatibility between the grub2-install binary and one of its linked libraries? I noticed that there was a mass rebuild for Fedora 31 and the grub2 package build failed due releng not having proper credentials: https://koji.fedoraproject.org/koji/buildinfo?buildID=1326365 I did a manual build to make sure that the package is rebuilt as a part of the mass rebuild and this version (grub2-2.02-96.fc31) works correctly to me: $ rpm -qa | grep grub2 grub2-tools-extra-2.02-94.fc31.ppc64le grub2-ppc64le-modules-2.02-94.fc31.noarch grub2-tools-2.02-94.fc31.ppc64le grub2-common-2.02-94.fc31.noarch grub2-ppc64le-2.02-94.fc31.ppc64le $ grub2-install -v --no-nvram --no-floppy /dev/sda1 ... grub2-install: info: /dev/sda2 is present. grub2-install: info: Looking for /dev/sda2. grub2-install: info: /dev/sda is a parent of /dev/sda2. grub2-install: info: /dev/sda2 starts from 10240. grub2-install: info: opening the device hd0. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 0. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 0. ... grub2-install: error: cannot find a GRUB drive for /dev/sda2. Check your device.map. $ rpm -qa | grep grub2 grub2-tools-extra-2.02-96.fc31.ppc64le grub2-tools-minimal-2.02-96.fc31.ppc64le grub2-ppc64le-modules-2.02-96.fc31.noarch grub2-ppc64le-2.02-96.fc31.ppc64le grub2-common-2.02-96.fc31.noarch grub2-tools-2.02-96.fc31.ppc64le $ grub2-install -v --no-nvram --no-floppy /dev/sda1 ... grub2-install: info: /dev/sda1 is present. grub2-install: info: Looking for /dev/sda1. grub2-install: info: /dev/sda is a parent of /dev/sda1. grub2-install: info: /dev/sda1 starts from 2048. grub2-install: info: opening the device hd0. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 26214400. grub2-install: info: drive = 0. grub2-install: info: the size of hd0 is 26214400. ... Installation finished. No error reported. We'll keep an eye on openQA and see if it goes back to working in the next successful compose, then. Thanks! I did a manual test from Fedora-Rawhide-20190804.n.0 and things seems to work again from /tmp/program.log ... 11:10:56,677 INF program: Running... nvram --print-config=boot-device 11:10:56,810 INF program: 11:10:56,811 DBG program: Return code: 0 11:10:56,812 INF program: Running... ofpathname /dev/vda1 11:10:57,331 INF program: /pci@800000020000000/scsi@5 11:10:57,332 DBG program: Return code: 0 11:10:57,333 INF program: Running... nvram --update-config boot-device=/pci@800000020000000/scsi@5 11:10:57,443 DBG program: Return code: 0 11:10:57,444 INF program: Running in chroot '/mnt/sysroot'... grub2-install --no-nvram --no-floppy /dev/vda1 11:11:06,341 INF program: Installing for powerpc-ieee1275 platform. 11:11:06,342 INF program: Installation finished. No error reported. 11:11:06,342 DBG program: Return code: 0 11:11:06,431 INF program: Running in chroot '/mnt/sysroot'... grub2-set-default 948cfd0ae0ee4bd998f7be2f8a2dc22d-5.3.0-0.rc2.git4.1.fc31.ppc64le 11:11:06,619 DBG program: Return code: 0 11:11:06,622 INF program: Running in chroot '/mnt/sysroot'... grub2-mkconfig -o /boot/grub2/grub.cfg 11:11:13,871 INF program: Generating grub configuration file ... 11:11:13,873 INF program: Generating boot entries from BLS files... 11:11:13,874 INF program: done 11:11:13,874 DBG program: Return code: 0 11:11:21,892 INF program: Running in chroot '/mnt/sysroot'... depmod -a 5.3.0-0.rc2.git4.1.fc31.ppc64le 11:11:29,160 DBG program: Return code: 0 11:11:29,161 INF program: Running in chroot '/mnt/sysroot'... dracut -f /boot/initramfs-5.3.0-0.rc2.git4.1.fc31.ppc64le.img 5.3.0-0.rc2.git4.1.fc31.ppc64le 11:11:52,703 DBG program: Return code: 0 11:11:52,715 INF program: Running in chroot '/mnt/sysroot'... /bin/sh /tmp/ks-script-xfghxjx3 11:11:53,448 DBG program: Return code: 0 11:11:53,450 INF program: Running... /bin/sh /tmp/ks-script-axijc95t 11:11:53,858 DBG program: Return code: 0 11:11:53,861 INF program: Running... /bin/sh /tmp/ks-script-didv2f1_ 11:11:53,883 DBG program: Return code: 0 Also confirmed by openQA results with compose 20190802/04 https://openqa.stg.fedoraproject.org/group_overview/3 Then we can probably close this, thanks. |