Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1705927 - After migration Fedora 29 to Fedora 30 the reboot gets stuck at grub> prompt - no menu entries
Summary: After migration Fedora 29 to Fedora 30 the reboot gets stuck at grub> prompt ...
Keywords:
Status: CLOSED DUPLICATE of bug 1652806
Alias: None
Product: Fedora
Classification: Fedora
Component: grub2
Version: 30
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ---
Assignee: Peter Jones
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-05-03 08:44 UTC by H.-P. Sorge
Modified: 2019-10-15 07:37 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-15 07:37:19 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description H.-P. Sorge 2019-05-03 08:44:59 UTC
Description of problem:
- prepare upgrade 
- dnf system-upgrade download --refresh --releasever=30
- ...
- dnf system-upgrade reboot
- no grub menu entries are being listed.


Version-Release number of selected component (if applicable):
grubby version 8.28


How reproducible:
dnf upgrade in fedora 30

Steps to Reproduce:
1. dnf upgrade
2. less  /boot/grub2/grub.cfg -> latest kernel menu entry not listed

Actual results:
After upgrade 29->30 a System restart stops at grub> prompt. 
The grub menu is not being shown. 

Any further dnf updates fail to add the latest kernel in grub menu. 


Expected results:
New kernel version in grub menu.


Additional info:
Upgrade 29->30:
The /boot/grub2/grub.cfg.rpmsave was gcreated.
The new /boot/grub2/grub.cfg has NO menu entries.
Restoring grub.cfg from grub.cfg.rpmsave and adding the latest kernel menu entry manually fixed this case. 

Subsequent dnf update:
dnf update does not add a new entry nor does it remove the oldest kernelentry.

Comment 1 H.-P. Sorge 2019-05-03 08:50:46 UTC
... just a remark:

"Restoring grub.cfg from grub.cfg.rpmsave and adding the latest kernel menu entry manually fixed this case."

was done via a FC30 USB rescue image.

Comment 2 David Koppelman 2019-05-05 15:05:51 UTC
I've encountered the same problem, grub.cfg without menu entries, and had boot success after applying the workaround from comment 1. I had to re-edit grub.cfg after the 5.0.11 upgrade or else it would boot to 5.0.10.

This is on a i386 system. The upgrade was initiated on the command line.

Comment 3 Bruno Larini 2019-05-07 21:20:10 UTC
Same problem here, exactly as told by David.

Comment 4 Bruno Larini 2019-05-07 21:27:18 UTC
(In reply to Bruno Larini from comment #3)
> Same problem here, exactly as told by David.

I should add, even after booting on the restored grub.cfg.rpmsave and running "grub2-mkconfig -o /boot/grub2/grub.cfg" no menuentry was added to the file.
For those stuck on the "grub>" prompt, use "configfile /boot/grub2/grub.cfg.rpmsave" to boot.

Comment 5 Martin Tessun 2019-05-11 16:53:01 UTC
Looking through the grub2-scripts, looks like they need /sbin/new-kernel-pkg, which is only available in grubby-deprecated.

So I installed grubby-deprecated and run the installkernel again, e.g.:
sudo /bin/kernel-install -v add 5.0.13-300.fc30.x86_64 /lib/modules/5.0.13-300.fc30.x86_64/vmlinuz

and grub2 was correctly updated.

So at least a workaround is to install grubby-deprecated for now.

Comment 6 Martin Tessun 2019-05-11 16:57:55 UTC
(In reply to Martin Tessun from comment #5)
> Looking through the grub2-scripts, looks like they need
> /sbin/new-kernel-pkg, which is only available in grubby-deprecated.
> 
> So I installed grubby-deprecated and run the installkernel again, e.g.:
> sudo /bin/kernel-install -v add 5.0.13-300.fc30.x86_64
> /lib/modules/5.0.13-300.fc30.x86_64/vmlinuz
> 
> and grub2 was correctly updated.
> 
> So at least a workaround is to install grubby-deprecated for now.

Maybe linking /sbin/installkernel (from package grubby) to new-kernel-pkg might help as well, but I did not test it.

Comment 7 Javier Martinez Canillas 2019-05-24 11:30:50 UTC
This very likely is a duplicate of Bug #1652806. Take a look to https://fedoraproject.org/wiki/Common_F30_bugs#GRUB_boot_menu_is_not_populated_after_an_upgrade.

Comment 8 Martin Tessun 2019-06-01 12:34:12 UTC
(In reply to Javier Martinez Canillas from comment #7)
> This very likely is a duplicate of Bug #1652806. Take a look to
> https://fedoraproject.org/wiki/
> Common_F30_bugs#GRUB_boot_menu_is_not_populated_after_an_upgrade.

No. No grub menu entries are created in case you do not install grubby-deprecated, as /sbin/installkernel is missing for the grub2-scripts.
With that not a single entry is added.

The F30 bug you mention does not apply as I still have a grub.cfg, just the newer kernels are missing. So I was able to boot my old F29 kernel, but the newer kernels didn't get added (postinstall invokes /bin/kernel-install, which finally invokes /sbin/installkernel, which is not installed unless grubby-deprecated is installed.

Once grubby-deprecated is installed, kernels are again added to grub.cfg as usual. In addition this is a UEFI system and always had grub2, never grub.

[tessun@kirk ~]$ cat /sbin/installkernel 
#!/bin/bash
if [[ -x /usr/libexec/installkernel/installkernel ]] ; then
	exec /usr/libexec/installkernel/installkernel "${@}"
elif [[ -x /usr/libexec/installkernel/installkernel-bls ]] ; then
	exec /usr/libexec/installkernel/installkernel-bls "${@}"
fi
echo "installkernel is not installed correctly." >>/dev/stderr
exit 1
[tessun@kirk ~]$ rpm -q -f /usr/libexec/installkernel/installkernel
grubby-deprecated-8.40-30.fc30.x86_64
[tessun@kirk ~]$ 


Or without grubby-deprecated installed:
[root@kirk ~]# rpm -e --nodeps  grubby-deprecated grubby 
[root@kirk ~]# dnf install grubby
Last metadata expiration check: 0:01:44 ago on Sat 01 Jun 2019 14:30:42 CEST.
Dependencies resolved.
================================================================================================================================================================
 Package                             Architecture                        Version                                      Repository                           Size
================================================================================================================================================================
Installing:
 grubby                              x86_64                              8.40-30.fc30                                 fedora                               36 k

Transaction Summary
================================================================================================================================================================
Install  1 Package

Total download size: 36 k
Installed size: 64 k
Is this ok [y/N]: y
Downloading Packages:
grubby-8.40-30.fc30.x86_64.rpm                                                                                                  608 kB/s |  36 kB     00:00    
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                           281 kB/s |  36 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                        1/1 
  Installing       : grubby-8.40-30.fc30.x86_64                                                                                                             1/1 
  Running scriptlet: grubby-8.40-30.fc30.x86_64                                                                                                             1/1 
  Verifying        : grubby-8.40-30.fc30.x86_64                                                                                                             1/1 

Installed:
  grubby-8.40-30.fc30.x86_64                                                                                                                                    

Complete!
[root@kirk ~]# cat /sbin/installkernel 
#!/bin/bash
if [[ -x /usr/libexec/installkernel/installkernel ]] ; then
	exec /usr/libexec/installkernel/installkernel "${@}"
elif [[ -x /usr/libexec/installkernel/installkernel-bls ]] ; then
	exec /usr/libexec/installkernel/installkernel-bls "${@}"
fi
echo "installkernel is not installed correctly." >>/dev/stderr
exit 1
[root@kirk ~]# rpm -q -f /usr/libexec/installkernel/installkernel
error: file /usr/libexec/installkernel/installkernel: No such file or directory
[root@kirk ~]#

Comment 9 Martin Tessun 2019-06-01 12:51:05 UTC
In addition grub2-mkconfig never creates any linux menuentries. It always looks empty like this:


### BEGIN /etc/grub.d/10_linux ###
insmod part_gpt
insmod ext2
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  eabe445d-7d21-47d8-9887-86f1116534d7
else
  search --no-floppy --fs-uuid --set=root eabe445d-7d21-47d8-9887-86f1116534d7
fi
insmod part_gpt
insmod fat
set boot='hd0,gpt1'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=boot --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1  FA14-5091
else
  search --no-floppy --fs-uuid --set=boot FA14-5091
fi

# This section was generated by a script. Do not modify the generated file - all changes
# will be lost the next time file is regenerated. Instead edit the BootLoaderSpec files.
#
# The blscfg command parses the BootLoaderSpec files stored in /boot/loader/entries and
# populates the boot menu. Please refer to the Boot Loader Specification documentation
# for the files format: https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/.

set default_kernelopts="root=/dev/mapper/fedora_kirk-root ro rd.driver.blacklist=nouveau rd.lvm.lv=fedora_kirk/root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=fedora_kirk/swap rhgb quiet  "

insmod blscfg
blscfg
### END /etc/grub.d/10_linux ###


===========================================

[root@kirk ~]# grub2-mkconfig | grep menuen
Generating grub configuration file ...
if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
  menuentry_id_option=""
export menuentry_id_option
Found theme: /boot/grub2/themes/breeze/theme.txt
Found Windows Boot Manager on /dev/sda1@/EFI/Microsoft/Boot/bootmgfw.efi
menuentry 'Windows Boot Manager (on /dev/sda1)' --class windows --class os $menuentry_id_option 'osprober-efi-FA14-5091' {
Adding boot menu entry for EFI firmware configuration
menuentry 'System setup' $menuentry_id_option 'uefi-firmware' {
done
[root@kirk ~]# 



That said the workaround with kernel-install from grubby is nice, but a new creation of a grub.cfg file results in no linux entries at all.

So from my PoV there are two bugs:
- grubby needds binaries from grubby-deprecated
- grub2 kernel-install scripts from grub2-mkconfig don't work at all.

Comment 10 Javier Martinez Canillas 2019-06-03 07:24:04 UTC
(In reply to Martin Tessun from comment #8)
> (In reply to Javier Martinez Canillas from comment #7)
> > This very likely is a duplicate of Bug #1652806. Take a look to
> > https://fedoraproject.org/wiki/
> > Common_F30_bugs#GRUB_boot_menu_is_not_populated_after_an_upgrade.
> 
> No. No grub menu entries are created in case you do not install

I was answering to the original reporter, who mentions a different issue than you.

> grubby-deprecated, as /sbin/installkernel is missing for the grub2-scripts.
> With that not a single entry is added.
> 

It's installed by the grubby package as well.

$ rpm -qf /usr/sbin/installkernel
grubby-8.40-30.fc30.x86_64

> The F30 bug you mention does not apply as I still have a grub.cfg, just the

I didn't say that the known bug applied to you.

> newer kernels are missing. So I was able to boot my old F29 kernel, but the
> newer kernels didn't get added (postinstall invokes /bin/kernel-install,
> which finally invokes /sbin/installkernel, which is not installed unless

That's not correct, kernel-install invokes new-kernel-pkg if GRUB_ENABLE_BLSCFG isn't set to true in /etc/default/grub. But if GRUB_ENABLE_BLSCFG=true it just needs the kernel-install plugins.

> grubby-deprecated is installed.
> 
> Once grubby-deprecated is installed, kernels are again added to grub.cfg as
> usual. In addition this is a UEFI system and always had grub2, never grub.
>

I think the problem is that you don't have GRUB_ENABLE_BLSCFG=true in /etc/default/grub.
 
> [tessun@kirk ~]$ cat /sbin/installkernel 
> #!/bin/bash
> if [[ -x /usr/libexec/installkernel/installkernel ]] ; then
> 	exec /usr/libexec/installkernel/installkernel "${@}"
> elif [[ -x /usr/libexec/installkernel/installkernel-bls ]] ; then
> 	exec /usr/libexec/installkernel/installkernel-bls "${@}"
> fi
> echo "installkernel is not installed correctly." >>/dev/stderr
> exit 1
> [tessun@kirk ~]$ rpm -q -f /usr/libexec/installkernel/installkernel
> grubby-deprecated-8.40-30.fc30.x86_64
> [tessun@kirk ~]$ 
> 
> 
> Or without grubby-deprecated installed:
> [root@kirk ~]# rpm -e --nodeps  grubby-deprecated grubby 
> [root@kirk ~]# dnf install grubby
> Last metadata expiration check: 0:01:44 ago on Sat 01 Jun 2019 14:30:42 CEST.
> Dependencies resolved.
> =============================================================================
> =============================================================================
> ======
>  Package                             Architecture                       
> Version                                      Repository                     
> Size
> =============================================================================
> =============================================================================
> ======
> Installing:
>  grubby                              x86_64                             
> 8.40-30.fc30                                 fedora                         
> 36 k
> 
> Transaction Summary
> =============================================================================
> =============================================================================
> ======
> Install  1 Package
> 
> Total download size: 36 k
> Installed size: 64 k
> Is this ok [y/N]: y
> Downloading Packages:
> grubby-8.40-30.fc30.x86_64.rpm                                              
> 608 kB/s |  36 kB     00:00    
> -----------------------------------------------------------------------------
> -----------------------------------------------------------------------------
> ------
> Total                                                                       
> 281 kB/s |  36 kB     00:00     
> Running transaction check
> Transaction check succeeded.
> Running transaction test
> Transaction test succeeded.
> Running transaction
>   Preparing        :                                                        
> 1/1 
>   Installing       : grubby-8.40-30.fc30.x86_64                             
> 1/1 
>   Running scriptlet: grubby-8.40-30.fc30.x86_64                             
> 1/1 
>   Verifying        : grubby-8.40-30.fc30.x86_64                             
> 1/1 
> 
> Installed:
>   grubby-8.40-30.fc30.x86_64                                                
> 
> 
> Complete!
> [root@kirk ~]# cat /sbin/installkernel 
> #!/bin/bash
> if [[ -x /usr/libexec/installkernel/installkernel ]] ; then
> 	exec /usr/libexec/installkernel/installkernel "${@}"
> elif [[ -x /usr/libexec/installkernel/installkernel-bls ]] ; then
> 	exec /usr/libexec/installkernel/installkernel-bls "${@}"
> fi
> echo "installkernel is not installed correctly." >>/dev/stderr
> exit 1
> [root@kirk ~]# rpm -q -f /usr/libexec/installkernel/installkernel
> error: file /usr/libexec/installkernel/installkernel: No such file or
> directory
> [root@kirk ~]#

Yes, because /usr/libexec/installkernel/installkernel isn't used but instead /usr/libexec/installkernel/installkernel-bls:

rpm -qf /usr/libexec/installkernel/installkernel-bls
grubby-8.40-30.fc30.x86_64

Comment 11 Javier Martinez Canillas 2019-06-03 07:28:42 UTC
(In reply to Martin Tessun from comment #9)
> In addition grub2-mkconfig never creates any linux menuentries. It always

That's expected:

https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault

Comment 12 Martin Tessun 2019-06-03 15:02:56 UTC
(In reply to Javier Martinez Canillas from comment #11)
> (In reply to Martin Tessun from comment #9)
> > In addition grub2-mkconfig never creates any linux menuentries. It always
> 
> That's expected:
> 
> https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault

So running "grub2-switch-to-blscfg" did not change my grub.cfg

So I created it newly:
# cp /boot/efi/EFI/fedora/grub.cfg /boot/efi/EFI/fedora/grub.cfg_20190603
# grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

With my "backup" in place I did the reboot and it showed up all the kernels.

So what I believe is going wrong during the update is that the grub.cfg is not adjusted accordingly.
I can add both grub.cfg (the one from F29-F30 upgrade and the one newly created), but at least for me the blscfg was never applied during upgrade - as grubby-deprecated is also not installed, new kernel entries do not show up (while the old ones still got deleted).

So in case you had just one F29 kernel you would not be able to boot into Linux, as grub.cfg does not know any kernel after the upgrade. In my case you could still see the F29 kernels, as I keep 3 kernels installed, so I ended up with just two F29 kernels after the upgrade.
With the blscdg, I have all my kernels back without fiddling around with the grub.cfg anymore, which really is convenient.

That said, I believe the bug is more that grub.cfg isn't updated correctly for bls during the F29-F30 upgrade.

Comment 13 Martin Tessun 2019-06-03 15:13:26 UTC
(In reply to Javier Martinez Canillas from comment #10)
> 
> I think the problem is that you don't have GRUB_ENABLE_BLSCFG=true in
> /etc/default/grub.

Well not I did it, but grub2-tools posttrans :)

$ rpm -q --scripts grub2-tools
preinstall scriptlet (using /bin/sh):
if [ -f /boot/grub2/user.cfg ]; then
    if grep -q '^GRUB_PASSWORD=' /boot/grub2/user.cfg ; then
	sed -i 's/^GRUB_PASSWORD=/GRUB2_PASSWORD=/' /boot/grub2/user.cfg
    fi
elif [ -f /boot/efi/EFI/fedora/user.cfg ]; then
    if grep -q '^GRUB_PASSWORD=' /boot/efi/EFI/fedora/user.cfg ; then
	sed -i 's/^GRUB_PASSWORD=/GRUB2_PASSWORD=/' \
	    /boot/efi/EFI/fedora/user.cfg
    fi
elif [ -f /etc/grub.d/01_users ] && \
	grep -q '^password_pbkdf2 root' /etc/grub.d/01_users ; then
    if [ -f /boot/efi/EFI/fedora/grub.cfg ]; then
	# on EFI we don't get permissions on the file, but
	# the directory is protected.
	grep '^password_pbkdf2 root' /etc/grub.d/01_users | \
		sed 's/^password_pbkdf2 root \(.*\)$/GRUB2_PASSWORD=\1/' \
	    > /boot/efi/EFI/fedora/user.cfg
    fi
    if [ -f /boot/grub2/grub.cfg ]; then
	install -m 0600 /dev/null /boot/grub2/user.cfg
	chmod 0600 /boot/grub2/user.cfg
	grep '^password_pbkdf2 root' /etc/grub.d/01_users | \
		sed 's/^password_pbkdf2 root \(.*\)$/GRUB2_PASSWORD=\1/' \
	    > /boot/grub2/user.cfg
    fi
fi
posttrans scriptlet (using /bin/sh):

if [ -f /etc/default/grub ]; then
    ! grep -q '^GRUB_ENABLE_BLSCFG=.*' /etc/default/grub && \
      /sbin/grub2-switch-to-blscfg --backup-suffix=.rpmsave &>/dev/null || :
fi
$ 


Anyways the grub.cfg wasn't updated accordingly as stated in my previous comment which lead to no F30 kernels showing up to boot.
So that was the real culprit in the end then?

Comment 14 Javier Martinez Canillas 2019-06-03 15:29:32 UTC
(In reply to Martin Tessun from comment #12)
> (In reply to Javier Martinez Canillas from comment #11)
> > (In reply to Martin Tessun from comment #9)
> > > In addition grub2-mkconfig never creates any linux menuentries. It always
> > 
> > That's expected:
> > 
> > https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault
> 
> So running "grub2-switch-to-blscfg" did not change my grub.cfg
>

I wonder what could go wrong there. I just tested on an EFI and legacy BIOS install and it worked correctly for me, on system upgrade the grub.cfg was re-generated with a BLS configuration.

Comment 15 Martin Tessun 2019-06-04 07:11:29 UTC
(In reply to Javier Martinez Canillas from comment #14)
> (In reply to Martin Tessun from comment #12)
> > (In reply to Javier Martinez Canillas from comment #11)
> > > (In reply to Martin Tessun from comment #9)
> > > > In addition grub2-mkconfig never creates any linux menuentries. It always
> > > 
> > > That's expected:
> > > 
> > > https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault
> > 
> > So running "grub2-switch-to-blscfg" did not change my grub.cfg
> >
> 
> I wonder what could go wrong there. I just tested on an EFI and legacy BIOS
> install and it worked correctly for me, on system upgrade the grub.cfg was
> re-generated with a BLS configuration.

No idea either. I just updated my Laptop now as well from F29 to F30 and the bls config was applied as expected including the upgrade of grub.cfg

Comment 16 Jesper Brouer 2019-06-04 15:27:41 UTC
(In reply to Martin Tessun from comment #5)
> 
> So at least a workaround is to install grubby-deprecated for now.

For this workaround to work for me, I also had to edit /etc/default/grub and set:

 GRUB_ENABLE_BLSCFG=false

After changing this regenerating /boot/grub2/grub.cfg works as expected:

 sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Comment 17 Javier Martinez Canillas 2019-10-15 07:37:19 UTC

*** This bug has been marked as a duplicate of bug 1652806 ***


Note You need to log in before you can comment on or make changes to this bug.