Proposed as a Blocker for 35-beta by Fedora user pbrobinson using the blocker tracking app because:

 This stops various Raspberry Pi and probably some other arm hardware booting. We've explicitly kept the fat16 partition option, which is allowed as an option as part of the UEFI spec, because these devices don't detect the ef ID type even if it's the same format on disk.

Do we know when exactly this changed? Would help with pinning down what caused it.

(In reply to Adam Williamson from comment #2)
> Do we know when exactly this changed? Would help with pinning down what
> caused it.

I checked the nightlies as far back as I could, unfortunately the best I could work out was it happened sometime between 20210428 - 20210529.

Fedora-Workstation-Rawhide-20210428.n.1.armhfp.raw1 *       2048  1230847  1228800  600M  6 FAT16
Fedora-IoT-35-20210529.0.aarch64.raw1 *       2048 1028095 1026048  501M ef EFI (FAT-12/16/32)

Vojta, do you know about any changes that could be related to this issue?

This was actually intentional change in blivet 3.4.0[1], we are now using parted PARTITION_ESP flag for MBR EFI partitions to set the correct partition id (EF), see https://bugzilla.redhat.com/show_bug.cgi?id=1930486

I'm moving this bug blivet, we can simply revert the patch, if this is really causing the RPi to not boot.

Paul: Did you try to turn the ESP flag off on the EFI partition ("set 1 esp off" in parted)? Does it fix the boot issue?

Chris: You are the reporter of the original bug to change this, can you take a look at this bug? Do you have some examples of systems that need the ESP flag set to be able to boot?


[1] https://github.com/storaged-project/blivet/pull/933

Oops. Yeah I'm definitely the instigator, but it spreading to pi images is not intended.

In bug 1930486, that is a UEFI x86_64 VM, with an existing MBR disk. And the installer creates an ESP (it has an EFI/ dir, and the assorted bootloaders) with type code 0x06. It's correct that it should be 0xEF.

And in the case of non-UEFI aarch64 images, the partition in question is not an ESP so it shouldn't get type code 0xEF. I guess ideally there'd be a specific kickstart type, ARMBOOTLDR or whatever, so that  the two bootloader volumes can be distinguished, and created by blivet correctly.

However, distinguishing between these two cases is probably sufficiently rare that the easiest thing to do in the near term is just revert PR 933. Another alternative is to not support UEFI on MBR, and goto fail. Just because the UEFI spec says you can do it, doesn't obligate anyone to support that scenario.

(In reply to Vojtech Trefny from comment #5)
> This was actually intentional change in blivet 3.4.0[1], we are now using
> parted PARTITION_ESP flag for MBR EFI partitions to set the correct
> partition id (EF), see https://bugzilla.redhat.com/show_bug.cgi?id=1930486
> 
> I'm moving this bug blivet, we can simply revert the patch, if this is
> really causing the RPi to not boot.
> 
> Paul: Did you try to turn the ESP flag off on the EFI partition ("set 1 esp
> off" in parted)? Does it fix the boot issue?

Unfortunately not, after running parted:

Device     Boot   Start      End  Sectors  Size Id Type
/dev/sdb1  *       2048  1230847  1228800  600M  b W95 FAT32



(In reply to Chris Murphy from comment #6)

> And in the case of non-UEFI aarch64 images, the partition in question is not

The aarch64 and arm images are UEFI. The change to 32-bit arm was made in F34 - https://fedoraproject.org/wiki/Changes/ARMv7UEFI

Ping - Can we revert this or somehow make it configurable for both use cases? This significantly impacts Rawhide testing for both armhfp and aarch64.

Are the failing Raspberry Pi's UEFI? UEFI spec says 0xEF is the correct type code for an EFI system partition on MBR partition map. Wikipedia says 0x06 is Compaq FAT16. If you change the 1st partition to type code 0x06 does that fix things? I'm uncertain I understand what's actually causing the problem. I take it this one image is intended to boot UEFI and non-UEFI ARM devices, hence MBR instead of GPT? Because if they're all UEFI devices, then maybe part of the confusion is that this is still using MBR scheme instead of GPT?

For what it's worth:

>doc/ChangeLog.dosfstools-2.x:12: - mkdosfs: by default, use FAT32 on devices >= 512MB

The images have FAT32, regardless of what the partition type code is. So if the problem is due to some boards not liking FAT32, and you need FAT16, then (a) the partition size needs to be reduced below 512M, not sure if that's really in SI, or IEC units; (b) use pykickstart 'part --mkfsoptions "-F 16"' to force FAT16.

Downloaded Fedora-Minimal-34-1.2.aarch64.raw and put it on a loop device, partition 1 type 0x06 according to dosfsck is FAT32. And likely have been FAT32 ever since the EFI System partition size was bumped to 600M.

Seems relevant.

Recognize efi partition (0xef) as a valid boot
https://github.com/raspberrypi/rpi-eeprom/issues/126

(In reply to Paul Whalen from comment #7)
> (In reply to Vojtech Trefny from comment #5)
> > This was actually intentional change in blivet 3.4.0[1], we are now using
> > parted PARTITION_ESP flag for MBR EFI partitions to set the correct
> > partition id (EF), see https://bugzilla.redhat.com/show_bug.cgi?id=1930486
> > 
> > I'm moving this bug blivet, we can simply revert the patch, if this is
> > really causing the RPi to not boot.
> > 
> > Paul: Did you try to turn the ESP flag off on the EFI partition ("set 1 esp
> > off" in parted)? Does it fix the boot issue?
> 
> Unfortunately not, after running parted:
> 
> Device     Boot   Start      End  Sectors  Size Id Type
> /dev/sdb1  *       2048  1230847  1228800  600M  b W95 FAT32
> 

So it still doesn't boot after this change? If not, than the partition type is not a problem, but the FAT32 probably is and we should start forcing FAT16 for EFI partitions?

(In reply to Chris Murphy from comment #11)
> Downloaded Fedora-Minimal-34-1.2.aarch64.raw and put it on a loop device,
> partition 1 type 0x06 according to dosfsck is FAT32. And likely have been
> FAT32 ever since the EFI System partition size was bumped to 600M.

Thanks Chris, confirmed that parted shows the partition as FAT32 in Fedora 34, but the partition ID is 06 so it works as expected (and fdisk thinks its FAT16)

I changed the Rawhide disk image partition ID 06 in fdisk and it now boots as well.

Before:
/dev/sdb1  *       2048  1230847  1228800  600M ef EFI (FAT-12/16/32)

After:
/dev/sdb1  *       2048  1230847  1228800  600M  6 FAT16