So, hum. We're basically poking about here:

https://github.com/rhboot/shim/blob/1ff4a36a23ac5c17144275ccb3e1e1061750a137/fallback.c#L1003

One thing I notice is that we hit this part:

console_print(L"Reset System\n");

and that does this:

gRT->ResetSystem(EfiResetCold, EFI_SUCCESS, 0, NULL);

note that's a *cold* reset. Exactly what that means for edk2-aarch64 I'm having fun trying to figure out, but it seems at least plausible that it does something similar to power off / power on. That's awkward for the scenario we're dealing with here: a VM with no persistent storage for EFI variables, which is expecting to boot successfully from the fallback path. I can see how it *could* lead to a loop like this - the fallback path boots, updates BootOrder, does a cold reset, BootOrder is gone again so we hit the fallback path, repeat ad infinitum...

If ever try_start_first_option() is hit and works, it'd break the loop, but clearly that's *not* happening in this case. I'm not sure if we're detecting the presence of a TPM and thus not even trying it, or trying it but it's failing (we'd need verbose messages to figure that out).

Here's a more detailed log extract:

[Bds]Booting UEFI Misc Device 3
 BlockSize : 512 
 LastBlock : 13FFFFF 
 BlockSize : 512 
 LastBlock : 1FFFFF 
 BlockSize : 512 
 LastBlock : 119B7FF 
FSOpen: Open '\EFI\BOOT\BOOTAA64.EFI' Success
[Bds] DevicePath expand: PciRoot(0x0)/Pci(0x5,0x0) -> PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x088DD077,0x800,0x64000)/\EFI\BOOT\BOOTAA64.EFI
[Security] 3rd party image[0] can be loaded after EndOfDxe: PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x088DD077,0x800,0x64000)/\EFI\BOOT\BOOTAA64.EFI.
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B FABC9040
Loading driver at 0x000F8501000 EntryPoint=0x000F8501120
Loading driver at 0x000F8501000 EntryPoint=0x000F8501120 
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF FABC9E98
FSOpen: Open '\EFI\BOOT\fbaa64.efi' Success
FSOpen: Open '\EFI\BOOT\fbaa64.efi' Success
[0m[37m[40mSystem BootOrder not found.  Initializing defaults.
FSOpen: Open 'EFI' Success
FSOpen: Open 'fedora' Success
FSOpen: Open 'BOOTAA64.CSV' Success
FSOpen: Open '\EFI\fedora\BOOTAA64.CSV' Success

Reset System

(then the whole thing happens again, over and over, till the test gives up and dies).

For now I'm gonna see if I can possibly tweak the openQA bits such that we run these tests *with* persistent storage for EFI vars, at least across the lifetime of each test.

Something else I forgot last night: this works on F28. It only fails on Rawhide.

Here's the logs from an F28 test:

[Bds]Booting UEFI Misc Device 3
 BlockSize : 512 
 LastBlock : 13FFFFF 
 BlockSize : 512 
 LastBlock : 1FFFFF 
 BlockSize : 512 
 LastBlock : 119B7FF 
FSOpen: Open '\EFI\BOOT\BOOTAA64.EFI' Success
[Bds] DevicePath expand: PciRoot(0x0)/Pci(0x5,0x0) -> PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x69E22731,0x800,0x64000)/\EFI\BOOT\BOOTAA64.EFI
[Security] 3rd party image[0] can be loaded after EndOfDxe: PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x69E22731,0x800,0x64000)/\EFI\BOOT\BOOTAA64.EFI.
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B FABC9040
Loading driver at 0x000F84FB000 EntryPoint=0x000F84FB120
Loading driver at 0x000F84FB000 EntryPoint=0x000F84FB120 
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF FABC9E98
FSOpen: Open '\EFI\BOOT\fbaa64.efi' Success
FSOpen: Open '\EFI\BOOT\fbaa64.efi' Success
[0m[37m[40mSystem BootOrder not found.  Initializing defaults.
FSOpen: Open 'EFI' Success
FSOpen: Open 'fedora' Success
FSOpen: Open 'BOOTAA64.CSV' Success
FSOpen: Open '\EFI\fedora\BOOTAA64.CSV' Success
Creating boot entry "Boot0001" with label "Fedora" for file "\EFI\fedora\shimaa64.efi"

FSOpen: Open '\EFI\fedora\shimaa64.efi' Success
[Security] 3rd party image[0] can be loaded after EndOfDxe: PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x69E22731,0x800,0x64000)/\EFI\fedora\shimaa64.efi.
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B F9260040
Loading driver at 0x000F8422000 EntryPoint=0x000F8422120
Loading driver at 0x000F8422000 EntryPoint=0x000F8422120 
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF F9260E98
FSOpen: Open '\EFI\fedora\grubaa64.efi' Success
[0m[30m[40m[2J[01;01H[0m[37m[40m[21;07HUse the ^ and v keys to change the selection.  

the difference is obvious: after we get to "System BootOrder not found.  Initializing defaults.", we actually log creating a new entry, and then immediately boot it (without the "Reset System" happening, so presumably `try_start_first_option()` ran and worked). For some reason, that's working in F28 but failing in Rawhide.

F28 has shim-aa64-13-4.aarch64 , Rawhide has shim-aa64-13-5.aarch64 . The former I think comes from https://koji.fedoraproject.org/koji/buildinfo?buildID=1054631 - shim-signed-13-4 - the latter from https://koji.fedoraproject.org/koji/buildinfo?buildID=1054372 - shim-13-5 . Both those seem to use the same shimaa64.efi source file (the SHA512 is the same in the `sources` file in both builds), but the BOOTAA64.EFI file winds up being a slightly different size between the two, not sure why.

pwhalen actually noted what looks like the cause of this, which I missed: in shim-13-5 , BOOTAA64.CSV and BOOTIA32.CSV and BOOTX64.CSV are all 0-length. In shim-signed-13-4 they are not. This can also be seen in the package git repos, of course ('master' branch of shim has zero-length files, 'f28' branch of shim-signed does not). Looks like an easy fix to just put the right files back in the shim package build, but of course only a few people can actually build it, so we need one of them to do it.

System BootOrder not found.  Initializing defaults.
FSOpen: Open 'EFI' Success
FSOpen: Open 'fedora' Success
FSOpen: Open 'BOOTAA64.CSV' Success
FSOpen: Open '\EFI\fedora\BOOTAA64.CSV' Success
Creating boot entry "Boot0001" with label "Fedora" for file "\EFI\fedora\shimaa64.efi"

FSOpen: Open '\EFI\fedora\shimaa64.efi' Success


shim-signed-15-2 looks to fix the issue, could we get this tagged into f29?

Wrong package- shim-15-3 also fixes this and is indeed tagged into f29. Thanks pjones!