Bug 1890085

Well. There's a lot of detail involved in keyboard layouts.

The description here has some odd ideas. How do you expect a keyboard layout specified in "the kernel boot options" to take effect "when editing the command line of the boot loader"? The boot loader comes before the kernel, not after. :)

The layout used when entering an encryption passphrase is decided by the initramfs environment, produced by dracut. It's demonstrably not always English:

https://openqa.fedoraproject.org/tests/703303#step/disk_guided_encrypted_postinstall/1

that's an openQA test of the most recent Fedora 33 install, doing an encrypted install in Russian. As you can see, when booting the installed system, the layout used for entering the passphrase is ru, not en.

Ditto Japanese: https://openqa.fedoraproject.org/tests/703300#step/disk_guided_encrypted_postinstall/1
and French: https://openqa.fedoraproject.org/tests/703325#step/disk_guided_encrypted_postinstall/1

there may be a Silverblue-specific issue here; we don't run the non-English language encrypted install tests on the Silverblue installer image (though perhaps we could). Can you test with a non-Silverblue installer image and see if you see the same behaviour? Thanks.

(In reply to Adam Williamson from comment #1)
To be more specific, I have now compared Workstation and Silverblue install images of the Fedora 33 development branch labeled 20201021.n.0 by setting up respective systems.
After keeping English as the system language but changing the default and only keyboard layout from English to German in the graphical installer in both cases and performing the installation, I have booted both systems in text mode after removing "rhgb quiet" from the boot options.
1. The Workstation system behaves correctly by unlocking the disk after entering the LUKS password exactly as entered in the graphical installer. When booting the system in graphical mode, then the LUKS unlock screen indicates the keyboard layout correctly as German (DE).
2. The Silverblue system, however, requires the LUKS password to be entered assuming an English layout in order to unlock the disk. Accordingly, the graphical unlock screen does not indicate the German keyboard layout. After completing the boot procedure, both systems apply the German keyboard layout to both of virtual consoles and graphical login screen.
I have to correct myself insofar as the kernel boot options of both systems indeed do not contain any kind of keyboard or language setting. If I remember correctly this used to be the case some time in the past between 1997 and 2020 but is no more the case nowadays.

OK, so basically this issue is specific to Silverblue. Or, probably, to ostree-based installs.

I have a hunch as to what's going on, I'll see later today if I can reproduce this and if it's broken the way I think it is.

OK, so I can confirm the bug here indeed: I did an install from the F33 RC2 (==Final) Silverblue installer image, in French, with encryption enabled, and the installer used fr layout when creating the passphrase, but the installed system used en layout when decrypting.

The installed system itself has a correct /etc/vconsole.conf . The initramfs environment - as inspected by booting with rd.break and looking around - does not. I'm not sure if this is the issue yet and if so why it doesn't have one.

OK, so I ran an equivalent install from the Server DVD (French, encryption enabled) and in that install, the initramfs environment has a /etc/vconsole.conf . So that's the difference. The question, I guess, is why.

I ran 'dracut -f -v --debug' on both installs, grepped the output for 'vconsole', and compared the results; they're identical. That's odd.

F33 final is signed off, so there's no point to FE status any more.

Unfortunately, this is intentional as far as Anaconda's handling, see https://github.com/rhinstaller/anaconda/blob/master/pyanaconda/modules/storage/bootloader/installation.py#L187

Reassigning - hope that's the right component?

well, the ostree package contains specific ostree tooling and isn't really an "ostree-based distributions" virtual component, but it got assigned to Colin so it's probably fine. :P

I think this is a dupe of https://github.com/fedora-silverblue/issue-tracker/issues/3, which is now addressable by https://github.com/coreos/rpm-ostree/pull/2170 though it's still in experimental mode. Having it work seamlessly though will require some work on the Anaconda side to have it use the new API.

Yes, that does look like the same issue indeed.

Jonathan, if you think the API will be good enough for using in f34, feel free to switch the bug back to anaconda.

Is now a good time to move the bug or did anything happen for F34?

https://github.com/fedora-silverblue/issue-tracker/issues/3#issuecomment-822689910 > Let's redirect this one to Anaconda

Assuming I want to fix this, I want to do... what? Make Anaconda run this in chroot?

rpm-ostree ex initramfs-etc /etc/vconsole.conf

Or this?

rpm-ostree initramfs --enable
(and then something else since that changes the default?)

Wizards, please advise.



---

Notes to self or anyone trying to implement this. There are two alternatives:

a) Modify RecreateInitrdsTask.run() and plug in a new method depending on payload type. Not sure where exactly would the new method go.

b) Create a new ostree-specific Task, add it to the list in generate_initramfs_with_tasks() and make its run() conditional on payload type.

Not sure which is better.

(In reply to Vladimír Slávik from comment #15)
> Assuming I want to fix this, I want to do... what? Make Anaconda run this in
> chroot?
> 
> rpm-ostree ex initramfs-etc /etc/vconsole.conf

Yes exactly, but rpm-ostree is a client and daemon which talk over D-Bus, so getting it set up will get messy. :) (We have a "peer" mode though that path has kinda fallen into disuse.)

It might be easier to actually not run it in a chroot and just use the rpm-ostree binary and system D-Bus already in the Anaconda environment (obviously, this introduces a version mismatch, though I don't think it will be hard for us to ensure config compatibility for this feature as long as the skew isn't massive). You'd start the daemon using something like `rpm-ostree start-daemon --sysroot /sysroot` and then you *should* be able to just use `rpm-ostree ex initramfs-etc`.

The other thing is that the command is still marked as experimental, which isn't ideal before integrating it into Anaconda. I'll look at promoting it to `rpm-ostree initramfs-etc`, but it shouldn't block hacking on this meanwhile.

I tried to manually run the commands mentioned above, in the installation environment, once Rawhide Silverblue installation is finished.

- rpm-ostree start-daemon --sysroot /mns/sysroot -> ran for a bit then exited, journal has only one message that it's idle and will exit
- rpm-ostree ex initramfs-etc --track /etc/vconsole.conf -> error about system not booted via libostree
- rpm-ostree initramfs --enable -> also error

The error is the same with no daemon running, so I guess the client doesn't see it.

Then I rebooted into the new system and tried to run these again.

- rpm-ostree ex initramfs-etc --track /etc/vconsole.conf (also vconsole.conf) -> worked
- rpm-ostree initramfs --enable -> worked

Related bug about config file missing in initramfs - https://bugzilla.redhat.com/show_bug.cgi?id=1801539#c2 - fstrim not working through luks. Can be fixed the same way.

(In reply to Vladimír Slávik from comment #17)
> I tried to manually run the commands mentioned above, in the installation
> environment, once Rawhide Silverblue installation is finished.
> 
> - rpm-ostree start-daemon --sysroot /mns/sysroot -> ran for a bit then
> exited, journal has only one message that it's idle and will exit

Ahh right OK. So instead, you can try doing `systemctl edit rpm-ostreed` (or add the equivalent drop-in) to override `ExecStart` to be `/usr/bin/rpm-ostree start-daemon --sysroot /mnt/sysroot`. That way, you shouldn't have to start the daemon manually at all. We can probably clean this up to make it an env var like RPM_OSTREE_SYSROOT instead.

We discussed this recently upstream: https://github.com/coreos/rpm-ostree/pull/2854#discussion_r640720977. Long-term, it's possible that we'll have rpm-ostree "client commands" just take a `--sysroot` arg directly and not involve a daemon architecture at all. That should simplify things like this a lot too.

For prototyping and dirty hacking, you can also keep running `rpm-ostree start-daemon --sysroot /mnt/sysroot` manually, but using RPMOSTREE_DEBUG_DISABLE_DAEMON_IDLE_EXIT=1, which is a debug option that prevents the daemon from auto-exiting on idle.

This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.

Hello!

Any update on this issue? I can confirm having this issue with Fedora SB 35. :)

initramfs-etc just recently got proposed as a stable command, I believe. If this is released before F35, perhaps we can finally resolve the bug, see comment 15.

Hello Jonathan, has there been any progress on making initramfs-etc possible as a client command?

Note to self: bug 1801539 can be solved the same way.

Hi Vladimír,

The `initramfs-etc` command is now out of experimental. However, no progress has been made on the "pure client" side.
Reading back, I think the issue you were hitting trying to stand up the daemon was

> - rpm-ostree ex initramfs-etc --track /etc/vconsole.conf -> error about system not booted via libostree

We should be able to hack around this.

Can you make sure that /run/ostree-booted is present in the chroot?

/run/ostree-booted is not present in the chroot

This message is a reminder that Fedora Linux 35 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '35'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 35 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Changing the affected Fedora version to F37, where I can still experience this bug (with fresh F37 Silverblube install), so that this bug is not automatically closed.

Confirmed in F37 Silverblue. 

Displaying the keyboard layout (similar to F37 Workstation) would surely help.

Confirmed also in F39 Silverblue (tried Gnome x86_64-Rawhide-20230303 and Kinoite x86_64-Rawhide-20230303).

In F37 Workstation the custom keyboard selection was both applied during setup AND shown on boot.
in all versions of Silverblue it is neither applied nor shown.

considering this thread's history, seems like it's time to update the version number once again :(

yup, let's kick it to 38 at least.

In today's prioritized bugs meeting, we agreed to accept this as a prioritized bug. Using the wrong keyboard layout during the boot process is a bad user experience
https://meetbot.fedoraproject.org/fedora-meeting-1/2023-04-19/fedora_prioritized_bugs_and_issues.2023-04-19-14.00.log.html#l-44

Another (maybe easier) option is to add arguments to the kernel command line to set the keymap. For example from https://www.freedesktop.org/software/systemd/man/latest/vconsole.conf.html:

```
... vconsole.keymap=fr
```

(In reply to Timothée Ravier from comment #34)
After adding "vconsole.keymap=de" to the boot parameters, the screen for entering the disk password indeed exhibits a keyboard symbol supplemented by language specifier "de", thanks!

(In reply to Timothée Ravier from comment #34)
> Another (maybe easier) option is to add arguments to the kernel command line
> to set the keymap. For example from
> https://www.freedesktop.org/software/systemd/man/latest/vconsole.conf.html:
> 
> ```
> ... vconsole.keymap=fr
> ```

Nice find! I didn't realize there was a karg for it.
So one thing Anaconda could much more easily do then is to inject the right karg at install time.

This bug is here for 3 years now, and it significantly degrades the user experience for Fedora Silverblue users who do not use English keyboard layout. Could we please finally fix it? Fix in F40 would be much welcome...

-- Dee'Kej --

(In reply to Timothée Ravier from comment #34)
If have checked adding "vconsole.keymap=de" to the boot options of system with a disk encrypted using a German keyboard layout and a suitable password, and this boot option definitely resoves the issue.

I believe vconsole.keymap= used to be present on kernel cmdline in the past, but it was then removed, and perhaps the reason was that when changing the default keyboard layout (on a non-atomic Fedora), it still overrode the change and forced the original keymap during encrypted disk unlocking. It's definitely something to check, before requesting it to be added back. There's also bug 1405539, which is somewhat related to this area. In general, this area of selecting and changing default keyboard layout is undercooked at this moment.

This also affect F39, I assume?

Hi, unfortunately it is. Installer development team don't have capacity to handle this bug :(.

I'm not sure, if I should file a new bug or if it is the same bug. I experienced this problem with Fedora 40 Workstation (not Silverblue). 

I changed the language in the installer to German, then installed with LUKS with password "testz". But I must type "testy" during boot to unlock the luks partition. 

Then I started gnome-disks and tried to change the passphrase. If I tried "testz" as old luks passphrase, I couldn't change the luks passphrase, but with "testy" (in the GUI, gnome-disk with German environment). 

I didn't experienced this problem with older Fedora Workstation. 

Please let me know, if I can supply further information. In the next days I will reinstall Fedora several times, so I'm happy to test this again.

that would be a different bug, but it seems strange, because we do have automated testing for that and it is not currently failing. see e.g. https://openqa.fedoraproject.org/tests/2552274 - that test installs in French, and for the encryption passphrase, it presses the keys that would produce the string 'weakpassword' on a US keyboard, which should give a different string in French (at least A becomes Q and W becomes Z, don't remember if anything else changes). It then presses the same keys when booting the installed system to unlock it. If the layout were different between the installer and the unlock prompt, the test would fail.

Theoretically the test can falsely pass if the layout is wrong at *both* points, but that's not what you describe, and we have a guard against that in the test (we have the system hit the keys marked 'qwerty' on a US keyboard when typing the username, and we check that the username actually comes out as 'azerty'; if not, the test fails).

Do let us know if you hit this consistently, though. Please file a new bug in that case.

oh, wait, I just saw that you said "Workstation live". The live images do not attempt to change the keyboard layout; especially under Wayland it is awkward to have the installer (which is just an app running on the desktop) fighting with the desktop for control of the keyboard layout. On lives if you want to use a different layout you are supposed to configure it via the desktop environment. The KEYBOARD page tells you this if you go to it, I think.

We are working on a new Workstation live installer flow which goes through a language/layout selection wizard on boot (before you have the opportunity to reach the desktop or the installer) but it's not ready for F40, it's currently in Rawhide aimed for F41.

Ok, thanks for the info. Sorry, for the noise. Looking forward for the new installer.