Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1655329

Summary: broken options to fedora-arm-image-installer
Product: [Fedora] Fedora Reporter: Torbjorn Jansson <torbjorn>
Component: arm-image-installerAssignee: Peter Robinson <pbrobinson>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 29CC: jan.kratochvil, pbrobinson, pcfe, pwhalen, torbjorn
Target Milestone: ---   
Target Release: ---   
Hardware: armv7hl   
OS: Linux   
Whiteboard:
Fixed In Version: arm-image-installer-2.13-1.fc30 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-06 04:09:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
rpi3 boot
none
journalctl --root=/mnt/ none

Description Torbjorn Jansson 2018-12-02 20:49:21 UTC
Description of problem:
fedora-arm-image-installer have two options that appears broken.

first one --norootpass creates an image that appears to get stuck for about 50 minutes before sometimes producing a login prompt, but it is impossible to logon.
checking systemd journal shows repeating issues starting services, and many complaints related to missing user accounts.

second issue is that --resizefs doesn't actually resize the root filesystem.
this worked fine in fedora28 version of fedora-arm-image-installer but not in fedora29


Version-Release number of selected component (if applicable):
arm-image-installer-2.5-1.fc29.noarch

How reproducible:
Yes it is 100% reproducible with my odroid hc2 and above two options specified (in addition to other options i need)


Steps to Reproduce:
Create image like this for example:
fedora-arm-image-installer --target=none --image=Fedora-Minimal-armhfp-29-1.2-sda.raw.xz --resizefs --norootpass --media=/dev/sdXXX --args "<args here like console and so on>"

odroid also needs the sd_fusing script to make a bootable image
if the uboot used does not contain this patch (this is the case with current fedora 29 uboot image):
http://git.denx.de/?p=u-boot.git;a=patch;h=9cd97c5b049a9a282dda0b1782cbb38d8cedb417
then you need to manually create a symlink so the correct dtb file is loaded.
like this:
ln -s exynos5422-odroidhc1.dtb exynos5422-odroidunknown.dtb


Actual results:
boot stuck for a very long time and never able to logon properly.

journal contains this section that repeats every 90 seconds or so
----
[ 1293.050223] systemd[1]: Failed to get initial list of names: Connection timed out
[ 1293.059841] systemd[1]: dbus.service: Main process exited, code=exited, status=1/FAILURE
[ 1293.072092] systemd[1]: dbus.service: Failed with result 'exit-code'.
[ 1293.082140] systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
[ 1293.092808] systemd[1]: sshd.service: Scheduled restart job, restart counter is at 5.
[ 1293.106343] systemd[1]: systemd-logind.service: Start operation timed out. Terminating.
[ 1293.123132] dbus-daemon[820]: dbus[820]: Unknown username "systemd-resolve" in message bus configuration file
[ 1293.139994] systemd[1]: NetworkManager.service: Service RestartSec=100ms expired, scheduling restart.
[ 1293.152811] dbus-daemon[820]: dbus[820]: Unknown username "systemd-timesync" in message bus configuration file
[ 1293.166311] dbus-daemon[820]: dbus[820]: Unknown username "systemd-network" in message bus configuration file
[ 1293.179637] dbus-daemon[820]: dbus[820]: Unknown username "polkitd" in message bus configuration file
[ 1293.191577] dbus-daemon[820]: dbus[820]: Unknown username "polkitd" in message bus configuration file
[ 1293.206002] dbus-daemon[820]: Failed to start message bus: Could not get UID and GID for username "dbus"
[ 1293.221191] systemd[1]: NetworkManager.service: Scheduled restart job, restart counter is at 4.
[ 1293.235797] systemd[1]: Stopped Network Manager.
[ 1293.244930] systemd[1]: Started D-Bus System Message Bus.
[ 1293.254343] systemd[1]: Starting Network Manager...
[ 1293.262796] systemd[1]: Stopped OpenSSH server daemon.
[ 1293.272751] systemd[1]: Stopped target sshd-keygen.target.
[ 1293.281229] systemd[1]: Stopping sshd-keygen.target.
[ 1293.288891] systemd[1]: Reached target sshd-keygen.target.
[ 1293.297969] systemd[1]: Starting OpenSSH server daemon...
[ 1293.307232] systemd[1]: systemd-logind.service: Failed with result 'timeout'.
[ 1293.317644] systemd[1]: Failed to start Login Service.
[ 1293.326278] systemd[1]: systemd-logind.service: Service has no hold-off time (RestartSec=0), scheduling restart.
[ 1293.340292] systemd[1]: systemd-logind.service: Scheduled restart job, restart counter is at 11.
[ 1293.353654] systemd[1]: Stopped Login Service.
[ 1293.361281] systemd[1]: Starting Login Service...
[ 1293.371696] NetworkManager[821]: <info>  [1529667196.1274] NetworkManager (version 1.12.4-1.fc29) is starting... (after a restart)
[ 1293.386568] NetworkManager[821]: <info>  [1529667196.1283] Read config: /etc/NetworkManager/NetworkManager.conf
[ 1293.405375] systemd-logind[823]: New seat seat0. 
-----


Expected results:
a working fedora image on sd card.

Additional info:

Comment 1 Peter Robinson 2018-12-03 07:47:20 UTC
> Version-Release number of selected component (if applicable):
> arm-image-installer-2.5-1.fc29.noarch

The latest stable version is 2.8, I believe all of the above are fixed with the latest version.

Comment 2 Torbjorn Jansson 2018-12-08 13:58:06 UTC
i have re-tested with:
arm-image-installer-2.8-1.fc29.noarch

--resizefs is fixed and now correctly resizes the root partition.

but the bigger issue of --norootpass producing broken images is still there.
same as before, missing users at boot in journal.
journal is easiest seen by adding: systemd.journald.forward_to_console=1 to kernel boot command line and then use the serial port to see whats going on.

Comment 3 Jan Kratochvil 2019-06-25 08:42:19 UTC
Cannot this be a duplicate of Bug 1692903?

Comment 4 Torbjorn Jansson 2019-06-25 16:26:45 UTC
well... i think it might be the other way around regarding the duplicate or something.

all i know is that if i pass --norootpass image is broken and without it boots just fine and i have several devices using resulting image.
so i'm not so sure it is selinux related, if it was then it would not matter if i specify --norootpass or not.

for completeness yes i have selinux disabled where the image is created but i don't see how that's relevant to the image generation.
whatever --norootpass does to the image it is not working out.

Comment 5 Jan Kratochvil 2019-06-25 16:38:23 UTC
(In reply to Torbjorn Jansson from comment #4)
> all i know is that if i pass --norootpass image is broken and without it
> boots just fine
...
> for completeness yes i have selinux disabled where the image is created

It would be great if you could test a fix for this issue posted upstream today:
  https://pagure.io/arm-image-installer/pull-request/38

Comment 6 Paul Whalen 2019-06-25 19:30:35 UTC
> for completeness yes i have selinux disabled where the image is created but
> i don't see how that's relevant to the image generation.
> whatever --norootpass does to the image it is not working out.

Testing with the latest - arm-image-installer-2.12-1.fc30.noarch

I can't reproduce the issue with Fedora 30 Minimal (with no initial-setup). I also tried with SE Linux in permissive with no change. 

Command used: 

sudo arm-image-installer --target=rpi3 --image=Fedora-Minimal-armhfp-30-1.2-sda.raw.xz --resizefs --norootpass --media=/dev/sdd --args "systemd.journald.forward_to_console=1" --addconsole

To remove the root password requirement the script uses "sed -i 's/root:x:/root::/' /tmp/root/etc/passwd"

Comment 7 Paul Whalen 2019-06-25 19:32:21 UTC
Created attachment 1584429 [details]
rpi3 boot

Comment 8 Paul Whalen 2019-06-25 19:46:09 UTC
OK, reproduced with selinux disabled on the host (tsk tsk)

Comment 9 Jan Kratochvil 2019-06-26 13:18:12 UTC
Created attachment 1584820 [details]
journalctl --root=/mnt/

(In reply to Paul Whalen from comment #6)
> Testing with the latest - arm-image-installer-2.12-1.fc30.noarch

Also: arm-image-installer-2.12-1.fc30.noarch

 
> I also tried with SE Linux in permissive with no change. 

On x86_64 host system one must have: getenforce == Disabled
Is it really so? getenforce == Permissive will still build a correct ARM image, only getenforce == Disabled will break the built ARM image.


> sudo arm-image-installer --target=rpi3
> --image=Fedora-Minimal-armhfp-30-1.2-sda.raw.xz --resizefs --norootpass
> --media=/dev/sdd --args "systemd.journald.forward_to_console=1" --addconsole

I have done as root:
  arm-image-installer --target=rpi3 --image=Fedora-Minimal-armhfp-30-1.2-sda.raw.xz --norootpass --media=/dev/sdb
  (sorry it was not literally your command but I believe it would be the same)
And then it shows login screen but when I type "root" it asks for "Password:" and I cannot login with any password I try.

I do not know how to catch that console afterwards so I fsck-ed+mounted the device afterwards and typed:
  journalctl --root=/mnt/
Which I am attaching. It starts with:
Apr 12 17:18:02 localhost audit[617]: AVC avc:  denied  { read } for  pid=617 comm="sh" name="passwd" dev="sda3" ino=35796 scontext=system_u:system_r:loadkeys_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=0


> To remove the root password requirement the script uses "sed -i
> 's/root:x:/root::/' /tmp/root/etc/passwd"

The problem is that the installing (=host=x86_64) system without SELinux will corrupt by that sed the SELinux context of /tmp/root/etc/passwd.

Comment 10 Jan Kratochvil 2019-06-26 13:23:01 UTC
(In reply to Paul Whalen from comment #8)
> OK, reproduced with selinux disabled on the host (tsk tsk)

Oops, OK, great; I did not have to spend time reproducing it again, sorry I did not refresh my browser window.

Comment 11 Fedora Update System 2019-06-26 18:54:20 UTC
FEDORA-2019-2dd9f78d69 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-2dd9f78d69

Comment 12 Fedora Update System 2019-06-26 18:54:22 UTC
FEDORA-2019-7cd0e1fc4b has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-7cd0e1fc4b

Comment 13 Fedora Update System 2019-06-27 01:41:46 UTC
arm-image-installer-2.13-1.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-7cd0e1fc4b

Comment 14 Fedora Update System 2019-06-27 02:43:48 UTC
arm-image-installer-2.13-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-2dd9f78d69

Comment 15 Fedora Update System 2019-07-06 04:09:15 UTC
arm-image-installer-2.13-1.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.