Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1379800

Summary: systemd does not create new machine-id file if none is present
Product: [Fedora] Fedora Reporter: Paul Whalen <pwhalen>
Component: systemdAssignee: systemd-maint
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 25CC: awilliam, gmarr, johannbg, lnykryn, msekleta, muadda, pbrobinson, pschindl, robatino, ssahani, s, systemd-maint, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: RejectedBlocker
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-12 10:07:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 245418    
Attachments:
Description Flags
"systemctl --all --failed" on Fedora-Cloud-Base-25-20160924.n.0.x86_64 none

Description Paul Whalen 2016-09-27 17:07:05 UTC
Description of problem:

Fedora ARM disk images are generated, then stripped of '/etc/machine-id' with the intention systemd will repopulate if none is present. This doesnt happen unless there is minimally an empty machine-id file. 

Version-Release number of selected component (if applicable):

systemd-231-4.fc25

How reproducible:
Always. 

Steps to Reproduce:
1. Boot any system, delete machine-id 'sudo rm /etc/machine-id', reboot
2. Machine-id is not recreated, journal (and initial-setup) services fail

Expected results:
machine-id recreated, no failing services

Comment 1 Jóhann B. Guðmundsson 2016-09-27 17:51:14 UTC
Hmm why are you under the assumption a new one is generated if missing? 

In anycase you need to run systemd-machine-id-setup to create the initial machine-id ( or new one ) if missing see [1] [2]

1. https://www.freedesktop.org/software/systemd/man/machine-id.html
https://www.freedesktop.org/software/systemd/man/systemd-machine-id-setup.html#

Comment 2 Peter Robinson 2016-09-27 18:27:48 UTC
(In reply to Jóhann B. Guðmundsson from comment #1)
> Hmm why are you under the assumption a new one is generated if missing? 

In the case of images and liveCDs you would not necessarily want the same machine-id across a lot of individual images. So as part of the build process we should really remove it and then it should generate a new one if it's missing so each deployed image/live-cd is unique.

Comment 3 Jóhann B. Guðmundsson 2016-09-27 18:59:16 UTC
But is not what you really want to run systemd-firstboot [1] at every firstboot so you flush it status and have it run again? 

1. https://www.freedesktop.org/software/systemd/man/systemd-firstboot.html

Comment 4 Peter Robinson 2016-09-27 19:08:34 UTC
(In reply to Jóhann B. Guðmundsson from comment #3)
> But is not what you really want to run systemd-firstboot [1] at every
> firstboot so you flush it status and have it run again? 

no, because in the boot process journald fails before we even get to that point.

Comment 5 Jóhann B. Guðmundsson 2016-09-27 19:18:37 UTC
Are you sure you are talking about systemd-firstboot here not Fedora's firstboot which are not the same thing?

Comment 6 Peter Robinson 2016-09-27 19:24:51 UTC
So if we put an empty /etc/machine-id file in place (say "touch /etc/machine-id" it will regenerate a new machine-id in the empty file, if the file isn't there it won't do anything. This seems like a bug to me as opposed to any particular service.

Comment 7 Jóhann B. Guðmundsson 2016-09-27 19:59:06 UTC
If the system ( or part of it ) is spectacular failing without that file that's more cause for concern than just an empty file.

In anycase can you try this type service unit as an workaround in the meantime

regenerate-machineid.service

[Unit]
Description=Generate New Machine ID
Documentation=man:systemd-firstboot(1)
DefaultDependencies=no
Conflicts=shutdown.target
After=systemd-remount-fs.service
Before=systemd-sysusers.service sysinit.target shutdown.target
ConditionPathIsReadWrite=/etc
ConditionPathExists=!/etc/machine-id

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/systemd-firstboot --setup-machine-id

[Install]
WantedBy=sysinit.target

Comment 8 Zbigniew Jędrzejewski-Szmek 2016-09-28 21:38:45 UTC
*** Bug 1379725 has been marked as a duplicate of this bug. ***

Comment 9 Zbigniew Jędrzejewski-Szmek 2016-09-28 23:14:45 UTC
Yeah, machine-id is supposed to be created if it is missing. Of course this only works if the FS is not readonly. I guess this might be some bad interaction with the FS being initially readonly and remounted to rw during boot. Remounting is done by a service, which runs later than the id setup code. It seems that this case was not taken into account properly.

Comment 10 Peter Robinson 2016-09-30 22:33:27 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #9)
> Yeah, machine-id is supposed to be created if it is missing. Of course this
> only works if the FS is not readonly. I guess this might be some bad
> interaction with the FS being initially readonly and remounted to rw during
> boot. Remounting is done by a service, which runs later than the id setup
> code. It seems that this case was not taken into account properly.

So why does it work if the file is touched and empty as it should still be a RO filesytem and hence not be able to write whether the file is there or not?

Or is it some things, I know at least journald fails, just checking for the presence of the file instead of it's actual contents?

Comment 11 Zbigniew Jędrzejewski-Szmek 2016-10-01 14:56:29 UTC
If the file is empty it temporarily bind-mounts a file with the right contents, and later on when the fs is rw, the real file is updated and the one on top is unmounted.

Comment 12 Geoffrey Marr 2016-10-03 15:23:57 UTC
Created attachment 1206921 [details]
"systemctl --all --failed" on Fedora-Cloud-Base-25-20160924.n.0.x86_64

Output of "systemctl --all --failed" on Fedora-Cloud-Base-25-20160924.n.0.x86_64. If "/etc/machine-id" is created manually and either the system rebooted or 'systemd-machine-id-setup' is run, the system will begin running 'journald.service' and the associated parts.

Comment 13 Geoffrey Marr 2016-10-03 15:29:33 UTC
Proposing as Fedora 25 Final blocker:

On Fedora-Cloud-Base-25-20160924.n.0.x86_64, this causes "journald.service" to not start thus causing the journal to not keep a log. This causes (at least) two testcases to fail and violates the following F25-Final Blocker criteria:

"All system services present after installation with one of the release-blocking package sets must start properly, unless they require hardware which is not present."

Comment 14 Adam Williamson 2016-10-03 17:53:35 UTC
pbrobinson has applied the 'create it as an empty file' workaround on all relevant kickstarts (by the looks of it):

https://pagure.io/fedora-kickstarts/c/73645f341ed34239e13c27ce8e7e8702487a6718?branch=f25

so I don't believe we need to block on this. But indeed systemd should handle the case of the file not existing correctly too, so it's still a bug.

Comment 15 Petr Schindler 2016-10-03 18:00:24 UTC
Discussed at 2016-10-03 blocker review meeting: [1]. 

This bug was rejected as Final blocker: this would be a blocker, but it has been worked around in fedora-kickstarts by ensuring the images contain an empty file, so we do not actually need to block on it so long as that works

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2016-10-03/

Comment 16 Geoffrey Marr 2016-10-05 03:50:13 UTC
Follow-up to 2016-10-03 blocker review meeting:

Tested with 'create empty file' workaround in F25-cloud-base-20161004, works great. No longer see issue and journal is updated like it should. Thanks.

Comment 17 Peter Robinson 2016-10-05 10:55:02 UTC
(In reply to Adam Williamson from comment #14)
> pbrobinson has applied the 'create it as an empty file' workaround on all
> relevant kickstarts (by the looks of it):

Correct, it has been worked around, I don't think we need to block on this, upstream does need to fix it :)

Comment 18 Fedora End Of Life 2017-11-16 19:03:25 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 19 Fedora End Of Life 2017-12-12 10:07:46 UTC
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.