Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1379725

Summary: systemd-journald.service loaded but failed
Product: [Fedora] Fedora Reporter: Geoffrey Marr <gmarr>
Component: systemdAssignee: systemd-maint
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 25CC: johannbg, lnykryn, msekleta, muadda, ssahani, s, systemd-maint, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-09-28 21:38:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Output of 'journalctl -aeb' command
none
Output of 'dmesg' after adding kernel-boot parameter none

Description Geoffrey Marr 2016-09-27 14:15:57 UTC
Created attachment 1205221 [details]
Output of 'journalctl -aeb' command

Description of problem:
Installed Fedora-Cloud-Base-25-20160924.n.0.x86_64-us-west-2-HVM-standard-0 (ami-4a815e2a) on Amazon EC2. Went to check the journal and noticed it stopped recording information a few minutes earlier. Ran 'systemctl --all --failed' and noted 7 failed units: 

● systemd-journald.service  
● systemd-tmpfiles-clean.service   
● systemd-tmpfiles-setup-dev.service 
● systemd-tmpfiles-setup.service     
● systemd-journald-audit.socket     
● systemd-journald-dev-log.socket    
● systemd-journald.socket           

Version-Release number of selected component (if applicable):
Fedora-Cloud-Base-25-20160924.n.0.x86_64-us-west-2-HVM-standard-0 (ami-4a815e2a)

How reproducible:
On each boot. Noticed the amount of failed units changes between 6 and 7.

Steps to Reproduce:
1. Install Fedora-Cloud-Base-25-20160924.n.0.x86_64-us-west-2-HVM-standard-0 (ami-4a815e2a) on Amazon EC2
2. Run 'journalctl -aeb' to note that the journal stops updating
3. Run 'systemctl --all --failed' to see the failed units

Comment 1 Zbigniew Jędrzejewski-Szmek 2016-09-27 16:00:05 UTC
The log you attached looks like journald was running in the initramfs, and was shut down in preparation for the switch to the main root file system. This is normal, and usually the next message would be from the new journald. It seems that in your case journald does not start properly, so there are no logs from after the switchroot. It's hard to say what is going on here without some logs.

You can try running (as root) /usr/lib/systemd/systemd-journald and see if it fails to start. If this fails, it will print out some messages. Otherwise, you can switch the log target during boot – add systemd.log_target=kmsg on the kernel command line. Then 'dmesg' should contain the logs.

Comment 2 Geoffrey Marr 2016-09-27 20:10:46 UTC
Created attachment 1205308 [details]
Output of 'dmesg' after adding kernel-boot parameter

Zbigniew,
I tried running /usr/lib/systemd/systemd-journald and it failed, so I added the kernel boot parameter and have attached the output in a file called 'dmesg.txt'. Let me know what else I can do to help.

Comment 3 Zbigniew Jędrzejewski-Szmek 2016-09-28 10:21:40 UTC
That's strange:
[   38.534076] systemd-journald[308]: Failed to open runtime journal: No such file or directory

This directory should be created automatically (it's trying to create a subdirectory of /run/log/journal, so most likely it's /run/log missing that is causing the error above). Maybe it's related to the this:

[   38.777170] systemd-tmpfiles[329]: [/usr/lib/tmpfiles.d/systemd.conf:26] Failed to replace specifiers: /run/log/journal/%m

Do you have /etc/machine-id file? This might be related to #1379800.

Comment 4 Geoffrey Marr 2016-09-28 14:51:10 UTC
Zbigniew,
There is no machine-id file created on firstboot. If I make the file (touch /etc/machine-id), then reboot the system, there are no failed units when 'systemctl --all --failed' is run. You're right, it looks like it is a part of #1379800.

Comment 5 Zbigniew Jędrzejewski-Szmek 2016-09-28 21:38:45 UTC
OK, so let's move this over to the other bug.

*** This bug has been marked as a duplicate of bug 1379800 ***