Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1153080

Summary: dracut reports /dev/root does not exist with virtio, and multipathd failed to get path uid
Product: [Fedora] Fedora Reporter: Michel Normand <normand>
Component: device-mapper-multipathAssignee: Ben Marzinski <bmarzins>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 21CC: agk, bmarzins, bugproxy, cjdick, dan, dracut-maint-list, dwysocha, fdinitto, gduarte, gkurz, hannsj_uhl, harald, heinzm, jonathan, lvm-team, msnitzer, normand, prajnoha, prockai, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: device-mapper-multipath-0.4.9-68.fc21.3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1226105 (view as bug list) Environment:
Last Closed: 2015-03-09 08:35:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1051573, 1226105    
Attachments:
Description Flags
console.log
none
myvm3.virtio.xml is failing libvirt config
none
myvm3.ibmvscsi.xml config with no failure
none
console.log with scsi device
none
console.log failure adding rd.debug none

Description Michel Normand 2014-10-15 15:12:51 UTC
Description of problem:
Trying a guest boot from netinst iso, configuring libvirt xml with virtio as per attached myvm3.virtio.xml

The boot is failing as reported by dracut and multipathd:
=== extract of console.log in attachment
[    8.460637] localhost multipathd[124]: vdb: add path (uevent)
[    8.460832] localhost multipathd[124]: vdb: spurious uevent, path already in pathvec
[    8.461349] localhost multipathd[124]: vdb: failed to get path uid
[    8.520261] localhost multipathd[124]: uevent trigger error
[   24.287084] localhost kernel: random: nonblocking pool is initialized
[  194.303501] localhost dracut-initqueue[511]: Warning: Could not boot.
[  194.307002] localhost systemd[1]: Received SIGRTMIN+20 from PID 513 (plymouthd).
[  194.308278] localhost dracut-initqueue[511]: Warning: /dev/root does not exist
[  194.313058] localhost systemd[1]: Starting Dracut Emergency Shell...
===

The iso was retrieved locally on ppckvm host from http://ppc.koji.fedoraproject.org/stage/f21_Alpha_RC1/21_Alpha/Server/ppc64le/iso/Fedora-Server-netinst-ppc64le-21_Alpha.iso


Version-Release number of selected component (if applicable):
dracut-038-29.git20140903.fc21

There is no such problem if libvirt config is modified from virtio to scsi
(as per attached myvm3.ibmvscsi.xml)

Comment 1 Michel Normand 2014-10-15 15:13:40 UTC
Created attachment 947245 [details]
console.log

Comment 2 Michel Normand 2014-10-15 15:16:10 UTC
Created attachment 947247 [details]
myvm3.virtio.xml is failing libvirt config

Comment 3 Michel Normand 2014-10-15 15:17:49 UTC
Created attachment 947248 [details]
myvm3.ibmvscsi.xml config with no failure

Comment 4 Michel Normand 2014-10-16 14:22:43 UTC
I am adding this bug as blocker of ppc64 Beta blocker bug #1146697 as reported failure related to Beta test https://fedoraproject.org/wiki/QA:Testcase_install_to_VirtIO

Comment 5 Harald Hoyer 2014-10-21 06:53:36 UTC
(In reply to Michel Normand from comment #1)
> Created attachment 947245 [details]
> console.log

cat /proc/cmdline
BOOT_IMAGE=/ppc/ppc64/vmlinuz ro


Well, there is no "root=" defined on the kernel command line, nor any other parameter, which tells dracut or anaconda what to do.

Comment 6 Dan HorĂ¡k 2014-10-23 18:50:37 UTC
Michel, how does the console log look when using the scsi disks?

Comment 7 Michel Normand 2014-11-12 15:40:17 UTC
Created attachment 956787 [details]
console.log with scsi device

in reply to comment #6 the new attached console_with_scsi.log is the console log with scsi disks. The devices (sda, sdb) are properly detected and boot is working. To be compared with previous console.log (with virtio device)

Comment 8 Harald Hoyer 2014-11-13 13:52:43 UTC
Can you please attach /run/initramfs/rdsosreport.txt from the failed boot or "cat" it and show the console.log? Preferably with "rd.debug" added to the kernel command line.

Comment 9 Michel Normand 2014-11-13 15:07:11 UTC
(In reply to Harald Hoyer from comment #8)
> Can you please attach /run/initramfs/rdsosreport.txt from the failed boot or
> "cat" it and show the console.log? Preferably with "rd.debug" added to the
> kernel command line.

The initially appended console.log already has the cat of /run/initramfs/rdsosreport.txt
But I will redo a test with suggested "rd.debug", and will append result.

Comment 10 Michel Normand 2014-11-13 15:49:53 UTC
Created attachment 957203 [details]
console.log failure adding rd.debug

as suggested in comment #8 redo the failing boot (with virtio devices) adding rd.debug. Upload the console log as a tarball because of huge debug traces.

Comment 11 Harald Hoyer 2014-11-20 11:45:28 UTC
[    7.721693] localhost kernel: virtio-pci 0000:00:04.0: enabling device (0100 -> 0103)
[    7.723111] localhost kernel: virtio-pci 0000:00:05.0: enabling device (0100 -> 0103)
[    7.918356] localhost kernel:  vda: vda1 vda2 vda3
[    7.922372] localhost kernel:  vdb: vdb1
[    7.935793] localhost multipathd[114]: vda: add path (uevent)
[    7.936034] localhost multipathd[114]: vda: failed to get path uid
[    8.000367] localhost multipathd[114]: uevent trigger error
[    8.225375] localhost multipathd[114]: vdb: add path (uevent)
[    8.230047] localhost multipathd[114]: vdb: failed to get path uid


Is multipath supposed to pickup virtio disks???? Why is multipath even included in the netinst installer initramfs?

1. reassigning to multipath
2. maybe reassign back to dracut to pickup commit
http://git.kernel.org/cgit/boot/dracut/dracut.git/commit/?id=778b3543609d8c9d32df7111229f4072d00d02f0

and add "rd.multipath=0" to the netinst iso initramfs.

Comment 12 IBM Bug Proxy 2015-02-06 10:11:08 UTC
------- Comment From mainamdar.com 2015-02-06 10:08 EDT-------
Any update on this?

Comment 13 Ben Marzinski 2015-02-06 15:26:17 UTC
Multipath can't work with vd.* devices. They don't have anything to use for a WWID.  It should not be used at all here. Reassigning to have it disabled.

Comment 14 Harald Hoyer 2015-02-13 10:45:32 UTC
(In reply to Ben Marzinski from comment #13)
> Multipath can't work with vd.* devices. They don't have anything to use for
> a WWID.  It should not be used at all here. Reassigning to have it disabled.

Then change your udev rules and multipathd. 

dracut cannot disable multipath on the fly only for vd* devices.

Comment 15 Chris 2015-02-17 20:45:37 UTC
Looks like I have the same problem.
I have installed RHEL7.0 onto a PC with VT technology enabled, Base package: Server with GUI, and added on the 3 Virtualization packages.
All running fine as expected. Im only using this for study purposes, it remains off any network.

I now want to install a RHEL7.0 KVM onto it. I must of tried 20 times now and everytime it fails during the install and falls into Dracut emergency shell, errors-

vda: add path (uevent)
vda: spurious uevent, path already in pathvec
vda: failed to get path uid
uevent: trigger error
.
Warning: could not boot
Warning: /dev/root does not exist
Starting Dracut Emergency Shell...

the journalctl log it produces shows up just 1 error of possible interest-
Failed to access perfctr msr (MSR c1 is 0)

I have tried this on 2 different PCs and 1 laptop which all have VT technology enabled, and ive successfully performed this process on all 3 of them with RHEL6.1-4

I've followed the Red Hat KVM official KVM install procedures to the letter and also tried a few ways ive been successul with RHEL6 with no luck.

Comment 16 Fedora Update System 2015-02-19 04:49:41 UTC
device-mapper-multipath-0.4.9-68.fc21.3 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/device-mapper-multipath-0.4.9-68.fc21.3

Comment 17 Ben Marzinski 2015-02-19 04:51:25 UTC
vd.* devices are now blacklisted by multipath in device-mapper-multipath-0.4.9-68.fc21.3

So they should never come up as valid multipath device paths, and multipath should completely ignore them.

Comment 18 Fedora Update System 2015-02-19 18:02:42 UTC
Package device-mapper-multipath-0.4.9-68.fc21.3:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing device-mapper-multipath-0.4.9-68.fc21.3'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-2306/device-mapper-multipath-0.4.9-68.fc21.3
then log in and leave karma (feedback).

Comment 19 Fedora Update System 2015-03-09 08:35:20 UTC
device-mapper-multipath-0.4.9-68.fc21.3 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.