Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 747632 - using iSCSI target as root filesystem results in kernel panic at boot
Summary: using iSCSI target as root filesystem results in kernel panic at boot
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dracut
Version: 16
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: dracut-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker
Depends On:
Blocks: F16Blocker, F16FinalBlocker
TreeView+ depends on / blocked
 
Reported: 2011-10-20 14:49 UTC by Tim Flink
Modified: 2011-10-25 16:10 UTC (History)
7 users (show)

Fixed In Version: dracut-013-16.fc16
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-10-25 03:44:24 UTC
Type: ---
Embargoed:

Attachments (Terms of Use)
console log from failed boot (deleted)
2011-10-20 14:49 UTC, Tim Flink 		no flags Details
fstab from system (deleted)
2011-10-20 15:05 UTC, Tim Flink 		no flags Details
grub configuration (deleted)
2011-10-20 15:19 UTC, Tim Flink 		no flags Details
anaconda log (deleted)
2011-10-20 15:20 UTC, Tim Flink 		no flags Details
anaconda program.log (deleted)
2011-10-20 15:20 UTC, Tim Flink 		no flags Details
anaconda storage.log (deleted)
2011-10-20 15:21 UTC, Tim Flink 		no flags Details
anaconda install.log (deleted)
2011-10-20 15:59 UTC, Tim Flink 		no flags Details
dracut.log after running failed new-kernel-pkg cmd (deleted)
2011-10-20 17:12 UTC, Tim Flink 		no flags Details
View All

Description Tim Flink 2011-10-20 14:49:07 UTC 
Description of problem:

I did a minimal i386 install with the following partition layout:
Local IDE:
 - 500M /boot
 - 2048 swap

iSCSI Target
 - 20G /

When the system reboots, I get a kernel panic when dracut tries to mount the root filesystem.

Note that this is a regular NIC and does not support iBFT -> /boot is on the local disk.

I have been unable to get into the dracut shell, even when I pass in rd.shell or rd.break as kernel options.

Boot Args:
BOOT_IMAGE=/vmlinuz-3.1.0-0.rc10.git0.1.fc16.i686.PAE root=/dev/sdb1 ro ifname=em1:00:07:e9:5c:e0:30 iscsi_initiator=iqn.1994-05.com.domain:01.0e9b92 rd.md=0 rd.lvm=0 rd.dm=0 ip=em1:dhcp KEYTABLE=us SYSFONT=latarcyrheb-sun16 netroot=iscsi:@192.168.0.5::3260::iqn.2009-02.net.tirfa:for.all rd.luks=0 LANG=en_US.UTF-8 console=ttyS0,38400n8 rd.debug rd.udev.debug


Version-Release number of selected component (if applicable):
dracut-013-15.fc16

How reproducible:
Every time, using LVM or not.

Steps to Reproduce:
1. do minimal install with / on iSCSI, /boot and swap on local disk
2. reboot
  
Actual results:
kernel panic on init

Expected results:
boot into working system

Additional info:
Comment 1 Tim Flink 2011-10-20 14:49:40 UTC 
Created attachment 529303 [details]
console log from failed boot
Comment 2 Tim Flink 2011-10-20 15:05:11 UTC 
Created attachment 529318 [details]
fstab from system
Comment 3 Tim Flink 2011-10-20 15:07:14 UTC 
Proposing as a blocker for Fedora 16 final as it violates the following final release criterion [1]:

The installer must be able to complete an installation using any network-attached storage devices (e.g. iSCSI, FCoE, Fibre Channel)

[1] http://fedoraproject.org/wiki/Fedora_16_Final_Release_Criteria
Comment 4 Harald Hoyer 2011-10-20 15:10:09 UTC 
[    3.748491] VFS: Cannot open root device "sdb1" or unknown-block(0,0)

This indicates, that no initramfs has been loaded and dracut was not even executed... 

Check your grub configuration!
Comment 5 Tim Flink 2011-10-20 15:19:16 UTC 
Created attachment 529322 [details]
grub configuration

This is the grub configuration on that system. There is neither an initramfs on /boot nor a initrd directive in the grub configuration
Comment 6 Tim Flink 2011-10-20 15:20:13 UTC 
Created attachment 529324 [details]
anaconda log
Comment 7 Tim Flink 2011-10-20 15:20:43 UTC 
Created attachment 529325 [details]
anaconda program.log
Comment 8 Tim Flink 2011-10-20 15:21:12 UTC 
Created attachment 529326 [details]
anaconda storage.log
Comment 9 David Lehman 2011-10-20 15:31:01 UTC 
Looks to me like everything anaconda was supposed to put in GRUB_CMDLINE_LINUX is there for the iscsi root. Apparently either there is no initrd or else grub2-mkconfig failed to find/recognize it. Tim, can you tell us if there's an initramfs to match your kernel in /boot?
Comment 10 Tim Flink 2011-10-20 15:46:56 UTC 
(In reply to comment #9)
> Looks to me like everything anaconda was supposed to put in GRUB_CMDLINE_LINUX
> is there for the iscsi root. Apparently either there is no initrd or else
> grub2-mkconfig failed to find/recognize it. Tim, can you tell us if there's an
> initramfs to match your kernel in /boot?

There are no initramfs files in /boot
Comment 11 Tim Flink 2011-10-20 15:59:18 UTC 
Created attachment 529337 [details]
anaconda install.log
Comment 12 David Lehman 2011-10-20 16:02:25 UTC 
From install.log:

mkinitrd failed
warning: %posttrans(kernel-PAE-3.1.0-0.rc10.git0.1.fc16.i686) scriptlet failed, exit status 1


Tim, can you reproduce and then run whatever's in the kernel's %posttrans from the command line so we can see what fails?
Comment 13 Tim Flink 2011-10-20 17:02:40 UTC 
(In reply to comment #12)
> Tim, can you reproduce and then run whatever's in the kernel's %posttrans from
> the command line so we can see what fails?

The two commands in %posttrans for kernel-PAE are:
/sbin/new-kernel-pkg --package kernel-PAE --mkinitrd --dracut --depmod --update 3.1.0-0.rc10.git0.1.fc16.i686.PAE || exit $?

/sbin/new-kernel-pkg --package kernel-PAE --rpmposttrans 3.1.0-0.rc10.git0.1.fc16.i686.PAE || exit $?


The failure is in the first command:

# /sbin/new-kernel-pkg --package kernel-PAE --mkinitrd --dracut --depmod --update 3.1.0-0.rc10.git0.1.fc16.i686.PAE

F: Failed to install wget
mkinitrd failed
Comment 14 Tim Flink 2011-10-20 17:12:19 UTC 
Created attachment 529343 [details]
dracut.log after running failed new-kernel-pkg cmd
Comment 15 Tim Flink 2011-10-20 17:47:43 UTC 
I re-ran the kernel %posttrans commands after adding the following to dracut.conf:
omit_dracutmodules+="livenet"

I got an initramfs and the system now boots with the iSCSI target as /.

Re-assigning to dracut.
Comment 16 Tim Flink 2011-10-20 18:13:22 UTC 
I tried the patch that wwoods posted:

http://article.gmane.org/gmane.linux.kernel.initramfs/2183

After reverting dracut.conf and verified that the kernel posttrans commands failed again. I applied the patch to /usr/share/dracut/modules.d/90livenet/module-setup.sh, re-ran the kernel posttrans commands and they completed successfully.
Comment 17 Fedora Update System 2011-10-21 10:59:48 UTC 
dracut-013-16.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/dracut-013-16.fc16
Comment 18 Adam Williamson 2011-10-21 17:28:40 UTC 
Discussed at 2011-10-21 blocker review meeting. Accepted as a blocker per criterion "The installer must be able to complete an installation using any network-attached storage devices (e.g. iSCSI, FCoE, Fibre Channel)" - technically the bug only affects minimal installations, but iSCSI is a case where you're probably quite likely to do a minimal install, and it seems a serious enough bug to take as a blocker.
Comment 19 Fedora Update System 2011-10-22 00:10:43 UTC 
Package dracut-013-16.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing dracut-013-16.fc16'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-14704
then log in and leave karma (feedback).
Comment 20 Fedora Update System 2011-10-25 03:44:24 UTC 
dracut-013-16.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 21 Tim Flink 2011-10-25 16:10:13 UTC 
Verified with an i686 minimal install of Fedora 16 using iSCSI target as /
Note You need to log in before you can comment on or make changes to this bug.