Bug 729340

Can you attach the bootup log of a successful boot please (I am curious if it worked with 2.6.38 or 2.6.39)? How does one go about reproducing this? Can you dump the xenstore keys using this little script inside the guest (the working and the non-working one please):

You might have to install the xenstore-ls tool - not exactly sure which package that is in FC15 or FC14. 

sh-4.1# more /a
A=1
DEV=`dmesg | grep vbd | sed s/.*vbd//`
RC=1
while (true)
do
 # /local/domain/0/backend/vbd/12/51760
 for a in $DEV
 do
        xenstore-ls /local/domain/0/backend/vbd/$A$a
        RC=$?
 done
 if [ $RC -eq 0 ]; then
        break;
 fi
 A=$(($A+1))
 if [ $A -eq 0 ]; then
        break;
 fi
done

You should get something like this:

frontend = "/local/domain/12/device/vbd/51712"
physical-device = "fc:a"
params = "/dev/vg_guest_1/data-ext4"
frontend-id = "12"
online = "1"
removable = "0"
bootable = "1"
state = "4"
dev = "xvda"
type = "phy"
mode = "w"
feature-flush-cache = "1"
sectors = "20971520"
info = "0"
sector-size = "512"
frontend = "/local/domain/12/device/vbd/51744"
physical-device = "fc:9"
params = "/dev/vg_guest_1/data-ext3"
frontend-id = "12"
online = "1"
removable = "0"
bootable = "1"
state = "4"
... and so on.

Ah, hadn't noticed that the previous working instance was 2.6.38. You guys skipped 2.6.39 in this test.

In which case I think you are hitting:
c80a420995e721099906607b07c09a24543b31d9:
Author: Stefano Stabellini <stefano.stabellini.com>
Date:   Thu Dec 2 17:55:00 2010 +0000

    xen-blkfront: handle Xen major numbers other than XENVBD

    This patch makes sure blkfront handles correctly virtual device numbers
    corresponding to Xen emulated IDE and SCSI disks: in those cases
    blkfront translates the major number to XENVBD and the minor number to a
    low xvd minor.

    Note: this behaviour is different from what old xenlinux PV guests used
    to do: they used to steal an IDE or SCSI major number and use it
    instead.

And for 3.1 we have this patch:
commit 196cfe2ae8fcdc03b3c7d627e7dfe8c0ce7229f9
Author: Stefan Bader <stefan.bader>
Date:   Thu Jul 14 15:30:22 2011 +0200

    xen-blkfront: Drop name and minor adjustments for emulated scsi devices
    
    These were intended to avoid the namespace clash when representing
    emulated IDE and SCSI devices. However that seems to confuse users
    more than expected (a disk defined as sda becomes xvde).
    So for now go back to the scheme which does no adjustments. This
    will break when mixing IDE and SCSI names in the configuration of
    guests but should be by now expected.
    
    Acked-by: Stefano Stabellini <stefano.stabellini.com>
    Signed-off-by: Stefan Bader <stefan.bader>
    Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk>

which fixes that. I think that is what you are hitting.

Created attachment 517524 [details]
Successful boot log

(In reply to comment #1)
> Can you attach the bootup log of a successful boot please (I am curious if it
> worked with 2.6.38 or 2.6.39)? How does one go about reproducing this? Can you
> dump the xenstore keys using this little script inside the guest (the working
> and the non-working one please):
> 
> You might have to install the xenstore-ls tool - not exactly sure which package
> that is in FC15 or FC14. 

This script only returns this output in an unending loop:

    xenstore-ls: xs_open: No such file or directory

$ dmesg | grep vbd | sed s/.*vbd//
/2049
/2064
/2080

Successful boot log attached, please ignore running yum afterwards.

(In reply to comment #2)
> Ah, hadn't noticed that the previous working instance was 2.6.38. You guys
> skipped 2.6.39 in this test.

There was no 2.6.39 kernel submitted for Fedora 15, or at least I cannot find such one:

    https://admin.fedoraproject.org/updates/search/kernel

Ugh, well then I think we are back to trying trying the patch:

commit 196cfe2ae8fcdc03b3c7d627e7dfe8c0ce7229f9
Author: Stefan Bader <stefan.bader>
Date:   Thu Jul 14 15:30:22 2011 +0200

    xen-blkfront: Drop name and minor adjustments for emulated scsi devices
    
    These were intended to avoid the namespace clash when representing
    emulated IDE and SCSI devices. However that seems to confuse users
    more than expected (a disk defined as sda becomes xvde).
    So for now go back to the scheme which does no adjustments. This
    will break when mixing IDE and SCSI names in the configuration of
    guests but should be by now expected.
    
    Acked-by: Stefano Stabellini <stefano.stabellini.com>
    Signed-off-by: Stefan Bader <stefan.bader>
    Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk>

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index b536a9c..238b941 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -123,8 +123,8 @@ static DEFINE_SPINLOCK(minor_lock);
 #define BLKIF_MINOR_EXT(dev) ((dev)&(~EXTENDED))
 #define EMULATED_HD_DISK_MINOR_OFFSET (0)
 #define EMULATED_HD_DISK_NAME_OFFSET (EMULATED_HD_DISK_MINOR_OFFSET / 256)
-#define EMULATED_SD_DISK_MINOR_OFFSET (EMULATED_HD_DISK_MINOR_OFFSET + (4 * 16))
-#define EMULATED_SD_DISK_NAME_OFFSET (EMULATED_HD_DISK_NAME_OFFSET + 4)
+#define EMULATED_SD_DISK_MINOR_OFFSET (0)
+#define EMULATED_SD_DISK_NAME_OFFSET (EMULATED_SD_DISK_MINOR_OFFSET / 256)
 
 #define DEV_NAME	"xvd"	/* name in /dev */
 

Can you apply it the kernel and try it out?

(In reply to comment #7)
> 
> Can you apply it the kernel and try it out?

I'll do this shortly!

Konrad,

I can confirm, that the patch fixes the issue! I created a fresh AMI and booted it successfully.

thanks for testing. I'll add this for the next build.
I'll push out a 2.6.40.2 update as soon as GregKH drops a 3.0.2 release upstream.

thanks for your help chasing this down Konrad.

kernel-2.6.40.3-0.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/kernel-2.6.40.3-0.fc15

Package kernel-2.6.40.3-0.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing kernel-2.6.40.3-0.fc15'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/kernel-2.6.40.3-0.fc15
then log in and leave karma (feedback).

kernel-2.6.40.3-0.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.