Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1447590

Summary: qemu curl driver hangs in a particular libguestfs file download
Product: Red Hat Enterprise Linux 7 Reporter: kuwei <kuwei>
Component: qemu-kvm-rhevAssignee: Paolo Bonzini <pbonzini>
Status: CLOSED ERRATA QA Contact: Suqin Huang <shuang>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.4CC: chayang, coli, juzhang, juzhou, knoel, michen, mrezanin, mxie, mzhan, pbonzini, ptoscano, rjones, tzheng, virt-maint, xiaodwan
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: V2V
Fixed In Version: qemu-kvm-rhev-2.9.0-6.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-02 04:38:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 910269, 1437393    
Attachments:
Description Flags
v2v.log
none
v2v.log for one hour none

Description kuwei@redhat.com 2017-05-03 09:23:21 UTC
Created attachment 1275853 [details]
v2v.log

Description of problem:
Failed to convert guest from vmware to kvm by virt-v2v with qemu-img-rhev-2.9.0-2.el7.x86_64

Version-Release number of selected component (if applicable):
qemu-img-rhev-2.9.0-2.el7.x86_64
libvirt-3.2.0-3.el7.x86_64
virt-v2v-1.36.3-3.el7.x86_64
libguestfs-1.36.3-3.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1:Convert a rhel7.3 guest from vmware to kvm,details pls refer to attachment v2vlog 
#virt-v2v -ic vpx://root.75.182/data/10.73.3.19/?no_verify=1 esx5.5-rhel7.3-x86_64  --password-file /tmp/passwd -on test
[   0.0] Opening the source -i libvirt -ic vpx://root.75.182/data/10.73.3.19/?no_verify=1 esx5.5-rhel7.3-x86_64
[   1.3] Creating an overlay to protect the source from being modified
[   1.7] Initializing the target -o libvirt -os default
[   1.7] Opening the overlay
^c
 
Result:  the above conversion  can't be finished 

Addition info
1.It is works fine below packages it:
qemu-kvm-rhev-2.8.0-6.el7.x86_64
libvirt-3.2.0-3.el7.x86_64
virt-v2v-1.36.3-3.el7.x86_64
libguestfs-1.36.3-3.el7.x86_64 

2.A same bug i think :Bug 1430287

Comment 3 kuwei@redhat.com 2017-05-03 10:49:53 UTC
Created attachment 1275872 [details]
v2v.log for one hour

Comment 4 Richard W.M. Jones 2017-05-03 13:34:12 UTC
I am able to reproduce this with:

* qemu-kvm-rhev-2.9.0-2.el7.x86_64
* Upstream qemu (e619b14746e5d)

Does NOT reproduce with:

* qemu-kvm-rhev-2.8.0-6.el7.x86_64

Also (as kuwei also reported to me on IRC) the file is stuck at 8K in
the download.

So it's an actual bug in upstream qemu.

I'm going to bisect this to see if I can isolate a particular commit.

Comment 5 Richard W.M. Jones 2017-05-03 14:54:38 UTC
ba3186c4e473963ba83b5792f3d02d4ac0a76ba5 is the first bad commit
commit ba3186c4e473963ba83b5792f3d02d4ac0a76ba5
Author: Paolo Bonzini <pbonzini>
Date:   Wed Feb 22 19:07:23 2017 +0100

    curl: do not use aio_context_acquire/release
    
    Now that all bottom halves and callbacks take care of taking the
    AioContext lock, we can migrate some users away from it and to a
    specific QemuMutex or CoMutex.
    
    Protect BDRVCURLState access with a QemuMutex.
    
    Reviewed-by: Stefan Hajnoczi <stefanha>
    Signed-off-by: Paolo Bonzini <pbonzini>
    Message-id: 20170222180725.28611-2-pbonzini
    Signed-off-by: Stefan Hajnoczi <stefanha>

:040000 040000 1a66f1866953d188bdd51318f470c43f6aedf06e d0f24bf0226fa8cc5d8311d74c5542fa225638ea M	block

Reverting this commit fixes the problem.

Comment 6 Richard W.M. Jones 2017-05-03 15:43:00 UTC
Started a discussion upstream:

https://lists.nongnu.org/archive/html/qemu-devel/2017-05/msg00580.html

As this is a clear regression, I'm marking it as such.

Comment 8 Richard W.M. Jones 2017-05-03 16:03:28 UTC
Paolo posted a fix which worked for me:

https://lists.nongnu.org/archive/html/qemu-devel/2017-05/msg00596.html

Comment 9 Richard W.M. Jones 2017-05-10 14:57:13 UTC
There have been several iterations of this patch.  The latest
I am testing is:

http://lists.nongnu.org/archive/html/qemu-devel/2017-05/msg02432.html

Comment 10 Paolo Bonzini 2017-05-10 16:03:03 UTC
Patches posted:
[PATCH v2 0/7] curl: locking cleanups/fixes, coroutine conversion, remove aio_poll
Message-Id: <20170510143205.32013-1-pbonzini>

Comment 14 Suqin Huang 2017-06-07 04:22:07 UTC
Result: Pass

# virt-v2v -ic vpx://root.75.182/data/10.73.3.19/?no_verify=1 esx5.5-rhel7.3-x86_64  --password-file /tmp/passwd -on shuang
[   0.0] Opening the source -i libvirt -ic vpx://root.75.182/data/10.73.3.19/?no_verify=1 esx5.5-rhel7.3-x86_64
[   1.2] Creating an overlay to protect the source from being modified
[   1.6] Initializing the target -o libvirt -os default
[   1.6] Opening the overlay
[  26.6] Inspecting the overlay
[ 179.2] Checking for sufficient free disk space in the guest
[ 179.2] Estimating space required on target for each disk
[ 179.2] Converting Red Hat Enterprise Linux Server 7.3 (Maipo) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[1615.0] Mapping filesystem data to avoid copying unused and blank areas
[1641.0] Closing the overlay
[1641.2] Checking if the guest needs BIOS or UEFI to boot
[1641.2] Assigning disks to buses
[1641.2] Copying disk 1/1 to /var/lib/libvirt/images/shuang-sda (raw)
    (100.00/100%)
[2316.9] Creating output metadata
Pool default refreshed

Domain shuang defined from /tmp/v2vlibvirt7f19ad.xml


Package: qemu-img-rhev-2.9.0-8.el7.x86_64.rpm

Comment 15 Suqin Huang 2017-06-07 04:25:34 UTC
According to comment14, update bug status to VERIFIED

Comment 17 errata-xmlrpc 2017-08-02 04:38:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392