Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1738721 - qemu-img convert silently ignores -o when used with -n
Summary: qemu-img convert silently ignores -o when used with -n
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: qemu
Version: 31
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Fedora Virtualization Maintainers
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1592916
TreeView+ depends on / blocked
 
Reported: 2019-08-07 23:12 UTC by Nir Soffer
Modified: 2019-09-04 10:50 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-09-04 10:50:34 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)

Description Nir Soffer 2019-08-07 23:12:13 UTC 
Description of problem

If using the -n option (do not create target file) using

    -o preallocation=falloc

has no effect.

Example flow:

1. Create target file with first block allocated. This is needed for supporting
   Glsuter storage with 4k sector size, see bug 1738657. If we let qemu create
   the image it will fail to detect the target image block size and fail to 
   write to the image during conversion.

$ dd if=/dev/zero bs=4096 count=1 of=dst
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00041392 s, 9.9 MB/s

$ truncate -s 1g dst

$ truncate -s 1g src

2. Convert src image to dst

$ qemu-img convert -n -f raw -O raw -o preallocation=falloc src dst

$ ls -lhs
total 4.0K
4.0K -rw-rw-r--. 1 nsoffer nsoffer 1.0G Aug  8 02:01 dst
   0 -rw-rw-r--. 1 nsoffer nsoffer 1.0G Aug  8 01:57 src

The destination image was not allocated.

If we let qemu create the target image, it is allocated:

$ qemu-img convert -f raw -O raw -o preallocation=falloc src dst

$ ls -lhs
total 1.1G
1.1G -rw-rw-r--. 1 nsoffer nsoffer 1.0G Aug  8 02:09 dst
   0 -rw-rw-r--. 1 nsoffer nsoffer 1.0G Aug  8 01:57 src

This behavior is not documented and is not useful for users. This blocks 4k
support in oVirt.

Version-Release number of selected component (if applicable):
qemu-img-4.1.0-0.1.rc2.fc29.x86_64 (from virt-preview)

How reproducible:
Always
Comment 1 Nir Soffer 2019-08-07 23:41:51 UTC 
A possible workaround is to allocate the target image before converting
using fallocate (or oVirt fallocate helper).

    fallocate --posix -l 1g dst
    qemu-img convert -n ...

But, if the target filesystem does not support fallocate() (.e.g NFS < 4.2)
fallocate --posix will fall back to manual allocation which is extremely slow.
If the file is mostly allocated, converting may be up to 2x slower, since we 
must pay for allocation twice; first when preallocating the entire image, and
then when writing data.

If qemu implements preallocation, we pay only once for the data, and once for
the unallocated areas.

I think we can managed allocation outside of qemu-img for now, but this will
be a performance regression in oVirt for copying to preallocated disks on 
legacy filesystems (NFS < 4.2).
Comment 2 Nir Soffer 2019-08-07 23:52:50 UTC 
The issue affects only raw images:

$ qemu-img create -f qcow2 dst 1g
Formatting 'dst', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16

$ qemu-img convert -n -f raw -O qcow2 -o preallocation=metadata src dst

$ ls -lhs
total 196K
196K -rw-rw-r--. 1 nsoffer nsoffer 193K Aug  8 02:49 dst
   0 -rw-rw-r--. 1 nsoffer nsoffer 1.0G Aug  8 01:57 src

$ qemu-img create -f qcow2 dst 1g
Formatting 'dst', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16

$ qemu-img convert -f raw -O qcow2 -o preallocation=metadata src dst

$ ls -lhs
total 324K
324K -rw-rw-r--. 1 nsoffer nsoffer 1.1G Aug  8 02:50 dst
   0 -rw-rw-r--. 1 nsoffer nsoffer 1.0G Aug  8 01:57 src
Comment 3 Nir Soffer 2019-08-07 23:58:21 UTC 
Oops, comment 2 show that it is broken also for qcow2 format.

This cannot be fixed by allocated outside of qemu-img.
oVirt does not use metadata preallocation yet for qcow2 images.
Comment 4 Kevin Wolf 2019-08-09 10:12:41 UTC 
This is in fact expected behaviour, for the most past. -o specified options for image creation, and -n skips image creation, so -o never takes effect. The part that is wrong in qemu-img is that it doesn't error out when you use both -n and -o. I've sent patch upstream to print a warning and add a deprecation notice so we can make this a hard error later.

Obviously, this doesn't help your use case. So let me suggest two solutions for your actual problem:

1. For qcow2 (and actually any non-raw image), the workaround with -n isn't needed at all because the image header will always be allocated. The problem you want to work around only exists with raw images.

2. For raw images, create the file first with 'qemu-img create -o preallocation=falloc ...', and only then allocate the first sector either with 'qemu-io -c "write -P0 0 4k"' or 'dd conv=notrunc'. Afterwards, you can use 'qemu-img convert -n' without using '-o' at the same time.
Comment 5 Ben Cotton 2019-08-13 17:19:22 UTC 
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.
Comment 6 Kevin Wolf 2019-09-04 10:19:49 UTC 
Upstream commit ffd8e8ffd5 makes qemu-img print a warning to make the behaviour more obvious.
Comment 7 Nir Soffer 2019-09-04 10:50:34 UTC 
Since this is expected behavior I think we can close the bug.
Note You need to log in before you can comment on or make changes to this bug.