+++ This bug was initially created as a clone of Bug #1798148 +++

Description of problem:

$ qemu-img create -f qcow2 test.qcow2 1M
$ qemu-img create -f qcow2 overlay.qcow2 -b test.qcow2 
$ qemu-img info overlay.qcow2 
image: overlay.qcow2
file format: qcow2
virtual size: 1 MiB (1048576 bytes)
disk size: 196 KiB
cluster_size: 65536
backing file: test.qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

But if you add overlay.qcow2 to a guest then you can no longer
start the guest:

$ virt-install --import --name test --disk path=overlay.qcw2,format=qcow2 

ERROR    Requested operation is not valid: format of backing image 'test.qcow2' of image '/var/tmp/overlay.qcow2' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting)

Note this is a regression over earlier behaviour, and also breaks
libguestfs tools such as virt-sparsify.

Version-Release number of selected component (if applicable):

libvirt-6.0.0-2.module+el8.2.0+5513+34927b6c.x86_64

How reproducible:

100%

Steps to Reproduce:
1. See above.

--- Additional comment from Rick Barry on 2020-02-04 19:32:45 UTC ---

Rich, if you plan to take this BZ you can either reassign it to yourself or assign it back to virt-maint and add the Triaged flag after you've determined the priority and set the ITR the way you want.

--- Additional comment from Peter Krempa on 2020-02-05 13:25:47 UTC ---

Note that the above scenario relied on format detection which was considered unsupported and insecure for a very long time (at least since 2012, I didn't look further).

Since libvirt wasn't able to tell qemu how to open the image until -blockdev was introduced the only recourse was to use selinux to prevent opening any unwanted backing images.

That means that for a very long time, any image which didn't have a format specified was considered to be raw from libvirt's point of view. This 'accidentally' worked in some cases, e.g. when the image didn't have any backing image. Saying 'accidentally' because we didn't have a way to fix it. Additionally in cases when selinux or any other security driver was not available, the insecure format probing could still be exploited.

Since -blockdev support was introduced it became possible to control the backing files of images which libvirt started to do.

In that case it turned out, that actually assuming that a non-specified format equals to RAW which we did for a long time is dangerous as it can lead to data corruption. To avoid any possibility of it we started reporting the error you mention in this bug.

The following bugs tracked some of the problems which occured before:

https://bugzilla.redhat.com/show_bug.cgi?id=1588373
If the image which didn't have the format specified has more than one backing image, selinux will not label them properly and startup will fail.

https://bugzilla.redhat.com/show_bug.cgi?id=1790101
Not specifying the format makes the image look corrupt as the backing image was QCOW2 and libvirt assumed root.

You can also see the questions asked on the upstream list:
https://www.redhat.com/archives/libvir-list/2020-January/msg00320.html

As of the reasons above we never wanted to allow the image format probing done by qemu in the first place but that became possible only using blockdev. As of such the behaviour we have now is expected as we want to require users to certify the image format one way or another.

Note that in addition to recording the format of the image in the overlay file when creating it:

qemu-img create -f qcow2 -F qcow2 -b /path/to/backing.qcow2 /path/to/overlay.qcow2)

(-F specifies the backing file format)

It's now also possible to express the format in the VM XML as well as the fact that backing.qcow2 has no backing store:

    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/path/to/overlay.qcow2'/>
      <backingStore type='file'>
        <format type='qcow2'/>
        <source file='/path/to/backing.qcow2'/>
        <backingStore/>
      </backingStore>
      <target dev='vdb' bus='virtio'/>
    </disk>

(note that you can also assume that if libvirt doesn't support the <backing> store on input, which can be also queried via the domcapabilities api [1] that the elements are ignored and the backing chain is probed)

Any new libvirt will thus reject any probed image which doesn't have the backing store specified and you can use one of the two approaches outlined above to avoid that issue.

[1] https://www.redhat.com/archives/libvirt-users/2020-January/msg00081.html
    https://libvirt.org/formatdomaincaps.html#featureBackingStoreInput

--- Additional comment from Richard W.M. Jones on 2020-02-06 11:06:01 UTC ---

This change breaks at least virt-customize and virt-sparsify.  I am still
analysing the extent of the problems and how easy it will be to fix.
I don't think that making this change in the middle of RHEL 8 is a good idea,
I think this will negatively affect a lot of customers.

--- Additional comment from Richard W.M. Jones on 2020-02-06 17:21:02 UTC ---

Easiest workaround seems to be to do the autodetection in libguestfs, so:
https://www.redhat.com/archives/libguestfs/2020-February/msg00013.html