Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1354335
Summary: | overlay of disk images does not specify the format of the backing file | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | Xianghua Chen <xchen> | ||||
Component: | libguestfs | Assignee: | Richard W.M. Jones <rjones> | ||||
Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | ||||
Severity: | medium | Docs Contact: | |||||
Priority: | medium | ||||||
Version: | 7.3 | CC: | berrange, dyuan, hhan, jferlan, jsuchane, lmen, ptoscano, pzhang, rbalakri, rjones, wshi, xuzhang, yafu, yanyang | ||||
Target Milestone: | rc | ||||||
Target Release: | --- | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | libguestfs-1.32.6-3.el7 | Doc Type: | If docs needed, set a value | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2016-11-03 18:02:16 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | |||||||
Bug Blocks: | 910269 | ||||||
Attachments: |
|
Reproducer (all commands run as root): # cd /var/tmp # virt-builder centos-7.2 # ll -Z centos-7.2.img -rw-r--r--. root root system_u:object_r:svirt_image_t:s0:c146,c485 centos-7.2.img # qemu-img create -f qcow2 -o backing_file=/var/tmp/centos-7.2.img,backing_fmt=raw overlay.qcow2 # ll -Z overlay.qcow2 -rw-r--r--. root root unconfined_u:object_r:user_tmp_t:s0 overlay.qcow2 # virt-inspector -a overlay.qcow2 libguestfs: error: could not create appliance through libvirt. Try running qemu directly without libvirt using this environment variable: export LIBGUESTFS_BACKEND=direct Original error from libvirt: internal error: process exited while connecting to monitor: 2016-07-11T08:42:06.554484Z qemu-kvm: -drive file=/tmp/libguestfsx18fpO/overlay1,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=unsafe: Could not open backing file: Could not open backing file: Could not open '/var/tmp/centos-7.2.img': Permission denied [code=1 int1=-1] # ll -Z overlay.qcow2 centos-7.2.img -rw-r--r--. root root system_u:object_r:svirt_image_t:s0:c146,c485 centos-7.2.img -rw-r--r--. qemu qemu system_u:object_r:virt_content_t:s0 overlay.qcow2 Libvirt XML generated by virt-inspector is below. Note that libguestfs generates a temporary overlay on top of the overlay, so the full chain of images is: /tmp/.../overlay1 -> /var/tmp/overlay.qcow2 -> /var/tmp/centos-7.2.img <?xml version="1.0"?> <domain type="kvm" xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0"> <name>guestfs-avt07jmbsov1y0sa</name> <memory unit="MiB">500</memory> <currentMemory unit="MiB">500</currentMemory> <cpu mode="host-passthrough"> <model fallback="allow"/> </cpu> <vcpu>1</vcpu> <clock offset="utc"> <timer name="rtc" tickpolicy="catchup"/> <timer name="pit" tickpolicy="delay"/> <timer name="hpet" present="no"/> </clock> <os> <type>hvm</type> <kernel>/var/tmp/.guestfs-0/appliance.d/kernel</kernel> <initrd>/var/tmp/.guestfs-0/appliance.d/initrd</initrd> <cmdline>panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color</cmdline> <bios useserial="yes"/> </os> <on_reboot>destroy</on_reboot> <devices> <controller type="scsi" index="0" model="virtio-scsi"/> <disk device="disk" type="file"> <source file="/tmp/libguestfslgut5O/overlay1"/> <target dev="sda" bus="scsi"/> <driver name="qemu" type="qcow2" cache="unsafe"/> <address type="drive" controller="0" bus="0" target="0" unit="0"/> </disk> <disk type="file" device="disk"> <source file="/tmp/libguestfslgut5O/overlay2"/> <target dev="sdb" bus="scsi"/> <driver name="qemu" type="qcow2" cache="unsafe"/> <address type="drive" controller="0" bus="0" target="1" unit="0"/> <shareable/> </disk> <serial type="unix"> <source mode="connect" path="/tmp/libguestfslgut5O/console.sock"/> <target port="0"/> </serial> <channel type="unix"> <source mode="connect" path="/tmp/libguestfslgut5O/guestfsd.sock"/> <target type="virtio" name="org.libguestfs.channel.0"/> </channel> </devices> <qemu:commandline> <qemu:env name="TMPDIR" value="/var/tmp"/> </qemu:commandline> </domain> I think i's not a svirt bug. The bug is caused by libguestfs generates a temporary overlay on top of the overlay without setting the backing_file format, which leads to libvirt even can not detect the base backing file. But the qemu can detect the whole backing-chain, so failed with permission denied when starting the guest. steps to reproduce with libvirt: 1.qemu-img create -f qcow2 test.img 1G Formatting 'test.img', fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 2.#qemu-img create -f qcow2 -o backing_file=test.img -F qcow2 test.s1 Formatting 'test.s1', fmt=qcow2 size=1073741824 backing_file=test.img backing_fmt=qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 3.#qemu-img create -f qcow2 -o backing_file=test.s1 test.s2 ------without setting the backing file format Formatting 'test.s2', fmt=qcow2 size=1073741824 backing_file=test.s1 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 4.The backing-chain is: test.s2->test.s1->test.img 5.Prepare a guest using test.s2: #virsh edit rhel7.3 ... <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/lib/libvirt/images/test/test.s2'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> ... 6.Start the guest: #virsh start rhel7.3 error: Failed to start domain mig1 error: internal error: qemu unexpectedly closed the monitor: 2016-07-21T10:12:59.392715Z qemu-kvm: -drive file=/var/lib/libvirt/images/test/test.s2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=none: Could not open backing file: Could not open backing file: Could not open '/var/lib/libvirt/images/test/test.img': Permission denied 7.Check the libvirtd.log, find the detect the format of test.s1 is wrong: #cat /var/log/libvirt/libvirtd.log ... 2016-07-21 10:04:46.209+0000: 31270: debug : virStorageFileGetMetadata:3318 : path=/var/lib/libvirt/images/test/test.s2 format=15 uid=107 gid=107 probe=0, report_broken=1 2016-07-21 10:04:46.209+0000: 31270: debug : virStorageFileGetMetadataRecurse:3201 : path=/var/lib/libvirt/images/test/test.s2 format=15 uid=107 gid=107 probe=0 2016-07-21 10:04:46.209+0000: 31270: debug : virStorageFileBackendFileInit:1497 : initializing FS storage file 0x7f86bc01ad10 (file:/var/lib/libvirt/images/test/test.s2)[107:107] 2016-07-21 10:04:46.211+0000: 31270: debug : virStorageFileGetMetadataInternal:814 : path=/var/lib/libvirt/images/test/test.s2, buf=0x7f86dc001e70, len=33280, *******meta->format=15********* ----------test.s2's format is qcow2 ... 2016-07-21 10:04:46.211+0000: 31270: debug : virStorageFileGetMetadataRecurse:3201 : path=/var/lib/libvirt/images/test/test.s1 format=1 uid=107 gid=107 probe=0 2016-07-21 10:04:46.211+0000: 31270: debug : virStorageFileBackendFileInit:1497 : initializing FS storage file 0x7f86dc016ca0 (file:/var/lib/libvirt/images/test/test.s1)[107:107] 2016-07-21 10:04:46.212+0000: 31270: debug : virStorageFileGetMetadataInternal:814 : path=/var/lib/libvirt/images/test/test.s1, buf=0x7f86dc01ca60, len=33280, ******meta->format=1******* ----------test.s1's format is raw The log shows libvirt get test.s1's format as raw, so it can not get the correct info in the test.s1's header file. Hi, jferlan, Since libvirt does not detect images format for the reason of security in default, would you help to check whether it is the expected results with libvirt please? (In reply to yafu from comment #3) > I think i's not a svirt bug. The bug is caused by libguestfs generates a > temporary overlay on top of the overlay without setting the backing_file > format, which leads to libvirt even can not detect the base backing file. > But the qemu can detect the whole backing-chain, so failed with permission > denied when starting the guest. Indeed, this is even captured with the attached log file: libguestfs: trace: disk_create "/tmp/libguestfsBW52iQ/overlay1" "qcow2" -1 "backingfile:/run/guestfs-autotest/images/tmp.qcow2" libguestfs: command: run: qemu-img libguestfs: command: run: \ create libguestfs: command: run: \ -f qcow2 libguestfs: command: run: \ -o backing_file=/run/guestfs-autotest/images/tmp.qcow2 libguestfs: command: run: \ /tmp/libguestfsBW52iQ/overlay1 Formatting '/tmp/libguestfsBW52iQ/overlay1', fmt=qcow2 size=8589934592 backing_file=/run/guestfs-autotest/images/tmp.qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 > steps to reproduce with libvirt: [SNIP] > 7.Check the libvirtd.log, find the detect the format of test.s1 is wrong: > #cat /var/log/libvirt/libvirtd.log > ... > 2016-07-21 10:04:46.209+0000: 31270: debug : > virStorageFileGetMetadata:3318 : path=/var/lib/libvirt/images/test/test.s2 > format=15 uid=107 gid=107 probe=0, report_broken=1 > 2016-07-21 10:04:46.209+0000: 31270: debug : > virStorageFileGetMetadataRecurse:3201 : > path=/var/lib/libvirt/images/test/test.s2 format=15 uid=107 gid=107 probe=0 > 2016-07-21 10:04:46.209+0000: 31270: debug : > virStorageFileBackendFileInit:1497 : initializing FS storage file > 0x7f86bc01ad10 (file:/var/lib/libvirt/images/test/test.s2)[107:107] > 2016-07-21 10:04:46.211+0000: 31270: debug : > virStorageFileGetMetadataInternal:814 : > path=/var/lib/libvirt/images/test/test.s2, buf=0x7f86dc001e70, len=33280, > *******meta->format=15********* ----------test.s2's format is qcow2 > ... > 2016-07-21 10:04:46.211+0000: 31270: debug : > virStorageFileGetMetadataRecurse:3201 : > path=/var/lib/libvirt/images/test/test.s1 format=1 uid=107 gid=107 probe=0 > 2016-07-21 10:04:46.211+0000: 31270: debug : > virStorageFileBackendFileInit:1497 : initializing FS storage file > 0x7f86dc016ca0 (file:/var/lib/libvirt/images/test/test.s1)[107:107] > 2016-07-21 10:04:46.212+0000: 31270: debug : > virStorageFileGetMetadataInternal:814 : > path=/var/lib/libvirt/images/test/test.s1, buf=0x7f86dc01ca60, len=33280, > ******meta->format=1******* ----------test.s1's format is raw > > The log shows libvirt get test.s1's format as raw, so it can not get the > correct info in the test.s1's header file. > > > Hi, jferlan, Since libvirt does not detect images format for the reason of > security in default, would you help to check whether it is the expected > results with libvirt please? Yes. It's deliberate that libvirt does not perform (backing) image type detection. This would create a security hole by allowing guests to overwrite a raw image with a qcow2 header and thus tricking the host to making an arbitrary host file accessible. In the default configuration of libvirt any backing file that does not have the format specified explicitly is treated as raw. Assigning back to libguestfs since the behavior in libvirt is deliberate. (In reply to Peter Krempa from comment #4) > (In reply to yafu from comment #3) > > I think i's not a svirt bug. The bug is caused by libguestfs generates a > > temporary overlay on top of the overlay without setting the backing_file > > format, which leads to libvirt even can not detect the base backing file. > > But the qemu can detect the whole backing-chain, so failed with permission > > denied when starting the guest. > > Indeed, this is even captured with the attached log file: > > libguestfs: trace: disk_create "/tmp/libguestfsBW52iQ/overlay1" "qcow2" -1 > "backingfile:/run/guestfs-autotest/images/tmp.qcow2" > libguestfs: command: run: qemu-img > libguestfs: command: run: \ create > libguestfs: command: run: \ -f qcow2 > libguestfs: command: run: \ -o > backing_file=/run/guestfs-autotest/images/tmp.qcow2 > libguestfs: command: run: \ /tmp/libguestfsBW52iQ/overlay1 > Formatting '/tmp/libguestfsBW52iQ/overlay1', fmt=qcow2 size=8589934592 > backing_file=/run/guestfs-autotest/images/tmp.qcow2 encryption=off > cluster_size=65536 lazy_refcounts=off refcount_bits=16 This is deliberate. Unless the user specifies the source file format, we don't set the backing_fmt for the overlay. IOW we (or rather, the user) desires autodetection in this case. ... I don't understand your explanation for why libvirt doesn't do autodetection, because by the same argument qemu would be wrong to do autodetection, yet qemu is doing autodetection. Anyway I will just say that ... > Yes. It's deliberate that libvirt does not perform (backing) image type > detection. This would create a security hole by allowing guests to overwrite > a raw image with a qcow2 header and thus tricking the host to making an > arbitrary host file accessible. ... this part is wrong. Since Nov 2014, qemu has detected this situation and prevents writes to the first sector of the disk if the format is raw and the disk format was probed. (See qemu commit 38f3ef574b48afc507c6f636ae4393fd36bda072). (In reply to Richard W.M. Jones from comment #5) > I don't understand your explanation for why libvirt doesn't do autodetection, > because by the same argument qemu would be wrong to do autodetection, yet > qemu is doing autodetection. Anyway I will just say that ... QEMU only does auto-detection by default because it needs to avoid breaking historical usage of QEMU. If QEMU didn't care about historic compat, it would not auto-detect at all. > > Yes. It's deliberate that libvirt does not perform (backing) image type > > detection. This would create a security hole by allowing guests to overwrite > > a raw image with a qcow2 header and thus tricking the host to making an > > arbitrary host file accessible. > > ... this part is wrong. Since Nov 2014, qemu has detected this > situation and prevents writes to the first sector of the disk if > the format is raw and the disk format was probed. > (See qemu commit 38f3ef574b48afc507c6f636ae4393fd36bda072). That commit was added as a safety net for applications/users who accidentally still invoke QEMU in a way which triggers auto-detection. That doesn't imply that auto-detection is a good/valid thing to permit / enable when launching. Any usage of QEMU should none the less always strive to *never* use auto-detection, which is what libvirt does. IOW just because QEMU added a safety net doesn't mean libvirt should now allow auto-detection. Rich, based on comment 4 and comment 6 this can't be fixed in libvirt and would be a resolution for this. Is it possible to fix it in libguestfs? I posted a workaround for this: https://www.redhat.com/archives/libguestfs/2016-August/msg00004.html Upstream commit c8c181e8d9729ea09e56431b568dceba3873af7b. Verified with the packages: libguestfs-1.32.6-4.el7.x86_64 virt-dib-1.32.6-4.el7.x86_64 Verify steps: 1. # virt-builder centos-7.2 2. # qemu-img create -f qcow2 -o backing_file=centos-7.2.img,backing_fmt=raw overlay.qcow2 Check the selinux context: # ll -Z overlay.qcow2 centos-7.2.img -rw-r--r--. root root unconfined_u:object_r:user_home_t:s0 centos-7.2.img -rw-r--r--. root root unconfined_u:object_r:home_root_t:s0 overlay.qcow2 4. # virt-inspector -a overlay.qcow2 5. Check the selinux context: # ll -Z overlay.qcow2 centos-7.2.img -rw-r--r--. root root system_u:object_r:virt_content_t:s0 centos-7.2.img -rw-r--r--. root root system_u:object_r:virt_content_t:s0 overlay.qcow2 Command in step 4 finished successfully. So verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2576.html |
Created attachment 1178262 [details] log.virt-inspector-backing_file Description of problem: virt-inspector failed to inspect new image created by qemu-img --backing-file if the selinux context of original file is not virt_content_t. Please refer to full log in attachment. Version-Release number of selected component (if applicable): libguestfs-1.32.5-9.el7.x86_64 qemu-kvm-1.5.3-116.el7.x86_64 qemu-img-1.5.3-116.el7.x86_64 libvirt-2.0.0-1.el7.x86_64 How reproducible: Always Steps to Reproduce: 1. Copy a guest image from nfs server # cp /tmp/images/RHEL-Server-6.8-32-hvm.raw images/RHEL-Server-6.8-32-hvm.raw # ll -Z images/RHEL-Server-6.8-32-hvm.raw -rw-r--r--. root root unconfined_u:object_r:home_root_t:s0 images/RHEL-Server-6.8-32-hvm.raw 2. # qemu-img create -f qcow2 -o backing_file=/home/git/libguestfs.autotest/client/tests/libguestfs/images/RHEL-Server-6.8-32-hvm.raw,backing_fmt=raw /var/run/guestfs-autotest/images/tmp.qcow2 # ll -Z images/tmp.qcow2 -rw-r--r--. root root unconfined_u:object_r:home_root_t:s0 images/tmp.qcow2 3. # virt-inspector -a images/tmp.qcow2 libguestfs: error: could not create appliance through libvirt. Try running qemu directly without libvirt using this environment variable: export LIBGUESTFS_BACKEND=direct Original error from libvirt: internal error: process exited while connecting to monitor: 2016-07-08T08:13:56.786262Z qemu-kvm: -drive file=/tmp/libguestfsDzDycQ/overlay1,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=unsafe: Could not open backing file: Could not open backing file: Could not open '/home/git/libguestfs.autotest/client/tests/libguestfs/images/RHEL-Server-6.8-32-hvm.raw': Permission denied [code=1 int1=-1] 4. If I have executed libguestfs command first on original guest image before step 3: # virt-inspector -a images/RHEL-Server-6.8-32-hvm.raw # ll -Z images/RHEL-Server-6.8-32-hvm.raw -rw-r--r--. qemu qemu system_u:object_r:virt_content_t:s0 images/RHEL-Server-6.8-32-hvm.raw Ps.: Or just use chcon to instead executing libguestfs command: # chcon -t virt_content_t RHEL-Server-6.8-32-hvm.raw Then execute the command in step 3, it will success. Actual results: As above output. Expected results: Command in step 3 should be finished successfully. Additional info: