Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1043249
Summary: | Libguestfs fails to create appliance (through 'libvirt' backend): cannot open disk image from /tmp | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | [Community] Virtualization Tools | Reporter: | Kashyap Chamarthy <kchamart> | ||||||
Component: | libvirt | Assignee: | Libvirt Maintainers <libvirt-maint> | ||||||
Status: | CLOSED DUPLICATE | QA Contact: | |||||||
Severity: | unspecified | Docs Contact: | |||||||
Priority: | unspecified | ||||||||
Version: | unspecified | CC: | berrange, crobinso, kchamart, mbooth, ptoscano, rbalakri, rjones | ||||||
Target Milestone: | --- | ||||||||
Target Release: | --- | ||||||||
Hardware: | Unspecified | ||||||||
OS: | Unspecified | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | Doc Type: | Bug Fix | |||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | |||||||||
: | 1045039 (view as bug list) | Environment: | |||||||
Last Closed: | 2016-04-19 17:45:31 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Bug Depends On: | |||||||||
Bug Blocks: | 910269, 1045039 | ||||||||
Attachments: |
|
Description
Kashyap Chamarthy
2013-12-15 12:23:32 UTC
There should be a file: ~/.cache/libvirt/qemu/log/guestfs-pamlcgfvpwkqi1zh.log Can you attach it here as it will probably have more information. That said, I believe this is going to be a regular permissions problem, although the error message printed by libvirt/qemu could certainly be better. It's not SELinux, but is it possible that root simply can't open the disk image because of something to do with the media (VFAT?) or where it is mounted? You could try running (as root): md5sum /run/media/kashyap/2bad4c53-36d1-4f61-b89f-018011ee5912/ostack-images/ostack-compute.qcow2 (In reply to Richard W.M. Jones from comment #1) > There should be a file: > > ~/.cache/libvirt/qemu/log/guestfs-pamlcgfvpwkqi1zh.log I don't see such a file. I retried, but still I don't see the cache log: ==== kashyap@~$ export LIBGUESTFS_BACKEND=libvirt kashyap@~$ export LIBGUESTFS_DEBUG=1 kashyap@~$ export LIBGUESTFS_TRACE=1 kashyap@~$ sudo virt-df -a /run/media/kashyap/2bad4c53-36d1-4f61-b89f-018011ee5912/ostack-images/ostack-compute.qcow2 -h libguestfs: error: could not create appliance through libvirt: internal error: process exited while connecting to monitor: qemu-system-x86_64: -drive file=/tmp/libguestfsTYoexl/snapshot2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=writeback: could not open disk image /tmp/libguestfsTYoexl/snapshot2: Permission denied [code=1 domain=10] kashyap@~$ ls ~/.cache/libvirt/qemu/log kashyap@~$ ==== Re-ran virt-df as just user, the log file is present: ====== kashyap@~$ cat ~/.cache/libvirt/qemu/log/guestfs-i8blgo23hpd03kaj.log 2013-12-16 10:52:30.313+0000: starting up LC_ALL=C PATH=/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/kashyap/.local/bin:/home/kashyap/bin HOME=/home/kashyap USER=kashyap LOGNAME=kashyap QEMU_AUDIO_DRV=none TMPDIR=/var/tmp /usr/bin/qemu-kvm -name guestfs-i8blgo23hpd03kaj -S -machine pc-i440fx-1.6,accel=kvm,usb=off -cpu host,+kvmclock -m 500 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 9713bc40-4724-4670-a2e3-c814fd5bf777 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/home/kashyap/.config/libvirt/qemu/lib/guestfs-i8blgo23hpd03kaj.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot -no-acpi -kernel /var/tmp/.guestfs-1000/kernel.9033 -initrd /var/tmp/.guestfs-1000/initrd.9033 -append panic=1 console=ttyS0 udevtimeout=600 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/tmp/libguestfss0Taph/snapshot2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=writeback -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/tmp/libguestfss0Taph/snapshot1,if=none,id=drive-scsi0-0-1-0,format=qcow2,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0 -chardev socket,id=charserial0,path=/tmp/libguestfss0Taph/console.sock -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/tmp/libguestfss0Taph/guestfsd.sock -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 Domain id=2 is tainted: custom-argv Domain id=2 is tainted: host-cpu qemu: terminating on signal 15 from pid 9036 2013-12-16 10:52:34.591+0000: shutting down ====== > > Can you attach it here as it will probably have more > information. > > That said, I believe this is going to be a regular permissions > problem, although the error message printed by libvirt/qemu could > certainly be better. Agreed. > It's not SELinux, but is it possible that > root simply can't open the disk image because of something to > do with the media (VFAT?) No, it's ext4. > or where it is mounted? You could try > running (as root): > > md5sum > /run/media/kashyap/2bad4c53-36d1-4f61-b89f-018011ee5912/ostack-images/ostack- > compute.qcow2 $ root@~$ md5sum /run/media/kashyap/2bad4c53-36d1-4f61-b89f-018011ee5912/ostack-images/ostack-compute.qcow2 812f043f78fdafdba4a1705e98eb1954 /run/media/kashyap/2bad4c53-36d1-4f61-b89f-018011ee5912/ostack-images/ostack-compute.qcow2 Created attachment 837215 [details]
guestfs-pamlcgfvpwkqi1zh.log - from /var/log/libvirt/qemu
Actually, I was forgetting something. When libvirt runs as root, it will run qemu as user qemu.qemu, so if qemu.qemu cannot access the disk then you'll have a problem. You could try: su - qemu md5sum /run/media/kashyap/2bad4c53-36d1-4f61-b89f-018011ee5912/ostack-images/ostack-compute.qcow2 or just examine the permissions on the file *and every parent directory* to see if qemu would be allowed to open that file. (In reply to Richard W.M. Jones from comment #4) > Actually, I was forgetting something. When libvirt runs as root, > it will run qemu as user qemu.qemu, so if qemu.qemu cannot access > the disk then you'll have a problem. You could try: > > su - qemu > md5sum > /run/media/kashyap/2bad4c53-36d1-4f61-b89f-018011ee5912/ostack-images/ostack- > compute.qcow2 > > or just examine the permissions on the file *and every > parent directory* to see if qemu would be allowed to open > that file. Right - the disk image itself has qemu:qemu on the media: $ ls -lash ostack-compute.qcow2 11G -rw-r--r--. 1 qemu qemu 15G Dec 13 11:12 ostack-compute.qcow2 Parent dir: $ ls -lash /run/media/kashyap/2bad4c53-36d1-4f61-b89f-018011ee5912/ \ | grep ostack-images 4.0K drwxr-xr-x. 3 kashyap kashyap 4.0K Dec 15 20:14 ostack-images The media itself: $ ls -lash /run/media/kashyap/ total 4.0K 0 drwxr-x---+ 3 root root 60 Dec 16 09:52 . 0 drwxr-xr-x. 3 root root 60 Dec 16 09:34 .. 4.0K drwxr-xr-x. 13 500 500 4.0K Dec 14 22:10 2bad4c53-36d1-4f61- b89f-018011ee5912 Feel free to close this as permissions issue. Dan: At the moment libvirt runs qemu as qemu.qemu (or a setting chosen from /etc/libvirt/qemu.conf), if run as root. I'm not disputing that this is the best thing to do for regular virtual machines. However it's somewhat unexpected from a libguestfs point of view, since in normal (non-root) cases qemu runs as the user, and people often run libguestfs / virt tools as root as a "just do it" toggle. I have in fact run into this issue myself, and have forgetten about it too. Added to this, qemu upstream now produces better error messages. However nothing in the error says that qemu is running as a non-root user. So, I think it should be possible to override the user/group setuid stuff programmatically, not just through a config file. Or perhaps we can tell libvirt to ignore the config file? What do you think? You should already be able to override this with a security label <seclabel type='static' model='dac' relabel='no'> <label>root:qemu</label> </seclabel> IIRC you have to have either the user or group remain as 'qemu' but the other can be changed to say 'root'. Another instance of it with 'libvirt' backend: $ virt-builder fedora-20 [ 2.0] Downloading: http://libguestfs.org/download/builder/fedora-20.xz ####################################################################### 100.0% [ 813.0] Creating disk image: fedora-20.img [ 813.0] Uncompressing: http://libguestfs.org/download/builder/fedora-20.xz [ 828.0] Opening the new disk virt-builder: libguestfs error: could not create appliance through libvirt. Try using the direct backend to run qemu directly without libvirt, by setting the LIBGUESTFS_BACKEND=direct environment variable.: internal error: process exited while connecting to monitor: qemu-system-x86_64: -drive file=/root/fedora-20.img,if=none,id=drive-scsi0-0-0-0,format=raw,cache=unsafe: could not open disk image /root/fedora-20.img: Could not open '/root/f edora-20.img': Permission denied [code=1 domain=10] $ getenforce Permissive $ rpm -q libguestfs-tools-c libguestfs-tools-c-1.24.2-1.fc20.x86_64 $ ls -lash ~/.cache/virt-builder/ total 175M 4.0K drwxr-xr-x. 2 root root 4.0K Dec 19 15:52 . 4.0K dr-xr-x---. 5 root root 4.0K Dec 19 15:38 .. 175M -rw-r--r--. 1 root root 175M Dec 19 15:52 fedora-20.1 With direct QEMU appliance, works fine: $ export LIBGUESTFS_BACKEND=direct $ virt-builder fedora-20 [ 1.0] Downloading: http://libguestfs.org/download/builder/fedora-20.xz [ 2.0] Creating disk image: fedora-20.img [ 3.0] Uncompressing: http://libguestfs.org/download/builder/fedora-20.xz [ 16.0] Opening the new disk [ 25.0] Setting a random seed [ 25.0] Random root password: bRijv0QzPEGcNfWf [did you mean to use --root-password?] [ 25.0] Finishing off Output: fedora-20.img Total usable space: 5.2G Free space: 4.5G (86%) Created attachment 838980 [details]
0001-launch-libvirt-Run-qemu-as-root-when-libguestfs-runs.patch
I tried the attached patch which does a couple of things:
(1) Use root:root DAC label.
(2) Remove the code for chown/chmod the sockets.
However this doesn't work because qemu cannot connect to
the libvirt *monitor* :
qemu-system-x86_64: -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/guestfs-5keemr09uswfgev8.monitor,server,nowait: Failed to bind socket: Permission denied
qemu-system-x86_64: -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/guestfs-5keemr09uswfgev8.monitor,server,nowait: chardev: opening backend "socket" failed
I have verified that the DAC label is effective, ie. that qemu
really does run as root.root. I did this by:
$ while true; do ps -eo pid,uid,gid,euid,egid,args | grep '[q]emu'; done
[...]
14270 0 0 0 0 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name guestfs-5keemr09uswfgev8 -S -machine pc-i440fx-1.7,accel=kvm,usb=off -cpu host,+kvmclock -m 500 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid ae97eda2-7bb3-43ed-8e7f-9b4a5d27f1ae -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/guestfs-5keemr09uswfgev8.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot -no-acpi -kernel /home/rjones/d/libguestfs/tmp/.guestfs-0/kernel.12917 -initrd /home/rjones/d/libguestfs/tmp/.guestfs-0/initrd.12917 -append panic=1 console=ttyS0 udevtimeout=600 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_network=1 TERM=xterm-256color -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/root/f19,if=none,id=drive-scsi0-0-0-0,format=raw,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/home/rjones/d/libguestfs/tmp/libguestfsfjWgfU/snapshot1,if=none,id=drive-scsi0-0-1-0,format=qcow2,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0 -chardev socket,id=charserial0,path=/home/rjones/d/libguestfs/tmp/libguestfsfjWgfU/console.sock -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/home/rjones/d/libguestfs/tmp/libguestfsfjWgfU/guestfsd.sock -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -netdev user,id=usernet,net=169.254.0.0/16 -device virtio-net-pci,netdev=usernet
I have also checked and it's not SELinux.
I don't understand why a root.root process would have problems opening
any file ...
Apparently, libvirt strips Linux process capabilities from qemu. I think the actionable bit here was filed as https://bugzilla.redhat.com/show_bug.cgi?id=1045039 , so just duping to that. Please reopen if I missed something though *** This bug has been marked as a duplicate of bug 1045039 *** |