FWIW, the QEMU package in Fedora apparently ships with a sysctl file already, see BZ 1290589.

We are currently discussing an upstream solution that will avoid having to set this sysctl. It will most likely consist of

a) A gcc patch that allows to set specific ELF file header
b) A QEMU patch that will set this ELF file header using gcc on the qemu-system-s390x binary
c) A kernel patch that will look out for this ELF file header and setup pgste accordingly.

Folks at IBM are currently working on this.

We now have the kernel patch upstream:

commit 23fefe119ceb5fb0c7d3321010620010a4eddb18
Author: Martin Schwidefsky <schwidefsky.com>
Date:   Wed Jun 7 14:10:24 2017 +0200

    s390/kvm: avoid global config of vm.alloc_pgste=1

While the QEMU part will be easy, we are still missing a GCC patch.

As discussed during the zKVM meeting, it might make sense to temporarily set vm.allocate_pgste=1 like fedora does, to make working with zKVM easier during development. This can then be reverted once we have proper gcc + qemu support in place.

We have a new ld linker option upstream (binutils):
https://sourceware.org/binutils/docs/ld/S_002f390-ELF.html

commit b4cbbe8f7294070cc93a71ace78f134965ddad82
Author: Andreas Krebbel <krebbel.ibm.com>
Date:   Thu Jun 8 17:24:50 2017 +0200

    S/390: Add support for pgste marker


In summary, to get rid of this temporary solution, we need:
- binutils >= 2.29 (or relevant backport)
- kernels >= 4.12 (or relevant backport)

QEMU patch is currently being discussed upstream:
https://lists.gnu.org/archive/html/qemu-devel/2017-08/msg04160.html

Fix included in qemu-kvm-2.9.0-22.el7_4

Reproduced this bug with qemu-kvm-2.9.0-21.el7a.s390x:

Host kernel: kernel-4.11.0-30.el7a.s390x

# /usr/libexec/qemu-kvm 
ioctl(KVM_CREATE_VM) failed: 22 Invalid argument
Host kernel setup problem detected. Please verify:
- for kernels supporting the switch_amode or user_mode parameters, whether
  user space is running in primary address space
- for kernels supporting the vm.allocate_pgste sysctl, whether it is enabled
failed to initialize KVM: Invalid argument
Back to tcg accelerator.
VNC server running on ::1:5900
#

Result: Qemu-kvm command line could not boot up.

Verified this issue with qemu-kvm-2.9.0-22.el7a.s390x:

Host kernel: kernel-4.11.0-30.el7a.s390x

# /usr/libexec/qemu-kvm -no-shutdown
VNC server running on ::1:5900

Result: Qemu-kvm command line will not quit.

Besides, I installed a guest and it could finish successfully.

/usr/libexec/qemu-kvm -S -name avocado-vt-vm1 -sandbox off -machine s390-ccw-virtio -nodefaults -vga none -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_0No0v2/monitor-qmpmonitor1-20170920-033800-mlIN5A31,server,nowait -mon chardev=qmp_id_qmpmonitor1,mode=control -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/avocado_0No0v2/monitor-catch_monitor-20170920-033800-mlIN5A31,server,nowait -mon chardev=qmp_id_catch_monitor,mode=control -chardev socket,id=serial_id_serial0,path=/var/tmp/avocado_0No0v2/serial-serial0-20170920-033800-mlIN5A31,server,nowait -device sclpconsole,chardev=serial_id_serial0 -device virtio-scsi-ccw,id=virtio_scsi_ccw0 -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=unsafe,format=qcow2,file=/root/staf-kvm-devel/vt_test_images/ALT-Server-7.4-s390x-virtio-scsi.qcow2 -device scsi-hd,id=image1,drive=drive_image1 -device virtio-net-ccw,mac=9a:47:48:49:4a:4b,id=idNUNb5u,netdev=idAgqOHS -netdev tap,id=idAgqOHS,vhost=on,vhostfd=23,fd=22 -m 802 -smp 2,cores=1,threads=1,sockets=2 -cpu host -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=unsafe,media=cdrom,file=/root/staf-kvm-devel/workspace/var/lib/avocado/data/avocado-vt/isos/linux/RHEL-ALT-7.4-20170913.1-Server-s390x-dvd1.iso -device scsi-cd,id=cd1,drive=drive_cd1 -drive id=drive_unattended,if=none,snapshot=off,aio=threads,cache=unsafe,media=cdrom,file=/root/staf-kvm-devel/vt_test_images/rhel-alt-74-s390x/ks.iso -device scsi-cd,id=unattended,drive=drive_unattended -kernel /root/staf-kvm-devel/vt_test_images/rhel-alt-74-s390x/kernel.img -append ksdevice=link inst.repo=cdrom:/dev/sr0 inst.ks=cdrom:/dev/sr1:/ks.cfg nicdelay=60 biosdevname=0 net.ifnames=0 console=ttysclp0 -initrd /root/staf-kvm-devel/vt_test_images/rhel-alt-74-s390x/initrd.img -vnc :0 -rtc base=utc,clock=host,driftfix=slew -boot strict=on -no-shutdown -enable-kvm


Hi, David

Since I just test this bug on a beaker system which should be a zVM, and I install kvm guest on that environment.  Could I call it VERIFIED pass now?

Thanks,
Qunfang

Hi Qunfang,

yes, that should be enough. Testing under z/VM is sufficient.

As soon as you can run

$ /usr/libexec/qemu-kvm --enable-kvm

Without getting said error, the fix works. (If it works, you will get "VNC server running on ::1:5900")

And that is the case when you can run a KVM guest!

Thanks for verifying!

David

Okay, thanks for confirmation. :)

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3169