1365500 – CPU feature cmt not found with 2.0.0-1

Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1365500 - CPU feature cmt not found with 2.0.0-1

Summary: CPU feature cmt not found with 2.0.0-1

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	libvirt
Sub Component:
Version:	7.3
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Jiri Denemark
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:	1355857
Blocks:
TreeView+	depends on / blocked

Reported:	2016-08-09 11:53 UTC by Jiri Denemark
Modified:	2018-02-06 10:34 UTC (History)
CC List:	7 users (show)
Fixed In Version:	libvirt-2.0.0-8.el7
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:	1355857
Environment:
Last Closed:	2016-11-03 18:52:02 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Debian BTS	830857	0	None	None	None	2016-08-09 11:53:35 UTC
Red Hat Product Errata	RHSA-2016:2577	0	normal	SHIPPED_LIVE	Moderate: libvirt security, bug fix, and enhancement update	2016-11-03 12:07:06 UTC

Description Jiri Denemark 2016-08-09 11:53:35 UTC

+++ This bug was initially created as a clone of Bug #1355857 +++

Description of problem:

after upgrading libvirt-daemon from 1.3.5-1 to 2.0.0-1 I am unable to 
start a virtual machine which is configured with <cpu mode='host-model'>. With libvirt-daemon 1.3.5-1 this worked just fine. Now you receive an error like that one below when turning a machine (which was live migrated to the updated host) off and on again.

Starting the same machine with these settings works as expected:
  <cpu mode='custom' match='exact'>
    <model fallback='allow'>Haswell-noTSX</model>
  </cpu>

I think that this patch is causing the error for our CPUs (see below): 
http://libvirt.org/git/?p=libvirt.git;a=commit;h=f294b83ee632a6330f3a3045fbb5bcb9d9951c03

Commenting this out in /usr/share/libvirt/cpu_map.xml solves the issue 
as a workaround:

<!--    <feature name='cmt'>
      <cpuid eax_in='0x07' ebx='0x00001000'/>
    </feature> -->

Version-Release number of selected component (if applicable):
2.0.0-1

How reproducible:
100%


Steps to Reproduce:
1. Create a virtual machine with host-model CPU:

...
  <cpu mode='host-model'>
    <model fallback='allow'/>
  </cpu>
...

2. Start virtual machine


Actual results:
Start fails with this error:
"qemu-system-x86_64: CPU feature cmt not found"

Expected results:
Virtual machine should start without errors

Additional info:
virsh capabilities XML shows 'cmt' feature as supported by host CPU. The bug does not reproduce on hosts where 'cmt' feature is not advertised.

Comment 2 Jiri Denemark 2016-08-10 12:37:41 UTC

This bug is now fixed upstream by

commit dbb14bb0f1316d92c89541fca816c91dce0dc8fb
Refs: v2.1.0-108-gdbb14bb
Author:     Jiri Denemark <jdenemar>
AuthorDate: Tue Jun 28 10:51:41 2016 +0200
Commit:     Jiri Denemark <jdenemar>
CommitDate: Wed Aug 10 14:25:24 2016 +0200

    cpu_x86: Introduce x86FeatureIsMigratable

    Signed-off-by: Jiri Denemark <jdenemar>
---
 src/cpu/cpu_x86.c | 37 +++++++++++++++++++++++--------------
 1 file changed, 23 insertions(+), 14 deletions(-)

commit 1ac897a15da11d1bfca2642bce3b0beaad32bcf1
Refs: v2.1.0-109-g1ac897a
Author:     Jiri Denemark <jdenemar>
AuthorDate: Tue Jun 28 11:12:41 2016 +0200
Commit:     Jiri Denemark <jdenemar>
CommitDate: Wed Aug 10 14:25:24 2016 +0200

    cpu_x86: Properly drop non-migratable features

    By removing a non-migratable feature in a for loop we would fail to drop
    every second non-migratable feature if the features array contained
    several of them in a row.

    Signed-off-by: Jiri Denemark <jdenemar>
---
 src/cpu/cpu_x86.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

commit 58ba240df89a7dd7bff16ebb65d9aadd49fb8bbb
Refs: v2.1.0-110-g58ba240
Author:     Jiri Denemark <jdenemar>
AuthorDate: Tue Aug 9 15:03:20 2016 +0200
Commit:     Jiri Denemark <jdenemar>
CommitDate: Wed Aug 10 14:25:24 2016 +0200

    tests: Add a test for host-model CPU with CMT feature

    The generated command line wouldn't work since QEMU doesn't know what
    'cmt' is. The following patch will fix this issue.

    https://bugzilla.redhat.com/show_bug.cgi?id=1355857

    Signed-off-by: Jiri Denemark <jdenemar>
---
 tests/qemuxml2argvdata/qemuxml2argv-cpu-host-model-cmt.args | 22 ++++++++++++++++++++++
 tests/qemuxml2argvdata/qemuxml2argv-cpu-host-model-cmt.xml  | 19 +++++++++++++++++++
 tests/qemuxml2argvtest.c                                    |  1 +
 tests/testutilsqemu.c                                       |  1 +
 4 files changed, 43 insertions(+)

commit 300f668c665f1ec0f834917fe8a58b5991322441
Refs: v2.1.0-111-g300f668
Author:     Jiri Denemark <jdenemar>
AuthorDate: Tue Aug 9 15:15:20 2016 +0200
Commit:     Jiri Denemark <jdenemar>
CommitDate: Wed Aug 10 14:25:24 2016 +0200

    cpu_x86: Fix host-model CPUs on hosts with CMT

    Since the introduction of CMT features (commit v1.3.5-461-gf294b83)
    starting a domain with host-model CPU on a host which supports CMT fails
    because QEMU complains about unknown 'cmt' feature:

        qemu-system-x86_64: CPU feature cmt not found

    https://bugzilla.redhat.com/show_bug.cgi?id=1355857

    Signed-off-by: Jiri Denemark <jdenemar>
---
 src/cpu/cpu_x86.c                                           | 8 ++++++--
 tests/qemuxml2argvdata/qemuxml2argv-cpu-host-model-cmt.args | 2 +-
 2 files changed, 7 insertions(+), 3 deletions(-)

Comment 5 chhu 2016-08-18 09:22:07 UTC

Reproduced with package: 
libvirt-2.0.0-4.el7.x86_64, 
qemu-kvm-rhev-2.6.0-20.el7.x86_64, 
kernel:  3.10.0-492.el7.x86_64

Tested with libvirt-2.0.0-5.el7.x86_64

Test with xml cpu mode='host-model' pass, but test with xml: cpu mode='custom' match='minimum' failed with the same error.

Steps:
1. Start VM with xml: cpu mode='host-model': PASS
1) model fallback='allow'
# virsh dumpxml r7.3-raw | grep "<cpu" -A 5
  <cpu mode='host-model'>
    <model fallback='allow'/>
  </cpu>

# virsh start r7.3-raw
Domain r7.3-raw started

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 8     r7.3-raw                       running

2) model fallback='forbid'
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     r7.3-raw                       shut off

# virsh start r7.3-raw
Domain r7.3-raw started

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 9     r7.3-raw                       running

# virsh dumpxml r7.3-raw | grep "<cpu" -A 5
  <cpu mode='host-model'>
    <model fallback='forbid'/>
  </cpu>


2. Start VM with xml: cpu mode='custom' match='minimum': Failed

# virsh dumpxml r7.3-raw | grep "<cpu" -A 5
  <cpu mode='custom' match='minimum'>
    <model fallback='allow'>Haswell-noTSX</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='ds'/>
  </cpu>

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     r7.3-raw                       shut off

# virsh start r7.3-raw
error: Failed to start domain r7.3-raw
error: internal error: qemu unexpectedly closed the monitor: 2016-08-18T08:55:33.155930Z qemu-kvm: CPU feature cmt not found

Comment 6 chhu 2016-08-18 09:52:13 UTC

Hi, Jiri

I tested with xml cpu mode='host-model', VM started successfully, but when tested with xml: cpu mode='custom' match='minimum', VM failed to start and got the same error. Please see more details in Comment5, do you think this need to be fixed ? Could you please help to confirm? 

Thanks,
chhu

Comment 7 Jiri Denemark 2016-08-22 19:24:49 UTC

Ah yeah, the upstream series should take care of this already, I'll try to look at ways to fix this part downstream without having to backport all the downstream patches.

Comment 8 chhu 2016-09-01 03:38:02 UTC

Did more test for this bug:

Tested with packages:
libvirt-2.0.0-6.el7.x86_64
qemu-kvm-rhev-2.6.0-22.el7.x86_64


1. Start the VM, check the dumpxml，did operations: Start VM, managed save, migration(7.3<=>7.3)：PASS

<cpu mode='host-model'>
  <model fallback='allow'/>
</cpu>


2. Start the VM, check the dumpxml and the qemu cmdline.

1） mode=custom, match=minimum: failed, updated in comment 5.
<cpu mode='custom' match='minimum'>   
  <model fallback='allow'>Haswell-noTSX</model>
</cpu>

2) mode=custom, match= strict/exact: start VM with xml below:
<cpu mode='custom' match='strict'>  
  <model fallback='allow'>Haswell-noTSX</model>
  <vendor>Intel</vendor>
  <feature policy='require' name='cmt'/>
  ......
</cpu>
<cpu mode='custom' match='exact'>
  <model fallback='allow'>Haswell-noTSX</model>
  <vendor>Intel</vendor>
  <feature policy='require' name='cmt'/>  
</cpu>

Met the same error message: 
" qemu-kvm: CPU feature cmt not found" 
As the cmt in not known by qemu-kvm command line, so the error message is reasonable. 

Changed to <feature policy='disable' name='cmt'/>, start VM  successfully, qemu command line is without "-cmt", this is related to Bug 1371039 - CPU features with policy 'disable' should not be supported in the guest.


3. Start the VM, with cpu mode: host-model/customer, with different policy, check the dumpxml and the qemu cmdline.

- Feature policy: force/ require/ optional/ disable/ forbid
<cpu mode='host-model'>
  <model fallback='allow'/>
  <feature policy='disable' name='cmt'/>
</cpu>

<cpu mode='custom' match='exact'>
  <model fallback='allow'>Haswell-noTSX</model>
  <vendor>Intel</vendor>
  <feature policy='disable' name='cmt'/>
</cpu>


1) policy=forbid: met "error: unsupported configuration: guest and host CPU are not compatible: Host CPU provides forbidden features: cmt" => this is as design.

2) policy=require/ force/optional: met error message: "qemu-kvm: CPU feature cmt not found" =>  as the cmt in not known by qemu-kvm command line, so the error message is reasonable.

3) policy=disable: started the vm successfully and the qemu command line is without "-cmt". Normally, the qemu command line should include '-cmt'. 
This is related to Bug 1371039 - CPU features with policy 'disable' should not be supported in the guest.


4. Tried to start a VM with cpu mode: host-passthrough with different policy, met the same error: "qemu-kvm: CPU feature cmt not found".

-policy: disable/require/optional/force/forbid

  <cpu mode='host-passthrough'>
    <feature policy='disable' name='cmt'/>
  </cpu>

# virsh start r7t
error: Failed to start domain r7t
error: internal error: qemu unexpectedly closed the monitor: 2016-08-26T07:40:24.934930Z qemu-kvm: CPU feature cmt not found

Issues:
1) policy=require/ force/optional: met error message: "qemu-kvm: CPU feature cmt not found" =>  as the cmt in not known by qemu-kvm command line, so the error message is reasonable.

2) For policy='disable', the VM should be started successfully, now met error:"qemu-kvm: CPU feature cmt not found" 
  
3) For policy='forbid', should met error message as below:
"error: unsupported configuration: guest and host CPU are not compatible: Host CPU provides forbidden features: cmt"
Now met error:"qemu-kvm: CPU feature cmt not found"

@Jiri:
As QEMU complains about unknown 'cmt' feature, not sure if above Issues: 2)3) should be fixed. Could you please help to confirm?  Thank you!

Comment 9 chhu 2016-09-05 10:41:36 UTC

Updated another test issue which may related to this bug here:

Imporve the error message:
Migrate with 'invtsc' get qemu error:
"unable to execute QEMU command 'migrate': State blocked by non-migratable device 'cpu'" 
The error message should be reported by libvirt as:
"error: Requested operation is not valid: domain has CPU feature: invtsc"

How reproducible:
100%

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.6.0-22.el7.x86_64
libvirt-2.0.0-6.el7.x86_64
kernel: kernel-3.10.0-495.el7.x86_64

Steps to reproduce:
1. Start a VM with xml below successfully.
# virsh start r7-s1
Domain r7-s1 started

# virsh dumpxml r7-s1|grep "<cpu" -A 6
  <cpu mode='custom' match='minimum'>
    <model fallback='forbid'>Skylake-Client</model>
  </cpu>

# ps -ef|grep r7-s1
qemu     17490     1 11 22:59 ?        00:00:12 /usr/libexec/qemu-kvm -name guest=r7-s1...... -cpu Skylake-Client,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,+monitor,+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+osxsave,+tsc_adjust,+clflushopt,+xsaves,+pdpe1gb,+invtsc -m 1024

2. Try to do migration test: met error: "unable to execute QEMU command 'migrate': State blocked by non-migratable device 'cpu'"

# virsh migrate r7-s1 qemu+ssh://10.19.176.166/system --live --verbose
root.176.166's password: 
error: internal error: unable to execute QEMU command 'migrate': State blocked by non-migratable device 'cpu'


Hi, Jiri

Will the issue in this and above comments be fixed? As it's rhel7.3 snapshot-2 now, and this bug is with rhel-7.3.0 +, could you please help to provide the fixed patches for us to test? Many thanks!

Comment 10 Jiri Denemark 2016-09-05 11:06:21 UTC

CPU with match='minimum' is a lot like host-model, but it wasn't treated as such :-( and my upstream series fixes this. However, it's not something we can backport. Anyway, this part of the bug (invtsc added for minimum match CPUs) should not be a regression and as such, I don't think it's worth another downstream only hack.

Comment 11 chhu 2016-09-07 05:45:48 UTC

(In reply to chhu from comment #8)
> Did more test for this bug:
> 
> Tested with packages:
> libvirt-2.0.0-6.el7.x86_64
> qemu-kvm-rhev-2.6.0-22.el7.x86_64
> 
> 
> 1. Start the VM, check the dumpxml，did operations: Start VM, managed save,
> migration(7.3<=>7.3)：PASS
> 
> <cpu mode='host-model'>
>   <model fallback='allow'/>
> </cpu>
> 
> 
> 2. Start the VM, check the dumpxml and the qemu cmdline.
> 
> 1） mode=custom, match=minimum: failed, updated in comment 5.
> <cpu mode='custom' match='minimum'>   
>   <model fallback='allow'>Haswell-noTSX</model>
> </cpu>
> 
> 2) mode=custom, match= strict/exact: start VM with xml below:
> <cpu mode='custom' match='strict'>  
>   <model fallback='allow'>Haswell-noTSX</model>
>   <vendor>Intel</vendor>
>   <feature policy='require' name='cmt'/>
>   ......
> </cpu>
> <cpu mode='custom' match='exact'>
>   <model fallback='allow'>Haswell-noTSX</model>
>   <vendor>Intel</vendor>
>   <feature policy='require' name='cmt'/>  
> </cpu>
> 
> Met the same error message: 
> " qemu-kvm: CPU feature cmt not found" 
> As the cmt in not known by qemu-kvm command line, so the error message is
> reasonable. 
> 
> Changed to <feature policy='disable' name='cmt'/>, start VM  successfully,
> qemu command line is without "-cmt", this is related to Bug 1371039 - CPU
> features with policy 'disable' should not be supported in the guest.
> 
> 
> 3. Start the VM, with cpu mode: host-model/customer, with different policy,
> check the dumpxml and the qemu cmdline.
> 
> - Feature policy: force/ require/ optional/ disable/ forbid
> <cpu mode='host-model'>
>   <model fallback='allow'/>
>   <feature policy='disable' name='cmt'/>
> </cpu>
> 
> <cpu mode='custom' match='exact'>
>   <model fallback='allow'>Haswell-noTSX</model>
>   <vendor>Intel</vendor>
>   <feature policy='disable' name='cmt'/>
> </cpu>
> 
> 
> 1) policy=forbid: met "error: unsupported configuration: guest and host CPU
> are not compatible: Host CPU provides forbidden features: cmt" => this is as
> design.
> 
> 2) policy=require/ force/optional: met error message: "qemu-kvm: CPU feature
> cmt not found" =>  as the cmt in not known by qemu-kvm command line, so the
> error message is reasonable.
> 
> 3) policy=disable: started the vm successfully and the qemu command line is
> without "-cmt". Normally, the qemu command line should include '-cmt'. 
> This is related to Bug 1371039 - CPU features with policy 'disable' should
> not be supported in the guest.
> 
> 
> 4. Tried to start a VM with cpu mode: host-passthrough with different
> policy, met the same error: "qemu-kvm: CPU feature cmt not found".
> 
> -policy: disable/require/optional/force/forbid
> 
>   <cpu mode='host-passthrough'>
>     <feature policy='disable' name='cmt'/>
>   </cpu>
> 
> # virsh start r7t
> error: Failed to start domain r7t
> error: internal error: qemu unexpectedly closed the monitor:
> 2016-08-26T07:40:24.934930Z qemu-kvm: CPU feature cmt not found
> 
> Issues:
> 1) policy=require/ force/optional: met error message: "qemu-kvm: CPU feature
> cmt not found" =>  as the cmt in not known by qemu-kvm command line, so the
> error message is reasonable.
> 
> 2) For policy='disable', the VM should be started successfully, now met
> error:"qemu-kvm: CPU feature cmt not found" 
>   
> 3) For policy='forbid', should met error message as below:
> "error: unsupported configuration: guest and host CPU are not compatible:
> Host CPU provides forbidden features: cmt"
> Now met error:"qemu-kvm: CPU feature cmt not found"
> 
> @Jiri:
> As QEMU complains about unknown 'cmt' feature, not sure if above Issues:
> 2)3) should be fixed. Could you please help to confirm?  Thank you!

Tested with mbm_local/mbm_total on Broadwell met the similar results as test cmt on Skylake-Client, will retest mbm_local/mbm_total after this bug is fixed.

Comment 12 chhu 2016-09-07 05:57:11 UTC

(In reply to chhu from comment #9)
> Updated another test issue which may related to this bug here:
> 
> Imporve the error message:
> Migrate with 'invtsc' get qemu error:
> "unable to execute QEMU command 'migrate': State blocked by non-migratable
> device 'cpu'" 
> The error message should be reported by libvirt as:
> "error: Requested operation is not valid: domain has CPU feature: invtsc"
> 
> How reproducible:
> 100%
> 
> Version-Release number of selected component (if applicable):
> qemu-kvm-rhev-2.6.0-22.el7.x86_64
> libvirt-2.0.0-6.el7.x86_64
> kernel: kernel-3.10.0-495.el7.x86_64
> 
> Steps to reproduce:
> 1. Start a VM with xml below successfully.
> # virsh start r7-s1
> Domain r7-s1 started
> 
> # virsh dumpxml r7-s1|grep "<cpu" -A 6
>   <cpu mode='custom' match='minimum'>
>     <model fallback='forbid'>Skylake-Client</model>
>   </cpu>
> 
> # ps -ef|grep r7-s1
> qemu     17490     1 11 22:59 ?        00:00:12 /usr/libexec/qemu-kvm -name
> guest=r7-s1...... -cpu
> Skylake-Client,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,+monitor,+ds_cpl,+vmx,+smx,
> +est,+tm2,+xtpr,+pdcm,+osxsave,+tsc_adjust,+clflushopt,+xsaves,+pdpe1gb,
> +invtsc -m 1024
> 
> 2. Try to do migration test: met error: "unable to execute QEMU command
> 'migrate': State blocked by non-migratable device 'cpu'"
> 
> # virsh migrate r7-s1 qemu+ssh://10.19.176.166/system --live --verbose
> root.176.166's password: 
> error: internal error: unable to execute QEMU command 'migrate': State
> blocked by non-migratable device 'cpu'
> 
> 
> Hi, Jiri
> 
> Will the issue in this and above comments be fixed? As it's rhel7.3
> snapshot-2 now, and this bug is with rhel-7.3.0 +, could you please help to
> provide the fixed patches for us to test? Many thanks!

According to Jiri's reply:"Anyway, this part of the bug (invtsc added for minimum match CPUs) should not be a regression and as such, I don't think it's worth another downstream only hack."

I add the issues in this comment to bug1372581.

Comment 13 chhu 2016-09-07 09:35:19 UTC

(In reply to chhu from comment #11)
> (In reply to chhu from comment #8)
> > Did more test for this bug:
> > 
> > Tested with packages:
> > libvirt-2.0.0-6.el7.x86_64
> > qemu-kvm-rhev-2.6.0-22.el7.x86_64
> > 
> > 
> > 1. Start the VM, check the dumpxml，did operations: Start VM, managed save,
> > migration(7.3<=>7.3)：PASS
> > 
> > <cpu mode='host-model'>
> >   <model fallback='allow'/>
> > </cpu>
> > 
> > 
> > 2. Start the VM, check the dumpxml and the qemu cmdline.
> > 
> > 1） mode=custom, match=minimum: failed, updated in comment 5.
> > <cpu mode='custom' match='minimum'>   
> >   <model fallback='allow'>Haswell-noTSX</model>
> > </cpu>
> > 
> > 2) mode=custom, match= strict/exact: start VM with xml below:
> > <cpu mode='custom' match='strict'>  
> >   <model fallback='allow'>Haswell-noTSX</model>
> >   <vendor>Intel</vendor>
> >   <feature policy='require' name='cmt'/>
> >   ......
> > </cpu>
> > <cpu mode='custom' match='exact'>
> >   <model fallback='allow'>Haswell-noTSX</model>
> >   <vendor>Intel</vendor>
> >   <feature policy='require' name='cmt'/>  
> > </cpu>
> > 
> > Met the same error message: 
> > " qemu-kvm: CPU feature cmt not found" 
> > As the cmt in not known by qemu-kvm command line, so the error message is
> > reasonable. 
> > 
> > Changed to <feature policy='disable' name='cmt'/>, start VM  successfully,
> > qemu command line is without "-cmt", this is related to Bug 1371039 - CPU
> > features with policy 'disable' should not be supported in the guest.
> > 
> > 
> > 3. Start the VM, with cpu mode: host-model/customer, with different policy,
> > check the dumpxml and the qemu cmdline.
> > 
> > - Feature policy: force/ require/ optional/ disable/ forbid
> > <cpu mode='host-model'>
> >   <model fallback='allow'/>
> >   <feature policy='disable' name='cmt'/>
> > </cpu>
> > 
> > <cpu mode='custom' match='exact'>
> >   <model fallback='allow'>Haswell-noTSX</model>
> >   <vendor>Intel</vendor>
> >   <feature policy='disable' name='cmt'/>
> > </cpu>
> > 
> > 
> > 1) policy=forbid: met "error: unsupported configuration: guest and host CPU
> > are not compatible: Host CPU provides forbidden features: cmt" => this is as
> > design.
> > 
> > 2) policy=require/ force/optional: met error message: "qemu-kvm: CPU feature
> > cmt not found" =>  as the cmt in not known by qemu-kvm command line, so the
> > error message is reasonable.
> > 
> > 3) policy=disable: started the vm successfully and the qemu command line is
> > without "-cmt". Normally, the qemu command line should include '-cmt'. 
> > This is related to Bug 1371039 - CPU features with policy 'disable' should
> > not be supported in the guest.
> > 
> > 
> > 4. Tried to start a VM with cpu mode: host-passthrough with different
> > policy, met the same error: "qemu-kvm: CPU feature cmt not found".
> > 
> > -policy: disable/require/optional/force/forbid
> > 
> >   <cpu mode='host-passthrough'>
> >     <feature policy='disable' name='cmt'/>
> >   </cpu>
> > 
> > # virsh start r7t
> > error: Failed to start domain r7t
> > error: internal error: qemu unexpectedly closed the monitor:
> > 2016-08-26T07:40:24.934930Z qemu-kvm: CPU feature cmt not found
> > 
> > Issues:
> > 1) policy=require/ force/optional: met error message: "qemu-kvm: CPU feature
> > cmt not found" =>  as the cmt in not known by qemu-kvm command line, so the
> > error message is reasonable.
> > 
> > 2) For policy='disable', the VM should be started successfully, now met
> > error:"qemu-kvm: CPU feature cmt not found" 
> >   
> > 3) For policy='forbid', should met error message as below:
> > "error: unsupported configuration: guest and host CPU are not compatible:
> > Host CPU provides forbidden features: cmt"
> > Now met error:"qemu-kvm: CPU feature cmt not found"
> > 
> > @Jiri:
> > As QEMU complains about unknown 'cmt' feature, not sure if above Issues:
> > 2)3) should be fixed. Could you please help to confirm?  Thank you!
> 
> Tested with mbm_local/mbm_total on Broadwell met the similar results as test
> cmt on Skylake-Client, will retest mbm_local/mbm_total after this bug is
> fixed.

According to Jiri's reply:"CPU with match='minimum' is a lot like host-model, but it wasn't treated as such :-( and my upstream series fixes this. However, it's not something we can backport."

File a new Bug 1373849 to record the issue.

Comment 14 chhu 2016-09-07 09:49:00 UTC

According to the comment 12,13, the bug is verified.

Comment 15 Jiri Denemark 2016-09-07 14:06:41 UTC

Oh, you got me wrong, the cmt/mbm_local/mbm_total thing should be fixed as it is a regression. The invtsc issue with minimum match CPUs is not worth a RHEL-only hack since it's been broken ever since we introduced invtsc.

Comment 16 Jiri Denemark 2016-09-07 14:06:53 UTC

*** Bug 1373849 has been marked as a duplicate of this bug. ***

Comment 17 Jiri Denemark 2016-09-07 14:07:45 UTC

The additional patch: http://post-office.corp.redhat.com/archives/rhvirt-patches/2016-September/msg00204.html

Comment 19 Jiri Denemark 2016-09-09 09:27:52 UTC

So the additional patch fixes just one of the paths where custom/minimum CPUs are updated according to a host CPU and unfortunately it was not the path which is used when building a command line for QEMU. The patch only fixes "virsh dumpxml --update-cpu".

In other words, domains custom/minimum CPUs will still fail to start on hosts with CMT. I'm reopening bug 1373849 to track this.

Comment 20 chhu 2016-09-12 02:25:47 UTC

Tested for the patch in Comment18: 

Reproduced on package:
libvirt-2.0.0-6.el7.x86_64

Verified on package:
libvirt-2.0.0-8.el7.x86_64

Test steps:
1. Try to start a VM with cpu mode custom and match=minimum on host with CMT: Failed

# virsh dumpxml r7| grep "<cpu" -A 4
  <cpu mode='custom' match='minimum'>
    <model fallback='allow'>Broadwell</model>
  </cpu>
# virsh start r7
error: Failed to start domain r7
error: internal error: qemu unexpectedly closed the monitor: 2016-09-12T02:02:06.688799Z qemu-kvm: CPU feature cmt not found


2. Check the "virsh dumpxml --update-cpu": PASS

1) On libvirt-2.0.0-6.el7.x86_64: there are cmt, mbm_total, mbm_local listed.
# virsh dumpxml --update-cpu r7
<domain type='kvm'>
  <name>r7</name>
.....
  <cpu mode='custom' match='exact'>
    <model fallback='forbid'>Broadwell</model>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='ds'/>
    <feature policy='require' name='acpi'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='tm'/>
    <feature policy='require' name='pbe'/>
    <feature policy='require' name='dtes64'/>
    <feature policy='require' name='monitor'/>
    <feature policy='require' name='ds_cpl'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='smx'/>
    <feature policy='require' name='est'/>
    <feature policy='require' name='tm2'/>
    <feature policy='require' name='xtpr'/>
    <feature policy='require' name='pdcm'/>
    <feature policy='require' name='dca'/>
    <feature policy='require' name='osxsave'/>
    <feature policy='require' name='f16c'/>
    <feature policy='require' name='rdrand'/>
    <feature policy='require' name='arat'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='cmt'/>
    <feature policy='require' name='xsaveopt'/>
    <feature policy='require' name='mbm_total'/>
    <feature policy='require' name='mbm_local'/>
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='abm'/>
    <feature policy='require' name='invtsc'/>
  </cpu>

2) on libvirt-2.0.0-8.el7.x86_64, the cmt, mbm_total, mbm_local are moved.

# virsh dumpxml --update-cpu r7
<domain type='kvm'>
  <name>r7</name>
......
  <cpu mode='custom' match='exact'>
    <model fallback='allow'>Broadwell</model>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='ds'/>
    <feature policy='require' name='acpi'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='tm'/>
    <feature policy='require' name='pbe'/>
    <feature policy='require' name='dtes64'/>
    <feature policy='require' name='monitor'/>
    <feature policy='require' name='ds_cpl'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='smx'/>
    <feature policy='require' name='est'/>
    <feature policy='require' name='tm2'/>
    <feature policy='require' name='xtpr'/>
    <feature policy='require' name='pdcm'/>
    <feature policy='require' name='dca'/>
    <feature policy='require' name='osxsave'/>
    <feature policy='require' name='f16c'/>
    <feature policy='require' name='rdrand'/>
    <feature policy='require' name='arat'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='xsaveopt'/>
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='abm'/>
    <feature policy='require' name='invtsc'/>
  </cpu>

According to comment 12, 19, this bug is verified.

Comment 22 errata-xmlrpc 2016-11-03 18:52:02 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2577.html

Note You need to log in before you can comment on or make changes to this bug.