This is particularly useful on operating systems that don't ship
Python as part of the base system (eg. FreeBSD) while still working
just as well as it did before on Linux.

While at it, make it explicit that our scripts are only going to
work with Python 2, and remove the usage of unbuffered I/O, which
as far as I can tell has no effect on the output files.
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>

Available since QEMU 2.10.0 (specifically commit
v2.9.0-2233-g53f9a6f45f).
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

The features were added to QEMU by commit v2.4.0-1690-gf7fda28094 as
Skylake Server features.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

Add a new CPU model called 'EPYC' to model processors from AMD EPYC
family (which includes EPYC 76xx,75xx,74xx, 73xx and 72xx).

The following features bits have been added/removed compare to Opteron_G5

Added: monitor, movbe, rdrand, mmxext, ffxsr, rdtscp, cr8legacy, osvw,
       fsgsbase, bmi1, avx2, smep, bmi2, rdseed, adx, smap, clfshopt, sha
       xsaveopt, xsavec, xgetbv1, arat

Removed: xop, fma4, tbm

The patch is depend on EPYC CPU model supported introduced in qemu [1]

[1] https://patchwork.kernel.org/patch/9902205/

Cc: Tom Lendacky <Thomas.Lendacky@amd.com>
Signed-off-by: NBrijesh Singh <brijesh.singh@amd.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

CPU features unknown to a hypervisor will not be present in dataDisabled
even though the features won't naturally be enabled because.
Thus any features we asked for which are not in dataEnabled should be
considered disabled.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

All CPU features which QEMU does not know about but libvirt knows them
(currently "cmt" is the only one) are implicitly disabled by QEMU and
should be present in x86_64-cpuid-*-disabled.xml.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Commit v3.1.0-26-gd60012b4 started filtering hle and rtm features from
broken Intel Haswell CPUs. QEMU implemented similar functionality and
thus it doesn't report rtm and hle features as enabled for Core i5-4670T
CPU anymore.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Generated with

    (cd tests/cputestdata; ./cpu-cpuid.py diff x86_64-cpuid-*.json)
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The new command can be used to generate test data for virCPUUpdateLive.

When "cpu-cpuid.py diff x86-cpuid-Something.json" is run, it reads raw
CPUID data stored in x86-cpuid-Something.xml and CPUID data from QEMU
stored in x86-cpuid-Something.json to produce two more CPUID files:
x86-cpuid-Something-enabled.xml and x86-cpuid-Something-disabled.xml.

- x86-cpuid-Something-enabled.xml will contain CPUID bits present in
    x86-cpuid-Something.json (i.e., enabled by QEMU for the "host" CPU)

- x86-cpuid-Something-disabled.xml will contain all CPUID bits from
    x86-cpuid-Something.xml which are not present in
    x86-cpuid-Something.json (i.e., CPUID bits which the host CPU
    supports, but QEMU does not enable them for the "host" CPU)
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The new script is going to be more general and the original
functionality can be requested by "cpu-cpuid.py convert".
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Let's make the object local to the parseFeatureWords function which uses
it.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

cpuBaseline is responsible for computing a baseline CPU while feature
expansion is done by virCPUExpandFeatures. The cpuBaselineXML wrapper
(used by hypervisor drivers to implement virConnectBaselineCPU API)
calls cpuBaseline followed by virCPUExpandFeatures if requested by
VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES flag.

The features in the three changed test files had to be sorted using
"sort -k 3" because virCPUExpandFeatures returns a sorted list of
features.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

This element is only allowed for host CPUs.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

All existing Haswell CPUID data were gathered from CPUs with broken TSX.
Let's add new data for Haswell with correct TSX implementation.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

All Intel Haswell processors (except Xeon E7 v3 with stepping >= 4) have
TSX disabled by microcode update. As not all CPUs are guaranteed to be
patched with microcode updates we need to explicitly disable TSX on
affected CPUs to avoid its accidental usage.

https://bugzilla.redhat.com/show_bug.cgi?id=1406791Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The original test didn't use family/model numbers to make better
decisions about the CPU model and thus mis-detected the model in the two
cases which are modified in this commit. The detected CPU models now
match those obtained from raw CPUID data.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Converted by running the following command, renaming the files as
*.new, and committing only the *.new files.

    (cd tests/cputestdata; ./cpu-convert.py *.json)
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Instantiating "host" CPU and querying it using qom-get has been the only
way of probing host CPU via QEMU until 2.9.0 implemented
query-cpu-model-expansion for x86_64. Even though libvirt never really
used the old way its result can be easily converted into the one
produced by query-cpu-model-expansion. Thus we can reuse the original
test data and possible get new data from hosts where QEMU does not
support the new QMP command.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

While "x86" is a CPU sub driver name, it is not a recognized name of any
architecture known to libvirt. Let's use "x86_64" prefix which can be
used with virArch APIs.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

We want to remove the file created by

    json <<<"$data" >$fname.json

in case it was empty.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Guest CPU definitions with mode='custom' and missing <vendor> are
expected to run on a host CPU from any vendor as long as the required
CPU model can be used as a guest CPU on the host. But even though no CPU
vendor was explicitly requested we would sometimes force it due to a bug
in virCPUUpdate and virCPUTranslate.

The bug would effectively forbid cross vendor migrations even if they
were previously working just fine.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The API is no longer used anywhere else since it was replaced by a much
saner work flow utilizing new APIs that work on virCPUDefPtr directly:
virCPUCompare, virCPUUpdate, and virCPUTranslate.

Not testing the new work flow caused some bugs to be hidden. This patch
reveals them, but doesn't attempt to fix them. To make sure all test
still pass after this patch, all affected test results are modified to
pretend the tests succeeded. All of the bugs will be fixed in the
following commits and the artificial modifications will be reverted.

The following is the list of bugs in the new CPU model work flow:

- a guest CPU with mode='custom' and missing <vendor/> gets the vendor
  copied from host's CPU (the vendor should only be copied to host-model
  CPUs):
    DO_TEST_UPDATE("x86", "host", "min", VIR_CPU_COMPARE_IDENTICAL)
    DO_TEST_UPDATE("x86", "host", "pentium3", VIR_CPU_COMPARE_IDENTICAL)
    DO_TEST_GUESTCPU("x86", "host-better", "pentium3", NULL, 0)

- when a guest CPU with mode='custom' needs to be translated into
  another model because the original model is not supported by a
  hypervisor, the result will have its vendor set to the vendor of the
  original CPU model as specified in cpu_map.xml even if the original
  guest CPU XML didn't contain <vendor/>:
    DO_TEST_GUESTCPU("x86", "host", "guest", model486, 0)
    DO_TEST_GUESTCPU("x86", "host", "guest", models, 0)
    DO_TEST_GUESTCPU("x86", "host-Haswell-noTSX", "Haswell-noTSX",
                     haswell, 0)

- legacy POWERx_v* model names are not recognized:
    DO_TEST_GUESTCPU("ppc64", "host", "guest-legacy", ppc_models, 0)
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

In some cases preferred model doesn't really do anything since the
result remains the same even if it is removed.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Using a preferred CPU model which is not in the list of CPU models
supported by a hypervisor does not make sense.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Guest CPUs with match='minimum' should always be updated to match host
CPU model. Trying to get different results by supplying preferred models
does not make sense.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The reworked API is now called virCPUUpdate and it should change the
provided CPU definition into a one which can be consumed by the QEMU
command line builder:

    - host-passthrough remains unchanged
    - host-model is turned into custom CPU with a model and features
      copied from host
    - custom CPU with minimum match is converted similarly to host-model
    - optional features are updated according to host's CPU
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Some Intel processor families (e.g. the Intel Xeon processor E5 v3
family) introduced some PQos (Platform Qos) features, including CMT
(Cache Monitoring technology) and MBM (Memory Bandwidth Monitoring),
to monitor or control shared resource. This patch add them into x86
part of cpu_map.xml to be used for applications based on libvirt to
get cpu capabilities. For example, Nova in OpenStack schedules guests
based on the CPU features that the host has.
Signed-off-by: NQiaowei Ren <qiaowei.ren@intel.com>

Our current detection code uses just the number of CPU features which
need to be added/removed from the CPU model to fully describe the CPUID
data. The smallest number wins. But this may sometimes generate wrong
results as one can see from the fixed test cases. This patch modifies
the algorithm to prefer the CPU model with matching signature even if
this model results in a longer list of additional features.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The CPU model was implemented in QEMU by commit f6f949e929.

The change to i7-5600U is wrong since it's a 5th generation CPU, i.e.,
Broadwell rather than Skylake, but that's just the result of our CPU
detection code (which is fixed by the following commit).
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The actual CPU model in the data files is Penryn which makes the file
name look rather strange. Well, one of them contains Nehalem, but that's
a bug which will be fixed soon.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Implemented in QEMU by commit 28b8e4d0bf93ba176b4b7be819d537383c5a9060.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

This was implemented in QEMU by commit 0bb0b2d2fe7f645dda.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>