Since:

  commit 9f4e35dc
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Mon Mar 18 17:31:21 2019 +0000

    network: improve error report when firewall chain creation fails

We cache an error when failing to create the top level firewall chains.
This commit failed to account for fact that we may invoke
networkPreReloadFirewallRules() many times while libvirtd is running.
For example when firewalld is restarted.

When this happens the original failure may no longer occurr and we'll
successfully create our top level chains. We failed to clear the cached
error resulting in us failing to start virtual networks.
Reviewed-by: NLaine Stump <laine@laine.org>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The table included in the sample output for 'list --title' is
unnecessarily wide, which causes man to complain:

  warning [p 8, 0.5i]: can't break line

Make the table narrower.

Spotted by Lintian (manpage-has-errors-from-man tag).
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>
Acked-by: NMichal Privoznik <mprivozn@redhat.com>

Apparently "allow(s) to frobnicate" is not correct English, and
either "allow(s) one to frobnicate" or "allow(s) frobnicating"
should be used instead.

Spotted by Lintian (spelling-error-in-{binary,manpage} tags).
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Spotted by Lintian (manpage-has-bad-whatis-entry tag).
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>
Reviewed-by: NMartin Kletzander <mkletzan@redhat.com>

We need commit 6280c94f306d in order to fix our generated
man pages.
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>
Reviewed-by: NMartin Kletzander <mkletzan@redhat.com>

Signed-off-by: NAndrea Bolognani <abologna@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

My earlier commit be46f613 was incomplete. It removed caching of
microcode version in the CPU driver, which means the capabilities XML
will see the correct microcode version. But it is also cached in the
QEMU capabilities cache where it is used to detect whether we need to
reprobe QEMU. By missing the second place, the original commit
be46f613 made the situation even worse since libvirt would report
correct microcode version while still using the old host CPU model
(visible in domain capabilities XML).
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Support for kqemu was dropped in libvirt by commit 8e91a400 and even
back then we never set these capabilities when doing QMP probing.

Since no QEMU we aim to support has these, drop them completely.
Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NAndrea Bolognani <abologna@redhat.com>

This is a zero-cost workaround for a bug in GCC 8.3.0 which causes the
compilation to fail, because the compiler thinks that the value might be used
uninitialized even though it clearly cannot be.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Though it used to be called "Mac OS X" and "OS X" in the past,
it was never "MacOS X" nor "OS-X", and it's just "macOS" now.
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>

We have occasionally failed to document certain categories
of changes in the release notes, yet still left the
corresponding sections in the file even though they were
completely empty.
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>

In e17d1038 these functions were mistakenly moved into an #ifdef
block, but remained used outside of it leaving the build broken
for platforms where #ifdef evaluated to false.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

Model specific registers are a thing only on x86. Also, the
/dev/cpu/0/msr path exists only on Linux and the fallback
mechanism (asking KVM) exists on Linux and FreeBSD only.

Therefore, move the function within #ifdef that checks all
aforementioned constraints and provide a dummy stub for all
other cases.

This fixes the build on my arm box, mingw-* builds, etc.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

The NVIDIA V100 GPU has an onboard RAM that is mapped into the
host memory and accessible as normal RAM via an NVLink2 bridge. When
passed through in a guest, QEMU puts the NVIDIA RAM window in a
non-contiguous area, above the PCI MMIO area that starts at 32TiB.
This means that the NVIDIA RAM window starts at 64TiB and go all the
way to 128TiB.

This means that the guest might request a 64-bit window, for each PCI
Host Bridge, that goes all the way to 128TiB. However, the NVIDIA RAM
window isn't counted as regular RAM, thus this window is considered
only for the allocation of the Translation and Control Entry (TCE).
For more information about how NVLink2 support works in QEMU,
refer to the accepted implementation [1].

This memory layout differs from the existing VFIO case, requiring its
own formula. This patch changes the PPC64 code of
@qemuDomainGetMemLockLimitBytes to:

- detect if we have a NVLink2 bridge being passed through to the
guest. This is done by using the @ppc64VFIODeviceIsNV2Bridge function
added in the previous patch. The existence of the NVLink2 bridge in
the guest means that we are dealing with the NVLink2 memory layout;

- if an IBM NVLink2 bridge exists, passthroughLimit is calculated in a
different way to account for the extra memory the TCE table can alloc.
The 64TiB..128TiB window is more than enough to fit all possible
GPUs, thus the memLimit is the same regardless of passing through 1 or
multiple V100 GPUs.

Further reading explaining the background
[1] https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg03700.html
[2] https://www.redhat.com/archives/libvir-list/2019-March/msg00660.html
[3] https://www.redhat.com/archives/libvir-list/2019-April/msg00527.htmlSigned-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: NErik Skultety <eskultet@redhat.com>

The NVLink2 support in QEMU implements the detection of NVLink2
capable devices by verifying the attributes of the VFIO mem region
QEMU allocates for the NVIDIA GPUs. To properly allocate an
adequate amount of memLock, Libvirt needs this information before
a QEMU instance is even created, thus querying QEMU is not
possible and opening a VFIO window is too much.

An alternative is presented in this patch. Making the following
assumptions:

- if we want GPU RAM to be available in the guest, an NVLink2 bridge
must be passed through;

- an unknown PCI device can be classified as a NVLink2 bridge
if its device tree node has 'ibm,gpu', 'ibm,nvlink',
'ibm,nvlink-speed' and 'memory-region'.

This patch introduces a helper called @ppc64VFIODeviceIsNV2Bridge
that checks the device tree node of a given PCI device and
check if it meets the criteria to be a NVLink2 bridge. This
new function will be used in a follow-up patch that, using the
first assumption, will set up the rlimits of the guest
accordingly.
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

In 2878278c we've added new cpu model but we've forgot to
distribute the XML file it comes in.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This does not cause a problem in usual scenarios thanks to us allowing
CAP_DAC_OVERRIDE for the qemu process, however in some scenarios this might be
an issue because the directory is created with mkdtemp(3) which explicitly
creates that with 0700 permissions and qemu running as non-root cannot access
that.

The scenarios include:
 - Builds without CAPNG
 - Running libvirtd in certain container configurations [1]
 - and possibly others.

[1] https://github.com/kubevirt/kubevirt/pull/2181#issuecomment-481840304Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

The new virHostCPUGetMSR internal API will try to read the MSR from
/dev/cpu/0/msr and if it is not possible (the device does not exist or
libvirt is running unprivileged), it will fallback to asking KVM for the
MSR using KVM_GET_MSRS ioctl.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The script just parses whatever cpu-gather.sh printed out.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

This patch adds an inline python code for reading MSR features. Since
reading MSRs is a privileged operation, we have to read them from
/dev/cpu/*/msr if it is readable (i.e., the script runs as root) or
fallback to using KVM ioctl which can be done by any user that can start
virtual machines.

The python code is inlined rather than provided in a separate script
because whenever there's an issue with proper detection of CPU features,
we ask the reporter to run cpu-gather.sh script to give us all data we
need to know about the host CPU. Asking them to run several scripts
would likely result in one of them being ignored or forgotten.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The parseMapFeature for parsing features from CPU map XML can be easily
generalized to support more feature types.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Let's make sure the current CPUID specific code is only applied to CPUID
features.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

This will let us simplify the code since the dictionary keys will match
attribute names in various XMLs.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

leaf["eax"] & eax > 0 check works correctly only if there's at most 1
bit set in eax. Luckily that's been always the case, but fixing this
could save us from future surprises.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function will have to deal with both CPUID and MSR features.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

We don't really need to parse CPU data from QEMU older than 2.9 (i.e.,
before query-cpu-model-expansion) at this point. But even if there's a
need to do so, we can always use an older version of this script to do
the conversion.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

They are static and we will need to call them a little bit closer to the
beginning of the file.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The structure can only be used for CPUID data now. Adding a type
indicator and moving the data into a union will let us store alternative
data types.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function now works on virCPUx86DataItem and it's called
virCPUx86DataItemMatch.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function is renamed as virCPUx86DataItemMatchMasked to reflect the
change in parameter types.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function now works on virCPUx86DataItem and it's renamed as
virCPUx86DataItemAndBits.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The parameters changed from virCPUx86CPUID to virCPUx86DataItem and the
function is now called virCPUx86DataItemClearBits.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function is renamed as virCPUx86DataItemSetBits and it works on
virCPUx86DataItem now.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

virCPUx86DataSorter already compares two virCPUx86DataItem structs.
Let's add a tiny wrapper around it called virCPUx86DataCmp and use it
instead of open coded comparisons.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The while loop just copied half of virCPUx86DataAddItem.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Renamed as virCPUx86VendorToData.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

It's called virCPUx86DataAdd now.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>