Since:

  commit 9f4e35dc
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Mon Mar 18 17:31:21 2019 +0000

    network: improve error report when firewall chain creation fails

We cache an error when failing to create the top level firewall chains.
This commit failed to account for fact that we may invoke
networkPreReloadFirewallRules() many times while libvirtd is running.
For example when firewalld is restarted.

When this happens the original failure may no longer occurr and we'll
successfully create our top level chains. We failed to clear the cached
error resulting in us failing to start virtual networks.
Reviewed-by: NLaine Stump <laine@laine.org>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Spotted by Lintian (manpage-has-bad-whatis-entry tag).
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>
Reviewed-by: NMartin Kletzander <mkletzan@redhat.com>

We need commit 6280c94f306d in order to fix our generated
man pages.
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>
Reviewed-by: NMartin Kletzander <mkletzan@redhat.com>

My earlier commit be46f613 was incomplete. It removed caching of
microcode version in the CPU driver, which means the capabilities XML
will see the correct microcode version. But it is also cached in the
QEMU capabilities cache where it is used to detect whether we need to
reprobe QEMU. By missing the second place, the original commit
be46f613 made the situation even worse since libvirt would report
correct microcode version while still using the old host CPU model
(visible in domain capabilities XML).
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Support for kqemu was dropped in libvirt by commit 8e91a400 and even
back then we never set these capabilities when doing QMP probing.

Since no QEMU we aim to support has these, drop them completely.
Signed-off-by: NJán Tomko <jtomko@redhat.com>
Reviewed-by: NAndrea Bolognani <abologna@redhat.com>

In e17d1038 these functions were mistakenly moved into an #ifdef
block, but remained used outside of it leaving the build broken
for platforms where #ifdef evaluated to false.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

Model specific registers are a thing only on x86. Also, the
/dev/cpu/0/msr path exists only on Linux and the fallback
mechanism (asking KVM) exists on Linux and FreeBSD only.

Therefore, move the function within #ifdef that checks all
aforementioned constraints and provide a dummy stub for all
other cases.

This fixes the build on my arm box, mingw-* builds, etc.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>

The NVIDIA V100 GPU has an onboard RAM that is mapped into the
host memory and accessible as normal RAM via an NVLink2 bridge. When
passed through in a guest, QEMU puts the NVIDIA RAM window in a
non-contiguous area, above the PCI MMIO area that starts at 32TiB.
This means that the NVIDIA RAM window starts at 64TiB and go all the
way to 128TiB.

This means that the guest might request a 64-bit window, for each PCI
Host Bridge, that goes all the way to 128TiB. However, the NVIDIA RAM
window isn't counted as regular RAM, thus this window is considered
only for the allocation of the Translation and Control Entry (TCE).
For more information about how NVLink2 support works in QEMU,
refer to the accepted implementation [1].

This memory layout differs from the existing VFIO case, requiring its
own formula. This patch changes the PPC64 code of
@qemuDomainGetMemLockLimitBytes to:

- detect if we have a NVLink2 bridge being passed through to the
guest. This is done by using the @ppc64VFIODeviceIsNV2Bridge function
added in the previous patch. The existence of the NVLink2 bridge in
the guest means that we are dealing with the NVLink2 memory layout;

- if an IBM NVLink2 bridge exists, passthroughLimit is calculated in a
different way to account for the extra memory the TCE table can alloc.
The 64TiB..128TiB window is more than enough to fit all possible
GPUs, thus the memLimit is the same regardless of passing through 1 or
multiple V100 GPUs.

Further reading explaining the background
[1] https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg03700.html
[2] https://www.redhat.com/archives/libvir-list/2019-March/msg00660.html
[3] https://www.redhat.com/archives/libvir-list/2019-April/msg00527.htmlSigned-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: NErik Skultety <eskultet@redhat.com>

The NVLink2 support in QEMU implements the detection of NVLink2
capable devices by verifying the attributes of the VFIO mem region
QEMU allocates for the NVIDIA GPUs. To properly allocate an
adequate amount of memLock, Libvirt needs this information before
a QEMU instance is even created, thus querying QEMU is not
possible and opening a VFIO window is too much.

An alternative is presented in this patch. Making the following
assumptions:

- if we want GPU RAM to be available in the guest, an NVLink2 bridge
must be passed through;

- an unknown PCI device can be classified as a NVLink2 bridge
if its device tree node has 'ibm,gpu', 'ibm,nvlink',
'ibm,nvlink-speed' and 'memory-region'.

This patch introduces a helper called @ppc64VFIODeviceIsNV2Bridge
that checks the device tree node of a given PCI device and
check if it meets the criteria to be a NVLink2 bridge. This
new function will be used in a follow-up patch that, using the
first assumption, will set up the rlimits of the guest
accordingly.
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

In 2878278c we've added new cpu model but we've forgot to
distribute the XML file it comes in.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This does not cause a problem in usual scenarios thanks to us allowing
CAP_DAC_OVERRIDE for the qemu process, however in some scenarios this might be
an issue because the directory is created with mkdtemp(3) which explicitly
creates that with 0700 permissions and qemu running as non-root cannot access
that.

The scenarios include:
 - Builds without CAPNG
 - Running libvirtd in certain container configurations [1]
 - and possibly others.

[1] https://github.com/kubevirt/kubevirt/pull/2181#issuecomment-481840304Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

The new virHostCPUGetMSR internal API will try to read the MSR from
/dev/cpu/0/msr and if it is not possible (the device does not exist or
libvirt is running unprivileged), it will fallback to asking KVM for the
MSR using KVM_GET_MSRS ioctl.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

They are static and we will need to call them a little bit closer to the
beginning of the file.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The structure can only be used for CPUID data now. Adding a type
indicator and moving the data into a union will let us store alternative
data types.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function now works on virCPUx86DataItem and it's called
virCPUx86DataItemMatch.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function is renamed as virCPUx86DataItemMatchMasked to reflect the
change in parameter types.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function now works on virCPUx86DataItem and it's renamed as
virCPUx86DataItemAndBits.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The parameters changed from virCPUx86CPUID to virCPUx86DataItem and the
function is now called virCPUx86DataItemClearBits.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function is renamed as virCPUx86DataItemSetBits and it works on
virCPUx86DataItem now.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

virCPUx86DataSorter already compares two virCPUx86DataItem structs.
Let's add a tiny wrapper around it called virCPUx86DataCmp and use it
instead of open coded comparisons.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The while loop just copied half of virCPUx86DataAddItem.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Renamed as virCPUx86VendorToData.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

It's called virCPUx86DataAdd now.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The new name is virCPUx86DataAddItem.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

It is called virCPUx86DataSorter since the function will work on any CPU
data type.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

It is now called virCPUx86DataGet.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function is now called virCPUx86DataNext to reflect its purpose: it
is an iterator over CPU data (both CPUID and MSR in the near future).
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Although vendor string is always reported by CPUID, the container struct
is used for consistency and thus "cpuid" name is not a good fit anymore.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The following patches introduce CPU features read from MSR in addition
to those queried via CPUID instruction. Let's introduce a container
struct which will be able to describe either feature type.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Introduced in QEMU 3.1.0 by commit
c7a88b52f62b30c04158eeb07f73e3f72221b6a8
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Vim has trouble figuring out the filetype automatically because
the name doesn't follow existing conventions; annotations like
the ones we already have in Makefile.ci help it out.
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

Firstly, virCommandRun() does report an error on failure (which
in most cases is more accurate than what we overwrite it with).
Secondly, usually errno is not set (or gets overwritten in the
cleanup code) which makes virReportSystemError() report useless
error messages. Drop all virReportSystemError() calls in cases
like this (I've found three occurrences).
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

Introduced-by: e0fae78a
Spotted-by: Lintian
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>

The latter is deprecated and will be removed soon. The advised
replacement is '-overcommit mem-lock=on|off'.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Added in QEMU commit of v3.0.0-rc0~48^2~9 (then fixed by
v3.1.0-rc0~119^2~37) QEMU is replacing '-realtime mlock' with
'-overcommit mem-lock'. Add a capability to tell if we're dealing
new new enough qemu to use the replacement.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The '-realtime mlock' cmd line argument was introduced in QEMU
commit v1.5.0-rc0~190 which matches minimal QEMU version we
require. Therefore, the capability will always be present.

Apparently, nearly none of our xml2argv test cases had the
capability hence slightly bigger change under qemuxml2argvdata/.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The 'bandwidths' variable is allocated using VIR_RESIZE_N so it has to
be freed as well.

==118315== 8 bytes in 1 blocks are definitely lost in loss record 299 of 2,401
==118315==    at 0x4C29DAD: malloc (vg_replace_malloc.c:308)
==118315==    by 0x4C2C100: realloc (vg_replace_malloc.c:836)
==118315==    by 0x52C3FAF: virReallocN (viralloc.c:245)
==118315==    by 0x52C4079: virExpandN (viralloc.c:294)
==118315==    by 0x532BBA8: virResctrlAllocParseProcessMemoryBandwidth (virresctrl.c:1156)
==118315==    by 0x532BBA8: virResctrlAllocParseMemoryBandwidthLine (virresctrl.c:1211)
==118315==    by 0x532BBA8: virResctrlAllocParse (virresctrl.c:1414)
==118315==    by 0x532BBA8: virResctrlAllocGetGroup (virresctrl.c:1446)
==118315==    by 0x532C11D: virResctrlAllocGetDefault (virresctrl.c:1464)
==118315==    by 0x532D15E: virResctrlAllocAssign (virresctrl.c:1923)
==118315==    by 0x532D15E: virResctrlAllocCreate (virresctrl.c:2042)
==118315==    by 0x31E1ABEE: qemuProcessResctrlCreate (qemu_process.c:2596)
==118315==    by 0x31E1ABEE: qemuProcessLaunch (qemu_process.c:6444)
==118315==    by 0x31E1E341: qemuProcessStart (qemu_process.c:6721)
==118315==    by 0x31E81315: qemuDomainObjStart.constprop.50 (qemu_driver.c:7288)
==118315==    by 0x31E81A65: qemuDomainCreateWithFlags (qemu_driver.c:7341)
==118315==    by 0x54DDB4B: virDomainCreate (libvirt-domain.c:6534)
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>