When debugging tests under GDB/valgrind there is a significant
delay each time an execve is done as they scan shared libraries
once again. For tests which use many mock libraries, we have
been invoking execve many times which makes the debug experience
horrible. This changes our framework to activate the full
set of mock libraries in one single execve.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

bhyveargv2xmlmock calls virBhyveCapsBuild which in turn
calls virCPUProbeHost, probing the real host CPU. This
causes a test failure if the host CPU happens to contain
the 'arch-capabilities' feature as it triggers a call
to virHostCPUGetMSR() which fails on FreeBSD.

Fortunately we already have convenient code for mocking
the host CPU probing.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

There is no need to require users to produce iSCSI disk source
following our ordering of children elements. In fact, we don't
even accept our own order in the schema :(.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NLaine Stump <laine@redhat.com>

Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: NErik Skultety <eskultet@redhat.com>

Now that we delete the images elsewhere it's not required. Additionally
it's safe to do as we never released an upstream version which required
this being in place.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

The canonicalize_file_name(path) is equivalent to calling
realpath(path, NULL). Passing NULL for the second arg of
realpath is not standardized behaviour, however, Linux,
FreeBSD > 6.4 and macOS > 10.5 all support this critical
extension.

This leaves Windows which doesn't provide realpath at all.
The g_canonicalize_filename() function doesn't expand
symlinks, so is not strictly equivalent to realpath()
but is close enough for our Windows portability needs
right now.
Reviewed-by: NFabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

A few places were importing dirname.h without actually using it.
Reviewed-by: NFabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The last_component() method is a GNULIB custom function
that returns a pointer to the base name in the path.
This is similar to g_path_get_basename() but without the
malloc. The extra malloc is no trouble for libvirt's
needs so we can use g_path_get_basename().
Reviewed-by: NFabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

g_get_real_time() returns the time since epoch in microseconds.
It uses gettimeofday() internally while libvirt used clock_gettime
because it is declared async signal safe. In practice gettimeofday
is also async signal safe *provided* the timezone parameter is
NULL. This is indeed the case in g_get_real_time().
Reviewed-by: NFabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The g_pattern_match function_simple is an acceptably close
approximation of fnmatch for libvirt's needs.

In contrast to fnmatch(), the '/' character can be matched
by the wildcards, there are no '[...]' character ranges and
'*' and '?' can not be escaped to include them literally in
a pattern.
Reviewed-by: NFabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Eliminate direct use of normal setenv/unsetenv calls in
favour of GLib's wrapper. This eliminates two gnulib
modules
Reviewed-by: NFabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

We don't need this for any functional purpose, but when debugging hosts
it is useful to know what binary a given capabilities XML document is
associated with.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

As part of a goal to eliminate Perl from libvirt build tools,
rewrite the check-file-access.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

As part of a goal to eliminate Perl from libvirt build tools,
rewrite the group-qemu-caps.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Tested-by: NCole Robinson <crobinso@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

It will be used to represent the type of a filesystem pool in ESXi.
Signed-off-by: NPino Toscano <ptoscano@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

Previous patch made it possible for the QEMU driver to check if
a given PCI hostdev is unassigned, by checking if dev->info->type is
VIR_DOMAIN_DEVICE_ADDRESS_TYPE_UNASSIGNED, meaning that this device
shouldn't be part of the actual guest launch.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

This patch introduces a new PCI hostdev address type called
'unassigned'. This new type gives users the option to add
PCI hostdevs to the domain XML in an 'unassigned' state, meaning
that the device exists in the domain, is managed by Libvirt
like any regular PCI hostdev, but the guest does not have
access to it.

This adds extra options for managing PCI device binding
inside Libvirt, for example, making all the managed PCI hostdevs
declared in the domain XML to be detached from the host and bind
to the chosen driver and, at the same time, allowing just a
subset of these devices to be usable by the guest.

Next patch will use this new address type in the QEMU driver to
avoid adding unassigned devices to the QEMU launch command line.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Move the validation of vmcoreinfo from qemuBuildVMCoreInfoCommandLine()
to qemuDomainDefValidateFeatures(), allowing for validation
at domain define time.

qemuxml2xmltest.c was changed to account for this caps being
now validated at this earlier stage.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Move smartcard validation being done by qemuBuildSmartcardCommandLine()
to the existing qemuDomainSmartcardDefValidate() function. This
function is called by qemuDomainDeviceDefValidate(), allowing smartcard
validation in domain define time.

Tests were adapted to consider the new caps being needed in
this earlier stage.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Move the NVDIMM validation from qemuBuildMachineCommandLine()
to a new function in qemu_domain.c, qemuDomainDeviceDefValidateMemory(),
which is called by qemuDomainDeviceDefValidate(). This allows
NVDIMM validation to occur in domain define time.

It also increments memory hotplug validation, which can be seen
by the failures in the hotplug tests in qemuxml2xmltest.c that
needed to be adjusted after the move.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

In testXLInitDriver() a dummy driver structure is filled and it
is freed later in testXLFreeDriver(). However, it is sufficient
to unref just driver->config because that results in
libxlDriverConfigDispose() being called which unrefs
driver->config->caps. There is no need to unref it again in
testXLFreeDriver() - in fact it's undesired.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

When generating domain capabilities, we need to fake host CPU to
get reproducible result. We do this by copying a pre-existent CPU
config and setting VIR_TEST_MOCK_FAKE_HOST_CPU env variable which
is then consumed by qemucpumock. However, we forget to free the
CPU copy afterwards.

 2,196 (2,016 direct, 180 indirect) bytes in 18 blocks are definitely lost in loss record 291 of 297
    at 0x4838B86: calloc (vg_replace_malloc.c:762)
    by 0x57CB6A0: g_malloc0 (in /usr/lib64/libglib-2.0.so.0.6000.7)
    by 0x4A0F72D: virCPUDefNew (cpu_conf.c:87)
    by 0x4A0FAC7: virCPUDefCopyWithoutModel (cpu_conf.c:235)
    by 0x4A0FBBE: virCPUDefCopy (cpu_conf.c:273)
    by 0x10E3C0: testUtilsHostCpusGetDefForArch (testutilshostcpus.h:157)
    by 0x10E3C0: fakeHostCPU (domaincapstest.c:61)
    by 0x10E3C0: fillQemuCaps (domaincapstest.c:86)
    by 0x10E3C0: test_virDomainCapsFormat (domaincapstest.c:234)
    by 0x10F4BC: virTestRun (testutils.c:146)
    by 0x10DE93: doTestQemuInternal (domaincapstest.c:301)
    by 0x10E13D: doTestQemu (domaincapstest.c:332)
    by 0x1124CF: testQemuCapsIterate (testutilsqemu.c:635)
    by 0x10DCE3: mymain (domaincapstest.c:435)
    by 0x10FD8B: virTestMain (testutils.c:916)
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

Assuming that the backing image format is raw is wrong when doing image
detection:

1) In -drive mode qemu will still probe the image format of the backing
   image. This means it will try to open a backing file of the image
   which will fail if a more advanced security model is in use.

2) In blockdev mode the image will be opened as raw actually which is
   wrong since it might be qcow. Not opening the backing images will
   also end up in the guest seeing corrupted data.

Rather than attempt to solve various corner cases when us assuming the
storage file being raw and actually being right forbid startup when the
guest image doesn't have the format specified in the metadata.

https://bugzilla.redhat.com/show_bug.cgi?id=1588373Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

EXP_WARN and ALLOW_PROBE flags for the testStorageChain cases are no
longer used so we can remove them.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Pass in 'true' as '@report_broken' of virStorageFileGetMetadata to make
it fail in the tests. The most important code paths (when starting the
VM) expect this function to fail rather than silently return partial
data. Switch the test to exercise this more important code path.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

With NVMe disks, one can start a blockjob with a NVMe disk
that is not visible in domain XML (at least right away). Usually,
it's fairly easy to override this limitation of
qemuDomainGetMemLockLimitBytes() - for instance for hostdevs we
temporarily add the device to domain def, let the function
calculate the limit and then remove the device. But it's not so
easy with virStorageSourcePtr - in some cases they don't
necessarily are attached to a disk. And even if they are it's
done later in the process and frankly, I find it too complicated
to be able to use the simple trick we use with hostdevs.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

Now, that we have everything prepared, we can generate command
line for NVMe disks.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

This capability tracks if qemu is capable of:

  -drive file.driver=nvme

The feature was added in QEMU's commit of v2.12.0-rc0~104^2~2.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
ACKed-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

The device configs (which are actually the same one config)
come from a NVMe disk of mine.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
ACKed-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

To simplify implementation, some restrictions are added. For
instance, an NVMe disk can't go to any bus but virtio and has to
be type of 'disk' and can't have startupPolicy set.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

There is this class of PCI devices that act like disks: NVMe.
Therefore, they are both PCI devices and disks. While we already
have <hostdev/> (and can assign a NVMe device to a domain
successfully) we don't have disk representation. There are three
problems with PCI assignment in case of a NVMe device:

1) domains with <hostdev/> can't be migrated

2) NVMe device is assigned whole, there's no way to assign only a
   namespace

3) Because hypervisors see <hostdev/> they don't put block layer
   on top of it - users don't get all the fancy features like
   snapshots

NVMe namespaces are way of splitting one continuous NVDIMM memory
into smaller ones, effectively creating smaller NVMe-s (which can
then be partitioned, LVMed, etc.)

Because of all of this the following XML was chosen to model a
NVMe device:

  <disk type='nvme' device='disk'>
    <driver name='qemu' type='raw'/>
    <source type='pci' managed='yes' namespace='1'>
      <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </source>
    <target dev='vda' bus='virtio'/>
  </disk>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

In commit 45270337 forgot to make sure that tests pass.
Add the missing capability to fix the test.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

qemuBuildSoundCodecStr() validates if a given QEMU binary
supports the sound codec. This validation can be moved to
qemu_domain.c to be executed in domain define time.

The codec validation was moved to the existing
qemuDomainDeviceDefValidateSound() function.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Move QEMU caps validation of QEMU_CAPS_OBJECT_USB_AUDIO and
QEMU_CAPS_DEVICE_ICH9_INTEL_HDA to a new function in qemu_domain.c,
qemuDomainDeviceDefValidateSound(). This function is called by
qemuDomainDeviceDefValidate() to validate the sound device
in domain define time.

qemuxml2xmltest.c was adjusted to add the now required caps for
domain definition.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Move the SPICE caps validation from qemuBuildGraphicsSPICECommandLine()
to a new function called qemuDomainDeviceDefValidateSPICEGraphics().
This function is called by qemuDomainDeviceDefValidateGraphics(),
which in turn is called by qemuDomainDefValidate(), validating the graphics
parameters in domain define time.

This validation move exposed a flaw in the 'default-video-type' tests
for PPC64, AARCH64 and s390 archs. The XML was considering 'spice' as
the default video type, which isn't true for those architectures.
This was flying under the radar until now because the SPICE validation
was being made in 'virsh start' time, while the XML validation done in
qemuxml2xmltest.c considers define time.

All other tests were adapted to consider SPICE validation in this
earlier stage.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Move the VNC cap validation from qemuBuildGraphicsVNCCommandLine()
to qemuDomainDeviceDefValidateGraphics(). This function is called by
qemuDomainDefValidate(), validating the graphics parameters in domain
define time.

Tests were adapted to consider SDL validation in this earlier stage.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

There are validations for SDL, VNC, SPICE and EGL_HEADLESS
around several BuildGraphics*CommandLine in qemu_command.c. This
patch starts to move all of them to qemu_domain.c, inside the
existent qemuDomainDeviceDefValidateGraphics() function. This
function is called by qemuDomainDefValidate(), validating the
graphics parameters in domain define time.

In this patch we'll move the SDL validation code from
qemuBuildGraphicsSDLCommandLine(). Tests were adapted to consider
SDL validation in this earlier stage.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>

Move the pcihole64 validation being done by
qemuBuildGlobalControllerCommandLine() to the existing function
qemuDomainDeviceDefValidateControllerPCI(), which provides
domain define time validation.

The existing pcihole64 validations in qemu_domain.c were replaced
by the ones moved from qemu_command.c. The reason is that they
are more specific, allowing VIR_DOMAIN_CONTROLLER_MODEL_PCI_ROOT
and VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT to have distinct validation,
with exclusive QEMU caps and machine types.

Tests were adapted to consider the new caps being needed in
this earlier stage.
Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb413@gmail.com>