There were numerous places where numatune configuration (and thus
domain config as well) was changed in different ways.  On some
places this even resulted in persistent domain definition not to be
stable (it would change with daemon's restart).

In order to uniformly change how numatune config is dealt with, all
the internals are now accessible directly only in numatune_conf.c and
outside this file accessors must be used.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Since there was already public virDomainNumatune*, I changed the
private virNumaTune to match the same, so all the uses are unified and
public API is kept:

s/vir\(Domain\)\?Numa[tT]une/virDomainNumatune/g

then shrunk long lines, and mainly functions, that were created after
that:

sed -i 's/virDomainNumatuneMemPlacementMode/virDomainNumatunePlacement/g'

And to cope with the enum name, I haad to change the constants as
well:

s/VIR_NUMA_TUNE_MEM_PLACEMENT_MODE/VIR_DOMAIN_NUMATUNE_PLACEMENT/g

Last thing I did was at least a little shortening of already long
name:

s/virDomainNumatuneDef/virDomainNumatune/g
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

There are many places with numatune-related code that should be put
into special numatune_conf and this patch creates a basis for that.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

In XML format, by definition, order of fields should not matter, so
order of parsing the elements doesn't affect the end result.  When
specifying guest NUMA cells, we depend only on the order of the 'cell'
elements.  With this patch all older domain XMLs are parsed as before,
but with the 'id' attribute they are parsed and formatted according to
that field.  This will be useful when we have tuning settings for
particular guest NUMA node.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Excerpt from the virCommandAddArgBuffer() description: "Correctly
transfers memory errors or contents from buf to cmd."
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

to ease the review of commits to follow.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

This patch adds support for the QEMU vhost-user feature to libvirt.
vhost-user enables the communication between a QEMU virtual machine
and other userspace process using the Virtio transport protocol.
It uses a char dev (e.g. Unix socket) for the control plane,
while the data plane based on shared memory.

The XML looks like:

<interface type='vhostuser'>
    <mac address='52:54:00:3b:83:1a'/>
    <source type='unix' path='/tmp/vhost.sock' mode='server'/>
    <model type='virtio'/>
</interface>
Signed-off-by: NMichele Paolino <m.paolino@virtualopensystems.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1119173 documents that
commit eaba79d2 was flawed in the implementation of the
VIR_DOMAIN_BLOCK_JOB_ABORT_ASYNC flag when it comes to completing
a blockcopy.  Basically, the qemu pivot action is async (the QMP
command returns immediately, but the user must wait for the
BLOCK_JOB_COMPLETE event to know that all I/O related to the job
has finally been flushed), but the libvirt command was documented
as synchronous by default.  As active block commit will also be
using this code, it is worth fixing now.

* src/qemu/qemu_driver.c (qemuDomainBlockJobImpl): Don't skip wait
loop after pivot.
Signed-off-by: NEric Blake <eblake@redhat.com>

In many places we define a variable as a 'const char *' when in fact
we modify it just a few lines below. Or even free it. We should not do
that.

There's one exception though, in xenSessionFree() xenapi_utils.c. We
are freeing the xen_session structure which is defined in
xen/api/xen_common.h public header. The structure contains session_id
which is type of 'const char *' when in fact it should have been just
'char *'. So I'm leaving this unmodified, just noticing the fact in
comment.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Rename linuxDomainInterfaceStats to virNetInterfaceStats in order
to allow adding platform specific implementations without
making consumer worrying about specific implementation to be used.

Also, rename util/virstatslinux.c to util/virstats.c so placing
other platform specific implementations into this file don't
look unexpected from the file name.

4cc1f1a0 introduced a crash when doing a
block copy as virStorageSourceInitChainElement was called on
"disk->mirror" that is still NULL at that point instead of "mirror"
which temporarily holds the mirror source struct until it's fully
initialized. This resulted into a crash as a NULL was dereferenced.

Reported by: Shanzi Yu <shyu@redhat.com>

Commit id '3ea661de' refactored the code to use the 'disk->src->path'
instead of getting the path from virDomainDiskGetSource().  The one
call to qemuOpenFile() didn't use the disk source path, rather it used
the path as passed from the caller (in this case 'vda') - this caused
a failure with the virt-test/tp-libvirt as follows:

$ virsh domblkinfo virt-tests-vm1 vda
error: cannot stat file '/home/virt-test/shared/data/images/jeos-20-64.qcow2': Bad file descriptor

$

While preparing to add a capability for active commit, I noticed
that the existing code was abusing int for boolean values.

* src/conf/capabilities.h (_virCapsGuestFeature, _virCapsHost)
(virCapabilitiesNew, virCapabilitiesAddGuestFeature): Improve
types.
* src/conf/capabilities.c (virCapabilitiesNew)
(virCapabilitiesAddGuestFeature): Adjust signature.
* src/bhyve/bhyve_capabilities.c (virBhyveCapsBuild): Update
clients.
* src/esx/esx_driver.c (esxCapsInit): Likewise.
* src/libxl/libxl_conf.c (libxlMakeCapabilities): Likewise.
* src/lxc/lxc_conf.c (virLXCDriverCapsInit): Likewise.
* src/openvz/openvz_conf.c (openvzCapsInit): Likewise.
* src/parallels/parallels_driver.c (parallelsBuildCapabilities):
Likewise.
* src/phyp/phyp_driver.c (phypCapsInit): Likewise.
* src/qemu/qemu_capabilities.c (virQEMUCapsInit)
(virQEMUCapsInitGuestFromBinary): Likewise.
* src/security/virt-aa-helper.c (get_definition): Likewise.
* src/test/test_driver.c (testBuildCapabilities): Likewise.
* src/uml/uml_conf.c (umlCapsInit): Likewise.
* src/vbox/vbox_tmpl.c (vboxCapsInit): Likewise.
* src/vmware/vmware_conf.c (vmwareCapsInit): Likewise.
* src/xen/xen_hypervisor.c (xenHypervisorBuildCapabilities):
Likewise.
* src/xenapi/xenapi_driver.c (getCapsObject): Likewise.
* tests/qemucaps2xmltest.c (testGetCaps): Likewise.
* tests/testutils.c (virTestGenericCapsInit): Likewise.
* tests/testutilslxc.c (testLXCCapsInit): Likewise.
* tests/testutilsqemu.c (testQemuCapsInit): Likewise.
* tests/testutilsxen.c (testXenCapsInit): Likewise.
* tests/vircaps2xmltest.c (buildVirCapabilities): Likewise.
* tests/vircapstest.c (buildNUMATopology): Likewise.
* tests/vmx2xmltest.c (testCapsInit): Likewise.
* tests/xml2vmxtest.c (testCapsInit): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

This saves a few lines of code and catches the error when:
<spice autoport ='yes' defaultMode='any' ..>
  <channel name='main' mode='secure'/>
</spice>
is specified with spice_tls = 0 in qemu.conf.

Instead of this error in qemuBuildGraphicsSPICECommandLine:
error: unsupported configuration: spice secure channels set in XML
configuration, but TLS port is not provided

an error is reported in qemuProcessSPICEAllocatePorts:
error: unsupported configuration: Auto allocation of spice TLS port
requested but spice TLS is disabled in qemu.conf

Inspired by:
https://www.redhat.com/archives/libvir-list/2014-June/msg01408.html

When creating cgroups for vcpu and emulator threads whilst starting a
domain, we explicitly skip creating those cgroups in case priv->cgroup
is NULL (cgroups not supported) because SetAffinity() serves the same
purpose.  If the host supports only some cgroups (the ones we need are
either unmounted or disabled in qemu.conf), we error out with weird
message even though we could continue starting the domain.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1097028Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

The default graphics channel mode is 'any', so as to defaultMode attribute.
If defaultMode and channel mode are all the default value 'any',
qemuConnectDomainXMLToNative will set TLSPort.
But in qemuBuildGraphicsSPICECommandLine, if spice_tls is not enabled, libvirtd
will report an error to tell the user that spice TLS is disabled in qemu.conf.

So qemuConnectDomainXMLToNative should check spice_tls is enabled,
then decide to allocate an tlsPort number to this graphics.

If user specified defaultMode is 'secure', qemuConnectDomainXMLToNative
could allocate tlsPort, and then let qemuBuildGraphicsSPICECommandLine reports
the spice_tls disabled error.

The related bug is:
https://bugzilla.redhat.com/show_bug.cgi?id=1113868Signed-off-by: NJincheng Miao <jmiao@redhat.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Now that cgroups/security driver/locking driver support labelling of
individual images and tolerate network storage we don't have to refrain
from passing all image files to it. This allows removing the checking
code as we already make sure that the snapshot function won't be called
with unsupported options.

Now that security, cgroup and locking APIs support working on individual
images and we track the backing chain security info on a per-image basis
we can finally kill swapping the disk source in virDomainDiskDef and use
the virStorageSource directly.

Until now we were changing information about the disk source via
multiple steps of copying data. Now that we changed to a pointer to
store the disk source we might use it to change the approach to track
the data.

Additionally this will allow proper tracking of the backing chain.

When pivoting to a new disk source after a block commit (and possibly
after a soon-to-be-added active block commit) we changed just a few
fields to the new target. In case we'd copy a network disk to a local
file we'd not change the type properly.

To avoid such problems, switch to tracking of the source via changing of
the complete source struct to the one tracking the mirroring info.

Use the source struct and the corresponding function so that we can
avoid using the path separately. Now that
qemuDomainPrepareDiskChainElementPath isn't use anywhere, we can safely
remove it.

Additionally, the removal fixes a misaligned comment as the removed
function was added under a comment for a different function.

Instead of just returning the parent path, return the complete parent
source structure.

Only the top level gets writes, so the rest of the backing chain
requires only read-only access.

Add functions that will allow to set all the required cgroup stuff on
individual images taking a virStorageSourcePtr. Also convert functions
designed to setup whole backing chain to take advantage of the change.

Qemu will fallback to aio=threads when the cache mode doesn't use
O_DIRECT, even if aio=native was explictly set.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1086704Signed-off-by: NGiuseppe Scrivano <gscrivan@redhat.com>

In the future we might need to track state of individual images. Move
the readonly and shared flags to the virStorageSource struct so that we
can keep them in a per-image basis.

Add a wrapper that determines the correct uid and gid for a certain
storage file and domain.

It will help re-using the function.

Now that we are able to select images from the backing chain via indexed
access we should also convert possible network sources to
qemu-compatible strings before passing them to qemu.

Now that we are able to select images from the backing chain via indexed
access we should also convert possible network sources to
qemu-compatible strings before passing them to qemu.

The qemu block info function relied on working with local storage. Break
this assumption by adding support for remote volumes. Unfortunately we
still need to take a hybrid approach as some of the operations require a
filedescriptor.

Previously you'd get:
 $ virsh domblkinfo gl vda
 error: cannot stat file '/img10': Bad file descriptor

Now you get some stats:
 $ virsh domblkinfo gl vda
 Capacity:       10485760
 Allocation:     197120
 Physical:       197120

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1110198

Add startup auditing and also hotplug auditing for said devices.

For the regular dump operation we migrate the VM to a file. This won't
work when the VM has passthrough devices assigned. Rather than reporting
a cryptic error from qemu run our check whether it can be migrated.

This does not influence the memory-only dump that is allowed with
passthrough devices.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=874418

To allow changing the name that is recorded in the top of the current
image chain used in a block pull/rebase operation, we need to specify
the backing name to qemu. This is done via the "backing-file" attribute
to the block-stream commad.

To allow changing the name that is recorded in the overlay of the TOP
image used in a block commit operation, we need to specify the backing
name to qemu. This is done via the "backing-file" attribute to the
block-commit command.

This command allows to change the backing file name recorded in the
metadata of a qcow (or other) image. The capability also notifies that
the "block-stream" and "block-commit" commands understand the
"backing-file" attribute.

Pass the virStorageSource struct to the auditing function and check if
storage is local before auditing.

Replace the authType, chap, and cephx unions in virStoragePoolSource
with a single pointer to a virStorageAuthDefPtr.  Adjust all users of
the previous chap/cephx and secret unions with the source->auth data.

Replace the inline "auth" struct in virStorageSource with a pointer
to a virStorageAuthDefPtr and utilize between the domain_conf, qemu_conf,
and qemu_command sources for finding the auth data for a domain disk

Use the probing functionality added in the last patch to turn on
a capability bit when active commit is present, and gate active
commit on that capability.

For my own reference: the difference between BLOCKJOB_SYNC and
BLOCKJOB_ASYNC is whether qemu generated an event at the
conclusion of blockpull; basically, RHEL 6.2 was the only release
of qemu that has the sync semantics and lacks the event.  RHEL
6.3 added blockcopy, but also picked up on the upstream style
of qemu generating events.  As no one is likely to backport
active commit to RHEL 6.2, it's safe for blockcommit to always
require async blockjob support.

Modifying qemucapabilitiestest is painful; the .replies files would
be so much easier if they had comments correlating which command
generated the given reply.  Maybe I'll fix that up later...

* src/qemu/qemu_capabilities.h (QEMU_CAPS_ACTIVE_COMMIT): New
capability.
* src/qemu/qemu_driver.c (qemuDomainBlockCommit): Use the new bit
* src/qemu/qemu_capabilities.c (virQEMUCaps): Name the new bit.
(virQEMUCapsProbeQMPCommands): Set it.
* tests/qemucapabilitiesdata/caps_1.3.1-1.replies: Update.
* tests/qemucapabilitiesdata/caps_1.4.2-1.replies: Likewise.
* tests/qemucapabilitiesdata/caps_1.5.3-1.replies: Likewise.
* tests/qemucapabilitiesdata/caps_1.6.0-1.replies: Likewise.
* tests/qemucapabilitiesdata/caps_1.6.50-1.replies: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>