https://bugzilla.redhat.com/show_bug.cgi?id=1019053

When we migrate vms concurrently, there's a chance that libvirtd on
destination assigns the same port for different migrations, which will
lead to migration failure during prepare phase on destination. So we use
virPortAllocator here to solve the problem.
Signed-off-by: NWang Yufei <james.wangyufei@huawei.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
(cherry picked from commit 0196845d)

Conflicts:
  missing support for changing the migration listen address
	src/qemu/qemu_migration.c

We were unconditionally removing the device from the host list, when it
should only be done on error.

This fixes USB collision detection when hotplugging the same device to
two guests.

(cherry picked from commit 586b0ed8)

If we hit a collision, we free the USB device while it is still part
of our temporary USBDeviceList. When the list is unref'd, the device
is free'd again.

Make the initial device freeing dependent on whether it is present
in the temporary list or not.

(cherry picked from commit 5953a737)

Similar to what Jiri did for cgroup setup/teardown in 05e149f9, push
it all into the device handler functions so we can do the necessary prep
work before claiming the device.

This also fixes hotplugging USB devices by product/vendor (virt-manager's
default behavior):

https://bugzilla.redhat.com/show_bug.cgi?id=1016511
(cherry picked from commit ee414b5d)

They aren't used outside of qemu_hotplug.c

(cherry picked from commit 79776aa5)

https://bugzilla.redhat.com/show_bug.cgi?id=1025108

So far qemuSetupHostdevCGroup was called very early during hotplug, even
before we knew the device we were about to hotplug was actually
available. By calling the function later, we make sure QEMU won't be
allowed to access devices used by other domains.

Another important effect of this change is that hopluging USB devices
specified by vendor and product (but not by their USB address) works
again. This was broken since v1.0.5-171-g7d763aca, when the call to
qemuFindHostdevUSBDevice was moved after the call to
qemuSetupHostdevCGroup, which then used an uninitialized USB address.

(cherry picked from commit 05e149f9)

To simplify future patches dealing with this code, simplify and refactor
some conditions to switch statements.

(cherry picked from commit 9d132989)

When doing an internal snapshot on a VM with sheepdog or RBD disks we
would not set a flag to mark the domain is using internal snapshots and
might end up creating a mixed snapshot. Move the setting of the variable
to avoid this problem.

(cherry picked from commit d8cf91ae)

QEMU 1.6.0 introduced new migration status: setup
Libvirt does not expect such string in QMP and refuses to migrate with error
"unexpected migration status in setup"

This patch fixes it.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
(cherry picked from commit d35ae414)

Since qemu-kvm 1.1 [1] (since 1.3. in upstream QEMU [2])
'-no-kvm-pit-reinjection' has been deprecated.
Use -global kvm-pit.lost_tick_policy=discard instead.

https://bugzilla.redhat.com/show_bug.cgi?id=978719

[1] http://git.kernel.org/cgit/virt/kvm/qemu-kvm.git/commit/?id=4e4fa39
[2] http://git.qemu.org/?p=qemu.git;a=commitdiff;h=c21fb4f

(cherry picked from commit 1569fa14)

Conflicts:
	tests/qemucapabilitiesdata/caps_1.2.2-1.caps
	tests/qemucapabilitiesdata/caps_1.2.2-1.replies
	tests/qemucapabilitiesdata/caps_1.3.1-1.caps
	tests/qemucapabilitiesdata/caps_1.3.1-1.replies
	tests/qemucapabilitiesdata/caps_1.4.2-1.caps
	tests/qemucapabilitiesdata/caps_1.4.2-1.replies
	tests/qemucapabilitiesdata/caps_1.5.3-1.caps
	tests/qemucapabilitiesdata/caps_1.5.3-1.replies
	tests/qemucapabilitiesdata/caps_1.6.0-1.caps
	tests/qemucapabilitiesdata/caps_1.6.0-1.replies
	tests/qemucapabilitiesdata/caps_1.6.50-1.caps
	tests/qemucapabilitiesdata/caps_1.6.50-1.replies
(qemucapabilitiestest is not backported)

Since 86d90b3a (yes, my patch; again) we are supporting NBD storage
migration. However, on error recovery path we got the steps reversed.
The correct order is: return NBD port to the virPortAllocator and then
either unlock the vm or remove it from the driver. Not vice versa.

==11192== Invalid write of size 4
==11192==    at 0x11488559: qemuMigrationPrepareAny (qemu_migration.c:2459)
==11192==    by 0x11488EA6: qemuMigrationPrepareDirect (qemu_migration.c:2652)
==11192==    by 0x114D1509: qemuDomainMigratePrepare3Params (qemu_driver.c:10332)
==11192==    by 0x519075D: virDomainMigratePrepare3Params (libvirt.c:7290)
==11192==    by 0x1502DA: remoteDispatchDomainMigratePrepare3Params (remote.c:4798)
==11192==    by 0x12DECA: remoteDispatchDomainMigratePrepare3ParamsHelper (remote_dispatch.h:5741)
==11192==    by 0x5212127: virNetServerProgramDispatchCall (virnetserverprogram.c:435)
==11192==    by 0x5211C86: virNetServerProgramDispatch (virnetserverprogram.c:305)
==11192==    by 0x520A8FD: virNetServerProcessMsg (virnetserver.c:165)
==11192==    by 0x520A9E1: virNetServerHandleJob (virnetserver.c:186)
==11192==    by 0x50DA78F: virThreadPoolWorker (virthreadpool.c:144)
==11192==    by 0x50DA11C: virThreadHelper (virthreadpthread.c:161)
==11192==  Address 0x1368baa0 is 576 bytes inside a block of size 688 free'd
==11192==    at 0x4A07F5C: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==11192==    by 0x5079A2F: virFree (viralloc.c:580)
==11192==    by 0x11456C34: qemuDomainObjPrivateFree (qemu_domain.c:267)
==11192==    by 0x50F41B4: virDomainObjDispose (domain_conf.c:2034)
==11192==    by 0x50C2991: virObjectUnref (virobject.c:262)
==11192==    by 0x50F4CFC: virDomainObjListRemove (domain_conf.c:2361)
==11192==    by 0x1145C125: qemuDomainRemoveInactive (qemu_domain.c:2087)
==11192==    by 0x11488520: qemuMigrationPrepareAny (qemu_migration.c:2456)
==11192==    by 0x11488EA6: qemuMigrationPrepareDirect (qemu_migration.c:2652)
==11192==    by 0x114D1509: qemuDomainMigratePrepare3Params (qemu_driver.c:10332)
==11192==    by 0x519075D: virDomainMigratePrepare3Params (libvirt.c:7290)
==11192==    by 0x1502DA: remoteDispatchDomainMigratePrepare3Params (remote.c:4798)
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
(cherry picked from commit 1f2f879e)

The following sequence

 1. Define a persistent QMEU guest
 2. Start the QEMU guest
 3. Stop libvirtd
 4. Kill the QEMU process
 5. Start libvirtd
 6. List persistent guests

At the last step, the previously running persistent guest
will be missing. This is because of a race condition in the
QEMU driver startup code. It does

 1. Load all VM state files
 2. Spawn thread to reconnect to each VM
 3. Load all VM config files

Only at the end of step 3, does the 'virDomainObjPtr' get
marked as "persistent". There is therefore a window where
the thread reconnecting to the VM will remove the persistent
VM from the list.

The easy fix is to simply switch the order of steps 2 & 3.

In addition to this though, we must only attempt to reconnect
to a VM which had a non-zero PID loaded from its state file.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit f26701f5)

The 'error' cleanup block in qemuProcessReconnect() had a
'return' statement in the middle of it. This caused a leak
of virConnectPtr & virQEMUDriverConfigPtr instances. This
was identified because netcf recently started checking its
refcount in libvirtd shutdown:

netcfStateCleanup:109 : internal error: Attempt to close netcf state driver with open connections
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit 54a24112)

Most of the usage of getuid()/getgid() is in cases where we are
considering what privileges we have. As such the code should be
using the effective IDs, not real IDs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit 9b0af092)

When running setuid, we must be careful about what env vars
we allow commands to inherit from us. Replace the
virCommandAddEnvPass function with two new ones which do
filtering

  virCommandAddEnvPassAllowSUID
  virCommandAddEnvPassBlockSUID

And make virCommandAddEnvPassCommon use the appropriate
ones
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit 9b8f307c)

Introduced by 1fa7946f.

https://bugzilla.redhat.com/show_bug.cgi?id=1019023
(cherry picked from commit 15fac93b)

After successful @cmd construction the memory where @keys points to is
part of @cmd. Avoid double freeing it.
(cherry picked from commit 3e8343e1)

The virConnectPtr is passed around loads of nwfilter code in
order to provide it as a parameter to the callback registered
by the virt drivers. None of the virt drivers use this param
though, so it serves no purpose.

Avoiding the need to pass a virConnectPtr means that the
nwfilterStateReload method no longer needs to open a bogus
QEMU driver connection. This addresses a race condition that
can lead to a crash on startup.

The nwfilter driver starts before the QEMU driver and registers
some callbacks with DBus to detect firewalld reload. If the
firewalld reload happens while the QEMU driver is still starting
up though, the nwfilterStateReload method will open a connection
to the partially initialized QEMU driver and cause a crash.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit 999d72fb)

We can dereference graphics[0] even if guest has no graphics device
configured. I screwed this up in a216e648

https://bugzilla.redhat.com/show_bug.cgi?id=1014088
(cherry picked from commit a924d9d0)

The qemuDomainChangeNet() is called when 'virsh update-device' is
invoked on a NIC. Currently, we fail to update the QoS even though
we have routines for that.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
(cherry picked from commit 9fa10d39)

The change in ef29de14 that introduced
better error logging from qemu introduced a warning from coverity about
unused return value from lseek. Silence this warning and fix typo in the
corresponding error message.

Reported by: John Ferlan

https://bugzilla.redhat.com/show_bug.cgi?id=1011330 (case A)

While activeScsiHostdevs and webSocketPorts were allocated in
qemuStateInitialize, they were not freed in qemuStateCleanup.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1011330 (case D)

qemuProcessStart created two references to virQEMUDriverConfigPtr before
calling fork():

    cfg = virQEMUDriverGetConfig(driver);
    ...
    hookData.cfg = virObjectRef(cfg);

However, the child only unreferenced hookData.cfg and the parent only
removed the cfg reference. That said, we don't need to increment the
reference counter when assigning cfg to hookData. Both the child and the
parent will correctly remove the reference on cfg (the child will do
that through hookData).
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The return value of virDomainControllerFind >=0 means that
the specific controller was found.
But some functions invoke it and treat 0 as not found.
This patch fix these incorrect invocation.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

If qemuParseCommandLine finds an arg it does not understand
it adds it to the QEMU passthrough custom arg list. If the
qemuParseCommandLine method hits an error for any reason
though, it just does 'VIR_FREE(cmd)' on the custom arg list.
This means all actual args / env vars are leaked. Introduce
a qemuDomainCmdlineDefFree method to be used for cleanup.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If the call to virDomainControllerInsert fails in
qemuParseCommandLine, the controller struct is leaked.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The 'qemuStringToArgvEnv' method splits up a string of command
line env/args to an 'arglist' array. It then copies env vars
to a 'progenv' array and args to a 'progargv' array. When
copyin the env vars, it NULL-ifies the element in 'arglist'
that is copied.

Upon OOM the 'virStringListFree' is called on progenv and
arglist. Unfortunately, because the elements in 'arglist'
related to env vars have been set to NULL, the call to
virStringListFree(arglist) doesn't free anything, even
though some non-NULL args vars still exist later in the
array.

To fix this leak, stop NULL-ifying the 'arglist' elements,
and change the cleanup code to only free elements in the
'arglist' array, not 'progenv'.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

In a number of places in qemuParseCommandLineDisk, an error
is reported, but no 'goto error' jump is used. This causes
failure to report OOM conditions to the caller.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If OOM occurs in qemuParseCommandLineDisk some intermediate
variables will be leaked when parsing Sheepdog or RBD disks.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The qemuBuildCommandLine code for parsing sound cards will leak
an intermediate variable if an OOM occurs. Move the free'ing of
the variable earlier to avoid the leak.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

In qemuParseNBDString, if the virURIParse fails, the
error is not reported to the caller. Instead execution
falls through to the non-URI codepath causing memory
leaks later on.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If qemuAddRBDHost fails due to parsing problems or OOM, then
qemuParseRBDString cleanup is skipped causing a memory leak.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

qemuDomainPCIAddressGetNextSlot has a loop for finding
compatible PCI buses. In the loop body it creates a
PCI address string, but never frees this. This causes
a leak if the loop executes more than one iteration,
or if a call in the loop body fails.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This resolves one of the issues listed in:

   https://bugzilla.redhat.com/show_bug.cgi?id=1003983

00:1E.0 is the location of this controller on at least some actual Q35
hardware, so we try to replicate the placement. The bridge should work
just as well in any other location though, so if 00:1E.0 isn't
available, just allow it to be auto-assigned anywhere appropriate.

This will make it simpler to add checks for other types of
controllers.

This is a prerequisite for patches to resolve:

   https://bugzilla.redhat.com/show_bug.cgi?id=1003983

This resolves one of the issues in:

   https://bugzilla.redhat.com/show_bug.cgi?id=1003983

This device is identical to qemu's "intel-hda" device (known as "ich6"
in libvirt), but has a different PCI device ID (which matches the ID
of the hda audio built into the ich9 chipset, of course). It's not
supported in earlier versions of qemu, so it requires a capability
bit.

I'm not sure why this code was written to compare the strings that it
had just retrieved from an enum->string conversion, rather than just
look at the original enum values, but this yields the same results,
and is much more efficient (especially as you add more devices).

This is a prerequisite for patches to resolve:

   https://bugzilla.redhat.com/show_bug.cgi?id=1003983

Part of the resolution to:

   https://bugzilla.redhat.com/show_bug.cgi?id=1003983

Although most devices available in qemu area defined as PCI devices,
and strictly speaking should only be attached via a PCI slot, in
practice qemu allows them to be attached to a PCIe slot and sometimes
this makes sense.

For example, The UHCI and EHCI USB controllers are usually attached
directly to the PCIe "root complex" (i.e. PCIe slots) on real
hardware, so that should be possible for a Q35-based qemu virtual
machine as well.

We still want to prefer a standard PCI slot when auto-assigning
addresses, though, and in general to disallow attaching PCI devices
via PCIe slots.

This patch makes that possible by adding a new
QEMU_PCI_CONNECT_TYPE_EITHER_IF_CONFIG flag. Three things are done
with this flag:

1) It is set for the "pcie-root" controller

2) qemuCollectPCIAddress() now has a set of nested switches that set
this "EITHER" flag for devices that we want to allow connecting to
pcie-root when specifically requested in the config.

3) qemuDomainPCIAddressFlagsCompatible() adds this new flag to the
"flagsMatchMask" if the address being checked came from config rather
than being newly auto-allocated by libvirt (this knowledge is
conveniently already available in the "fromConfig" arg).

Now any device having the EITHER flag set can be connected to
pcie-root if explicitly requested, but auto-allocated addresses for
those devices will still be standard PCI slots instead.

This patch only loosens the restrictions on devices that have been
specifically requested, but the setup is such that it should be fairly
easy to add new devices.

Replace them with switch cases. This will make it more efficient when
we add exceptions for more controller types, and other device types.

This is a prerequisite for patches to resolve:

   https://bugzilla.redhat.com/show_bug.cgi?id=1003983

The previous patches added infrastructure to report better errors from
monitor in some cases. This patch finalizes this "feature" by enabling
this enhanced error reporting on early phases of VM startup. In these
phases the possibility of qemu producing a useful error message is
really high compared to running it during the whole life cycle. After
the start up is complete, the feature is disabled to provide the usual
error messages so that users are not confused by possibly irrelevant
messages that may be in the domain log.

The original motivation to do this enhancement is to capture errors when
using VFIO device passthrough, where qemu reports errors after the
monitor is initialized and the existing error catching code couldn't
catch this producing a unhelpful message:

 # virsh start test
 error: Failed to start domain test
 error: Unable to read from monitor: Connection reset by peer

With this change, the message is changed to:

 # virsh start test
 error: Failed to start domain test
 error: internal error: early end of file from monitor: possible problem:
 qemu-system-x86_64: -device vfio-pci,host=00:1a.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: error, group 8 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
 qemu-system-x86_64: -device vfio-pci,host=00:1a.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to get group 8
 qemu-system-x86_64: -device vfio-pci,host=00:1a.0,id=hostdev0,bus=pci.0,addr=0x5: Device 'vfio-pci' could not be initialized