This wires up the XML changes in the previous patch to let SELinux
labeling honor user overrides, as well as affecting the live XML
configuration in one case where the user didn't specify anything
in the offline XML.

I noticed that the logs contained messages like this:

2011-12-05 23:32:40.382+0000: 26569: warning : SELinuxRestoreSecurityFileLabel:533 : cannot lookup default selinux label for /nfs/libvirt/images/dom.img

for all my domain images living on NFS.  But if we would just remember
that on domain creation that we were unable to set a SELinux label (due to
NFSv3 lacking labels, or NFSv4 not being configured to expose attributes),
then we could avoid wasting the time trying to clear the label on
domain shutdown.  This in turn is one less point of NFS failure,
especially since there have been documented cases of virDomainDestroy
hanging during an attempted operation on a failed NFS connection.

* src/security/security_selinux.c (SELinuxSetFilecon): Move guts...
(SELinuxSetFileconHelper): ...to new function.
(SELinuxSetFileconOptional): New function.
(SELinuxSetSecurityFileLabel): Honor override label, and remember
if labeling failed.
(SELinuxRestoreSecurityImageLabelInt): Skip relabeling based on
override.

Implement the parsing and formatting of the XML addition of
the previous commit.  The new XML doesn't affect qemu command
line, so we can now test round-trip XML->memory->XML handling.

I chose to reuse the existing structure, even though per-device
override doesn't use all of those fields, rather than create a
new structure, in order to reuse more code.

* src/conf/domain_conf.h (_virDomainDiskDef): Add seclabel member.
* src/conf/domain_conf.c (virDomainDiskDefFree): Free it.
(virSecurityLabelDefFree): New function.
(virDomainDiskDefFormat): Print it.
(virSecurityLabelDefFormat): Reduce output if model not present.
(virDomainDiskDefParseXML): Alter signature, and parse seclabel.
(virSecurityLabelDefParseXML): Split...
(virSecurityLabelDefParseXMLHelper): ...into new helper.
(virDomainDeviceDefParse, virDomainDefParseXML): Update callers.
* tests/qemuxml2argvdata/qemuxml2argv-seclabel-dynamic-override.args:
New file.
* tests/qemuxml2xmltest.c (mymain): Enhance test.
* tests/qemuxml2argvtest.c (mymain): Likewise.

Pure code motion; no semantic change.

* src/conf/domain_conf.h (virDomainSeclabelType)
(virSecurityLabelDefPtr): Declare earlier.
* src/conf/domain_conf.c (virSecurityLabelDefClear)
(virSecurityLabelDefParseXML): Move earlier.
(virDomainDefParseXML): Move seclabel parsing earlier.

A future patch will parse and output <seclabel> in more than one
location in a <domain> xml; make it easier to reuse code.

* src/conf/domain_conf.c (virSecurityLabelDefFree): Rename...
(virSecurityLabelDefClear): ...and make static.
(virSecurityLabelDefParseXML): Alter signature.
(virDomainDefParseXML, virDomainDefFree): Adjust callers.
(virDomainDefFormatInternal): Split output...
(virSecurityLabelDefFormat): ...into new helper.

* src/qemu/qemu_driver.c: implement the qemu driver support

Add a util function virDomainNetFind to find a domain's net def.

* daemon/remote.c: implement the server side support
* src/remote/remote_driver.c: implement the client side support
* src/remote/remote_protocol.x: definitions for the new entry points
* src/remote_protocol-structs: structure definitions

* src/libvirt.c: implement the main entry points

The APIs are used to set/get domain's network interface's parameters.
Currently supported parameters are bandwidth settings.

* include/libvirt/libvirt.h.in: new API and parameters definition
* python/generator.py: skip the Python API generation
* src/driver.h: add new entry to the driver structure
* src/libvirt_public.syms: export symbols

https://bugzilla.redhat.com/show_bug.cgi?id=770520

We had two nested loops both trying to use 'i' as the iteration
variable, which can result in an infinite loop when the inner
loop interferes with the outer loop.  Introduced in commit 93ab5859.

* src/qemu/qemu_driver.c (qemuDomainSetBlkioParameters): Don't
reuse iteration variable across two loops.

In order to avoid situation where a USB device is
in use by two domains, we must keep a list of already
attached devices like we do for PCI.

This patch adds max_files option to qemu.conf which can be used to
override system default limit on number of opened files that are
allowed for qemu user.

Upstream QEMU starts to support it from commit 2c74c2cb.

Remove the requirement that DHCP messages have to be broadcasted.
DHCP requests are most often sent via broadcast but can be directed
towards a specific DHCP server. For example 'dhclient' takes '-s <server>'
as a command line parameter thus allowing DHCP requests to be sent to a
specific DHCP server.

Add logic to assign addresses for devices with spapr-vio addresses.

We also do validation of addresses specified by the user, ie. ensuring
that there are not duplicate addresses on the bus.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>

For QEMU PPC64 we have a machine type ("pseries") which has a virtual
bus called "spapr-vio". We need to be able to create devices on this
bus, and as such need a way to specify the address for those devices.

This patch adds a new address type "spapr-vio", which achieves this.

The addressing is specified with a "reg" property in the address
definition. The reg is optional, if it is not specified QEMU will
auto-assign an address for the device.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>

Currently non-x86 guests must have <acpi/> defined in <features> to
prevent libvirt from running qemu with -no-acpi. Although it works, it
is a hack.

Instead add a capability flag which indicates whether qemu understands
the -no-acpi option. Use it to control whether libvirt emits -no-acpi.

Current versions of qemu always display -no-acpi in their help output,
so this patch has no effect. However the development version of qemu
has been modified such that -no-acpi is only displayed when it is
actually supported.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>

Set up the types for the numa functions and insert them into the
virDriver structure definition.

This patch also sets cgroup cpuset parameters for numatune.

The RPC code had several latent memory leaks and an attempt to
free the wrong string, but thankfully nothing triggered them
(blkiotune was the only one returning a string, and always as
the last parameter).  Also, our cleanups for rpcgen ended up
nuking a line of code that renders VIR_TYPED_PARAM_INT broken,
because it was the only use of 'i' in a function, even though
it was a member usage rather than a standalone declaration.

* daemon/remote.c (remoteSerializeTypedParameters): Free the
correct array element.
(remoteDispatchDomainGetSchedulerParameters)
(remoteDispatchDomainGetSchedulerParametersFlags)
(remoteDispatchDomainBlockStatsFlags)
(remoteDispatchDomainGetMemoryParameters): Don't leak strings.
* src/rpc/genprotocol.pl: Don't nuke member-usage of 'buf' or 'i'.

The lifetime of the virDomainEventState object is tied to
the lifetime of the driver, which in stateless drivers is
tied to the lifetime of the virConnectPtr.

If we add & remove a timer when allocating/freeing the
virDomainEventState object, we can get a situation where
the timer still triggers once after virDomainEventState
has been freed. The timeout callback can't keep a ref
on the event state though, since that would be a circular
reference.

The trick is to only register the timer when a callback
is registered with the event state & remove the timer
when the callback is unregistered.

The demo for the bug is to run

  while true ; do date ; ../tools/virsh -q -c test:///default 'shutdown test; undefine test; dominfo test' ; done

prior to this fix, it will frequently hang and / or
crash, or corrupt memory

Currently all drivers using domain events need to provide a callback
for handling a timer to dispatch events in a clean stack. There is
no technical reason for dispatch to go via driver specific code. It
could trivially be dispatched directly from the domain event code,
thus removing tedious boilerplate code from all drivers

Also fix the libxl & xen drivers to pass 'true' when creating the
virDomainEventState, since they run inside the daemon & thus always
expect events to be present.

* src/conf/domain_event.c, src/conf/domain_event.h: Internalize
  dispatch of events from timer callback
* src/libxl/libxl_driver.c, src/lxc/lxc_driver.c,
  src/qemu/qemu_domain.c, src/qemu/qemu_driver.c,
  src/remote/remote_driver.c, src/test/test_driver.c,
  src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c: Remove all timer dispatch functions

The virDomainEventCallbackList and virDomainEventQueue APIs are
now solely helpers used internally by virDomainEventState APIs.
Remove their decls from domain_event.h since no driver code should
need to use them any more.

* src/conf/domain_event.c: Make virDomainEventCallbackList and
  virDomainEventQueue APIs static & remove some unused APIs
* src/conf/domain_event.h, src/libvirt_private.syms: Remove
  virDomainEventCallbackList and virDomainEventQueue APIs

No caller of the domain events APIs should need to poke at the
struct internals. Thus they should all be removed from the
header file

* src/conf/domain_event.h: Remove struct definitions
* src/conf/domain_event.c: Add struct definitions

* src/libxl/libxl_driver.c, src/lxc/lxc_driver.c,
  src/qemu/qemu_driver.c, src/remote/remote_driver.c,
  src/test/test_driver.c, src/uml/uml_driver.c,
  src/vbox/vbox_tmpl.c, src/xen/xen_driver.c: Convert
  to threadsafe APIs

While virDomainEventState has APIs for managing removal of callbacks,
while locked, adding callbacks in the first place requires direct
access to the virDomainEventCallbackList structure. This is not
threadsafe since it is bypassing the virDomainEventState locks

* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Add APIs for managing callbacks
  via virDomainEventState.

When registering a callback for a particular event some callers
need to know how many callbacks already exist for that event.
While it is possible to ask for a count, this is not free from
race conditions when threaded. Thus the API for registering
callbacks should return the count of callbacks. Also rename
virDomainEventStateDeregisterAny to virDomainEventStateDeregisterID

* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Return count of callbacks when
  registering callbacks
* src/libxl/libxl_driver.c, src/libxl/libxl_driver.c,
  src/qemu/qemu_driver.c, src/remote/remote_driver.c,
  src/remote/remote_driver.c, src/uml/uml_driver.c,
  src/vbox/vbox_tmpl.c, src/xen/xen_driver.c: Update
  for change in APIs

The Xen & VBox drivers deal with callbacks & dispatching of
events directly. All the other drivers use a timer to dispatch
events from a clean stack state, rather than deep inside the
drivers. Convert Xen & VBox over to virDomainEventState so
that they match behaviour of other drivers

* src/conf/domain_event.c: Return count of remaining
  callbacks when unregistering event callback
* src/vbox/vbox_tmpl.c, src/xen/xen_driver.c,
  src/xen/xen_driver.h: Convert to virDomainEventState

If only iptables rules are created then two unnecessary ebtables chains
are also created. This patch fixes this and prevents these chains from
being created. They have been cleaned up properly, though.

A generic error code was returned, if the user aborted a migration job.
This made it hard to distinguish between a user requested abort and an
error that might have occured. This patch introduces a new error code,
which is returned in the specific case of a user abort, while leaving
all other failures with their existing code. This makes it easier to
distinguish between failure while mirgrating and an user requested
abort.

 * include/libvirt/virterror.h: - add new error code
 * src/util/virterror.c: - add message for the new error code
 * src/qemu/qemu_migration.h: - Emit operation aborted error instead of
                                operation failed, on migration abort

If managed save fails at the right point in time, then the save
image can end up with 0 bytes in length (no valid header), and
our attempts in commit 55d88def to detect and skip invalid save
files missed this case.

* src/qemu/qemu_driver.c (qemuDomainSaveImageOpen): Also unlink
empty file as corrupt.  Reported by Dennis Householder.

Currently, on device detach, we parse given XML, find the device
in domain object, free it and try to restore security labels.
However, in some cases (e.g. usb hostdev) parsed XML contains
less information than freed device. In usb case it is bus & device
IDs. These are needed during label restoring as a symlink into
/dev/bus is generated from them. Therefore don't drop device
configuration until security labels are restored.

In commit 6f84e110 I mistakenly set default migration speed to
33554432 Mb!  The units of migMaxBandwidth is Mb, with conversion
handled in qemuMonitor{JSON,Text}SetMigrationSpeed().

Also, remove definition of QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX since
it is no longer used after reverting commit ef1065cf.

If an async job run on a domain will stop the domain at the end of the
job, a concurrently run query job can hang in qemu monitor and nothing
can be done with that domain from this point on. An attempt to start
such domain results in "Timed out during operation: cannot acquire state
change lock" error.

However, quite a few things have to happen at the right time... There
must be an async job running which stops a domain at the end. This race
was reported with dump --crash but other similar jobs, such as
(managed)save and migration, should be able to trigger this bug as well.
While this async job is processing its last monitor command, that is a
query-migrate to which qemu replies with status "completed", a new
libvirt API that results in a query job must arrive and stay waiting
until the query-migrate command finishes. Once query-migrate is done but
before the async job closes qemu monitor while stopping the domain, the
other thread needs to wake up and call qemuMonitorSend to send its
command to qemu. Before qemu gets a chance to respond to this command,
the async job needs to close the monitor. At this point, the query job
thread is waiting for a condition that no-one will ever signal so it
never finishes the job.

* src/qemu/qemu_hostdev.c (qemuDomainReAttachHostdevDevices):
pciDeviceListFree(pcidevs) in the end free()s the device even if
it's in use by other domain, which can cause a race.

How to reproduce:

<script>

virsh nodedev-dettach pci_0000_00_19_0
virsh start test
virsh attach-device test hostdev.xml
virsh start test2

for i in {1..5}; do
        echo "[ -- ${i}th time --]"
        virsh nodedev-reattach pci_0000_00_19_0
done

echo "clean up"
virsh destroy test
virsh nodedev-reattach pci_0000_00_19_0
</script>

Device pci_0000_00_19_0 dettached

Domain test started

Device attached successfully

error: Failed to start domain test2
error: Requested operation is not valid: PCI device 0000:00:19.0 is in use by domain test

[ -- 1th time --]
Device pci_0000_00_19_0 re-attached

[ -- 2th time --]
Device pci_0000_00_19_0 re-attached

[ -- 3th time --]
Device pci_0000_00_19_0 re-attached

[ -- 4th time --]
Device pci_0000_00_19_0 re-attached

[ -- 5th time --]
Device pci_0000_00_19_0 re-attached

clean up
Domain test destroyed

Device pci_0000_00_19_0 re-attached

The patch also fixes another problem, there won't be error like
"qemuDomainReAttachHostdevDevices: Not reattaching active
device 0000:00:19.0" in daemon log if some device is in active.
As pciResetDevice and pciReattachDevice won't be called for
the device anymore. This is sensible as we already reported
error when preparing the device if it's active. Blindly trying
to pciResetDevice & pciReattachDevice on the device and getting
an error is just redundant.

This patch fixes two problems:
    1) The device will be reattached to host even if it's not
       managed, as there is a "pciDeviceSetManaged".
    2) The device won't be reattached to host with original
       driver properly. As it doesn't honor the device original
       properties which are maintained by driver->activePciHostdevs.