Cancelling the QEMU migration may cause QEMU to flush pending
data on the migration socket. This may in turn block QEMU if
nothing reads from the other end of the socket. Closing the
socket before cancelling QEMU migration avoids this possible
deadlock.

* src/qemu/qemu_migration.c: Close sockets before cancelling
  migration on failure

The 'nbytes' variable was not re-initialized to the
buffer size on each iteration of the tunnelled migration
loop. While saferead() will ensure a full read, except
on EOF, it is clearer to use the real buffer size

* src/qemu/qemu_migration.c: Always read full buffer of data

The doTunnelMigrate method forgot to set the bandwidth
resource restriction

* src/qemu/qemu_migration.c: Set resource restriction

The qemuMigrationWaitForCompletion method contains a loop which
repeatedly queries QEMU to check migration progress, and also
processes job signals (pause, setspeed, setbandwidth, cancel).

The tunnelled migration loop does not currently support this
functionality, but should. Refactor the code to allow it to
be used with tunnelled migration.

Implement the v3 migration protocol, which has two extra
steps, 'begin' on the source host and 'confirm' on the
source host. All other methods also gain both input and
output cookies to allow bi-directional data passing at
all stages.

The QEMU peer2peer migration method gains another impl
to provide the v3 migration. This finally allows migration
cookies to work with tunnelled migration, which is required
for Spice seamless migration & the lock manager transfer

* src/qemu/qemu_driver.c: Wire up migrate v3 APIs
* src/qemu/qemu_migration.c, src/qemu/qemu_migration.h: Add
  begin & confirm methods, and peer2peer impl of v3

Merge the doNonTunnelMigrate2 and doTunnelMigrate2 methods
into one doPeer2PeerMigrate2 method, since they are substantially
the same. With the introduction of v3 migration, this will be
even more important, to avoid massive code duplication.

* src/qemu/qemu_migration.c: Merge tunnel & non-tunnel migration

The v2 migration protocol was accidentally missing out the
finish step, when prepare succeeded, but returned an invalid
URI

* src/libvirt.c: Teardown VM if prepare returns invalid URI

To facilitate the introduction of the v3 migration protocol,
the doTunnelMigrate method is refactored into two pieces. One
piece is intended to mirror the flow of virDomainMigrateVersion2,
while the other is the helper for setting up sockets and processing
the data.

Previously socket setup would be done before the 'prepare' step,
so errors could be dealt with immediately, avoiding need to shut
off the destination QEMU. In the new split, socket setup is done
after the 'prepare' step. This is not a serious problem, since
the control flow already requires calling 'finish' to tear down
the destination QEMU upon several errors.

* src/qemu/qemu_migration.c:

Use the graphics information from the QEMU migration cookie to
issue a 'client_migrate_info' monitor command to QEMU. This causes
the SPICE client to automatically reconnect to the target host
when migration completes

* src/qemu/qemu_migration.c: Set data for SPICE client relocation
  before starting migration on src
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  new qemuMonitorGraphicsRelocate() command

Extend the QEMU migration cookie structure to allow information
about the destination host graphics setup to be passed by to
the source host. This will enable seamless migration of any
connected graphics clients

* src/qemu/qemu_migration.c: Add graphics info to migration
  cookies
* daemon/libvirtd.c: Always initialize gnutls to enable
  x509 cert parsing in QEMU

The migration protocol has support for a 'cookie' parameter which
is an opaque array of bytes as far as libvirt is concerned. Drivers
may use this for passing around arbitrary extra data they might
need during migration. The QEMU driver needs to do a few things:

 - Pass hostname/uuid to allow strict protection against localhost
   migration attempts
 - Pass SPICE/VNC server port from the target back to the source to
   allow seamless relocation of client sessions
 - Pass lock driver state from source to destination

This patch introduces the basic glue for handling cookies
but only includes the host/guest UUID & name.

* src/libvirt_private.syms: Export virXMLParseStrHelper
* src/qemu/qemu_migration.c, src/qemu/qemu_migration.h: Parsing
  and formatting of migration cookies
* src/qemu/qemu_driver.c: Pass in cookie parameters where possible
* src/remote/remote_protocol.h, src/remote/remote_protocol.x: Change
  cookie max length to 16384 bytes

The qemuMigrationPrepareTunnel method should not unlock the
qemu driver, since that is the caller's job.

* src/qemu/qemu_migration.c: Fix qemuMigrationPrepareTunnel
  unlocking of QEMU driver

* src/remote/remote_protocol.x: Define wire protocol for migration
  protocol v3
* daemon/remote.c: Server side dispatch
* src/remote/remote_driver.c: Client side serialization
* src/remote/remote_protocol.c, src/remote/remote_protocol.h,
  daemon/remote_dispatch_args.h, daemon/remote_dispatch_prototypes.h,
  daemon/remote_dispatch_ret.h, daemon/remote_dispatch_table.h:
  Re-generate files
* src/remote_protocol-structs: Declare new ABIs

Migration just seems to go from bad to worse. We already had to
introduce a second migration protocol when adding the QEMU driver,
since the one from Xen was insufficiently flexible to cope with
passing the data the QEMU driver required.

It turns out that this protocol still has some flaws that we
need to address. The current sequence is

 *  Src: DumpXML
          - Generate XML to pass to dst

 *  Dst: Prepare
          - Get ready to accept incoming VM
          - Generate optional cookie to pass to src

 *  Src: Perform
          - Start migration and wait for send completion
          - Kill off VM if successful, resume if failed

 *  Dst: Finish
          - Wait for recv completion and check status
          - Kill off VM if unsuccessful

The problems with this are:

 - Since the first step is a generic 'DumpXML' call, we can't
   add in other migration specific data. eg, we can't include
   any VM lease data from lock manager plugins
 - Since the first step is a generic 'DumpXML' call, we can't
   emit any 'migration begin' event on the source, or have
   any hook that runs right at the start of the process
 - Since there is no final step on the source, if the Finish
   method fails to receive all migration data & has to kill
   the VM, then there's no way to resume the original VM
   on the source

This patch attempts to introduce a version 3 that uses the
improved 5 step sequence

 *  Src: Begin
          - Generate XML to pass to dst
          - Generate optional cookie to pass to dst

 *  Dst: Prepare
          - Get ready to accept incoming VM
          - Generate optional cookie to pass to src

 *  Src: Perform
          - Start migration and wait for send completion
          - Generate optional cookie to pass to dst

 *  Dst: Finish
          - Wait for recv completion and check status
          - Kill off VM if failed, resume if success
          - Generate optional cookie to pass to src

 *  Src: Confirm
          - Kill off VM if success, resume if failed

The API is designed to allow both input and output cookies
in all methods where applicable. This lets us pass around
arbitrary extra driver specific data between src & dst during
migration. Combined with the extra 'Begin' method this lets
us pass lease information from source to dst at the start of
migration

Moving the killing of the source VM out of Perform and
into Confirm, means we can now recover if the dst host
can't successfully Finish receiving migration data.

The hvsupport.html.in file is forever out of date. By annotating
the driver struct tables in each driver with version information,
we can auto-generate the hvsupport.html.in file. Annotating the
drivers will be mandatory for new patches, ensuring hvsupport.html.in
is never out of date again.

* docs/hvsupport.html.in: Delete
* hvsupport.pl: Script to generate hvsupport.html.in
* Makefile.am: Autogenerate hvsupport.html.in

Add many version number annotations to the internal driver
tables, to allow hvsupport.html to display more accurate
information

Change all the driver struct initializers to use the
C99 style, leaving out unused fields. This will make
it possible to add new APIs without changing every
driver. eg change:

    qemudDomainResume, /* domainResume */
    qemudDomainShutdown, /* domainShutdown */
    NULL, /* domainReboot */
    qemudDomainDestroy, /* domainDestroy */

to

    .domainResume = qemudDomainResume,
    .domainShutdown = qemudDomainShutdown,
    .domainDestroy = qemudDomainDestroy,

And get rid of any existing C99 style initializersr which
set NULL, eg change

     .listPools          = vboxStorageListPools,
     .numOfDefinedPools  = NULL,
     .listDefinedPools   = NULL,
     .findPoolSources    = NULL,
     .poolLookupByName   = vboxStoragePoolLookupByName,

to

     .listPools          = vboxStorageListPools,
     .poolLookupByName   = vboxStoragePoolLookupByName,

Fix some driver names:

  s/virDrvCPUCompare/virDrvCompareCPU/
  s/virDrvCPUBaseline/virDrvBaselineCPU/
  s/virDrvQemuDomainMonitorCommand/virDrvDomainQemuMonitorCommand/
  s/virDrvSecretNumOfSecrets/virDrvNumOfSecrets/
  s/virDrvSecretListSecrets/virDrvListSecrets/

And some driver struct field names:

  s/getFreeMemory/nodeGetFreeMemory/

A qemu domain can get paused when libvirtd is stopped (e.g., because of
I/O error) so we should check its current state when reconnecting to it.

Only in drivers which use virDomainObj, drivers that query hypervisor
for domain status need to be updated separately in case their hypervisor
supports this functionality.

The reason is also saved into domain state XML so if a domain is not
running (i.e., no state XML exists) the reason will be lost by libvirtd
restart. I think this is an acceptable limitation.

Reason is currently always set to 0 (i.e., *_UNKNOWN).

This API is supposed to replace virDomainGetInfo when the only purpose
of calling it is getting current domain status.

This has been present since the introduction of phypAttachDevice
in commit 444fd07a.

* src/phyp/phyp_driver.c (phypAttachDevice): Don't dereference
NULL.

virFDStreamClose used a mutex after it was freed, and failed
to destroy that mutex on its last use.

* src/fdstream.c (virFDStreamFree): Inline into sole caller...
(virFDStreamClose): ...to avoid use-after-free and leak.
Reported by Matthias Bolte.

Move common code from Property and Parameter into new Member class.

Rename the other base class to Type.

Detect it based on usage as parameter, return type and member of other
object types.

Detect it based on usage as parameter and return type too.

Don't make all object and enum types (de)serializable by default.
Detect this from the input file instead.

Several vSphere API methods are called on global objects like the
FileManager, the PerformanceManager or the SearchIndex. The generator
input file allows to mark such methods and the generator generates
such method in a way that automatically handles marked parameter. This
is done by some special macros. Those were manually written and this
patch moves them to the generator.

Accept all types on deserialization in order to accept all Event subtypes.

This will be used for the upcoming domain event support.

Steps to reproduce this problem:
1. # ./autogen.sh --without-libvirtd
2. # make dist
3. # rpmbuild --nodeps --define "_sourcedir `pwd`" -ba libvirt.spec
   ...
   make  check-TESTS
   make[1]: Entering directory `/home/wency/rpmbuild/BUILD/libvirt-0.9.1/tests'
   make[1]: *** No rule to make target `test_conf.sh', needed by `check-TESTS'.  Stop.
   make[1]: Leaving directory `/home/wency/rpmbuild/BUILD/libvirt-0.9.1/tests'
   make: *** [check-am] Error 2
   error: Bad exit status from /var/tmp/rpm-tmp.7Rb9PV (%check)

Steps to reproduce this problem:
1. # ./autogen.sh --without-libvirtd
2. # make dist
3. # rpmbuild --nodeps --define "_sourcedir `pwd`" -ba libvirt.spec
   ...
   make[2]: Entering directory `/home/wency/rpmbuild/BUILD/libvirt-0.9.1/daemon'
   make[2]: *** No rule to make target `probes.d', needed by `probes.h'.  Stop.
   make[2]: Leaving directory `/home/wency/rpmbuild/BUILD/libvirt-0.9.1/daemon'
   make[1]: *** [all-recursive] Error 1
   make[1]: Leaving directory `/home/wency/rpmbuild/BUILD/libvirt-0.9.1'
   make: *** [all] Error 2
   error: Bad exit status from /var/tmp/rpm-tmp.WkAD7a (%build)

kill, SIGTERM and SIGKILL require additional header.

Reported on FreeBSD only.

Commit aaf20355 was incomplete here and
missed to remove some parts.