If libvirtd is restarted when a job is running, the new libvirtd process
needs to know about that to be able to recover and rollback the
operation.

Query commands are safe to be called during long running jobs (such as
migration). This patch makes them all work without the need to
special-case every single one of them.

The patch introduces new job.asyncCond condition and associated
job.asyncJob which are dedicated to asynchronous (from qemu monitor
point of view) jobs that can take arbitrarily long time to finish while
qemu monitor is still usable for other commands.

The existing job.active (and job.cond condition) is used all other
synchronous jobs (including the commands run during async job).

Locking schema is changed to use these two conditions. While asyncJob is
active, only allowed set of synchronous jobs is allowed (the set can be
different according to a particular asyncJob) so any method that
communicates to qemu monitor needs to check if it is allowed to be
executed during current asyncJob (if any). Once the check passes, the
method needs to normally acquire job.cond to ensure no other command is
running. Since domain object lock is released during that time, asyncJob
could have been started in the meantime so the method needs to recheck
the first condition. Then, normal jobs set job.active and asynchronous
jobs set job.asyncJob and optionally change the list of allowed job
groups.

Since asynchronous jobs only set job.asyncJob, other allowed commands
can still be run when domain object is unlocked (when communicating to
remote libvirtd or sleeping). To protect its own internal synchronous
commands, the asynchronous job needs to start a special nested job
before entering qemu monitor. The nested job doesn't check asyncJob, it
only acquires job.cond and sets job.active to block other jobs.

EnterMonitor and ExitMonitor methods are very similar to their
*WithDriver variants; consolidate them into EnterMonitorInternal and
ExitMonitorInternal to avoid (mainly future) code duplication.

This avoids code duplication and also avoids relying on good luck that
ignore_value(virDomainObjUnref(obj)) doesn't set errno.

Incrementally running 'make syntax-check' on a tree previously
built after commit 62dee6fa but before 44036460 fails sc_po_check
(because the generated qemu_dispatch.h gained translatable strings).
This is a followup to commit addaa537 for that scenario.

* cfg.mk (sc_po_check): Add another prereq.
($(srcdir)/daemon/qemu_dispatch.h): Add rule.

* docs/todo.pl: Skip closed bugs

The UML inotify handler would kill off guests when certain
conditions arise, but it forgot to remove transient guests
from the list of domains

* src/uml/uml_driver.c: Cleanup transient guests

* src/uml/uml_driver.c: Add audit hooks

* src/lxc/lxc_driver.c: Add auditing hooks

Since a host can run several different virtualization types at
the same time, audit messages should allow domains to be identified.
Add a 'virt={qemu,kvm,uml,lxc,...}' key to domain audit messages

* src/conf/domain_audit.c: Identify virt type of guest

When passing through filesystems from the host to a guest, the
host filesystem passed must be audited

* src/conf/domain_audit.{c,h}: Add virDomainAuditFS

The LXC and UML drivers can both make use of auditing. Move
the qemu_audit.{c,h} files to src/conf/domain_audit.{c,h}

* src/conf/domain_audit.c: Rename from src/qemu/qemu_audit.c
* src/conf/domain_audit.h: Rename from src/qemu/qemu_audit.h
* src/Makefile.am: Remove qemu_audit.{c,h}, add domain_audit.{c,h}
* src/qemu/qemu_audit.h, src/qemu/qemu_cgroup.c,
  src/qemu/qemu_command.c, src/qemu/qemu_driver.c,
  src/qemu/qemu_hotplug.c, src/qemu/qemu_migration.c,
  src/qemu/qemu_process.c: Update for changed audit API names

To avoid using or freeing uninitialized memory or pointers.

connectDBus is only used if HAVE_DBUS is set. Therefore mark
it as potentially unused.

V2: Remove policy kit references from virNetServer and use DBus APIs
    directly, if available.

Given a PID, the QEMU driver reads /proc/$PID/cmdline and
/proc/$PID/environ to get the configuration. This is fed
into the ARGV->XML convertor to build an XML configuration
for the process.

/proc/$PID/exe is resolved to identify the full command
binary path

After checking for name/uuid uniqueness, an attempt is
made to connect to the monitor socket. If successful
then 'info status' and 'info kvm' are issued to determine
whether the CPUs are running and if KVM is enabled.

* src/qemu/qemu_driver.c: Implement virDomainQemuAttach
* src/qemu/qemu_process.h, src/qemu/qemu_process.c: Add
  qemuProcessAttach to connect to the monitor of an
  existing QEMU process

When attaching to an external QEMU process, it is neccessary
to check if the process is using KVM or not. This can be done
using a monitor command

* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  API for checking if KVM is enabled

To enable attaching to externally launched QEMU, we need
to be able to reverse engineer a guest XML config based
on the argv for a PID in /proc

* src/qemu/qemu_command.c, src/qemu/qemu_command.h: Add
  qemuParseCommandLinePid which extracts QEMU config from
  argv in /proc, given a PID number

When converting QEMU argv into a virDomainDefPtr, also extract
the pidfile, monitor character device config and the monitor
mode.

* src/qemu/qemu_command.c, src/qemu/qemu_command.h: Extract
  pidfile & monitor config from QEMU argv
* src/qemu/qemu_driver.c, tests/qemuargv2xmltest.c: Add extra
  params when calling qemuParseCommandLineString

Avoid re-formatting the pidfile path everytime we need it. Create
it once when starting the guest, and preserve it until the guest
is shutdown.

* src/libvirt_private.syms, src/util/util.c,
  src/util/util.h: Add virFileReadPidPath
* src/qemu/qemu_domain.h: Add pidfile field
* src/qemu/qemu_process.c: Store pidfile path in qemuDomainObjPrivate

This command allows libvirt to attach to an existing QEMU
instance.

 $ qemu-kvm -cdrom ~/demo.iso \
     -monitor unix:/tmp/demo,server,nowait \
     -name foo \
     -uuid cece4f9f-dff0-575d-0e8e-01fe380f12ea  &
 $ QEMUPID=$!
 $ virsh qemu-attach $QEMUPID

This tweaks the RPC generator to cope with some naming
conventions used for the QEMU specific APIs

* daemon/remote.c: Server side dispatcher
* src/remote/remote_driver.c: Client side dispatcher
* src/remote/qemu_protocol.x: Wire protocol definition
* src/rpc/gendispatch.pl: Use '$structprefix' in method
  names, fix QEMU flags and fix dispatcher method names

Introduce a new API in libvirt-qemu.so

 virDomainPtr virDomainQemuAttach(virConnectPtr domain,
                                  unsigned long long pid,
                                  unsigned int flags);

This allows libvirtd to attach to an existing, externally
launched QEMU process. This is useful for QEMU developers who
prefer to launch QEMU themselves for debugging/devel reasons,
but still want the benefit of libvirt based tools like
virt-top, virt-viewer, etc

* include/libvirt/libvirt-qemu.h: Define virDomainQemuAttach
* src/driver.h, src/libvirt-qemu.c, src/libvirt_qemu.syms:
  Driver glue for virDomainQemuAttach

* src/qemu/qemu.conf: Add blkio controller into qemu.conf.

* src/qemu/qemu_driver.c (doCoreDump): Guarantee fd is closed.

* src/util/virtaudit.[ch]: Rename...
* src/util/viraudit.[ch]: ...to match virAudit* API.
* src/Makefile.am (UTIL_SOURCES): Reflect rename.
* daemon/libvirtd.c: Likewise.
* po/POTFILES.in: Likewise.
* src/libvirt_private.syms: Likewise.
* src/qemu/qemu_audit.c: Likewise.

Set StrictHostKeyChecking=no to auto-accept new ssh host keys if the
no_verify extra parameter was specified.  This won't disable host key
checking for already known hosts.  Includes a test and documentation.

* tools/virsh.pod: move the description which was misplaced

* src/conf/domain_conf.c: caller must free returned value of
  virXPathString in useserial

Initialize ptr data.datastorePathWithoutFileName as NULL, otherwise
it might cause crash when trying to free it in cleanup.

Since we are going to add some libvirt-qemu.so entry points in
0.9.4, we might as well start checking for RPC stability, just
as for libvirt.so.

* src/Makefile.am (PROTOCOL_STRUCTS): New variable.
(remote_protocol-structs): Rename...
(%_protocol-structs): ...and make more generic.
* src/qemu_protocol-structs: New file.

log2() is heavy when ffs() can do the same thing.  But ffs()
requires gnulib support for mingw.

This patch solves this linker error on Fedora 14.

/usr/bin/ld: libvirt_lxc-domain_conf.o: undefined reference to symbol 'log2@@GLIBC_2.2.5'
/usr/bin/ld: note: 'log2@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line
/lib64/libm.so.6: could not read symbols: Invalid operation
collect2: ld returned 1 exit status

* .gnulib: Update to latest, for ffs.
* bootstrap.conf (gnulib_modules): Import ffs.
* src/conf/domain_conf.c (virDomainDefParseXML): Use ffs instead
of log2.
Reported by Dave Allan.

The drivers were accepting domain configs without checking if those
were actually meant for them. For example the LXC driver happily
accepts configs with type QEMU.

Add a check for the expected domain types to the virDomainDefParse*
functions.

Detected in valgrind run:

==9184== 1 bytes in 1 blocks are definitely lost in loss record 1 of 19
==9184==    at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==9184==    by 0x3073715F78: xdr_array (xdr_array.c:97)
==9184==    by 0x4CF97C9: xdr_remote_domain_get_security_label_ret (remote_protocol.c:1696)
==9184==    by 0x4D08741: virNetMessageDecodePayload (virnetmessage.c:286)
==9184==    by 0x4D00F78: virNetClientProgramCall (virnetclientprogram.c:318)
==9184==    by 0x4CE3887: call (remote_driver.c:3933)
==9184==    by 0x4CF71C6: remoteDomainGetSecurityLabel (remote_driver.c:1580)
==9184==    by 0x4CCA480: virDomainGetSecurityLabel (libvirt.c:7340)
==9184==    by 0x41993A: cmdDominfo (virsh.c:2414)
==9184==    by 0x411E92: vshCommandRun (virsh.c:12730)
==9184==    by 0x4211ED: main (virsh.c:14076)
==9184==
==9184== 2 bytes in 1 blocks are definitely lost in loss record 2 of 19
==9184==    at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==9184==    by 0x3073715F78: xdr_array (xdr_array.c:97)
==9184==    by 0x4CF974F: xdr_remote_node_get_security_model_ret (remote_protocol.c:1713)
==9184==    by 0x4D08741: virNetMessageDecodePayload (virnetmessage.c:286)
==9184==    by 0x4D00F78: virNetClientProgramCall (virnetclientprogram.c:318)
==9184==    by 0x4CE3887: call (remote_driver.c:3933)
==9184==    by 0x4CF6F96: remoteNodeGetSecurityModel (remote_driver.c:1648)
==9184==    by 0x4CBF799: virNodeGetSecurityModel (libvirt.c:7382)
==9184==    by 0x4197D7: cmdDominfo (virsh.c:2394)
==9184==    by 0x411E92: vshCommandRun (virsh.c:12730)
==9184==    by 0x4211ED: main (virsh.c:14076)
==9184==
==9184== 8 bytes in 1 blocks are definitely lost in loss record 3 of 19
==9184==    at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==9184==    by 0x3073715F78: xdr_array (xdr_array.c:97)
==9184==    by 0x4CF9729: xdr_remote_node_get_security_model_ret (remote_protocol.c:1710)
==9184==    by 0x4D08741: virNetMessageDecodePayload (virnetmessage.c:286)
==9184==    by 0x4D00F78: virNetClientProgramCall (virnetclientprogram.c:318)
==9184==    by 0x4CE3887: call (remote_driver.c:3933)
==9184==    by 0x4CF6F96: remoteNodeGetSecurityModel (remote_driver.c:1648)
==9184==    by 0x4CBF799: virNodeGetSecurityModel (libvirt.c:7382)
==9184==    by 0x4197D7: cmdDominfo (virsh.c:2394)
==9184==    by 0x411E92: vshCommandRun (virsh.c:12730)
==9184==    by 0x4211ED: main (virsh.c:14076)
==9184==
==9184== LEAK SUMMARY:
==9184==    definitely lost: 11 bytes in 3 blocks

* src/remote/remote_driver.c: Avoid leak on remoteDomainGetSecurityLabel
  and remoteNodeGetSecurityModel.

The shell version would output 40 extra spaces for a test with
a multiple of 40 sub-tests, and the C version can use the same
printf optimization for avoiding a loop over single space output
as the shell version.

* tests/testutils.c (virtTestMain): Avoid loop for alignment.
* tests/test-lib.sh: Fix formatting when counter is multiple of 40.

Kernel cmdline args can be passed to xen pv domains even when a
bootloader is specified.  The current config-to-sxpr mapping
ignores cmdline when bootloader is present.

Since the xend sub-driver is used with many xen toolstack versions,
this patch takes conservative approach of adding an else block to
existing !def->os.bootloader, and only appends sxpr if def->os.cmdline
is non-NULL.

V2: Fix existing testcase broken by this patch and add new testcases

If virDomainSaveConfig() failed, we will return NULL to source,
and the vm is still available to restart during confirm() step in
v3 protocol. So we should kill it off in qemuMigrationFinish().

In v2 protocol, we should not set vm to NULL, because we hold
a reference of vm and should unrefernce it.