The xenstore database sometimes has stale domain IDs which are not
present in the hypervisor anymore. Filter these out to avoid causing
confusion

* src/xen/xs_internal.c: Filter domain IDs against HV's list
* src/xen/xen_hypervisor.h, src/xen/xen_hypervisor.c: Add new
  xenHypervisorHasDomain() method for checking ID validity

The xenUnifiedNumOfDomains and xenUnifiedListDomains methods work
together as a pair, so it is critical they both apply the same
logic. With the current mis-matched logic it is possible to sometimes
get into a state when you miss certain active guests.

* src/xen/xen_driver.c: Change xenUnifiedNumOfDomains ordering to
  match xenUnifiedListDomains.

Sometimes getaddrinfo returns IPv4 addresses before IPv6 addresses.
IPv6 sockets default to attempting to bind to IPv4 addresses too.
So if the IPv4 address is activated first, then binding to IPv6
will unneccessarily fail.

* daemon/libvirtd.c: Bind to IPv6 and IPv4 addresses separately

The numactl package is not applicable for s390[x] arches, so do
not enable it as a build dep.

* libvirt.spec.in: Exclude numactl on s390[x]

* src/qemu/qemu_monitor.c: add error check for qemuMonitorOpenXXX
  returned file descriptor

When running qemu:///system instance, libvirtd runs as root,
but QEMU may optionally be configured to run non-root. When
then saving a guest to a state file, the file is initially
created as root, and thus QEMU cannot write to it. It is also
missing labelling required to allow access via SELinux.

* src/qemu/qemu_driver.c: Set ownership on save image before
  running migrate command in virDomainSave impl. Call out to
  security driver to set save image labelling
* src/security/security_driver.h: Add driver APIs for setting
  and restoring saved state file labelling
* src/security/security_selinux.c: Implement saved state file
  labelling for SELinux

* src/qemu/qemu.conf:  disables the mac_filter config switch by default
  to match existing convention, also document the option

* src/qemu/qemu_conf.c: don't initialize ebtables if
  disabled

* src/phyp/phyp_driver.c: a 10s timeout on socket availability was way
  too long, reduced to 1ms

* src/qemu/qemu_monitor.c src/qemu/qemu_monitor_text.c: cast size_t to
  int when passing to '%d'

Finish changes intended to be part of commit
6c708023

This implements the virConnectIsSecure, virConnectIsEncrypted,
virDomainIsPersistent, virDomainIsActive, virNetworkIsActive,
virNetworkIsPersistent, virStoragePoolIsActive,
virStoragePoolIsPersistent, virInterfaceIsActive APIs in
(nearly) all drivers. Exceptions are:

 phyp: missing domainIsActive/Persistent
 esx: missing domainIsPersistent
 opennebula: missing domainIsActive/Persistent

* src/remote/remote_protocol.x: Define remote wire ABI for newly
  added APIs.
* daemon/remote_dispatch*.h: Re-generated from remote_protocol.x
* src/esx/esx_driver.c, src/lxc/lxc_driver.c, src/network/bridge_driver.c,
  src/opennebula/one_driver.c, src/openvz/openvz_conf.c,
  src/openvz/openvz_driver.c, src/phyp/phyp_driver.c,
  src/remote/remote_driver.c, src/storage/storage_driver.c,
  src/test/test_driver.c, src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c, src/xen/xen_driver.h, src/xen/xen_inotify.c,
  src/xen/xen_inotify.h: Implement all the new APIs where possible

Introduce a number of new APIs to  expose some boolean properties
of objects, which cannot otherwise reliably determined, nor are
aspects of the XML configuration.

 * virDomainIsActive: Checking virDomainGetID is not reliable
   since it is not possible to distinguish between error condition
   and inactive domain for ID of -1.
 * virDomainIsPersistent: Check whether a persistent config exists
   for the domain

 * virNetworkIsActive: Check whether the network is active
 * virNetworkIsPersistent: Check whether a persistent config exists
   for the network

 * virStoragePoolIsActive: Check whether the storage pool is active
 * virStoragePoolIsPersistent: Check whether a persistent config exists
   for the storage pool

 * virInterfaceIsActive: Check whether the host interface is active

 * virConnectIsSecure: whether the communication channel to the
   hypervisor is secure
 * virConnectIsEncrypted: whether any network based commnunication
   channels are encrypted

NB, a channel can be secure, even if not encrypted, eg if it does
not involve the network, like a UNIX socket, or pipe.

 * include/libvirt/libvirt.h.in: Define public API
 * src/driver.h: Define internal driver API
 * src/libvirt.c: Implement public API entry point
 * src/libvirt_public.syms: Export API symbols
 * src/esx/esx_driver.c, src/lxc/lxc_driver.c,
   src/interface/netcf_driver.c, src/network/bridge_driver.c,
   src/opennebula/one_driver.c, src/openvz/openvz_driver.c,
   src/phyp/phyp_driver.c, src/qemu/qemu_driver.c,
   src/remote/remote_driver.c, src/test/test_driver.c,
   src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
   src/xen/xen_driver.c: Stub out driver tables

* daemon/libvirtd.c tools/virsh.c: Steve Grubb <sgrubb@redhat.com> found
  a few more issues

* src/libvirt.c src/lxc/lxc_conf.c src/lxc/lxc_container.c
  src/lxc/lxc_controller.c src/node_device/node_device_hal.c
  src/openvz/openvz_conf.c src/qemu/qemu_driver.c
  src/qemu/qemu_monitor_text.c src/remote/remote_driver.c
  src/storage/storage_backend_disk.c src/storage/storage_driver.c
  src/util/logging.c src/xen/sexpr.c src/xen/xend_internal.c
  src/xen/xm_internal.c: Steve Grubb <sgrubb@redhat.com> sent a code
  review and those are the fixes correcting the problems

Some monitor commands may take a very long time to complete. It is
not desirable to block other incoming API calls forever. With this
change, if an existing API call is holding the job lock, additional
API calls will not wait forever. They will time out after a short
period of time, allowing application to retry later.

* include/libvirt/virterror.h, src/util/virterror.c: Add new
  VIR_ERR_OPERATION_TIMEOUT error code
* src/qemu/qemu_driver.c: Change to a timed condition variable
  wait for acquiring the monitor job lock

QEMU monitor commands may sleep for a prolonged period of time.
If the virDomainObjPtr or qemu driver lock is held this will
needlessly block execution of many other API calls. it also
prevents asynchronous monitor events from being dispatched
while a monitor command is executing, because deadlock will
ensure.

To resolve this, it is neccessary to release all locks while
executing a monitor command. This change introduces a flag
indicating that a monitor job is active, and a condition
variable to synchronize access to this flag. This ensures that
only a single thread can be making a state change or executing
a monitor command at a time, while still allowing other API
calls to be completed without blocking

* src/qemu/qemu_driver.c: Release driver and domain lock when
  running monitor commands. Re-add locking to disk passphrase
  callback
* src/qemu/THREADS.txt: Document threading rules

Change the QEMU monitor file handle watch to poll for both
read & write events, as well as EOF. All I/O to/from the
QEMU monitor FD is now done in the event callback thread.

When the QEMU driver needs to send a command, it puts the
data to be sent into a qemuMonitorMessagePtr object instance,
queues it for dispatch, and then goes to sleep on a condition
variable. The event thread sends all the data, and then waits
for the reply to arrive, putting the response / error data
back into the qemuMonitorMessagePtr and notifying the condition
variable.

There is a temporary hack in the disk passphrase callback to
avoid acquiring the domain lock.  This avoids a deadlock in
the command processing, since the domain lock is still held
when running monitor commands. The next commit will remove
the locking when running commands & thus allow re-introduction
of locking the disk passphrase callback

* src/qemu/qemu_driver.c: Temporarily don't acquire lock in
  disk passphrase callback. To be reverted in next commit
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Remove
  raw I/O functions, and a generic qemuMonitorSend() for
  invoking a command
* src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Remove all low level I/O, and use the new qemuMonitorSend()
  API. Provide a qemuMonitorTextIOProcess() method for detecting
  command/reply/prompt boundaries in the monitor data stream

Use ssh keyfiles from the current user's home directory instead of trying
to use keyfiles from a hardcoded /home/user directory. Fallback to
username/password authentication if keyfiles are not available or keyfile
authentication failed.

Add reference counting on the virDomainObjPtr objects. With the
forthcoming asynchronous QEMU monitor, it will be neccessary to
release the lock on virDomainObjPtr while waiting for a monitor
command response. It is neccessary to ensure one thread can't
delete a virDomainObjPtr while another is waiting. By introducing
reference counting threads can make sure objects they are using
are not accidentally deleted while unlocked.

* src/conf/domain_conf.h, src/conf/domain_conf.c: Add
  virDomainObjRef/Unref APIs, remove virDomainObjFree
* src/openvz/openvz_conf.c: replace call to virDomainObjFree
  with virDomainObjUnref

In preparation of the monitor I/O process becoming fully asynchronous,
it is neccessary to ensure all access to internals of the qemuMonitorPtr
object is protected by a mutex lock.

* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Add mutex for locking
  monitor.
* src/qemu/qemu_driver.c: Add locking around all monitor commands

Change the QEMU driver to not directly invoke the text mode monitor
APIs. Instead add a generic wrapper layer, which will eventually
invoke either the text or JSON protocol code as needed. Pass an
qemuMonitorPtr object into the monitor APIs instead of virDomainObjPtr
to complete the de-coupling of the monitor impl from virDomainObj
data structures

* src/qemu/qemu_conf.h: Remove qemuDomainObjPrivate definition
* src/qemu/qemu_driver.c: Add qemuDomainObjPrivate definition.
  Pass qemuMonitorPtr into all monitor APIs instead of the
  virDomainObjPtr instance.
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Add thin
  wrappers for all qemuMonitorXXX command APIs, calling into
  qemu_monitor_text.c/h
* src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Rename qemuMonitor -> qemuMonitorText & update to accept
  qemuMonitorPtr instead of virDomainObjPtr

Decouple the monitor code from the virDomainDefPtr structure
by moving the disk encryption lookup code back into the
qemu_driver.c file. Instead provide a function callback to
the monitor code which can be invoked to retrieve encryption
data as required.

* src/qemu/qemu_driver.c: Add findDomainDiskEncryption,
  and findVolumeQcowPassphrase. Pass address of the method
  findVolumeQcowPassphrase into qemuMonitorOpen()
* src/qemu/qemu_monitor.c: Associate a disk
  encryption function callback with the qemuMonitorPtr
  object.
* src/qemu/qemu_monitor_text.c: Remove findDomainDiskEncryption
  and findVolumeQcowPassphrase.

Introduce a new qemuDomainObjPrivate object which is used to store
the private QEMU specific data associated with each virDomainObjPtr
instance. This contains a single member, an instance of the new
qemuMonitorPtr object which encapsulates the QEMU monitor state.
The internals of the latter are private to the qemu_monitor* files,
not to be shown to qemu_driver.c

* src/qemu/qemu_conf.h: Definition of qemuDomainObjPrivate.
* src/qemu/qemu_driver.c: Register a functions for creating
  and freeing qemuDomainObjPrivate instances with the domain
  capabilities. Remove the qemudDispatchVMEvent() watch since
  I/O watches are now handled by the monitor code itself. Pass
  a new qemuHandleMonitorEOF() callback into qemuMonitorOpen
  to allow notification when the monitor quits.
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Introduce
  the 'qemuMonitor' object. Temporarily add new APIs
  qemuMonitorWrite, qemuMonitorRead, qemuMonitorWaitForInput
  to allow text based monitor impl to perform I/O.
* src/qemu/qemu_monitor_text.c: Call APIs for reading/writing
  to monitor instead of accessing the file handle directly.

The qemu_driver.c code should not contain any code that interacts
with the QEMU monitor at a low level. A previous commit moved all
the command invocations out. This change moves out the code which
actually opens the monitor device.

* src/qemu/qemu_driver.c: Remove qemudOpenMonitor & methods called
  from it.
* src/Makefile.am: Add qemu_monitor.{c,h}
* src/qemu/qemu_monitor.h: Add qemuMonitorOpen()
* src/qemu/qemu_monitor.c: All code for opening the monitor

* src/util/threads.h, src/util/threads-pthread.c,
  src/libvirt_private.syms: Add virCondWaitUntil()

* src/util/threads-pthread.c: pthreads APIs do not set errno, instead
  the return value is the positive errno. Set errno based on the return
  value in the wrappers

* src/util/pci.c, src/util/pci.h: Make the pciDeviceList struct
  opaque to callers of the API. Add accessor methods for managing
  devices in the list
* src/qemu/qemu_driver.c: Update to use APIs instead of directly
  accessing pciDeviceList fields

* src/formatdomain.html.in: Add <channel> element documentation

* docs/formatdomain.html.in: refactors the existing character device
  documentation to make it explicit which directives configure the guest
  interface, and which configure the host interface.

As it was basically unimplemented and more confusing than useful
at the moment.
* src/libvirt_private.syms: remove from internal symbols list
* src/qemu/qemu_bridge_filter.c src/util/ebtables.c: remove code and
  one use of the unimplemented function

In case of an error the domains hash and the driver mutex may leak.

* src/opennebula/one_driver.c: free/destroy domains hash and driver
  mutex in error cases

It also demonstrates how to use the libvirt.openAuth() method.

* examples/python/Makefile.am: add esxlist.py to EXTRA_DIST
* examples/python/README: add some notes about esxlist.py
* examples/python/esxlist.py: the new example

- Make reading ID from file working for IDs > 127
- Fix inverse error check for writing ID to file
- Use feof() to distinguish EOF from real error of fread()
- Don't interpret libssh2 error codes as number of bytes

Also reset UUID to all 0 instead of all 48 (== '0') in phypUUIDTable_RemLpar()

phypNumDomainsGeneric() and phypListDomainsGeneric() return 0 in case
of an error. This makes it impossible to distinguish between an actual
error and no domains being defined on the hypervisor. It also turn the
no domains situation into an error. Return -1 in case of an error to
fix this problem.