The event loop implementation is used by more than just the
daemon, so move it into the shared area.

* daemon/event.c, src/util/event_poll.c: Renamed
* daemon/event.h, src/util/event_poll.h: Renamed
* tools/Makefile.am, tools/console.c, tools/virsh.c: Update
  to use new virEventPoll APIs
* daemon/mdns.c, daemon/mdns.c, daemon/Makefile.am: Update
  to use new virEventPoll APIs

$ ./configure
...
$ make
...
  GEN    libvirt.syms
...
$ ./configure --with-driver-modules
...
$ make
...

libvirt.syms doesn't get regenerated but it should as it should
contain virDriverLoadModule now.

* src/Makefile.am (libvirt.syms): Depend on configure changes.
Reported by Matthias Bolte.

The introduction of the v3 migration protocol, along with
support for migration cookies, will significantly expand
the size of the migration code. Move it all to a separate
file to make it more manageable

The functions are not moved 100%. The API entry points
remain in the main QEMU driver, but once the public
virDomainPtr is resolved to the internal virDomainObjPtr,
all following code is moved.

This will allow the new v3 API entry points to call into the
same shared internal migration functions

* src/qemu/qemu_domain.c, src/qemu/qemu_domain.h: Add
  qemuDomainFormatXML helper method
* src/qemu/qemu_driver.c: Remove all migration code
* src/qemu/qemu_migration.c, src/qemu/qemu_migration.h: Add
  all migration code.

Move the qemudStartVMDaemon and qemudShutdownVMDaemon
methods into a separate file, renaming them to
qemuProcessStart, qemuProcessStop. All helper methods
called by these are also moved & renamed to match

* src/Makefile.am: Add qemu_process.c/.h
* src/qemu/qemu_command.c: Add qemuDomainAssignPCIAddresses
* src/qemu/qemu_command.h: Add VNC port min/max
* src/qemu/qemu_domain.c, src/qemu/qemu_domain.h: Add
  domain event queue helpers
* src/qemu/qemu_driver.c, src/qemu/qemu_driver.h: Remove
  all QEMU process startup/shutdown functions
* src/qemu/qemu_process.c, src/qemu/qemu_process.h: Add
  all QEMU process startup/shutdown functions

The name convention of device mapper disk is different, and 'parted'
can't be used to delete a device mapper disk partition. e.g.

Name                 Path
-----------------------------------------
3600a0b80005ad1d7000093604cae912fp1 /dev/mapper/3600a0b80005ad1d7000093604cae912fp1

Error: Expecting a partition number.

This patch introduces 'dmsetup' to fix it.

Changes:
  - New function "virIsDevMapperDevice" in "src/utils/utils.c"
  - remove "is_dm_device" in "src/storage/parthelper.c", use
    "virIsDevMapperDevice" instead.
  - Requires "device-mapper" for 'with-storage-disk" in "libvirt.spec.in"
  - Check "dmsetup" in 'configure.ac' for "with-storage-disk"
  - Changes on "src/Makefile.am" to link against libdevmapper
  - New entry for "virIsDevMapperDevice" in "src/libvirt_private.syms"

Changes from v1 to v3:
  - s/virIsDeviceMapperDevice/virIsDevMapperDevice/g
  - replace "virRun" with "virCommand"
  - sort the list of util functions in "libvirt_private.syms"
  - ATTRIBUTE_NONNULL(1) for virIsDevMapperDevice declaration.

e.g.

Name                 Path
-----------------------------------------
3600a0b80005ad1d7000093604cae912fp1 /dev/mapper/3600a0b80005ad1d7000093604cae912fp1

Vol /dev/mapper/3600a0b80005ad1d7000093604cae912fp1 deleted

Name                 Path
-----------------------------------------

When built as modules, the connection drivers live
in $LIBDIR/libvirt/drivers. Now we add lock manager
drivers, we need to distinguish. So move the existing
modules to 'connection-driver'

* src/Makefile.am: Move module install dir
* src/driver.c: Move module search dir

The current security driver usage requires horrible code like

    if (driver->securityDriver &&
        driver->securityDriver->domainSetSecurityHostdevLabel &&
        driver->securityDriver->domainSetSecurityHostdevLabel(driver->securityDriver,
                                                              vm, hostdev) < 0)

This pair of checks for NULL clutters up the code, making the driver
calls 2 lines longer than they really need to be. The goal of the
patchset is to change the calling convention to simply

  if (virSecurityManagerSetHostdevLabel(driver->securityDriver,
                                        vm, hostdev) < 0)

The first check for 'driver->securityDriver' being NULL is removed
by introducing a 'no op' security driver that will always be present
if no real driver is enabled. This guarentees driver->securityDriver
!= NULL.

The second check for 'driver->securityDriver->domainSetSecurityHostdevLabel'
being non-NULL is hidden in a new abstraction called virSecurityManager.
This separates the driver callbacks, from main internal API. The addition
of a virSecurityManager object, that is separate from the virSecurityDriver
struct also allows for security drivers to carry state / configuration
information directly. Thus the DAC/Stack drivers from src/qemu which
used to pull config from 'struct qemud_driver' can now be moved into
the 'src/security' directory and store their config directly.

* src/qemu/qemu_conf.h, src/qemu/qemu_driver.c: Update to
  use new virSecurityManager APIs
* src/qemu/qemu_security_dac.c,  src/qemu/qemu_security_dac.h
  src/qemu/qemu_security_stacked.c, src/qemu/qemu_security_stacked.h:
  Move into src/security directory
* src/security/security_stack.c, src/security/security_stack.h,
  src/security/security_dac.c, src/security/security_dac.h: Generic
  versions of previous QEMU specific drivers
* src/security/security_apparmor.c, src/security/security_apparmor.h,
  src/security/security_driver.c, src/security/security_driver.h,
  src/security/security_selinux.c, src/security/security_selinux.h:
  Update to take virSecurityManagerPtr object as the first param
  in all callbacks
* src/security/security_nop.c, src/security/security_nop.h: Stub
  implementation of all security driver APIs.
* src/security/security_manager.h, src/security/security_manager.c:
  New internal API for invoking security drivers
* src/libvirt.c: Add missing debug for security APIs

Add vboxArrayGetWithUintArg to handle new signature variations. Also
refactor vboxArrayGet* implementation to use a common helper function.

Deal with the incompatible changes in the VirtualBox 4.0 API. This
includes major changes in virtual machine and storage medium lookup,
in RDP server property handling, in session/lock handling and other
minor areas.

VirtualBox 4.0 also dropped the old event API and replaced it with a
completely new one. This is not fixed yet and will be addressed in
another patch. Therefore, currently the domain events are supported
for VirtualBox 3.x only.

Based on initial work from Jean-Baptiste Rouault.

This fixes the build from a tarball and makes autobuild.sh
work again.

This should actually have been part of this earlier commit:

  esx: Move VMX handling code out of the driver directory
  42b2f35d

Reported by Eric Blake.

All other drivers are explicitly linked to gnulib. The VMware
driver lacked this, resulting in mdir_name being an undefine
symbol.

Explicitly link the VMware driver to gnulib to fix this.

Now the VMware driver doesn't depend on the ESX driver anymore.

Add a WITH_VMX option that depends on WITH_ESX and WITH_VMWARE.
Also add a libvirt_vmx.syms file.

Move some escaping functions from esx_util.c to vmx.c.

Adapt the test suite, ESX and VMware driver to the new code layout.

* daemon/Makefile.am: Avoid spurious space before tabs.
* src/Makefile.am: Likewise.
* examples/dominfo/Makefile.am: Likewise.
* examples/domsuspend/Makefile.am: Likewise.
* tools/Makefile.am: Likewise.
* src/datatypes.h (VIR_CONNECT_MAGIC): Likewise.
* src/internal.h (TODO): Likewise.
* src/qemu/qemu_monitor.h (QEMU_MONITOR_MIGRATE): Likewise.
* src/xen/xen_hypervisor.c (XEN_V2_OP_GETAVAILHEAP): Likewise.
* src/xen/xs_internal.h: Likewise.

* configure.ac (dlopen): Cygwin dlopen is in libc; avoid spurious
failure.
(XDR_CFLAGS): Define when needed.
* src/Makefile.am (libvirt_driver_remote_la_CFLAGS): Use it.

* src/Makefile.am (VMWARE_DRIVER_SOURCES): Fix vmware_driver.h.
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>

Don't require dlopen, but link to ole32 and oleaut32 on Windows.

Don't expose g_pVBoxFuncs anymore. It was only used to get the
version of the API. Make VBoxCGlueInit return the version instead.
This simplifies the implementation of the MSCOM glue layer.

Get the VirtualBox version from the registry.

Add a dummy implementation of the nsIEventQueue to the MSCOM glue
as there seems to be no direct equivalent with MSCOM. It might be
implemented using the normal window message loop. This requires
additional investigation.

* src/Makefile.am (QEMU_DRIVER_SOURCES): Ship qemu_cgroup.h.

The QEMU driver file is far too large. Move all the hotplug
helper code out into a separate file. No functional change.

* src/qemu/qemu_hotplug.c, src/qemu/qemu_hotplug.h,
  src/Makefile.am: Add hotplug helper file
* src/qemu/qemu_driver.c: Delete hotplug code

The QEMU driver file is far too large. Move all the hostdev
helper code out into a separate file. No functional change.

* src/qemu/qemu_hostdev.c, src/qemu/qemu_hostdev.h,
  src/Makefile.am: Add hostdev helper file
* src/qemu/qemu_driver.c: Delete hostdev code

The QEMU driver file is far too large. Move all the cgroup
helper code out into a separate file. No functional change.

* src/qemu/qemu_cgroup.c, src/qemu/qemu_cgroup.h,
  src/Makefile.am: Add cgroup helper file
* src/qemu/qemu_driver.c: Delete cgroup code

The QEMU driver file is far too large. Move all the audit
helper code out into a separate file. No functional change.

* src/qemu/qemu_audit.c, src/qemu/qemu_audit.h,
  src/Makefile.am: Add audit helper file
* src/qemu/qemu_driver.c: Delete audit code

Move the code for handling the QEMU virDomainObjPtr private
data, and custom XML namespace into a separate file

* src/qemu/qemu_domain.c, src/qemu/qemu_domain.h: New file
  for private data & namespace code
* src/qemu/qemu_driver.c, src/qemu/qemu_driver.h: Remove
  private data & namespace code
* src/qemu/qemu_driver.h, src/qemu/qemu_command.h: Update
  includes
* src/Makefile.am: Add src/qemu/qemu_domain.c

The qemu_conf.c code is doing three jobs, driver config file
loading, QEMU capabilities management and QEMU command line
management. Move the command line code into its own file

* src/qemu/qemu_command.c, src/qemu/qemu_command.h: New
  command line management code
* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h: Delete command
  line code
* src/qemu/qemu_conf.h, src/qemu_conf.c: Adapt for API renames
* src/Makefile.am: add src/qemu/qemu_command.c
* src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_text.c: Add
  import of qemu_command.h

The qemu_conf.c code is doing three jobs, driver config file
loading, QEMU capabilities management and QEMU command line
management. Move the capabilities code into its own file

* src/qemu/qemu_capabilities.c, src/qemu/qemu_capabilities.h: New
  capabilities management code
* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h: Delete capabilities
  code
* src/qemu/qemu_conf.h: Adapt for API renames
* src/Makefile.am: add src/qemu/qemu_capabilities.c

* src/util/threadpool.c, src/util/threadpool.h: Thread pool
  implementation
* src/Makefile.am: Build thread pool
* src/libvirt_private.syms: Export public functions

This introduces a new set of APIs in src/util/command.h
to use for invoking commands. This is intended to replace
all current usage of virRun and virExec variants, with a
more flexible and less error prone API.

* src/util/command.c: New file.
* src/util/command.h: New header.
* src/Makefile.am (UTIL_SOURCES): Build it.
* src/libvirt_private.syms: Export symbols internally.
* tests/commandtest.c: New test.
* tests/Makefile.am (check_PROGRAMS): Run it.
* tests/commandhelper.c: Auxiliary program.
* tests/commanddata/test2.log - test15.log: New expected outputs.
* cfg.mk (useless_free_options): Add virCommandFree.
(msg_gen_function): Add virCommandError.
* po/POTFILES.in: New translation.
* .x-sc_avoid_write: Add exemption.
* tests/.gitignore: Ignore new built file.

* bootstrap.conf (gnulib_modules): Add configmake.
* daemon/Makefile.am (libvirtd_CFLAGS): Drop defines provided by
gnulib.
* src/Makefile.am (INCLUDES): Likewise.
* tests/Makefile.am (INCLUDES): Likewise.
* tools/Makefile.am (virsh_CFLAGS): Likewise.
* daemon/libvirtd.c (qemudInitPaths, usage, main): Update
clients.
* src/cpu/cpu_map.c (CPUMAPFILE): Likewise.
* src/driver.c (DEFAULT_DRIVER_DIR): Likewise.
* src/internal.h (_): Likewise.
* src/libvirt.c (virInitialize): Likewise.
* src/lxc/lxc_conf.h (LXC_CONFIG_DIR, LXC_STATE_DIR, LXC_LOG_DIR):
Likewise.
* src/lxc/lxc_conf.c (lxcCapsInit, lxcLoadDriverConfig):
Likewise.
* src/network/bridge_driver.c (NETWORK_PID_DIR)
(NETWORK_STATE_DIR, DNSMASQ_STATE_DIR, networkStartup): Likewise.
* src/nwfilter/nwfilter_driver.c (nwfilterDriverStartup):
Likewise.
* src/qemu/qemu_conf.c (qemudLoadDriverConfig): Likewise.
* src/qemu/qemu_driver.c (qemudStartup): Likewise.
* src/remote/remote_driver.h (LIBVIRTD_PRIV_UNIX_SOCKET)
(LIBVIRTD_PRIV_UNIX_SOCKET_RO, LIBVIRTD_CONFIGURATION_FILE)
(LIBVIRT_PKI_DIR): Likewise.
* src/secret/secret_driver.c (secretDriverStartup): Likewise.
* src/security/security_apparmor.c (VIRT_AA_HELPER): Likewise.
* src/security/virt-aa-helper.c (main): Likewise.
* src/storage/storage_backend_disk.c (PARTHELPER): Likewise.
* src/storage/storage_driver.c (storageDriverStartup): Likewise.
* src/uml/uml_driver.c (TEMPDIR, umlStartup): Likewise.
* src/util/hooks.c (LIBVIRT_HOOK_DIR): Likewise.
* tools/virsh.c (main): Likewise.
* docs/hooks.html.in: Likewise.

To avoid the need for duplicating implementations of virStream
drivers, provide a generic implementation that can handle any
FD based stream. This code is copied from the existing impl
in the QEMU driver, with the locking moved into the stream
impl, and addition of a read callback

The FD stream code will refuse to operate on regular files or
block devices, since those can't report EAGAIN properly when
they would block on I/O

* include/libvirt/virterror.h, include/libvirt/virterror.h: Add
  VIR_FROM_STREAM error domain
* src/qemu/qemu_driver.c: Remove code obsoleted by the new
  generic streams driver.
* src/fdstream.h, src/fdstream.c, src/fdstream.c,
  src/libvirt_private.syms: Generic reusable FD based streams

Suggested by danpb, as it's not up-to-date anymore and
lacks many functions that were added to libvirtd.

Move existing routines about virSysinfoDef to an util module,
add a new entry point virSysinfoRead() to read the host values
with dmidecode

* src/conf/domain_conf.c src/conf/domain_conf.h src/util/sysinfo.c
  src/util/sysinfo.h: move to a new module, add virSysinfoRead()
* src/Makefile.am: handle the new module build
* src/libvirt_private.syms: new internal symbols
* include/libvirt/virterror.h src/util/virterror.c: defined a new
  error code for that module
* po/POTFILES.in: add new file for translations

The libvirt_util.la library was mistakenly linked into libvirtd
directly. Since libvirt_util.la is already linked to libvirt.so,
this resulted in libvirtd getting two copies of the code and
more critically 2 copies of static global variables.

Testing in turn exposed a issue with loadable modules. The
gnulib replacement functions are not exported to loadable
modules. Rather than trying to figure out the name sof all
gnulib functions & export them, just linkage all loadable
modules against libgnu.la statically.

* daemon/Makefile.am: Remove linkage of libvirt_util.la
  and libvirt_driver.la
* src/Makefile.am: Link driver modules against libgnu.la
* src/libvirt.c: Don't try to load modules which were
  compiled out
* src/libvirt_private.syms: Export all other internal
  symbols that are required  by drivers

Integrate with libaudit.so for auditing of important operations.
libvirtd gains a couple of config entries for auditing. By
default it will enable auditing, if its enabled on the host.
It can be configured to force exit if auditing is disabled
on the host. It will can also send audit messages via libvirt
internal logging API

Places requiring audit reporting can use the VIR_AUDIT
macro to report data. This is a no-op unless auditing is
enabled

* autobuild.sh, mingw32-libvirt.spec.in: Disable audit
  on mingw
* configure.ac: Add check for libaudit
* daemon/libvirtd.aug, daemon/libvirtd.conf,
  daemon/test_libvirtd.aug, daemon/libvirtd.c: Add config
  options to enable auditing
* include/libvirt/virterror.h, src/util/virterror.c: Add
  VIR_FROM_AUDIT source
* libvirt.spec.in: Enable audit
* src/util/virtaudit.h, src/util/virtaudit.c: Simple internal
  API for auditing messages

Since bugs due to double-closed file descriptors are difficult to track down in a multi-threaded system, I am introducing the VIR_CLOSE(fd) macro to help avoid mistakes here.

There are lots of places where close() is being used. In this patch I am only cleaning up usage of close() in src/conf where the problems were.

I also dare to declare close() as being deprecated in libvirt code base (HACKING).

* src/Makefile.am (libvirt.def, libvirt_qemu.def): '\}' and '\t'
are not required by POSIX.  Use '}' and literal tab instead.
(install-data-local): Avoid sed -i.
* tests/read-bufsiz: Likewise.
Reported by Mitchell Hashimoto.

* src/Makefile.am (EXTRA_DIST): Ensure 'make distcheck' and
'rpmbuild' can reproduce a build.
* daemon/Makefile.am (DAEMON_SOURCES): Likewise.

'./autobuild.sh' with lcov installed discovered that our
coverage support has been bit-rotting for a while.  This
restores it back to a successful state, although I have
not yet spent any time looking through the resulting files to
look for low-hanging fruit in the unit test coverage front.

* configure.ac: Clear COMPILER_FLAGS at right place.
* Makefile.am (cov): Newer genhtml no longer likes plain -s.
* m4/compiler-flags.m4 (gl_COMPILER_FLAGS): Don't AC_SUBST
COMPILER_FLAGS; it is a shell variable for use in configure only.
* src/Makefile.am (AM_CFLAGS, AM_LDFLAGS): New variables, to make
it easier to provide global flag additions.  Use throughout, to
uniformly apply coverage flags.
* .gitignore: Globally ignore gcov output.
* daemon/.gitignore: Simplify.
* src/.gitignore: Likewise.
* tests/.gitignore: Likewise.