Update the libvirtd config handling code to use virReportError
instead of the virConfError custom macro
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When the guest changes its memory balloon applications may want
to know what the new value is, without having to periodically
poll on XML / domain info. Introduce a "balloon change" event
to let apps see this

* include/libvirt/libvirt.h.in: Define the
  virConnectDomainEventBalloonChangeCallback callback
  and VIR_DOMAIN_EVENT_ID_BALLOON_CHANGE constant
* python/libvirt-override-virConnect.py,
  python/libvirt-override.c: Wire up helpers for new event
* daemon/remote.c: Helper for serializing balloon event
* examples/domain-events/events-c/event-test.c,
  examples/domain-events/events-python/event-test.py: Add
  example of balloon event usage
* src/conf/domain_event.c, src/conf/domain_event.h: Handling
  of balloon events
* src/remote/remote_driver.c: Add handler of balloon events
* src/remote/remote_protocol.x: Define wire protocol for
  balloon events
* src/remote_protocol-structs: Likewise.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Domains configured with autostart may fail to start if the host
network stack has not been started.  E.g. when using bridged
networking autostarting a domain can fail with

libvirtd[1403]: 2012-06-20 13:23:49.833+0000: 1485: error :
qemuAutostartDomain:177 : Failed to autostart VM 'test': Cannot get
interface MTU on 'br0': No such device

The callback that is invoked when a new RPC client is
initialized does not have any opaque parameter. Add
one so that custom data can be passed into the callback
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

since they aren't. Detected by Debian's lintian.

The generator doesn't handle lists of virDomainSnapshotPtr, so
this commit requires a bit more work than some RPC additions.

* src/remote/remote_protocol.x
(REMOTE_PROC_DOMAIN_LIST_ALL_SNAPSHOTS)
(REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_ALL_CHILDREN): New RPC calls,
with corresponding structs.
* daemon/remote.c (remoteDispatchDomainListAllSnapshots)
(remoteDispatchDomainSnapshotListAllChildren): New functions.
* src/remote/remote_driver.c (remoteDomainListAllSnapshots)
(remoteDomainSnapshotListAllChildren): Likewise.
* src/remote_protocol-structs: Regenerate.

This patch wires up the RPC protocol handlers for
virConnectListAllDomains(). The RPC generator has no support for the way
how virConnectListAllDomains() returns the results so the handler code
had to be done manually.

The new api is handled by REMOTE_PROC_CONNECT_LIST_ALL_DOMAINS, with
number 273 and marked with high priority.

To facilitate future troubleshooting add a bunch more debugging
statements into important startup parts of libvirt

Previous commit

  commit 32a9aac2
  Author: William Jon McCann <william.jon.mccann@gmail.com>
  Date:   Thu May 3 12:36:27 2012 -0400

    Use XDG Base Directories instead of storing in home directory

Accidentally changed the umask when creating /var/run/libvirt
to 077. This prevents /var/run/libvirt being readable by non-root,
which is required for non-root to connect to libvirtd. Fix the
code so that umask 077 is only used for the non-privileged libvirtd
instance.

Only the non-privileged libvirtd instance uses $HOME. So avoid
running the code for migrating to XDG directories unless using
a non-privileged libvirtd

Commit 1c275e9a accidentally dropped the storage driver from
libvirtd, because it depended on a C preprocessor macro that
was not defined.  Furthermore, if you do './configure
--without-storage-dir --with-storage-disk' or any other combination
where you explicitly build a subset of storage backends excluding
the dir backend, then the build is broken.

Based on analysis by Osier Yang.

* configure.ac (WITH_STORAGE): Define top-level conditional.
* src/Makefile.am (mod_LTLIBRARIES): Build driver even when
storage_dir is disabled.
* daemon/libvirtd.c: Pick up storage driver for any backend, not
just dir.
* daemon/Makefile.am (libvirtd_LDADD): Likewise.

Some of our rules used $(PERL), while others used 'perl'.  Always
using the variable allows a developer to point to a different (often
better) perl than the default one found on $PATH.

* daemon/Makefile.am ($(srcdir)/remote_dispatch.h): s/perl/$(PERL).
* src/Makefile.am ($(srcdir)/remote/remote_client_bodies.h)
(PDWTAGS, %protocol.c, %_probes.stp): Likewise.

Without this fix, a VPATH build (such as used by ./autobuild.sh)
fails with messages like:

make[3]: Entering directory `/home/remote/eblake/libvirt-tmp2/build/daemon'
../../build-aux/augeas-gentest.pl libvirtd.conf ../../daemon/test_libvirtd.aug.in test_libvirtd.aug
cannot read libvirtd.conf: No such file or directory at ../../build-aux/augeas-gentest.pl line 38.

Since the test files are not part of the tarball, we can generate
them into the build dir, but rather than create a subdirectory
just for the test file, it is easier to test them directly in
libvirt.git/src.

* daemon/Makefile.am (AUG_GENTEST): Factor out definition.
(test_libvirtd.aug): Look for correct file.
* src/Makefile.am (AUG_GENTEST): Use $(PERL).
(qemu/test_libvirtd_qemu.aug, lxc/test_libvirtd_lxc.aug)
(locking/test_libvirt_sanlock.aug): Rename to avoid subdirectories.
(check-augeas-qemu, check-augeas-lxc, check-augeas-sanlock): Reflect
location of built tests.
* configure.ac (PERL): Substitute perl.

There was no rule forcing libvirt_qemu_probes.o to be built
before libvirt_qemu_probes.lo was used. Also libvirtd was
still referencing the .o file, rather than the .lo file.

Both the .lo and .o file must be listed as DEPENDENCIES,
otherwise libtool will unhelpfully delete the .o file
once the .lo file is created.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When adding new config file parameters, the corresponding
additions to the augeas lens' are constantly forgotten.
Also there are augeas test cases, these don't catch the
error, since they too are never updated.

To address this, the augeas test cases need to be auto-generated
from the example config files.

* build-aux/augeas-gentest.pl: Helper to generate an
  augeas test file, substituting in elements from the
  example config files
* src/Makefile.am, daemon/Makefile.am: Switch to
  auto-generated augeas test cases
* daemon/test_libvirtd.aug, daemon/test_libvirtd.aug.in,
  src/locking/test_libvirt_sanlock.aug,
  src/locking/test_libvirt_sanlock.aug.in,
  src/lxc/test_libvirtd_lxc.aug,
  src/lxc/test_libvirtd_lxc.aug.in,
  src/qemu/test_libvirtd_qemu.aug,
  src/qemu/test_libvirtd_qemu.aug.in: Remove example
  config file data, replacing with a ::CONFIG:: placeholder
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add nmissing 'host_uuid' entry to libvirtd.conf lens and
rename spice_passwd to spice_password in qemu.conf lens
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Instead of doing

  # example_config

use

  #example_config

so it is possible to programatically uncomment example config
options, as distinct from their comment/descriptions

Also delete rogue trailing comma not allowed by lens
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Remove the uid param from virGetUserConfigDirectory,
virGetUserCacheDirectory, virGetUserRuntimeDirectory,
and virGetUserDirectory

These functions were universally called with the
results of getuid() or geteuid(). To make it practical
to port to Win32, remove the uid parameter and hardcode
geteuid()
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

* daemon/libvirtd.c: Set custom driver module dir if the current
  binary name is 'lt-libvirtd' (indicating execution directly
  from GIT checkout)
* src/driver.c, src/driver.h, src/libvirt_driver_modules.syms: Add
  virDriverModuleInitialize to allow driver module location to
  be changed
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When building as driver modules, it is not possible for the QEMU
driver module to reference the DTrace/SystemTAP probes linked into
the main libvirt.so. Thus we need to move the QEMU probes into a
separate file 'libvirt_qemu_probes.d'. Also rename the existing
file from 'probes.d' to 'libvirt_probes.d' while we're at it

* daemon/Makefile.am, src/internal.h: Include libvirt_probes.h
  instead of probes.h
* src/Makefile.am: Add rules for libvirt_qemu_probes.d
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor_json.c,
  src/qemu/qemu_monitor_text.c: Include libvirt_qemu_probes.h
* src/libvirt_probes.d: Rename from probes.d
* src/libvirt_qemu_probes.d: QEMU specific probes formerly
  in probes.d
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The driver modules all use symbols which are defined in libvirt.so.
Thus for loading of modules to work, the binary that libvirt.so
is linked to must export its symbols back to modules. If the
libvirt.so itself is dlopen()d then the RTLD_GLOBAL flag must
be set. Unfortunately few, if any, programming languages use
the RTLD_GLOBAL flag when loading modules :-( This means is it
not practical to use driver modules for any libvirt client side
drivers (OpenVZ, VMWare, Hyper-V, Remote client, test).

This patch changes the build process so only server side drivers
are built as modules (Xen, QEMU, LXC, UML)

* daemon/libvirtd.c: Add missing load of 'interface' driver
* src/Makefile.am: Only build server side drivers as modules
* src/libvirt.c: Don't load any driver modules
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Adding syslog.target is obsolete, avahi.target does not
exist and dbus.target is also obsolete
Reported-by: NLennart Poettering <lpoetter@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Commit 2223ea98 removed the only use of 'server' param in
remoteDispatchAuthPolkit().  Mark the parameter with ATTRIBUTE_UNUSED
to fix the build when configuring with polkit0.

Due to a bug in editing /etc/sysconfig/libvirtd, VDSM was causing
libvirt processes to run with the following command line args

   /usr/sbin/libvirtd --listen '#' 'by vdsm'

While it correctly rejects any invalid option flags, libvirtd
was not rejecting any non-option command line arguments

* daemon/libvirtd.c: Reject non-option argv

There is no 'udev.target' unit in systemd (only 'udev.service')
yet libvirtd's unit file had a dep on one. There's no compelling
reason for a dep on udev, so remove it altogether.
Reported-by: NAvi Kivity <avi@redhat.com>

The current unprivileged user libvirtd sockets are in the abstract
namespace. This has a number of problems

 - You can't connect to them remotely using the nc/ssh tunnel
 - This is not portable for OS-X, BSD & probably others
 - Parent directory permissions don't apply

As defined in:
http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

This offers a number of advantages:
 * Allows sharing a home directory between different machines, or
sessions (eg. using NFS)
 * Cleanly separates cache, runtime (eg. sockets), or app data from
user settings
 * Supports performing smart or selective migration of settings
between different OS versions
 * Supports reseting settings without breaking things
 * Makes it possible to clear cache data to make room when the disk
is filling up
 * Allows us to write a robust and efficient backup solution
 * Allows an admin flexibility to change where data and settings are stored
 * Dramatically reduces the complexity and incoherence of the
system for administrators

This is part of the solution to the problem detailed in:

  https://bugzilla.redhat.com/show_bug.cgi?id=816465

and further detailed in

  https://www.redhat.com/archives/libvir-list/2012-May/msg00202.htm

A short explanation is included in the comments of the patch itself.

Note that this patch by itself breaks communication between lldpad and
libvirtd, so the other 3 patches in the series must be applied at the
same time as this patch.

DBus connection. The HAL device code further requires that
the DBus connection is integrated with the event loop and
provides such glue logic itself.

The forthcoming FirewallD integration also requires a
dbus connection with event loop integration. Thus we need
to pull the current event loop glue out of the HAL driver.

Thus we create src/util/virdbus.{c,h} files. This contains
just one method virDBusGetSystemBus() which obtains a handle
to the single shared system bus instance, with event glue
automagically setup.

* daemon/libvirtd-config.c (daemonConfigFree): fix memory leaks.

How to reproduce?

% make && make -C tests check TESTS=libvirtdconftest
% cd tests && valgrind -v --leak-check=full ./libvirtdconftest

actual result:

==11008== 185 bytes in 5 blocks are definitely lost in loss record 3 of 5
==11008==    at 0x4A05FDE: malloc (vg_replace_malloc.c:236)
==11008==    by 0x39CF07F6E1: strdup (strdup.c:43)
==11008==    by 0x406626: daemonConfigLoadOptions (libvirtd-config.c:438)
==11008==    by 0x406800: daemonConfigLoadData (libvirtd-config.c:492)
==11008==    by 0x403CCF: testCorrupt (libvirtdconftest.c:110)
==11008==    by 0x404FAD: virtTestRun (testutils.c:145)
==11008==    by 0x403A34: mymain (libvirtdconftest.c:219)
==11008==    by 0x404687: virtTestMain (testutils.c:700)
==11008==    by 0x39CF01ECDC: (below main) (libc-start.c:226)
==11008==
==11008== LEAK SUMMARY:
==11008==    definitely lost: 185 bytes in 5 blocks
Signed-off-by: NAlex Jia <ajia@redhat.com>

Using VIR_ERROR means the test suite can't catch error messages
easily. Use the proper error reporting APIs instead

Rename existing daemonConfigLoad API to daemonConfigLoadFile and
add an alternative daemonConfigLoadData

* daemon/libvirtd-config.c, daemon/libvirtd-config.h: Add
  daemonConfigLoadData and rename daemonConfigLoad to
  daemonConfigLoadFile
* daemon/libvirtd.c: Update for renamed API

To enable creation of unit tests, split the libvirtd config file
loading code out into separate files.

* daemon/libvirtd.c: Delete config loading code / structs
* daemon/libvirtd-config.c, daemon/libvirtd-config.h: Config
  file loading APIs
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

* configure.ac: Set WITH_SYSCTL only on Linux hosts
* daemon/Makefile.am: Conditionalize install-sysctl using WITH_SYSCTL
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Cc: Jason Helfman <jhelfman@e-e.com>

The code is splattered with a mix of

  sizeof foo
  sizeof (foo)
  sizeof(foo)

Standardize on sizeof(foo) and add a syntax check rule to
enforce it
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

 * Don't advertise information on the network without consent of
   the user, either through manual configuration, or a user
   interface that drives this option.
 * Since libvirtd must be configured for network access anyway
   (for all but ssh), this setting was not useful "out of the box",
   so changing this default setting does not remove "out of the box"
   functionality.

This patch introduces a new event type for the QMP event
SUSPEND:

    VIR_DOMAIN_EVENT_ID_PMSUSPEND

The event doesn't take any data, but considering there might
be reason for wakeup in future, the callback definition is:

typedef void
(*virConnectDomainEventSuspendCallback)(virConnectPtr conn,
                                        virDomainPtr dom,
                                        int reason,
                                        void *opaque);

"reason" is unused currently, always passes "0".

This patch introduces a new event type for the QMP event
WAKEUP:

    VIR_DOMAIN_EVENT_ID_PMWAKEUP

The event doesn't take any data, but considering there might
be reason for wakeup in future, the callback definition is:

typedef void
(*virConnectDomainEventWakeupCallback)(virConnectPtr conn,
                                       virDomainPtr dom,
                                       int reason,
                                       void *opaque);

"reason" is unused currently, always passes "0".

This patch introduces a new event type for the QMP event
DEVICE_TRAY_MOVED, which occurs when the tray of a removable
disk is moved (i.e opened or closed):

    VIR_DOMAIN_EVENT_ID_TRAY_CHANGE

The event's data includes the device alias and the reason
for tray status' changing, which indicates why the tray
status was changed. Thus the callback definition for the event
is:

enum {
    VIR_DOMAIN_EVENT_TRAY_CHANGE_OPEN = 0,
    VIR_DOMAIN_EVENT_TRAY_CHANGE_CLOSE,

\#ifdef VIR_ENUM_SENTINELS
    VIR_DOMAIN_EVENT_TRAY_CHANGE_LAST
\#endif
} virDomainEventTrayChangeReason;

typedef void
(*virConnectDomainEventTrayChangeCallback)(virConnectPtr conn,
                                           virDomainPtr dom,
                                           const char *devAlias,
                                           int reason,
                                           void *opaque);

Currently the libvirt.service unit file for systemd does not
specify any kill mode. So systemd kills off every process
inside its cgroup. ie all dnsmasq processes, all virtual
machines. This obviously not what we want. Set KillMode=process
so that it only kills the top level process of libvirtd

* daemon/libvirtd.service.in: Add KillMode=process
Reported-By: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>