Not all applications have an existing event loop they need
to integrate with. Forcing them to implement the libvirt
event loop integration APIs is an undue burden. This just
exposes our simple poll() based implementation for apps
to use. So instead of calling

   virEventRegister(....callbacks...)

The app would call

   virEventRegisterDefaultImpl()

And then have a thread somewhere calling

    static bool quit = false;
    ....
    while (!quit)
      virEventRunDefaultImpl()

* daemon/libvirtd.c, tools/console.c,
  tools/virsh.c: Convert to public event loop APIs
* include/libvirt/libvirt.h.in, src/libvirt_private.syms: Add
  virEventRegisterDefaultImpl and virEventRunDefaultImpl
* src/util/event.c: Implement virEventRegisterDefaultImpl
  and virEventRunDefaultImpl using poll() event loop
* src/util/event_poll.c: Add full error reporting
* src/util/virterror.c, include/libvirt/virterror.h: Add
  VIR_FROM_EVENTS

The event loop implementation is used by more than just the
daemon, so move it into the shared area.

* daemon/event.c, src/util/event_poll.c: Renamed
* daemon/event.h, src/util/event_poll.h: Renamed
* tools/Makefile.am, tools/console.c, tools/virsh.c: Update
  to use new virEventPoll APIs
* daemon/mdns.c, daemon/mdns.c, daemon/Makefile.am: Update
  to use new virEventPoll APIs

The daemon code calls virEventAddHandleImpl directly instead
of calling the wrapper virEventAddHandle.

* tools/console.c, daemon/libvirtd.c, daemon/mdns.c: Convert to
  use primary event APIs

In case of imminent crash or upon request (signal USR2),
dump the logging buffer to the libvirtd.log file for
post-mortem analysis
* daemon/libvirtd.c: create a sig_fatal() handler connected to
  SIGFPE SIGSEGV SIGILL SIGABRT SIGBUS and SIGUSR2, just dumping
  the log buffer using virLogEmergencyDumpAll

Syslog is not the best place to go search for libvirt error
logs, change it to a default file output libvirtd.log, but
still keep standard error if not run as a daemon.
Depending on whether it's run as root or user, the log is saved
in the local state dir or in $HOME/.libvirt.
* daemon/libvirtd.c: change default logging to go to libvirtd.log

Done mechanically with:
$ git grep -l '\bDEBUG0\? *(' | xargs -L1 sed -i 's/\bDEBUG0\? *(/VIR_&/'

followed by manual deletion of qemudDebug in daemon/libvirtd.c, along
with a single 'make syntax-check' fallout in the same file, and the
actual deletion in src/util/logging.h.

* src/util/logging.h (DEBUG, DEBUG0): Delete.
* daemon/libvirtd.h (qemudDebug): Likewise.
* global: Change remaining clients over to VIR_DEBUG counterpart.

Pure cosmetic change.

Add a hook to the error reporting APIs to allow specific
error messages to be filtered out. Wire up libvirtd to
remove VIR_ERR_NO_DOMAIN & similar error codes from the
logs. They are still logged at DEBUG level.

* daemon/libvirtd.c: Filter VIR_ERR_NO_DOMAIN and friends
* src/libvirt_private.syms, src/util/virterror.c,
  src/util/virterror_internal.h: Hook for changing error
  reporting level

This reverts the additions in commit

  abff683f

taking us back to state where all errors are fully logged
in both libvirtd and normal clients.

THe intent was to stop VIR_ERR_NO_DOMAIN (No such domain
with UUID XXXX) messages from client apps polluting syslog
The change affected all error codes, but more seriously,
it also impacted errors from internal libvirtd infrastructure
For example guest autostart no longer logged errors. The
libvirtd network code no longer logged some errors. This
makes debugging incredibly hard

* daemon/libvirtd.c: Remove error log priority filter
* src/util/virterror.c, src/util/virterror_internal.h: Remove
  callback for overriding log priority

Setting unix_sock_group to something else than default "root" in
/etc/libvirt/libvirtd.conf prevents system libvirtd from dumping core on
crash. This is because we used setgid(unix_sock_group) before binding to
/var/run/libvirt/libvirt-sock* and setgid() back to original group.
However, if a process changes its effective or filesystem group ID, it
will be forbidden from leaving core dumps unless fs.suid_dumpable sysctl
is set to something else then 0 (and it is 0 by default).

Changing socket's group ownership after bind works better. And we can do
so without introducing a race condition since we loosen access rights by
changing the group from root to something else.

We were doing so for child processes but not for libvirtd itself.

* daemon/libvirtd.c (qemudStartWorker, qemudStartEventLoop): Avoid
leaking pthread_attr resources.
* src/util/threads-pthread.c (virThreadCreate): Likewise.

* daemon/libvirtd.c (qemudFreeClient): Avoid a leak.
(qemudDispatchServer): Avoid null dereference.

* daemon/libvirtd.c (qemudRunLoop): Free any remaining client data.

This reverts commit

 Log all errors at level INFO to stop polluting syslog
 04bd0360.

and makes virRaiseErrorFull() log errors at debug priority
when called from inside libvirtd. This stops libvirtd from
polluting it's own log with client errors at error priority
that'll be reported and logged on the client side anyway.

* daemon/libvirtd.c (qemudRunLoop): Pass allocation size, not
current count, to VIR_SHRINK_N.
* docs/hacking.html.in: Update doc example.
* HACKING: Regenerate.

libvirtd no longer deals with SIGCHLD in its signal handler
since the QEMU driver switched to always daemonize processes.
Thus remove the sigaction for it, to avoid warning log
messages

* daemon/libvirtd.c: Don't catch SIGCHLD

* daemon/libvirtd.h (qemud_server): Change types of members
tracking array sizes, and add allocation trackers.
* daemon/event.c (virEventLoop): Likewise.
(virEventAddHandleImpl, virEventAddTimeoutImpl)
(virEventCleanupTimeouts, virEventCleanupHandles): Use
VIR_RESIZE_N instead of VIR_REALLOC_N.  Tweak debug messages to
match type changes.
* daemon/libvirtd.c (qemudDispatchServer, qemudRunLoop): Likewise.

Per the gettext developer:
http://lists.gnu.org/archive/html/bug-gnu-utils/2010-10/msg00019.html
http://lists.gnu.org/archive/html/bug-gnu-utils/2010-10/msg00021.html

gettext() doesn't work correctly on all platforms unless you have
called setlocale().  Furthermore, gnulib's gettext.h has provisions
for setting up a default locale, which is the preferred method for
libraries to use gettext without having to call textdomain() and
override the main program's default domain (virInitialize already
calls bindtextdomain(), but this is insufficient without the
setlocale() added in this patch; and a redundant bindtextdomain()
in this patch doesn't hurt, but serves as a good example for other
packages that need to bind a second translation domain).

This patch is needed to silence a new gnulib 'make syntax-check'
rule in the next patch.

* daemon/libvirtd.c (main): Setup locale and gettext.
* src/lxc/lxc_controller.c (main): Likewise.
* src/security/virt-aa-helper.c (main): Likewise.
* src/storage/parthelper.c (main): Likewise.
* tools/virsh.c (main): Fix exit status.
* src/internal.h (DEFAULT_TEXT_DOMAIN): Define, for gettext.h.
(_): Simplify definition accordingly.
* po/POTFILES.in: Add src/storage/parthelper.c.

* bootstrap.conf (gnulib_modules): Add configmake.
* daemon/Makefile.am (libvirtd_CFLAGS): Drop defines provided by
gnulib.
* src/Makefile.am (INCLUDES): Likewise.
* tests/Makefile.am (INCLUDES): Likewise.
* tools/Makefile.am (virsh_CFLAGS): Likewise.
* daemon/libvirtd.c (qemudInitPaths, usage, main): Update
clients.
* src/cpu/cpu_map.c (CPUMAPFILE): Likewise.
* src/driver.c (DEFAULT_DRIVER_DIR): Likewise.
* src/internal.h (_): Likewise.
* src/libvirt.c (virInitialize): Likewise.
* src/lxc/lxc_conf.h (LXC_CONFIG_DIR, LXC_STATE_DIR, LXC_LOG_DIR):
Likewise.
* src/lxc/lxc_conf.c (lxcCapsInit, lxcLoadDriverConfig):
Likewise.
* src/network/bridge_driver.c (NETWORK_PID_DIR)
(NETWORK_STATE_DIR, DNSMASQ_STATE_DIR, networkStartup): Likewise.
* src/nwfilter/nwfilter_driver.c (nwfilterDriverStartup):
Likewise.
* src/qemu/qemu_conf.c (qemudLoadDriverConfig): Likewise.
* src/qemu/qemu_driver.c (qemudStartup): Likewise.
* src/remote/remote_driver.h (LIBVIRTD_PRIV_UNIX_SOCKET)
(LIBVIRTD_PRIV_UNIX_SOCKET_RO, LIBVIRTD_CONFIGURATION_FILE)
(LIBVIRT_PKI_DIR): Likewise.
* src/secret/secret_driver.c (secretDriverStartup): Likewise.
* src/security/security_apparmor.c (VIRT_AA_HELPER): Likewise.
* src/security/virt-aa-helper.c (main): Likewise.
* src/storage/storage_backend_disk.c (PARTHELPER): Likewise.
* src/storage/storage_driver.c (storageDriverStartup): Likewise.
* src/uml/uml_driver.c (TEMPDIR, umlStartup): Likewise.
* src/util/hooks.c (LIBVIRT_HOOK_DIR): Likewise.
* tools/virsh.c (main): Likewise.
* docs/hooks.html.in: Likewise.

Similarly to deprecating close(), I am now deprecating fclose() and
introduce VIR_FORCE_FCLOSE() and VIR_FCLOSE(). Also, fdopen() is replaced with
VIR_FDOPEN().

Most of the files are opened in read-only mode, so usage of
VIR_FORCE_CLOSE() seemed appropriate. Others that are opened in write
mode already had the fclose()<  0 check and I converted those to
VIR_FCLOSE()<  0.

I did not find occurrences of possible double-closed files on the way.

* daemon/libvirtd.c

Using automated replacement with sed and editing I have now replaced all
occurrences of close() with VIR_(FORCE_)CLOSE() except for one, of
course. Some replacements were straight forward, others I needed to pay
attention. I hope I payed attention in all the right places... Please
have a look. This should have at least solved one more double-close
error.

With SystemTap 1.0 a part of the generated macros in probes.h
expands to:

volatile __typeof__(((name))) arg2 = (name);

GCC reports an 'invalid initialize' error when name has type
char[]. Therfore, add casts to char* to avoid this.

It is useful to know where the client is connecting from,
so include the socket address in probe data.

* daemon/libvirtd.h: Use virSocketAddr for storing client
  address and keep printable address handy for logging
* daemon/libvirtd.c: Include socket address in client
  connect/disconnect probes
* daemon/probes.d: Add socket address to probes
* examples/systemtap/client.stp: Print socket address
* src/util/network.h: Add sockaddr_un to virSocketAddr union

Adds initial support for dtrace static probes in libvirtd
daemon, assuming use of systemtap dtrace compat shim on
Linux. The probes are inserted for network client connect,
disconnect, TLS handshake states and authentication protocol
states.

This can be tested by running the xample program and then
attempting to connect with any libvirt client (virsh,
virt-manager, etc).

 # stap examples/systemtap/client.stp
  Client fd=44 connected readonly=0
  Client fd=44 auth polkit deny pid:24997,uid:500
  Client fd=44 disconnected
  Client fd=46 connected readonly=1
  Client fd=46 auth sasl allow test
  Client fd=46 disconnected

The libvirtd.stp file should also really not be required,
since it is duplicated info that is already available in
the main probes.d definition file. A script to autogenerate
the .stp file is needed, either in libvirtd tree, or better
as part of systemtap itself.

* Makefile.am: Add examples/systemtap subdir
* autobuild.sh: Disable dtrace for mingw32
* configure.ac: Add check for dtrace
* daemon/.gitignore: Ignore generated dtrace probe file
* daemon/Makefile.am: Build dtrace probe header & object
  files
* daemon/libvirtd.stp: SystemTAP convenience probeset
* daemon/libvirtd.c: Add connect/disconnect & TLS probes
* daemon/remote.c: Add SASL and PolicyKit auth probes
* daemon/probes.d: Master probe definition
* daemon/libvirtd.h: Add convenience macro for probes
  so that compilation is a no-op when dtrace is not available
* examples/systemtap/Makefile.am, examples/systemtap/client.stp
  Example systemtap script using dtrace probe markers
* libvirt.spec.in: Enable dtrace on F13/RHEL6
* mingw32-libvirt.spec.in: Force disable dtrace

Integrate with libaudit.so for auditing of important operations.
libvirtd gains a couple of config entries for auditing. By
default it will enable auditing, if its enabled on the host.
It can be configured to force exit if auditing is disabled
on the host. It will can also send audit messages via libvirt
internal logging API

Places requiring audit reporting can use the VIR_AUDIT
macro to report data. This is a no-op unless auditing is
enabled

* autobuild.sh, mingw32-libvirt.spec.in: Disable audit
  on mingw
* configure.ac: Add check for libaudit
* daemon/libvirtd.aug, daemon/libvirtd.conf,
  daemon/test_libvirtd.aug, daemon/libvirtd.c: Add config
  options to enable auditing
* include/libvirt/virterror.h, src/util/virterror.c: Add
  VIR_FROM_AUDIT source
* libvirt.spec.in: Enable audit
* src/util/virtaudit.h, src/util/virtaudit.c: Simple internal
  API for auditing messages

Refactor some daemon code to facilitate the introductioin of static
probes, sanitizing function exit paths in many places

* daemon/libvirtd.c: Pass the dname string into remoteCheckDN
  to let caller deal with failure paths. Add separate exit paths
  to remoteCheckCertificate for auth failure vs denial. Merge
  all exit paths in qemudDispatchServer to one cleanup block
* daemon/remote.c: Add separate exit paths to SASL & PolicyKit
  functions for auth failure vs denial

This address BZ # 556599:

  https://bugzilla.redhat.com/show_bug.cgi?id=556599

When libvirtd exits it is leaving UNIX domain sockets on
the filesystem. These need to be removed.

The qemudInitPaths() method has signficant code churn to
switch from using a pre-allocated buffer on the stack, to
dynamically allocating on the heap.

* daemon/libvirtd.c, daemon/libvirtd.h: Store a reference
  to the UNIX domain socket path and unlink it on shutdown

* daemon/libvirtd.c (remoteReadConfigFile): Diagnose an invalid
host UUID rather than silently exiting with status 7.

Allow for a host UUID in the capabilities XML. Local drivers
will initialize this from the SMBIOS data. If a sanity check
shows SMBIOS uuid is invalid, allow an override from the
libvirtd.conf configuration file

* daemon/libvirtd.c, daemon/libvirtd.conf: Support a host_uuid
  configuration option
* docs/schemas/capability.rng: Add optional host uuid field
* src/conf/capabilities.c, src/conf/capabilities.h: Include
  host UUID in XML
* src/libvirt_private.syms: Export new uuid.h functions
* src/lxc/lxc_conf.c, src/qemu/qemu_driver.c,
  src/uml/uml_conf.c: Set host UUID in capabilities
* src/util/uuid.c, src/util/uuid.h: Support for host UUIDs
* src/node_device/node_device_udev.c: Use the host UUID functions
* tests/confdata/libvirtd.conf, tests/confdata/libvirtd.out: Add
  new host_uuid config option to test

Some diagnostics had a hard-coded "libvirtd: " prefix, some used
"error: " and some used "argv[0]: ".  Always use "argv[0]: ".
* daemon/libvirtd.c (argv0): New global.
(main): Set it.
(version, usage): Remove argv0 parameter.  Use global; update callers.
(daemonForkIntoBackground): Use argv0:, not error:.
(qemudWritePidFile): Start each diagnostic with argv0:.
Suggested by Eric Blake.

* daemon/libvirtd.c (daemonForkIntoBackground, main): Mark strings
for translation.
(usage): Rework --help so that it is translatable, replacing
each embedded, configuration-dependent, macro with an `%s'.

libvirtd: don't ignore virInitialize failure
* daemon/libvirtd.c (main): Diagnose virInitialize failure
and exit nonzero.

Run this:
  git grep -l 'VIR_ERROR\s*("'|xargs perl -pi -e \
    's/(VIR_ERROR)\s*\((".*?"),/$1(_($2),/'

Run this:
  git grep -l 'VIR_ERROR0\s*("'|xargs perl -pi -e \
    's/(VIR_ERROR0)\s*\((".*?")\)/$1(_($2))/'

Change VIR_ERROR("%s", "..."
to     VIR_ERROR0("..."

and

Change VIR_ERROR("%s", _("...")
to     VIR_ERROR0(_("...")

Use this command:
  git grep -E -l 'VIR_ERROR\("%s", (_\()?"'|xargs perl -pi -e \
  's/VIR_ERROR\("%s", (_\()?"/VIR_ERROR0($1"/'

* daemon/libvirtd.c (main): Diagnose virInitialize failure
and exit nonzero.

Approximately 60 messages were marked.  Since these diagnostics are
intended solely for developers and maintainers, encouraging translation
is deemed to be counterproductive:
http://thread.gmane.org/gmane.comp.emulators.libvirt/25050/focus=25052

Run this command:
  git grep -l VIR_WARN|xargs perl -pi -e \
    's/(VIR_WARN0?)\s*\(_\((".*?")\)/$1($2/'