A lot of code in libvirtd's dispatcher used the style

    dom = get_nonnull_domain (conn, args->dom);

Instead of the normal libvirt style

    dom = get_nonnull_domain(conn, args->dom);

* daemon/remote.c: Remove all whitelist before function brackets

The systemtap directory for tapsets is called

  /usr/share/systemtap/tapset

Not

 /usr/share/systemtap/tapsets

* daemon/Makefile.am,libvirt.spec.in: s/tapsets/tapset/

Move the buffers to the heap allocated client/private data structs.

The daemon loops over the linked list of streams when a client
quits, closing any that the client hadn't already closed. Except
it didn't ever move to the next element in the list!

* daemon/stream.c: Fix loop over linked list of streams

* daemon/remote.c, src/remote/remote_driver.c: Implementation
  of storage vol upload/download APIs
* src/remote/remote_protocol.x: Wire protocol definition for
  upload/download
* daemon/remote_dispatch_args.h, daemon/remote_dispatch_prototypes.h,
  daemon/remote_dispatch_table.h, src/remote/remote_protocol.h,
  src/remote/remote_protocol.c: Re-generate

The Open Nebula driver has been unmaintained since it was first
introduced. The only commits have been for tree-wide cleanups.
It also has a major design flaw, in that it only knows about guests
that it has created itself, which makes it of very limited use.

Discussions wrt evolution of the VMWare ESX driver, concluded that
it should limit itself to single-node ESX operation and not try to
manage the multi-node architecture of VirtualCenter. Open Nebula
is a cluster like Virtual Center, not a single node system, so
the same reasoning applies.

The DeltaCloud project includes an Open Nebula driver and is a much
better fit architecturally, since it is explicitly targetting the
distributed multihost cluster scenario.

Thus this patch deletes the libvirt Open Nebula driver with the
recommendation that people use DeltaCloud for managing it instead.

* configure.ac: Remove probe for xmlrpc & --with-one arg
* daemon/Makefile.am, daemon/libvirtd.c, src/Makefile.am: Remove
  ONE driver build
* src/opennebula/one_client.c, src/opennebula/one_client.h,
  src/opennebula/one_conf.c, src/opennebula/one_conf.h,
  src/opennebula/one_driver.c, src/opennebula/one_driver.c: Delete
  files
* autobuild.sh, libvirt.spec.in, mingw32-libvirt.spec.in: Remove
  build rules for Open Nebula
* docs/drivers.html.in, docs/sitemap.html.in: Remove reference
  to OpenNebula
* docs/drvone.html.in: Delete file

Commit f44bfb7f was supposed to make sure no additional libvirt API (esp.
*Free) is called before remoteDispatchConnError() is called on error.
However, the patch missed two instances.

Child processes don't always reach _exit(); if they die from a
signal, then any messages should still be accurate.  Most users
either expect a 0 status (thankfully, if status==0, then
WIFEXITED(status) is true and WEXITSTATUS(status)==0 for all
known platforms) or were filtering on WIFEXITED before printing
a status, but a few were missing this check.  Additionally,
nwfilter_ebiptables_driver was making an assumption that works
on Linux (where WEXITSTATUS shifts and WTERMSIG just masks)
but fails on other platforms (where WEXITSTATUS just masks and
WTERMSIG shifts).

* src/util/command.h (virCommandTranslateStatus): New helper.
* src/libvirt_private.syms (command.h): Export it.
* src/util/command.c (virCommandTranslateStatus): New function.
(virCommandWait): Use it to also diagnose status from signals.
* src/security/security_apparmor.c (load_profile): Likewise.
* src/storage/storage_backend.c
(virStorageBackendQEMUImgBackingFormat): Likewise.
* src/util/util.c (virExecDaemonize, virRunWithHook)
(virFileOperation, virDirCreate): Likewise.
* daemon/remote.c (remoteDispatchAuthPolkit): Likewise.
* src/nwfilter/nwfilter_ebiptables_driver.c (ebiptablesExecCLI):
Likewise.

Bug https://bugzilla.redhat.com/show_bug.cgi?id=689374 reported libvirtd
crash during error dispatch.

The reason is that libvirtd uses remoteDispatchConnError() with non-NULL
conn parameter which means that virConnGetLastError() is used instead of
its thread safe replacement virGetLastError().

So when several libvirtd threads are reporting errors at the same time,
the errors can get mixed or corrupted or in case of bad luck libvirtd
itself crashes.

Since Daniel B. is going to rewrite this code from scratch on top of his
RPC infrastructure, I tried to come up with a minimal fix. Thus,
remoteDispatchConnError() now just ignores its conn argument and always
calls virGetLastError(). However, several callers had to be touched as
well, since no libvirt API is allowed to be called before dispatching
the error. Doing so would reset the error and we would have nothing to
dispatch. As a result of that, the code is not very nice but that
doesn't really make daemon/remote.c worse than it is now :-) And it will
all die soon, which is good.

The bug report also contains a reproducer in C which detects both mixed
up error messages and libvirtd crash. Before this patch, I was able to
crash libvirtd in about 20 seconds up to 3 minutes depending on number
of CPU cores (more is better) and luck.

* src/remote/remote_protocol.x: Define wire protocol
* daemon/remote.c, src/remote/remote_driver.c: Add new
  functions for virDomainMigrateSetSpeed API
* src/remote/remote_protocol.c, src/remote/remote_protocol.h,
  daemon/remote_dispatch_args.h, daemon/remote_dispatch_prototypes.h,
  daemon/remote_dispatch_table.h: Re-generate files

Add a new xen driver based on libxenlight [1], which is the primary
toolstack starting with Xen 4.1.0.  The driver is stateful and runs
privileged only.

Like the existing xen-unified driver, the libxenlight driver is
accessed with xen:// URI.  Driver selection is based on the status
of xend.  If xend is running, the libxenlight driver will not load
and xen:// connections are handled by xen-unified.  If xend is not
running *and* the libxenlight driver is available, xen://
connections are deferred to the libxenlight driver.

V6:
 - Address several code style issues noted by Daniel Veillard
 - Make drive work with xen:/// URI
 - Hold domain object reference while domain is injected in
   libvirt event loop.  Race found and fixed by Markus Groß.

V5:
 - Ensure events are unregistered when domain private data
   is destroyed.  Discovered and fixed by Markus Groß.

V4:
 - Handle restart of libvirtd, reconnecting to previously
   started domains
 - Rebased to current master
 - Tested against Xen 4.1 RC7-pre (c/s 22961:c5d121fd35c0)

V3:
  - Reserve vnc port within driver when autoport=yes

V2:
  - Update to Xen 4.1 RC6-pre (c/s 22940:5a4710640f81)
  - Rebased to current master
  - Plug memory leaks found by Stefano Stabellini and valgrind
  - Handle SHUTDOWN_crash domain death event

[1] http://lists.xensource.com/archives/html/xen-devel/2009-11/msg00436.html

This is the part allowing to dynamically resize the debug log
buffer from it's default 64kB size. The buffer is now dynamically
allocated.
It adds a new API virLogSetBufferSize() which resizes the buffer
If passed a zero size, the buffer is deallocated and we do the small
optimization of not formatting messages which are not output anymore.
On the daemon side, it just adds a new option log_buffer_size to
libvirtd.conf and call virLogSetBufferSize() if needed
* src/util/logging.h src/util/logging.c src/libvirt_private.syms:
  make buffer dynamic and add virLogSetBufferSize() internal API
* daemon/libvirtd.conf: document the new log_buffer_size option
* daemon/libvirtd.c: read and use the new log_buffer_size option

Remote protocol implementation of virDomainSetBlkioParameters and virDomainGetBlkioParameters.
Signed-off-by: NGui Jianfeng <guijianfeng@cn.fujitsu.com>

This patch implements the remote protocol to address the new API.
Signed-off-by: NTaku Izumi <izumi.taku@jp.fujitsu.com>

otherwise the user might not have enough permissions to access the
socket if root's umask is 077.

http://bugs.debian.org/614210

When building libvirt without libvirtd, I receive the following errors:
make[1]: Leaving directory `/home/wency/source/test/libvirt/src'
 (cd daemon && make  top_distdir=../libvirt-0.8.8 distdir=../libvirt-0.8.8/daemon \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[1]: Entering directory `/home/wency/source/test/libvirt/daemon'
make[1]: *** No rule to make target `libvirtd.8.in', needed by `distdir'.  Stop.

This bug was caused by commit 6db98a2d.
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Not all applications have an existing event loop they need
to integrate with. Forcing them to implement the libvirt
event loop integration APIs is an undue burden. This just
exposes our simple poll() based implementation for apps
to use. So instead of calling

   virEventRegister(....callbacks...)

The app would call

   virEventRegisterDefaultImpl()

And then have a thread somewhere calling

    static bool quit = false;
    ....
    while (!quit)
      virEventRunDefaultImpl()

* daemon/libvirtd.c, tools/console.c,
  tools/virsh.c: Convert to public event loop APIs
* include/libvirt/libvirt.h.in, src/libvirt_private.syms: Add
  virEventRegisterDefaultImpl and virEventRunDefaultImpl
* src/util/event.c: Implement virEventRegisterDefaultImpl
  and virEventRunDefaultImpl using poll() event loop
* src/util/event_poll.c: Add full error reporting
* src/util/virterror.c, include/libvirt/virterror.h: Add
  VIR_FROM_EVENTS

The event loop implementation is used by more than just the
daemon, so move it into the shared area.

* daemon/event.c, src/util/event_poll.c: Renamed
* daemon/event.h, src/util/event_poll.h: Renamed
* tools/Makefile.am, tools/console.c, tools/virsh.c: Update
  to use new virEventPoll APIs
* daemon/mdns.c, daemon/mdns.c, daemon/Makefile.am: Update
  to use new virEventPoll APIs

The daemon code calls virEventAddHandleImpl directly instead
of calling the wrapper virEventAddHandle.

* tools/console.c, daemon/libvirtd.c, daemon/mdns.c: Convert to
  use primary event APIs

In case of imminent crash or upon request (signal USR2),
dump the logging buffer to the libvirtd.log file for
post-mortem analysis
* daemon/libvirtd.c: create a sig_fatal() handler connected to
  SIGFPE SIGSEGV SIGILL SIGABRT SIGBUS and SIGUSR2, just dumping
  the log buffer using virLogEmergencyDumpAll

As the file may grow quite a bit especially with debug turned on.
* daemon/libvirtd.logrotate.in daemon/Makefile.am libvirt.spec.in:
  add new logrotate file for the daemon log

Syslog is not the best place to go search for libvirt error
logs, change it to a default file output libvirtd.log, but
still keep standard error if not run as a daemon.
Depending on whether it's run as root or user, the log is saved
in the local state dir or in $HOME/.libvirt.
* daemon/libvirtd.c: change default logging to go to libvirtd.log

Remove the <stdbool.h> header from all source files / headers
and just put it into internal.h

* src/internal.h: Add <stdbool.h>

Right now, 'man libvirtd' includes information that depends on
configure results, so it must be generated on the fly and live
in $(builddir); however, requiring pod2man on all end user
machines is overkill.  Meanwhile, 'man virsh' doesn't mention
any configure results, so it can be built at 'make dist' time.
If that situation changes in the future, we can generate virsh.1
in the same way that we generate libvirtd.8.

* daemon/Makefile.am (libvirtd.8.in): New rule, to run pod2man in
advance of distribution.
(libvirtd.8): Use only sed from tarball.
(EXTRA_DIST): Ship new file.
(libvirtd.pod): Delete unused rule.
(man8_MANS): Let automake know which section to use.
(CLEANFILES, MAINTAINERCLEANFILES): Adjust to new files.
* tools/Makefile.am (dist_man1_MANS): Distribute pre-built man
pages, fine since they don't require any substitution.
(virt-xml-validate.1, virt-pki-validate.1): Change input source.
(virsh.1): Build into srcdir.
(CLEANFILES, MAINTAINERCLEANFILES): Adjust to new build style.
* daemon/.gitignore: Update.
Reported by Diego Elio Pettenò.

Done mechanically with:
$ git grep -l '\bDEBUG0\? *(' | xargs -L1 sed -i 's/\bDEBUG0\? *(/VIR_&/'

followed by manual deletion of qemudDebug in daemon/libvirtd.c, along
with a single 'make syntax-check' fallout in the same file, and the
actual deletion in src/util/logging.h.

* src/util/logging.h (DEBUG, DEBUG0): Delete.
* daemon/libvirtd.h (qemudDebug): Likewise.
* global: Change remaining clients over to VIR_DEBUG counterpart.

To make it easier to investigate problems with async event
delivery, add two more debugging lines

* daemon/remote.c: Debug when an event is queued for dispatch
* src/remote/remote_driver.c: Debug when an event is received
  for processing

Done by editing the first three files, then running
'make -C src rpcgen', then editing src/remote_protocol-structs
to match.

* daemon/remote.c (remoteDispatchGetSysinfo): New function.
* src/remote/remote_driver.c (remoteGetSysinfo, remote_driver):
Client side serialization.
* src/remote/remote_protocol.x (remote_get_sysinfo_args)
(remote_get_sysinfo_ret): New types.
(REMOTE_PROC_GET_SYSINFO): New enum value.
* daemon/remote_dispatch_args.h: Regenerate.
* daemon/remote_dispatch_prototypes.h: Likewise.
* daemon/remote_dispatch_ret.h: Likewise.
* daemon/remote_dispatch_table.h: Likewise.
* src/remote/remote_protocol.c: Likewise.
* src/remote/remote_protocol.h: Likewise.
* src/remote_protocol-structs: Likewise.

Pure cosmetic change.

Fixes test failure that was overlooked after commit 1e1f7a89.

* daemon/Makefile.am (check-local): Let 'make check' fail on error.
* daemon/test_libvirtd.aug: Move qemu-specific option...
* src/qemu/test_libvirtd_qemu.aug: ...into correct test.
* src/qemu/libvirtd_qemu.aug: Parse new option.

Regression introduced in commit e6b68d74 (Nov 2010).

Prior to that point, handlesAlloc was always a multiple of
EVENT_ALLOC_EXTENT (10), and was an int (so even if the subtraction
had been able to wrap, a negative value would be less than the count
not try to free the handles array).  But after that point,
VIR_RESIZE_N made handlesAlloc grow geometrically (with a pattern of
10, 20, 30, 45 for the handles array) but still freed in multiples of
EVENT_ALLOC_EXTENT; and the count changed to size_t.  Which means that
after 31 handles have been created, then 30 handles destroyed,
handlesAlloc is 5 while handlesCount is 1, and since (size_t)(1 - 5)
is indeed greater than 1, this then tried to free 10 elements, which
had the awful effect of nuking the handles array while there were
still live handles.

Nuking live handles puts libvirtd in an inconsistent state, and was
easily reproducible by starting and then stopping 60 faqemu guests.

* daemon/event.c (virEventCleanupTimeouts, virEventCleanupHandles):
Avoid integer wrap-around causing us to delete the entire array
while entries are still active.
* tests/eventtest.c (mymain): Expose the bug.

Add a hook to the error reporting APIs to allow specific
error messages to be filtered out. Wire up libvirtd to
remove VIR_ERR_NO_DOMAIN & similar error codes from the
logs. They are still logged at DEBUG level.

* daemon/libvirtd.c: Filter VIR_ERR_NO_DOMAIN and friends
* src/libvirt_private.syms, src/util/virterror.c,
  src/util/virterror_internal.h: Hook for changing error
  reporting level

This reverts the additions in commit

  abff683f

taking us back to state where all errors are fully logged
in both libvirtd and normal clients.

THe intent was to stop VIR_ERR_NO_DOMAIN (No such domain
with UUID XXXX) messages from client apps polluting syslog
The change affected all error codes, but more seriously,
it also impacted errors from internal libvirtd infrastructure
For example guest autostart no longer logged errors. The
libvirtd network code no longer logged some errors. This
makes debugging incredibly hard

* daemon/libvirtd.c: Remove error log priority filter
* src/util/virterror.c, src/util/virterror_internal.h: Remove
  callback for overriding log priority

This bug has been present since before the time that commit
f8a519 (Dec 2008) tried to make the dispatch loop re-entrant.

Dereferencing eventLoop.handles outside the lock risks crashing, since
any other thread could have reallocated the array in the meantime.
It's a narrow race window, however, and one that would have most
likely resulted in passing bogus data to the callback rather than
actually causing a segv, which is probably why it has gone undetected
this long.

* daemon/event.c (virEventDispatchHandles): Cache data while
inside the lock, as the array might be reallocated once outside.

- Add augeas tests
- Clarify vnc_auto_unix_socket precedence in qemu.conf

Setting unix_sock_group to something else than default "root" in
/etc/libvirt/libvirtd.conf prevents system libvirtd from dumping core on
crash. This is because we used setgid(unix_sock_group) before binding to
/var/run/libvirt/libvirt-sock* and setgid() back to original group.
However, if a process changes its effective or filesystem group ID, it
will be forbidden from leaving core dumps unless fs.suid_dumpable sysctl
is set to something else then 0 (and it is 0 by default).

Changing socket's group ownership after bind works better. And we can do
so without introducing a race condition since we loosen access rights by
changing the group from root to something else.

* daemon/Makefile.am: Avoid spurious space before tabs.
* src/Makefile.am: Likewise.
* examples/dominfo/Makefile.am: Likewise.
* examples/domsuspend/Makefile.am: Likewise.
* tools/Makefile.am: Likewise.
* src/datatypes.h (VIR_CONNECT_MAGIC): Likewise.
* src/internal.h (TODO): Likewise.
* src/qemu/qemu_monitor.h (QEMU_MONITOR_MIGRATE): Likewise.
* src/xen/xen_hypervisor.c (XEN_V2_OP_GETAVAILHEAP): Likewise.
* src/xen/xs_internal.h: Likewise.

We were doing so for child processes but not for libvirtd itself.

* daemon/libvirtd.c (qemudStartWorker, qemudStartEventLoop): Avoid
leaking pthread_attr resources.
* src/util/threads-pthread.c (virThreadCreate): Likewise.