virRun gives pretty useful error output, let's not overwrite it unless there
is a good reason. Some places were providing more information about what
the commands were _attempting_ to do, however that's usually less useful from
a debugging POV than what actually happened.

as described at
http://wiki.debian.org/ToolChain/DSOLinking
https://fedoraproject.org/wiki/UnderstandingDSOLinkChange

otherwise the build fails on current Debian unstable with:

CCLD   libvirtd
/usr/bin/ld: ../src/.libs/libvirt_driver_lxc.a(libvirt_driver_lxc_la-lxc_container.o): undefined reference to symbol 'capng_apply'
/usr/bin/ld: note: 'capng_apply' is defined in DSO //usr/lib/libcap-ng.so.0 so try adding it to the linker command line

CCLD   libvirtd
/usr/bin/ld: ../src/.libs/libvirt_driver_storage.a(libvirt_driver_storage_la-storage_backend.o): undefined reference to symbol 'fgetfilecon'
/usr/bin/ld: note: 'fgetfilecon' is defined in DSO //lib/libselinux.so.1 so try adding it to the linker command line
//lib/libselinux.so.1: could not read symbols: Invalid operation

and similar errors.

* src/qemu/qemu_driver.c: qemuDomainUpdateDeviceFlags() is not disk
  specific as the message suggests

On cygwin:

  CC       libvirt_driver_security_la-security_dac.lo
security/security_dac.c: In function 'virSecurityDACSetProcessLabel':
security/security_dac.c:618: warning: format '%d' expects type 'int', but argument 7 has type 'uid_t' [-Wformat]

We've done this before (see src/util/util.c).

* src/security/security_dac.c (virSecurityDACSetProcessLabel): On
cygwin, uid_t is a 32-bit long.

On cygwin:

  CC        libvirt_util_la-cgroup.lo
util/cgroup.c: In function 'virCgroupKillRecursiveInternal':
util/cgroup.c:1458: warning: implicit declaration of function 'virCgroupNew' [-Wimplicit-function-declaration]

* src/util/cgroup.c (virCgroupKill): Don't build on platforms
where virCgroupNew is unsupported.

When building libvirt without libvirtd, I receive the following errors:
make[1]: Leaving directory `/home/wency/source/test/libvirt/src'
 (cd daemon && make  top_distdir=../libvirt-0.8.8 distdir=../libvirt-0.8.8/daemon \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[1]: Entering directory `/home/wency/source/test/libvirt/daemon'
make[1]: *** No rule to make target `libvirtd.8.in', needed by `distdir'.  Stop.

This bug was caused by commit 6db98a2d.
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

* docs/formatdomain.html.in: Fix typo.

as described at
http://wiki.debian.org/ToolChain/DSOLinking
https://fedoraproject.org/wiki/UnderstandingDSOLinkChange

otherwise the build fails on current Debian unstable with:

CCLD   virsh
/usr/bin/ld: virsh-virsh.o: undefined reference to symbol 'xmlSaveTree@@LIBXML2_2.6.8'
/usr/bin/ld: note: 'xmlSaveTree@@LIBXML2_2.6.8' is defined in DSO //usr/lib/libxml2.so.2 so try adding it to the linker command line
//usr/lib/libxml2.so.2: could not read symbols: Invalid operation

This is needed to detect situations when optional argument was
specified with non-integer value: '--int-opt foo'. To keep functions
uniform vshCommandOptString function was also changed, because it
returns tri-state value as well. Given result pointer is updated only
in case of success. If parsing fails, result is not updated at all.

This function should return pointer to const, because we don't want
to change command option value. Therefore we can ensure const-correctness.

Apparently some signals found on Unix are not exposed, this led
to a compilation failure
* src/util/logging.c: make code related to each signal dependant
  upon the definition of that signal

Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>

The way to detach a USB disk is the same as that to detach a SCSI
disk. Rename this function and we can use it to detach a USB disk.
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>

Currently it sounds like spice is completely unsupported, which is
confusing.

When I use newest libvirt to save a domain, libvirtd will be deadlock.
Here is the output of gdb:
(gdb) thread 3
[Switching to thread 3 (Thread 0x7f972a1fc710 (LWP 30265))]#0  0x000000351fe0e034 in __lll_lock_wait () from /lib64/libpthread.so.0
(gdb) bt
    at qemu/qemu_driver.c:2074
    ret=0x7f972a1fbbe0) at remote.c:2273
(gdb) thread 7
[Switching to thread 7 (Thread 0x7f9730bcd710 (LWP 30261))]#0  0x000000351fe0e034 in __lll_lock_wait () from /lib64/libpthread.so.0
(gdb) bt
(gdb) p *(virMutexPtr)0x6fdd60
$2 = {lock = {__data = {__lock = 2, __count = 0, __owner = 30261, __nusers = 1, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
    __size = "\002\000\000\000\000\000\000\000\065v\000\000\001", '\000' <repeats 26 times>, __align = 2}}
(gdb) p *(virMutexPtr)0x1a63ac0
$3 = {lock = {__data = {__lock = 2, __count = 0, __owner = 30265, __nusers = 1, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
    __size = "\002\000\000\000\000\000\000\000\071v\000\000\001", '\000' <repeats 26 times>, __align = 2}}
(gdb) info threads
  7 Thread 0x7f9730bcd710 (LWP 30261)  0x000000351fe0e034 in __lll_lock_wait () from /lib64/libpthread.so.0
  6 Thread 0x7f972bfff710 (LWP 30262)  0x000000351fe0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  5 Thread 0x7f972b5fe710 (LWP 30263)  0x000000351fe0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  4 Thread 0x7f972abfd710 (LWP 30264)  0x000000351fe0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
* 3 Thread 0x7f972a1fc710 (LWP 30265)  0x000000351fe0e034 in __lll_lock_wait () from /lib64/libpthread.so.0
  2 Thread 0x7f97297fb710 (LWP 30266)  0x000000351fe0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  1 Thread 0x7f9737aac800 (LWP 30260)  0x000000351fe0803d in pthread_join () from /lib64/libpthread.so.0

The reason is that we will try to lock some object in callback function, and we may call event API with locking the same object.
In the function virEventDispatchHandles(), we unlock eventLoop before calling callback function. I think we should
do the same thing in the function virEventCleanupTimeouts() and virEventCleanupHandles().
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Not all applications have an existing event loop they need
to integrate with. Forcing them to implement the libvirt
event loop integration APIs is an undue burden. This just
exposes our simple poll() based implementation for apps
to use. So instead of calling

   virEventRegister(....callbacks...)

The app would call

   virEventRegisterDefaultImpl()

And then have a thread somewhere calling

    static bool quit = false;
    ....
    while (!quit)
      virEventRunDefaultImpl()

* daemon/libvirtd.c, tools/console.c,
  tools/virsh.c: Convert to public event loop APIs
* include/libvirt/libvirt.h.in, src/libvirt_private.syms: Add
  virEventRegisterDefaultImpl and virEventRunDefaultImpl
* src/util/event.c: Implement virEventRegisterDefaultImpl
  and virEventRunDefaultImpl using poll() event loop
* src/util/event_poll.c: Add full error reporting
* src/util/virterror.c, include/libvirt/virterror.h: Add
  VIR_FROM_EVENTS

The event loop implementation is used by more than just the
daemon, so move it into the shared area.

* daemon/event.c, src/util/event_poll.c: Renamed
* daemon/event.h, src/util/event_poll.h: Renamed
* tools/Makefile.am, tools/console.c, tools/virsh.c: Update
  to use new virEventPoll APIs
* daemon/mdns.c, daemon/mdns.c, daemon/Makefile.am: Update
  to use new virEventPoll APIs

The daemon code calls virEventAddHandleImpl directly instead
of calling the wrapper virEventAddHandle.

* tools/console.c, daemon/libvirtd.c, daemon/mdns.c: Convert to
  use primary event APIs

* src/util/logging.c: fix virLogDumpAllFD() to avoid snprintf, simplify
  the code and provide more useful signal descriptions. Also remove an
  unused variable.

For qemu names the primary vga as "qxl-vga":

  1) if vram is specified for 2nd qxl device:

    -vga qxl -global qxl-vga.vram_size=$SIZE \
    -device qxl,id=video1,vram_size=$SIZE,...

  2) if vram is not specified for 2nd qxl device, (use the default
     set by global):

    -vga qxl -global qxl-vga.vram_size=$SIZE \
    -device qxl,id=video1,...

For qemu names all qxl devices as "qxl":

  1) if vram is specified for 2nd qxl device:

    -vga qxl -global qxl.vram_size=$SIZE \
    -device qxl,id=video1,vram_size=$SIZE ...

  2) if vram is not specified for 2nd qxl device:

    -vga qxl -global qxl-vga.vram_size=$SIZE \
    -device qxl,id=video1,...

"-global" is the only way to define vram_size for the primary qxl
device, regardless of how qemu names it, (It's not good a good
way, as original idea of "-global" is to set a global default for
a driver property, but to specify vram for first qxl device, we
have to use it).

For other qxl devices, as they are represented by "-device", could
specify it directly and seperately for each, and it overrides the
default set by "-global" if specified.

v1 - v2:
  * modify "virDomainVideoDefaultRAM" so that it returns 16M as the
    default vram_size for qxl device.

  * vram_size * 1024 (qemu accepts bytes for vram_size).

  * apply default vram_size for qxl device for which vram_size is
    not specified.

  * modify "graphics-spice" tests (more sensiable vram_size)

  * Add an argument of virDomainDefPtr type for qemuBuildVideoDevStr,
    to use virDomainVideoDefaultRAM in qemuBuildVideoDevStr).

v2 - v3:
  * Modify default video memory size for qxl device from 16M to 24M

  * Update codes to be consistent with changes on qemu_capabilities.*

Signed-off-by: NEric Blake <eblake@redhat.com>

In case of imminent crash or upon request (signal USR2),
dump the logging buffer to the libvirtd.log file for
post-mortem analysis
* daemon/libvirtd.c: create a sig_fatal() handler connected to
  SIGFPE SIGSEGV SIGILL SIGABRT SIGBUS and SIGUSR2, just dumping
  the log buffer using virLogEmergencyDumpAll

virLogEmergencyDumpAll() allows to dump the content of the
debug buffer from within a signal handler. It saves to all
log file or stderr if none is found
* src/util/logging.h src/util/logging.c: add the new API
  and cleanup the old virLogDump code
* src/libvirt_private.syms: exports it as a private symbol

* src/util/logging.c: the start pointer need to wrap around too

As the file may grow quite a bit especially with debug turned on.
* daemon/libvirtd.logrotate.in daemon/Makefile.am libvirt.spec.in:
  add new logrotate file for the daemon log

Syslog is not the best place to go search for libvirt error
logs, change it to a default file output libvirtd.log, but
still keep standard error if not run as a daemon.
Depending on whether it's run as root or user, the log is saved
in the local state dir or in $HOME/.libvirt.
* daemon/libvirtd.c: change default logging to go to libvirtd.log

Initially only the log actually written out by libvirt were
saved on the memory buffer, this patch forces all informations
including info and debug to be saved in memory too. This is
useful to get full data in case of crash.

This was also found while investigating

   https://bugzilla.redhat.com/show_bug.cgi?id=670848

An EOF on a domain's monitor socket results in an event being queued
to handle the EOF. The handler calls qemuProcessHandleMonitorEOF. If
it is a transient domain, this leads to a call to
virDomainRemoveInactive, which removes the domain from the driver's
hashtable and unref's it. Nowhere in this code is the qemu driver lock
acquired.

However, all modifications to the driver's domain hashtable *must* be
done while holding the driver lock, otherwise the hashtable can become
corrupt, and (even more likely) another thread could call a different
hashtable function and acquire a pointer to the domain that is in the
process of being destroyed.

To prevent such a disaster, qemuProcessHandleMonitorEOF must get the
qemu driver lock *before* it gets the DomainObj's lock, and hold it
until it is finished with the DomainObj. This guarantees that nobody
else modifies the hashtable at the same time, and that anyone who had
already gotten the DomainObj from the hashtable prior to this call has
finished with it before we remove/destroy it.

This was found while researching the root cause of:

https://bugzilla.redhat.com/show_bug.cgi?id=670848

virDomainUnref should only be called with the lock held for the
virDomainObj in question. However, when a transient qemu domain gets
EOF on its monitor socket, it queues an event which frees the monitor,
which unref's the virDomainObj without first locking it. If another
thread has already locked the virDomainObj, the modification of the
refcount could potentially be corrupted. In an extreme case, it could
also be potentially unlocked by virDomainObjFree, thus left open to
modification by anyone else who would have otherwise waited for the
lock (not to mention the fact that they would be accessing freed
data!).

The solution is to have qemuMonitorFree lock the domain object right
before unrefing it. Since the caller to qemuMonitorFree doesn't expect
this lock to be held, if the refcount doesn't go all the way to 0,
qemuMonitorFree must unlock it after the unref.

maybe sounds strange but I've used this signature for years.
see http://en.wikipedia.org/wiki/Family_name

Passwords are allowed to contain <, >, &, ', " characters.
Those need to be replaced by the corresponding entities.

Reported by Hereward Cooper.

In virFileOperation, the parent does a fallback to a non-fork
attempt if it detects that the child returned EACCES.  However,
the child was calling _exit(-EACCES), which does _not_ appear
as EACCES in the parent.

* src/util/util.c (virFileOperation): Correctly pass EACCES from
child to parent.

virSecurityDAC{Set,Restore}ChardevCallback expect virSecurityManagerPtr,
but are passed virDomainObjPtr instead. This makes
virSecurityDACSetChardevLabel set a wrong uid/gid on chardevs. This
patch fixes this behaviour.
Signed-off-by: NSoren Hansen <soren@linux2go.dk>

* .gnulib: Update to latest, for syntax-check improvement.
* .x-sc_prohibit_empty_lines_at_EOF: Add more exemptions.

* tests/xml2sexprdata/*.sexpr: Add backslash-newlines.
* tests/sexpr2xmldata/*.sexpr: Likewise.
* tests/qemuxml2argvdata/qemuxml2argv-disk-aio.args: Likewise.

This fixes a possible crash of libvirtd during its startup. When qemu
driver reconnects to running domains, it iterates over all domain
objects in a hash. When reconnecting to an associated qemu monitor
fails and the domain is transient, it's immediately removed from the
hash. Despite the fact that it's explicitly forbidden to do so. If
libvirtd is lucky enough, virHashForEach will access random memory when
the callback finishes and the deamon will crash.

Since it's trivial to fix virHashForEach to allow removal of hash
entries while iterating through them, I went this way instead of fixing
qemuReconnectDomain callback (and possibly others) to avoid deleting the
entries.

Replace the 'Unknown failure' error message with something a
little bit more descriptive.

* src/util/virterror.c: Improve error message

qemudDomainSaveImageStartVM was evil - it closed the incoming fd
argument on some, but not all, code paths, without informing the
caller about that action.  No wonder that this resulted in
double-closes: https://bugzilla.redhat.com/show_bug.cgi?id=672725

* src/qemu/qemu_driver.c (qemudDomainSaveImageStartVM): Alter
signature, to avoid double-close.
(qemudDomainRestore, qemudDomainObjRestore): Update callers.

Followup to commit 2222bd24