qemu driver in libvirt gained support for creating domain snapshots
almost a year ago in libvirt 0.8.0. Since then we enabled QMP support
for qemu >= 0.13.0 but QMP equivalents of {save,load,del}vm commands are
not implemented in current qemu (0.14.0) so the domain snapshot support
is not very useful.

This patch detects when the appropriate QMP command is not implemented
and tries to use human-monitor-command (aka HMP passthrough) to run
it's HMP equivalent.

To make it more obvious that it is only used for text monitor. The
naming also matches the style of qemuMonitorTextCommandWithFd.

So that it's obvious that they are supposed to be used with HMP commands.

JSON monitor command implementation can now just directly call text
monitor implementation and it will be automatically encapsulated into
QMP's human-monitor-command.

Some qemu monitor event handlers were issuing inadequate warning when
virDomainSaveStatus() failed. They copied the message from I/O error
handler without customizing it to provide better information on why
virDomainSaveStatus() was called.

For newer qemu-img, the help string for "backing file format" is
"[-F backing_fmt]".

Fix the wrong logic error by commit e997c268.

* src/storage/storage_backend.c

qemu-img silently disable "-e", so we can't use it for volume
encryption anymore, change it into "-o encryption=on" if qemu
supports "-o" option.

* src/qemu/qemu_audit.h (qemuAuditCgroupMajor)
(qemuAuditCgroupPath): Add parameter.
* src/qemu/qemu_audit.c (qemuAuditCgroupMajor)
(qemuAuditCgroupPath): Add 'acl=rwm' to cgroup audit entries.
* src/qemu/qemu_cgroup.c: Update clients.
* src/qemu/qemu_driver.c (qemudDomainSaveFlag): Likewise.

Adding audit points showed that we were granting too much privilege
to qemu; it should not need any mknod rights to recreate any
devices.  On the other hand, lxc should have all device privileges.
The solution is adding a flag parameter.

This also lets us restrict write access to read-only disks.

* src/util/cgroup.h (virCgroup*Device*): Adjust prototypes.
* src/util/cgroup.c (virCgroupAllowDevice)
(virCgroupAllowDeviceMajor, virCgroupAllowDevicePath)
(virCgroupDenyDevice, virCgroupDenyDeviceMajor)
(virCgroupDenyDevicePath): Add parameter.
* src/qemu/qemu_driver.c (qemudDomainSaveFlag): Update clients.
* src/lxc/lxc_controller.c (lxcSetContainerResources): Likewise.
* src/qemu/qemu_cgroup.c: Likewise.
(qemuSetupDiskPathAllow): Also, honor read-only disks.

Also add ATTRIBUTE_NONNULL markers.

* src/qemu/qemu_audit.h: The pattern qemuDomainXXXAudit is
inconsistent; prefer qemuAuditXXX instead.
* src/qemu/qemu_audit.c: Reflect the renames.
* src/qemu/qemu_driver.c: Likewise.
* src/qemu/qemu_hotplug.c: Likewise.
* src/qemu/qemu_migration.c: Likewise.
* src/qemu/qemu_process.c: Likewise.

Although the cgroup device ACL controller path can be worked out
by researching the code, it is more efficient to include that
information directly in the audit message.

* src/util/cgroup.h (virCgroupPathOfController): New prototype.
* src/util/cgroup.c (virCgroupPathOfController): Export.
* src/libvirt_private.syms: Likewise.
* src/qemu/qemu_audit.c (qemuAuditCgroup): Use it.

Device names can be manipulated, so it is better to also log
the major/minor device number corresponding to the cgroup ACL
changes that libvirt made.  This required some refactoring
of the relatively new qemu cgroup audit code.

Also, qemuSetupChardevCgroup was only auditing on failure, not success.

* src/qemu/qemu_audit.h (qemuDomainCgroupAudit): Delete.
(qemuAuditCgroup, qemuAuditCgroupMajor, qemuAuditCgroupPath): New
prototypes.
* src/qemu/qemu_audit.c (qemuDomainCgroupAudit): Rename...
(qemuAuditCgroup): ...and drop a parameter.
(qemuAuditCgroupMajor, qemuAuditCgroupPath): New functions, to
allow listing device major/minor in audit.
(qemuAuditGetRdev): New helper function.
* src/qemu/qemu_driver.c (qemudDomainSaveFlag): Adjust callers.
* src/qemu/qemu_cgroup.c (qemuSetupDiskPathAllow)
(qemuSetupHostUsbDeviceCgroup, qemuSetupCgroup)
(qemuTeardownDiskPathDeny): Likewise.
(qemuSetupChardevCgroup): Likewise, fixing missing audit.

* src/qemu/qemu_audit.c (qemuDomainHostdevAudit): Avoid use of
"type", which has a pre-defined meaning.
(qemuDomainCgroupAudit): Likewise, as well as "item".

I noticed these while testing 'make dist'.

Parsing ./../src/util/event.c
Function comment for virEventRegisterDefaultImpl lacks description of return value
Function comment for virEventRunDefaultImpl lacks description of return value
Parsing ./../src/util/virterror.c
Missing comment for function virSetErrorLogPriorityFunc

* src/util/event.c (virEventRegisterDefaultImpl)
(virEventRunDefaultImpl): Document return types.
* src/util/virterror.c (virSetErrorLogPriorityFunc): Provide docs.

* docs/formatdomain.html.in: Document virtio backend selection.

otherwise the user might not have enough permissions to access the
socket if root's umask is 077.

http://bugs.debian.org/614210

virRun gives pretty useful error output, let's not overwrite it unless there
is a good reason. Some places were providing more information about what
the commands were _attempting_ to do, however that's usually less useful from
a debugging POV than what actually happened.

as described at
http://wiki.debian.org/ToolChain/DSOLinking
https://fedoraproject.org/wiki/UnderstandingDSOLinkChange

otherwise the build fails on current Debian unstable with:

CCLD   libvirtd
/usr/bin/ld: ../src/.libs/libvirt_driver_lxc.a(libvirt_driver_lxc_la-lxc_container.o): undefined reference to symbol 'capng_apply'
/usr/bin/ld: note: 'capng_apply' is defined in DSO //usr/lib/libcap-ng.so.0 so try adding it to the linker command line

CCLD   libvirtd
/usr/bin/ld: ../src/.libs/libvirt_driver_storage.a(libvirt_driver_storage_la-storage_backend.o): undefined reference to symbol 'fgetfilecon'
/usr/bin/ld: note: 'fgetfilecon' is defined in DSO //lib/libselinux.so.1 so try adding it to the linker command line
//lib/libselinux.so.1: could not read symbols: Invalid operation

and similar errors.

* src/qemu/qemu_driver.c: qemuDomainUpdateDeviceFlags() is not disk
  specific as the message suggests

On cygwin:

  CC       libvirt_driver_security_la-security_dac.lo
security/security_dac.c: In function 'virSecurityDACSetProcessLabel':
security/security_dac.c:618: warning: format '%d' expects type 'int', but argument 7 has type 'uid_t' [-Wformat]

We've done this before (see src/util/util.c).

* src/security/security_dac.c (virSecurityDACSetProcessLabel): On
cygwin, uid_t is a 32-bit long.

On cygwin:

  CC        libvirt_util_la-cgroup.lo
util/cgroup.c: In function 'virCgroupKillRecursiveInternal':
util/cgroup.c:1458: warning: implicit declaration of function 'virCgroupNew' [-Wimplicit-function-declaration]

* src/util/cgroup.c (virCgroupKill): Don't build on platforms
where virCgroupNew is unsupported.

When building libvirt without libvirtd, I receive the following errors:
make[1]: Leaving directory `/home/wency/source/test/libvirt/src'
 (cd daemon && make  top_distdir=../libvirt-0.8.8 distdir=../libvirt-0.8.8/daemon \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[1]: Entering directory `/home/wency/source/test/libvirt/daemon'
make[1]: *** No rule to make target `libvirtd.8.in', needed by `distdir'.  Stop.

This bug was caused by commit 6db98a2d.
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

* docs/formatdomain.html.in: Fix typo.

as described at
http://wiki.debian.org/ToolChain/DSOLinking
https://fedoraproject.org/wiki/UnderstandingDSOLinkChange

otherwise the build fails on current Debian unstable with:

CCLD   virsh
/usr/bin/ld: virsh-virsh.o: undefined reference to symbol 'xmlSaveTree@@LIBXML2_2.6.8'
/usr/bin/ld: note: 'xmlSaveTree@@LIBXML2_2.6.8' is defined in DSO //usr/lib/libxml2.so.2 so try adding it to the linker command line
//usr/lib/libxml2.so.2: could not read symbols: Invalid operation

This is needed to detect situations when optional argument was
specified with non-integer value: '--int-opt foo'. To keep functions
uniform vshCommandOptString function was also changed, because it
returns tri-state value as well. Given result pointer is updated only
in case of success. If parsing fails, result is not updated at all.

This function should return pointer to const, because we don't want
to change command option value. Therefore we can ensure const-correctness.

Apparently some signals found on Unix are not exposed, this led
to a compilation failure
* src/util/logging.c: make code related to each signal dependant
  upon the definition of that signal

Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>

The way to detach a USB disk is the same as that to detach a SCSI
disk. Rename this function and we can use it to detach a USB disk.
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>

Currently it sounds like spice is completely unsupported, which is
confusing.

When I use newest libvirt to save a domain, libvirtd will be deadlock.
Here is the output of gdb:
(gdb) thread 3
[Switching to thread 3 (Thread 0x7f972a1fc710 (LWP 30265))]#0  0x000000351fe0e034 in __lll_lock_wait () from /lib64/libpthread.so.0
(gdb) bt
    at qemu/qemu_driver.c:2074
    ret=0x7f972a1fbbe0) at remote.c:2273
(gdb) thread 7
[Switching to thread 7 (Thread 0x7f9730bcd710 (LWP 30261))]#0  0x000000351fe0e034 in __lll_lock_wait () from /lib64/libpthread.so.0
(gdb) bt
(gdb) p *(virMutexPtr)0x6fdd60
$2 = {lock = {__data = {__lock = 2, __count = 0, __owner = 30261, __nusers = 1, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
    __size = "\002\000\000\000\000\000\000\000\065v\000\000\001", '\000' <repeats 26 times>, __align = 2}}
(gdb) p *(virMutexPtr)0x1a63ac0
$3 = {lock = {__data = {__lock = 2, __count = 0, __owner = 30265, __nusers = 1, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
    __size = "\002\000\000\000\000\000\000\000\071v\000\000\001", '\000' <repeats 26 times>, __align = 2}}
(gdb) info threads
  7 Thread 0x7f9730bcd710 (LWP 30261)  0x000000351fe0e034 in __lll_lock_wait () from /lib64/libpthread.so.0
  6 Thread 0x7f972bfff710 (LWP 30262)  0x000000351fe0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  5 Thread 0x7f972b5fe710 (LWP 30263)  0x000000351fe0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  4 Thread 0x7f972abfd710 (LWP 30264)  0x000000351fe0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
* 3 Thread 0x7f972a1fc710 (LWP 30265)  0x000000351fe0e034 in __lll_lock_wait () from /lib64/libpthread.so.0
  2 Thread 0x7f97297fb710 (LWP 30266)  0x000000351fe0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  1 Thread 0x7f9737aac800 (LWP 30260)  0x000000351fe0803d in pthread_join () from /lib64/libpthread.so.0

The reason is that we will try to lock some object in callback function, and we may call event API with locking the same object.
In the function virEventDispatchHandles(), we unlock eventLoop before calling callback function. I think we should
do the same thing in the function virEventCleanupTimeouts() and virEventCleanupHandles().
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Not all applications have an existing event loop they need
to integrate with. Forcing them to implement the libvirt
event loop integration APIs is an undue burden. This just
exposes our simple poll() based implementation for apps
to use. So instead of calling

   virEventRegister(....callbacks...)

The app would call

   virEventRegisterDefaultImpl()

And then have a thread somewhere calling

    static bool quit = false;
    ....
    while (!quit)
      virEventRunDefaultImpl()

* daemon/libvirtd.c, tools/console.c,
  tools/virsh.c: Convert to public event loop APIs
* include/libvirt/libvirt.h.in, src/libvirt_private.syms: Add
  virEventRegisterDefaultImpl and virEventRunDefaultImpl
* src/util/event.c: Implement virEventRegisterDefaultImpl
  and virEventRunDefaultImpl using poll() event loop
* src/util/event_poll.c: Add full error reporting
* src/util/virterror.c, include/libvirt/virterror.h: Add
  VIR_FROM_EVENTS

The event loop implementation is used by more than just the
daemon, so move it into the shared area.

* daemon/event.c, src/util/event_poll.c: Renamed
* daemon/event.h, src/util/event_poll.h: Renamed
* tools/Makefile.am, tools/console.c, tools/virsh.c: Update
  to use new virEventPoll APIs
* daemon/mdns.c, daemon/mdns.c, daemon/Makefile.am: Update
  to use new virEventPoll APIs

The daemon code calls virEventAddHandleImpl directly instead
of calling the wrapper virEventAddHandle.

* tools/console.c, daemon/libvirtd.c, daemon/mdns.c: Convert to
  use primary event APIs

* src/util/logging.c: fix virLogDumpAllFD() to avoid snprintf, simplify
  the code and provide more useful signal descriptions. Also remove an
  unused variable.

For qemu names the primary vga as "qxl-vga":

  1) if vram is specified for 2nd qxl device:

    -vga qxl -global qxl-vga.vram_size=$SIZE \
    -device qxl,id=video1,vram_size=$SIZE,...

  2) if vram is not specified for 2nd qxl device, (use the default
     set by global):

    -vga qxl -global qxl-vga.vram_size=$SIZE \
    -device qxl,id=video1,...

For qemu names all qxl devices as "qxl":

  1) if vram is specified for 2nd qxl device:

    -vga qxl -global qxl.vram_size=$SIZE \
    -device qxl,id=video1,vram_size=$SIZE ...

  2) if vram is not specified for 2nd qxl device:

    -vga qxl -global qxl-vga.vram_size=$SIZE \
    -device qxl,id=video1,...

"-global" is the only way to define vram_size for the primary qxl
device, regardless of how qemu names it, (It's not good a good
way, as original idea of "-global" is to set a global default for
a driver property, but to specify vram for first qxl device, we
have to use it).

For other qxl devices, as they are represented by "-device", could
specify it directly and seperately for each, and it overrides the
default set by "-global" if specified.

v1 - v2:
  * modify "virDomainVideoDefaultRAM" so that it returns 16M as the
    default vram_size for qxl device.

  * vram_size * 1024 (qemu accepts bytes for vram_size).

  * apply default vram_size for qxl device for which vram_size is
    not specified.

  * modify "graphics-spice" tests (more sensiable vram_size)

  * Add an argument of virDomainDefPtr type for qemuBuildVideoDevStr,
    to use virDomainVideoDefaultRAM in qemuBuildVideoDevStr).

v2 - v3:
  * Modify default video memory size for qxl device from 16M to 24M

  * Update codes to be consistent with changes on qemu_capabilities.*

Signed-off-by: NEric Blake <eblake@redhat.com>