The APIs names for accessing the domain list object are
very inconsistent. Rename them all to have a standard
virDomainObjList prefix.

Currently the virQEMUDriverPtr struct contains an wide variety
of data with varying access needs. Move all the static config
data into a dedicated virQEMUDriverConfigPtr object. The only
locking requirement is to hold the driver lock, while obtaining
an instance of virQEMUDriverConfigPtr. Once a reference is held
on the config object, it can be used completely lockless since
it is immutable.

NB, not all APIs correctly hold the driver lock while getting
a reference to the config object in this patch. This is safe
for now since the config is never updated on the fly. Later
patches will address this fully.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If a compression binary prints something to stderr, currently
it is discarded. However, it can contain useful data from
debugging POV, so we should catch it.

If a decompression binary prints something to stderr, currently
it is discarded. However, it can contain useful data from
debugging POV, so we should catch it.

Commit 34e8f63a introduced support for catching errors from
libvirt iohelper. However, at those times there wasn't such fancy
API as virCommandDoAsyncIO(), so everything has to be implemented
on our own. But since we do have the API now, we can use it and
drop our implementation then.

Currently, if we want to feed stdin, or catch stdout or stderr of a
virCommand we have to use virCommandRun(). When using virCommandRunAsync()
we have to register FD handles by hand. This may lead to code duplication.
Hence, introduce an internal API, which does this automatically within
virCommandRunAsync(). The intended usage looks like this:

    virCommandPtr cmd = virCommandNew*(...);
    char *buf = NULL;

    ...

    virCommandSetOutputBuffer(cmd, &buf);
    virCommandDoAsyncIO(cmd);

    if (virCommandRunAsync(cmd, NULL) < 0)
        goto cleanup;

    ...

    if (virCommandWait(cmd, NULL) < 0)
        goto cleanup;

    /* @buf now contains @cmd's stdout */
    VIR_DEBUG("STDOUT: %s", NULLSTR(buf));

    ...

cleanup:
    VIR_FREE(buf);
    virCommandFree(cmd);

Note, that both stdout and stderr buffers may change until virCommandWait()
returns.

With this change it's easy to spot a forgotten free if a new device
class is added.

libvirt.c calls curl_global_init() if WITH_CURL is defined and thus it
should be linked with libcurl. This fixes link failure in case neither
xenapi nor esx driver is enabled (they are the only users of libcurl).

QEMU is fully capable of handling VDI images and we just refuse to
work with them.  As qemu-img knows and supports this, there should be
no problem with this addition.

This is of course, just basic functionality, without searching for any
backing files, etc.

Some files have the magic shifted to some offset other than 0, so we
have to support that.  I also cleaned up some lines to be more
readable and added missing magic for iso file format.

Commit 4445e16b changed the signature
of esxConnectToHost and esxConnectToVCenter by replacing the esxPrivate
pointer with virConnectPtr. The esxPrivate pointer was then retrieved
again from virConnectPtr's privateData. This resulted in a NULL pointer
dereference, because the privateData pointer was not yet initialized at
the point where esxConnectToHost and esxConnectToVCenter are called.

This was fixed in commit b126715a that
moved the initialization of privateData before the problematic calls.

Simplify the logic by making the call to esxFreePrivate unconditional and
changing esxConnectToHost and esxConnectToVCenter back to take a esxPrivate
pointer directly. This allows to assign esxPrivate to the virConnectPtr's
privateData pointer as one of the last steps in esxOpen making it more
obvious that it is not initialized during the earlier steps of esxOpen.

Commit 6094ad7b (0.9.3 release) promoted several functions from
internal to public, but forgot to fix the documentation generator
to provide details about those functions.

For an example of what this fixes, look at:
file:///path/to/libvirt/docs/html/libvirt-libvirt.html#virEventAddHandle
before and after the patch.

* docs/apibuild.py (ignored_functions): Don't ignore functions
that were turned into official API.
* src/util/virevent.c: Fix comments to pass through parser.

Add support for QEMU -add-fd command line parameter detection.
This intentionally rejects qemu 1.2, where 'add-fd' QMP did
not allow full control of set ids, and where there was no command
line counterpart, but accepts qemu 1.3.
Signed-off-by: NEric Blake <eblake@redhat.com>

Add entry points for calling the qemu 'add-fd' and 'remove-fd'
monitor commands.  There is no entry point for 'query-fdsets';
the assumption is that a developer can use
 virsh qemu-monitor-command domain '{"execute":"query-fdsets"}'
when debugging issues, and that meanwhile, libvirt is responsible
enough to remember what fds it associated with what fdsets.
Likewise, on the 'add-fd' command, it is assumed that libvirt
will always pass a set id, rather than letting qemu autogenerate
the next available id number.

* src/qemu/qemu_monitor.c (qemuMonitorAddFd, qemuMonitorRemoveFd):
New functions.
* src/qemu/qemu_monitor.h (qemuMonitorAddFd, qemuMonitorRemoveFd):
New prototypes.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONAddFd)
(qemuMonitorJSONRemoveFd): New functions.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONAddFd)
(qemuMonitorJSONRemoveFd): New prototypes.

Way back when I started making changes for Coverity messages my first set
were to a bunch of CHECKED_RETURN errors.  In particular virAsprintf() had
a few callers that Coverity noted didn't check their return (although some
did check if the buffer being printed to was NULL or not).

It was suggested at the time as a further patch an ATTRIBUTE_RETURN_CHECK
should be added to virAsprintf(), see:

https://www.redhat.com/archives/libvir-list/2013-January/msg00120.html

This patch does that and fixes a few more instances not found by Coverity
that failed the check.

https://bugzilla.redhat.com/show_bug.cgi?id=905708

Only the first 12 bits should be set in the mask for this range. All
addresses between 172.16.0.0 and 172.31.255.255 are private.

When a disk-only snapshot is requested the domain is treated as if it
was offline. This forbids to mix memory checkpoints with the DISK_ONLY
flag.

This patch improves the error message and mentions the restriction in
the virsh man page.

Adding dots inside "exempli gratia" where missing.  While on that, I
took the liberty of changing it where found with simple grep.

Commit 60b176c3 introduced a bug that
when editing an XML with cputune similar to this:

...
  <vcpu placement='static' current='1'>2</vcpu>
  <cputune>
    <vcpupin vcpu="1" cpuset="0"/>
  </cputune>
...

results in formatted XML that looks like this:

...
  <vcpu placement='static' current='1'>2</vcpu>
  <cputune>
  </cputune>
...

That is caused by a condition depending on def->cputune.vcpupin being
set rather than checking def->cputune.nvcpupin.  Notice that nvcpupin
can be 0 and vcpupin can still be allocated since it's a pointer to an
array, so no harm done there.

I also changed it on other places in the code where it depended on the
wrong variable.

Setting the log output prefix to 0 is not supported and in fact results
in the following message:
warning : virLogParseOutputs:1021 : Ignoring invalid log output setting.

def->cpumask is a bitmap and needs to be freed by virBitmapFree.

https://bugzilla.redhat.com/show_bug.cgi?id=894723

Currently, if qemuProcessStart() succeeds, but it's decompression
binary that returns nonzero status, we don't kill the qemu process,
but remove it from internal domain list, leaving the qemu process
hanging around totally uncontrolled.

This patch resolves CVE-2013-0170:
https://bugzilla.redhat.com/show_bug.cgi?id=893450

When reading and dispatching of a message failed the message was freed
but wasn't removed from the message queue.

After that when the connection was about to be closed the pointer for
the message was still present in the queue and it was passed to
virNetMessageFree which tried to call the callback function from an
uninitialized pointer.

This patch removes the message from the queue before it's freed.

* rpc/virnetserverclient.c: virNetServerClientDispatchRead:
    - avoid use after free of RPC messages

https://bugzilla.redhat.com/show_bug.cgi?id=892289

It seems like with new udev within guest OS, the tray is locked,
so we need to:
- 'eject'
- wait for tray to open
- 'change'

Moreover, even when doing bare 'eject', we should check for
'tray_open' as guest may have locked the tray. However, the
waiting phase shouldn't be unbounded, so I've chosen 10 retries
maximum, each per 500ms. This should give enough time for guest
to eject a media and open the tray.

Commit f0aa4935 missed a file.

* src/libvirt_driver_modules.syms: Add editor hint.

Adjust the macros to free memory allocated during various calls to
perform the check if parameter is NULL prior to really freeing and to
set the pointer to NULL after done freeing.

Resolve a false positive from 'vboxIIDFromUUID_v2_x()'. The code sets
'iid->value = &iid->backing' unconditionally prior to calling 'nsIDFromChar()'.
The 'vboxIIDUnalloc_v2_x()' checks iid->value to not be &iid->backing. The
iid->backing is a static buffer within the initialized structure.

Since libxl provides the domain ID in the event handler callback,
find the domain object based on the ID.  This approach prevents
processing the callback on a domain that has already been reaped.

Also, similar to the xl implementation, ignore the SUSPEND shutdown
reason.  By calling libxl_domain_suspend(), we know a shutdown
event with SUSPEND reason will be generated, but it can be safely
ignored since any subsequent cleanup will be done by the callers.

libxlDoDomainSave() was removing non-persistent domains, but
required callers to have the virDomainObj locked.  Callers could
potentially unlock an already freed virDomainObj.  Move this
logic to the callers of libxlDoDomainSave().

I've noticed that libxl can invoke timeout reregister/modify hooks
after returning from libxl_ctx_free.  Explicitly remove the
timeouts before freeing the libxl ctx to avoid executing hooks on
stale objects.

It is possible to destroy and cleanup a VM, resulting in freeing the
libxlDomainObjPrivate object and associated libxl ctx, before all fds and
timeouts have been deregistered and destroyed.

Fix this race by incrementing the reference count on libxlDomainObjPrivate
for each fd and timeout registration.  Only when all fds and timeouts are
deregistered and destroyed will the libxlDomainObjPrivate be destroyed.

The libxl driver is racy in it's interactions with libxl and libvirt's
event loop.  The event loop can invoke callbacks after libxl has
deregistered the event, and possibly access freed data associated with
the event.

This patch fixes the race by converting libxlDomainObjPrivate to a
virObjectLockable, and locking it while executing libxl upcalls and
libvirt event loop callbacks.

Note that using the virDomainObj lock is not satisfactory since it may
be desirable to hold the virDomainObj lock even when libxl events such
as reading and writing to xenstore need processed.

xen-unstable changeset 26469 makes changes wrt modifying and deregistering
timeouts.

First, timeout modify callbacks will only be invoked with an
abs_t of {0,0}, i.e. make the timeout fire immediately.  Prior to this
commit, timeout modify callbacks were never invoked.

Second, timeout deregister hooks will no longer be called.

This patch makes changes in the libvirt libxl driver that should be
compatible before and after changeset 26469.

While at it, fix a potential overflow in the timeout register callback.

While working with a pmsuspend vs. snapshot issue, I noticed that
the state file in /var/run/libvirt/qemu/dom.xml contained a rather
suspicious "(null)" string, which does not round-trip well through
a libvirtd restart.  Had I been on a platform other than glibc
where printf("%s",NULL) crashes instead of printing (null), we might
have noticed the problem much sooner.

And in fixing that problem, I also noticed that we had several
missing states, because we were #defining several *_LAST names
to a value _different_ than what they were already given as enums
in libvirt.h.  Yuck.  I got rid of default: labels in the case
statements, because they get in the way of gcc's -Wswitch helping
us ensure we cover all enum values.

* src/conf/domain_conf.c (virDomainStateReasonToString)
(virDomainStateReasonFromString): Fill in missing domain states;
rewrite case statement to let compiler enforce checking.
(VIR_DOMAIN_NOSTATE_LAST, VIR_DOMAIN_RUNNING_LAST)
(VIR_DOMAIN_BLOCKED_LAST, VIR_DOMAIN_PAUSED_LAST)
(VIR_DOMAIN_SHUTDOWN_LAST, VIR_DOMAIN_SHUTOFF_LAST)
(VIR_DOMAIN_CRASHED_LAST): Drop dead defines.
(VIR_DOMAIN_PMSUSPENDED_LAST): Drop dead define.
(virDomainPMSuspendedReason): Add missing enum function.
(virDomainRunningReason, virDomainPausedReason): Add missing enum
value.
* src/conf/domain_conf.h (virDomainPMSuspendedReason): Declare
missing functions.
* src/libvirt_private.syms (domain_conf.h): Export them.

I got bit by 'make check' complaining that the sort order I got
by emacs' sort-lines function differed from expectations.

* src/libvirt_private.syms: Add emacs trailer.
* src/libvirt_atomic.syms: Likewise.
* src/libvirt_daemon.syms: Likewise.
* src/libvirt_esx.syms: Likewise.
* src/libvirt_libssh2.syms: Likewise.
* src/libvirt_linux.syms: Likewise.
* src/libvirt_openvz.syms: Likewise.
* src/libvirt_sasl.syms: Likewise.
* src/libvirt_vmx.syms: Likewise.
* src/libvirt_xenxs.syms: Likewise.

With our code, we fail to query for tray-open attribute currently.
That's because in HMP it is 'tray-open' and in QMP it's 'tray_open'.
It always has been. However, we got it exactly the opposite.

A logic bug meant we reported KVM was possible for every
architecture, merely based on whether the query-kvm command
exists. We should instead have been doing it based on whether
the query-kvm command returns 'present: 1'
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently QEMU capabilities are initialized before the QEMU driver
sets ownership on its various directories. The upshot is that if
you change the user/group in the qemu.conf file, libvirtd will fail
to probe QEMU the first time it is run after the config change.
Moving QEMU capabilities initialization to after the chown() calls
fixes this
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This previous commit

  commit 1a50ba2c
  Author: Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
  Date:   Mon Nov 26 15:17:13 2012 +0100

    qemu: Fix QMP Capabability Probing Failure

which attempted to make sure the QEMU process used for probing
ran as the right user id, caused serious performance regression
and unreliability in probing. The -daemonize switch in QEMU
guarantees that the monitor socket is present before the parent
process exits. This means libvirtd is guaranteed to be able to
connect immediately. By switching from -daemonize to the
virCommandDaemonize API libvirtd was no longer synchronized with
QEMU's startup process. The result was that the QEMU monitor
failed to open and went into its 200ms sleep loop. This happened
for all 25 binaries resulting in 5 seconds worth of sleeping
at libvirtd startup. In addition sometimes when libvirt connected,
QEMU would be partially initialized and crash causing total
failure to probe that binary.

This commit reverts the previous change, ensuring we do use the
-daemonize flag to QEMU. Startup delay is cut from 7 seconds
to 2 seconds on my machine, which is on a par with what it was
prior to the capabilities rewrite.

To deal with the fact that QEMU needs to be able to create the
pidfile, we switch pidfile location fron runDir to libDir, which
QEMU is guaranteed to be able to write to.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>