When iterating over USB host devices to setup cgroups, the
usbDevice object was leaked in both LXC and QEMU driers
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Rename all the pciDeviceXXX and pciXXXDevice APIs to have a
fixed virPCIDevice name prefix

The QEMU driver struct has a 'qemuVersion' field that was previously
used to cache the version lookup from capabilities. With the recent
QEMU capabilities rewrite the caching happens at a lower level so
this field is pointless. Removing it avoids worries about locking
when updating it.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use atomic ops to increment nextvmid and encapsulate it in a
method to prevent accidental non-atomic access

Switch virDomainObjList to inherit from virObjectLockable and
make all the APIs acquire/release the mutex when running. This
makes virDomainObjList completely self-locking and no longer
reliant on the hypervisor driver locks

The duplicate VM checking should be done atomically with
virDomainObjListAdd, so shoud not be a separate function.
Instead just use flags to indicate what kind of checks are
required.

This pair, used in virDomainCreateXML:

   if (virDomainObjListIsDuplicate(privconn->domains, def, 1) < 0)
     goto cleanup;
   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def, false)))
     goto cleanup;

Changes to

   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def,
                                   VIR_DOMAIN_OBJ_LIST_ADD_CHECK_LIVE,
                                   NULL)))
     goto cleanup;

This pair, used in virDomainRestoreFlags:

   if (virDomainObjListIsDuplicate(privconn->domains, def, 1) < 0)
     goto cleanup;
   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def, true)))
     goto cleanup;

Changes to

   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def,
                                   VIR_DOMAIN_OBJ_LIST_ADD_LIVE |
                                   VIR_DOMAIN_OBJ_LIST_ADD_CHECK_LIVE,
                                   NULL)))
     goto cleanup;

This pair, used in virDomainDefineXML:

   if (virDomainObjListIsDuplicate(privconn->domains, def, 0) < 0)
     goto cleanup;
   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def, false)))
     goto cleanup;

Changes to

   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def,
                                   0, NULL)))
     goto cleanup;

Otherwise, we get a lot of scary (but harmless) noise in the logs:

2013-02-05 15:35:48.555+0000: 8637: error : qemuMonitorJSONCheckError:353 : internal error unable to execute QEMU command 'add-fd': Parameter 'fdset-id' expects an existing fdset-id

one for every qemu 1.2 binary that we probe.

* src/qemu/qemu_monitor_json.c (qemuMonitorJSONAddFd): During
probe, avoid logging failures.

As a step towards making virDomainObjList thread-safe turn it
into an opaque virObject, preventing any direct access to its
internals.

As part of this a new method virDomainObjListForEach is
introduced to replace all existing usage of virHashForEach

The APIs names for accessing the domain list object are
very inconsistent. Rename them all to have a standard
virDomainObjList prefix.

Currently the virQEMUDriverPtr struct contains an wide variety
of data with varying access needs. Move all the static config
data into a dedicated virQEMUDriverConfigPtr object. The only
locking requirement is to hold the driver lock, while obtaining
an instance of virQEMUDriverConfigPtr. Once a reference is held
on the config object, it can be used completely lockless since
it is immutable.

NB, not all APIs correctly hold the driver lock while getting
a reference to the config object in this patch. This is safe
for now since the config is never updated on the fly. Later
patches will address this fully.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If a compression binary prints something to stderr, currently
it is discarded. However, it can contain useful data from
debugging POV, so we should catch it.

If a decompression binary prints something to stderr, currently
it is discarded. However, it can contain useful data from
debugging POV, so we should catch it.

Commit 34e8f63a introduced support for catching errors from
libvirt iohelper. However, at those times there wasn't such fancy
API as virCommandDoAsyncIO(), so everything has to be implemented
on our own. But since we do have the API now, we can use it and
drop our implementation then.

Currently, if we want to feed stdin, or catch stdout or stderr of a
virCommand we have to use virCommandRun(). When using virCommandRunAsync()
we have to register FD handles by hand. This may lead to code duplication.
Hence, introduce an internal API, which does this automatically within
virCommandRunAsync(). The intended usage looks like this:

    virCommandPtr cmd = virCommandNew*(...);
    char *buf = NULL;

    ...

    virCommandSetOutputBuffer(cmd, &buf);
    virCommandDoAsyncIO(cmd);

    if (virCommandRunAsync(cmd, NULL) < 0)
        goto cleanup;

    ...

    if (virCommandWait(cmd, NULL) < 0)
        goto cleanup;

    /* @buf now contains @cmd's stdout */
    VIR_DEBUG("STDOUT: %s", NULLSTR(buf));

    ...

cleanup:
    VIR_FREE(buf);
    virCommandFree(cmd);

Note, that both stdout and stderr buffers may change until virCommandWait()
returns.

With this change it's easy to spot a forgotten free if a new device
class is added.

libvirt.c calls curl_global_init() if WITH_CURL is defined and thus it
should be linked with libcurl. This fixes link failure in case neither
xenapi nor esx driver is enabled (they are the only users of libcurl).

QEMU is fully capable of handling VDI images and we just refuse to
work with them.  As qemu-img knows and supports this, there should be
no problem with this addition.

This is of course, just basic functionality, without searching for any
backing files, etc.

Some files have the magic shifted to some offset other than 0, so we
have to support that.  I also cleaned up some lines to be more
readable and added missing magic for iso file format.

Commit 4445e16b changed the signature
of esxConnectToHost and esxConnectToVCenter by replacing the esxPrivate
pointer with virConnectPtr. The esxPrivate pointer was then retrieved
again from virConnectPtr's privateData. This resulted in a NULL pointer
dereference, because the privateData pointer was not yet initialized at
the point where esxConnectToHost and esxConnectToVCenter are called.

This was fixed in commit b126715a that
moved the initialization of privateData before the problematic calls.

Simplify the logic by making the call to esxFreePrivate unconditional and
changing esxConnectToHost and esxConnectToVCenter back to take a esxPrivate
pointer directly. This allows to assign esxPrivate to the virConnectPtr's
privateData pointer as one of the last steps in esxOpen making it more
obvious that it is not initialized during the earlier steps of esxOpen.

Commit 6094ad7b (0.9.3 release) promoted several functions from
internal to public, but forgot to fix the documentation generator
to provide details about those functions.

For an example of what this fixes, look at:
file:///path/to/libvirt/docs/html/libvirt-libvirt.html#virEventAddHandle
before and after the patch.

* docs/apibuild.py (ignored_functions): Don't ignore functions
that were turned into official API.
* src/util/virevent.c: Fix comments to pass through parser.

Add support for QEMU -add-fd command line parameter detection.
This intentionally rejects qemu 1.2, where 'add-fd' QMP did
not allow full control of set ids, and where there was no command
line counterpart, but accepts qemu 1.3.
Signed-off-by: NEric Blake <eblake@redhat.com>

Add entry points for calling the qemu 'add-fd' and 'remove-fd'
monitor commands.  There is no entry point for 'query-fdsets';
the assumption is that a developer can use
 virsh qemu-monitor-command domain '{"execute":"query-fdsets"}'
when debugging issues, and that meanwhile, libvirt is responsible
enough to remember what fds it associated with what fdsets.
Likewise, on the 'add-fd' command, it is assumed that libvirt
will always pass a set id, rather than letting qemu autogenerate
the next available id number.

* src/qemu/qemu_monitor.c (qemuMonitorAddFd, qemuMonitorRemoveFd):
New functions.
* src/qemu/qemu_monitor.h (qemuMonitorAddFd, qemuMonitorRemoveFd):
New prototypes.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONAddFd)
(qemuMonitorJSONRemoveFd): New functions.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONAddFd)
(qemuMonitorJSONRemoveFd): New prototypes.

Way back when I started making changes for Coverity messages my first set
were to a bunch of CHECKED_RETURN errors.  In particular virAsprintf() had
a few callers that Coverity noted didn't check their return (although some
did check if the buffer being printed to was NULL or not).

It was suggested at the time as a further patch an ATTRIBUTE_RETURN_CHECK
should be added to virAsprintf(), see:

https://www.redhat.com/archives/libvir-list/2013-January/msg00120.html

This patch does that and fixes a few more instances not found by Coverity
that failed the check.

https://bugzilla.redhat.com/show_bug.cgi?id=905708

Only the first 12 bits should be set in the mask for this range. All
addresses between 172.16.0.0 and 172.31.255.255 are private.

When a disk-only snapshot is requested the domain is treated as if it
was offline. This forbids to mix memory checkpoints with the DISK_ONLY
flag.

This patch improves the error message and mentions the restriction in
the virsh man page.

Adding dots inside "exempli gratia" where missing.  While on that, I
took the liberty of changing it where found with simple grep.

Commit 60b176c3 introduced a bug that
when editing an XML with cputune similar to this:

...
  <vcpu placement='static' current='1'>2</vcpu>
  <cputune>
    <vcpupin vcpu="1" cpuset="0"/>
  </cputune>
...

results in formatted XML that looks like this:

...
  <vcpu placement='static' current='1'>2</vcpu>
  <cputune>
  </cputune>
...

That is caused by a condition depending on def->cputune.vcpupin being
set rather than checking def->cputune.nvcpupin.  Notice that nvcpupin
can be 0 and vcpupin can still be allocated since it's a pointer to an
array, so no harm done there.

I also changed it on other places in the code where it depended on the
wrong variable.

Setting the log output prefix to 0 is not supported and in fact results
in the following message:
warning : virLogParseOutputs:1021 : Ignoring invalid log output setting.

def->cpumask is a bitmap and needs to be freed by virBitmapFree.

https://bugzilla.redhat.com/show_bug.cgi?id=894723

Currently, if qemuProcessStart() succeeds, but it's decompression
binary that returns nonzero status, we don't kill the qemu process,
but remove it from internal domain list, leaving the qemu process
hanging around totally uncontrolled.

This patch resolves CVE-2013-0170:
https://bugzilla.redhat.com/show_bug.cgi?id=893450

When reading and dispatching of a message failed the message was freed
but wasn't removed from the message queue.

After that when the connection was about to be closed the pointer for
the message was still present in the queue and it was passed to
virNetMessageFree which tried to call the callback function from an
uninitialized pointer.

This patch removes the message from the queue before it's freed.

* rpc/virnetserverclient.c: virNetServerClientDispatchRead:
    - avoid use after free of RPC messages

https://bugzilla.redhat.com/show_bug.cgi?id=892289

It seems like with new udev within guest OS, the tray is locked,
so we need to:
- 'eject'
- wait for tray to open
- 'change'

Moreover, even when doing bare 'eject', we should check for
'tray_open' as guest may have locked the tray. However, the
waiting phase shouldn't be unbounded, so I've chosen 10 retries
maximum, each per 500ms. This should give enough time for guest
to eject a media and open the tray.

Commit f0aa4935 missed a file.

* src/libvirt_driver_modules.syms: Add editor hint.

Adjust the macros to free memory allocated during various calls to
perform the check if parameter is NULL prior to really freeing and to
set the pointer to NULL after done freeing.

Resolve a false positive from 'vboxIIDFromUUID_v2_x()'. The code sets
'iid->value = &iid->backing' unconditionally prior to calling 'nsIDFromChar()'.
The 'vboxIIDUnalloc_v2_x()' checks iid->value to not be &iid->backing. The
iid->backing is a static buffer within the initialized structure.

Since libxl provides the domain ID in the event handler callback,
find the domain object based on the ID.  This approach prevents
processing the callback on a domain that has already been reaped.

Also, similar to the xl implementation, ignore the SUSPEND shutdown
reason.  By calling libxl_domain_suspend(), we know a shutdown
event with SUSPEND reason will be generated, but it can be safely
ignored since any subsequent cleanup will be done by the callers.

libxlDoDomainSave() was removing non-persistent domains, but
required callers to have the virDomainObj locked.  Callers could
potentially unlock an already freed virDomainObj.  Move this
logic to the callers of libxlDoDomainSave().

I've noticed that libxl can invoke timeout reregister/modify hooks
after returning from libxl_ctx_free.  Explicitly remove the
timeouts before freeing the libxl ctx to avoid executing hooks on
stale objects.

It is possible to destroy and cleanup a VM, resulting in freeing the
libxlDomainObjPrivate object and associated libxl ctx, before all fds and
timeouts have been deregistered and destroyed.

Fix this race by incrementing the reference count on libxlDomainObjPrivate
for each fd and timeout registration.  Only when all fds and timeouts are
deregistered and destroyed will the libxlDomainObjPrivate be destroyed.