To avoid confusion between 'virCapsPtr' and 'qemuCapsPtr'
do some renaming of various fucntions/variables. All
instances of 'qemuCapsPtr' are renamed to 'qemuCaps'. To
avoid that clashing with the 'qemuCaps' typedef though,
rename the latter to virQEMUCaps.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To enable virCapabilities instances to be reference counted,
turn it into a virObject. All cases of virCapabilitiesFree
turn into virObjectUnref
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virCgroupPtr instance APIs are safe to use without locking
in the QEMU driver, since all internal state they rely on is
immutable. Update the comment to reflect this.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

We are wrapping it in ignore_value() anyway.

We are requesting for stderr catching for all cases in
virFileWrapperFdNew(). There is no need to have a separate
function just to report an error, esp. when we can do it in
virFileWrapperFdClose().

The qemuParseGlusterString() replaced dst->src without a VIR_FREE() of
what was in there before.

The qemuBuildCommandLine() did not properly free the boot_buf depending
on various usages.

The qemuParseCommandLineDisk() had numerous paths that didn't clean up
the virDomainDiskDefPtr def properly. Adjust the logic to go through an
error: label before cleanup in order to free the resource.

Annotate the fields in virQEMUDriverPtr to indicate the locking
rules for their use
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently the activePciHostdevs, inactivePciHostdevsd and
activeUsbHostdevs lists are all implicitly protected by the
QEMU driver lock. Now that the lists all inherit from the
virObjectLockable, we can make the locking explicit, removing
the dependency on the QEMU driver lock for correctness.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To allow modifications to the lists to be synchronized, convert
virPCIDeviceList and virUSBDeviceList into virObjectLockable
classes. The locking, however, will not be self-contained. The
users of these classes will have to call virObjectLock/Unlock
in the critical regions.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Rename all the usbDeviceXXX and usbXXXDevice APIs to have a
fixed virUSBDevice name prefix

When iterating over USB host devices to setup cgroups, the
usbDevice object was leaked in both LXC and QEMU driers
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Rename all the pciDeviceXXX and pciXXXDevice APIs to have a
fixed virPCIDevice name prefix

The QEMU driver struct has a 'qemuVersion' field that was previously
used to cache the version lookup from capabilities. With the recent
QEMU capabilities rewrite the caching happens at a lower level so
this field is pointless. Removing it avoids worries about locking
when updating it.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use atomic ops to increment nextvmid and encapsulate it in a
method to prevent accidental non-atomic access

The duplicate VM checking should be done atomically with
virDomainObjListAdd, so shoud not be a separate function.
Instead just use flags to indicate what kind of checks are
required.

This pair, used in virDomainCreateXML:

   if (virDomainObjListIsDuplicate(privconn->domains, def, 1) < 0)
     goto cleanup;
   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def, false)))
     goto cleanup;

Changes to

   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def,
                                   VIR_DOMAIN_OBJ_LIST_ADD_CHECK_LIVE,
                                   NULL)))
     goto cleanup;

This pair, used in virDomainRestoreFlags:

   if (virDomainObjListIsDuplicate(privconn->domains, def, 1) < 0)
     goto cleanup;
   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def, true)))
     goto cleanup;

Changes to

   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def,
                                   VIR_DOMAIN_OBJ_LIST_ADD_LIVE |
                                   VIR_DOMAIN_OBJ_LIST_ADD_CHECK_LIVE,
                                   NULL)))
     goto cleanup;

This pair, used in virDomainDefineXML:

   if (virDomainObjListIsDuplicate(privconn->domains, def, 0) < 0)
     goto cleanup;
   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def, false)))
     goto cleanup;

Changes to

   if (!(dom = virDomainObjListAdd(privconn->domains,
                                   privconn->caps,
                                   def,
                                   0, NULL)))
     goto cleanup;

Otherwise, we get a lot of scary (but harmless) noise in the logs:

2013-02-05 15:35:48.555+0000: 8637: error : qemuMonitorJSONCheckError:353 : internal error unable to execute QEMU command 'add-fd': Parameter 'fdset-id' expects an existing fdset-id

one for every qemu 1.2 binary that we probe.

* src/qemu/qemu_monitor_json.c (qemuMonitorJSONAddFd): During
probe, avoid logging failures.

As a step towards making virDomainObjList thread-safe turn it
into an opaque virObject, preventing any direct access to its
internals.

As part of this a new method virDomainObjListForEach is
introduced to replace all existing usage of virHashForEach

The APIs names for accessing the domain list object are
very inconsistent. Rename them all to have a standard
virDomainObjList prefix.

Currently the virQEMUDriverPtr struct contains an wide variety
of data with varying access needs. Move all the static config
data into a dedicated virQEMUDriverConfigPtr object. The only
locking requirement is to hold the driver lock, while obtaining
an instance of virQEMUDriverConfigPtr. Once a reference is held
on the config object, it can be used completely lockless since
it is immutable.

NB, not all APIs correctly hold the driver lock while getting
a reference to the config object in this patch. This is safe
for now since the config is never updated on the fly. Later
patches will address this fully.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If a compression binary prints something to stderr, currently
it is discarded. However, it can contain useful data from
debugging POV, so we should catch it.

If a decompression binary prints something to stderr, currently
it is discarded. However, it can contain useful data from
debugging POV, so we should catch it.

Add support for QEMU -add-fd command line parameter detection.
This intentionally rejects qemu 1.2, where 'add-fd' QMP did
not allow full control of set ids, and where there was no command
line counterpart, but accepts qemu 1.3.
Signed-off-by: NEric Blake <eblake@redhat.com>

Add entry points for calling the qemu 'add-fd' and 'remove-fd'
monitor commands.  There is no entry point for 'query-fdsets';
the assumption is that a developer can use
 virsh qemu-monitor-command domain '{"execute":"query-fdsets"}'
when debugging issues, and that meanwhile, libvirt is responsible
enough to remember what fds it associated with what fdsets.
Likewise, on the 'add-fd' command, it is assumed that libvirt
will always pass a set id, rather than letting qemu autogenerate
the next available id number.

* src/qemu/qemu_monitor.c (qemuMonitorAddFd, qemuMonitorRemoveFd):
New functions.
* src/qemu/qemu_monitor.h (qemuMonitorAddFd, qemuMonitorRemoveFd):
New prototypes.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONAddFd)
(qemuMonitorJSONRemoveFd): New functions.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONAddFd)
(qemuMonitorJSONRemoveFd): New prototypes.

https://bugzilla.redhat.com/show_bug.cgi?id=894723

Currently, if qemuProcessStart() succeeds, but it's decompression
binary that returns nonzero status, we don't kill the qemu process,
but remove it from internal domain list, leaving the qemu process
hanging around totally uncontrolled.

https://bugzilla.redhat.com/show_bug.cgi?id=892289

It seems like with new udev within guest OS, the tray is locked,
so we need to:
- 'eject'
- wait for tray to open
- 'change'

Moreover, even when doing bare 'eject', we should check for
'tray_open' as guest may have locked the tray. However, the
waiting phase shouldn't be unbounded, so I've chosen 10 retries
maximum, each per 500ms. This should give enough time for guest
to eject a media and open the tray.

With our code, we fail to query for tray-open attribute currently.
That's because in HMP it is 'tray-open' and in QMP it's 'tray_open'.
It always has been. However, we got it exactly the opposite.

A logic bug meant we reported KVM was possible for every
architecture, merely based on whether the query-kvm command
exists. We should instead have been doing it based on whether
the query-kvm command returns 'present: 1'
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently QEMU capabilities are initialized before the QEMU driver
sets ownership on its various directories. The upshot is that if
you change the user/group in the qemu.conf file, libvirtd will fail
to probe QEMU the first time it is run after the config change.
Moving QEMU capabilities initialization to after the chown() calls
fixes this
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This previous commit

  commit 1a50ba2c
  Author: Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
  Date:   Mon Nov 26 15:17:13 2012 +0100

    qemu: Fix QMP Capabability Probing Failure

which attempted to make sure the QEMU process used for probing
ran as the right user id, caused serious performance regression
and unreliability in probing. The -daemonize switch in QEMU
guarantees that the monitor socket is present before the parent
process exits. This means libvirtd is guaranteed to be able to
connect immediately. By switching from -daemonize to the
virCommandDaemonize API libvirtd was no longer synchronized with
QEMU's startup process. The result was that the QEMU monitor
failed to open and went into its 200ms sleep loop. This happened
for all 25 binaries resulting in 5 seconds worth of sleeping
at libvirtd startup. In addition sometimes when libvirt connected,
QEMU would be partially initialized and crash causing total
failure to probe that binary.

This commit reverts the previous change, ensuring we do use the
-daemonize flag to QEMU. Startup delay is cut from 7 seconds
to 2 seconds on my machine, which is on a par with what it was
prior to the capabilities rewrite.

To deal with the fact that QEMU needs to be able to create the
pidfile, we switch pidfile location fron runDir to libDir, which
QEMU is guaranteed to be able to write to.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently, there is no reason to hold qemu driver locked
throughout whole API execution. Moreover, we can use the
new qemuDomObjFromDomain() internal API to lookup domain then.

Hosts for rbd are ceph monitor daemons. These have fixed IP addresses,
so they are often referenced by IP rather than hostname for
convenience, or to avoid relying on DNS. Using IPv4 addresses as the
host name works already, but IPv6 addresses require rbd-specific
escaping because the colon is used as an option separator in the
string passed to qemu.

Escape these colons, and enclose the IPv6 address in square brackets
so it is distinguished from the port, which is currently mandatory.
Acked-by: NOsier Yang <jyang@redhat.com>
Signed-off-by: NJosh Durgin <josh.durgin@inktank.com>

https://bugzilla.redhat.com/show_bug.cgi?id=876829 complains that
if a guest is put into S3 state (such as via virsh dompmsuspend)
and then an external snapshot is taken, qemu forcefully transitions
the domain to paused, but libvirt doesn't reflect that change
internally.  Thus, a user has to use 'virsh suspend' to get libvirt
back in sync with qemu state, and if the user doesn't know this
trick, then the guest appears hung.

* src/qemu/qemu_driver.c (qemuDomainSnapshotCreateActiveExternal):
Track fact that qemu wakes up a suspended domain on migration.

The previous fix to avoid leaking securityDriverNames forgot to
handle the case of securityDriverNames being NULL, leading to
a crash
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When shutting down, the QEMU driver forgot to free the
securityDriverNames string list
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The autodestroy callback code has the following function
called from a hash iterator

  qemuDriverCloseCallbackRun(void *payload,
                             const void *name,
                             void *opaque)
  {
    ...
    char *uuidstr = name
    ...

    dom = closeDef->cb(data->driver, dom, data->conn);
    if (dom)
        virObjectUnlock(dom);

    virHashRemoveEntry(data->driver->closeCallbacks, uuidstr);
  }

The closeDef->cb function may well cause the current callback
to be removed, if it shuts down 'dom'. As such the use of
'uuidstr' in virHashRemoveEntry is accessing free'd memory.
We must make a copy of the uuid str before invoking the
callback to be safe.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This will allow storing additional topology data in the NUMA topology
definition.

This patch changes the storage type and fixes fallout of the change
across the drivers using it.

This patch also changes semantics of adding new NUMA cell information.
Until now the data were re-allocated and copied to the topology
definition. This patch changes the addition function to steal the
pointer to a pre-allocated structure to simplify the code.

The way in that memory balloon suppression was handled for S390
is flawed for a number or reasons.
1. Just preventing the default balloon to be created in the case
   of VIR_ARCH_S390[X] is not sufficient. An explicit memballoon
   element in the guest definition will still be honored, resulting
   both in a -balloon option and the allocation of a PCI bus address,
   neither being supported.
2. Prohibiting balloon for S390 altogether at a domain_conf level
   is no good solution either as there's work in progress on the QEMU
   side to implement a virtio-balloon device, although in
   conjunction with a new machine type. Suppressing the balloon
   should therefore be done at the QEMU driver level depending
   on the present capabilities.

Therefore we remove the conditional suppression of the default
balloon in domain_conf.c.
Further, we are claiming the memballoon device for virtio-s390
during device address assignment to prevent it from being considered
as a PCI device.
Finally, we suppress the generation of the balloon command line option
if this is a virtio-s390 machine.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Should have been done in commit 56fd5134 already, but was missed
due to oversight: qemuDomainSendKey didn't release the driver lock
in its cleanup section. This fixes an issue introduced by commit
8c5d2bad.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>