Adding a VNC WebSocket support for QEMU driver.  This functionality is
in upstream qemu from commit described as v1.3.0-982-g7536ee4, so the
capability is being recognized based on QEMU version for now.

The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.

After 78d7c3c5 we are strdup()-ing path to qemu-bridge-helper.
However, the check for its return value is missing. So it is
possible we've ignored the OOM error silently.

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Since the source is already translated before. This just adds the
checking. Move !disk->shared and !disk->src to improve the performance
a bit.

To support "shareable" for volume type disk, we have to translate
the source before trying to add the shared disk entry. To achieve
the goal, this moves the helper qemuTranslateDiskSourcePool into
src/qemu/qemu_conf.c, and introduce an internal only member (voltype)
for struct _virDomainDiskSourcePoolDef, to record the underlying
volume type for use when building the drive string.

Later patch will support "shareable" volume type disk.

Currently when getting an instance of virCgroupPtr we will
create the path in all cgroup controllers. Only at the virt
driver layer are we attempting to filter controllers. This
is bad because the mere act of creating the dirs in the
controllers can have a functional impact on the kernel,
particularly for performance.

Update the virCgroupForDriver() method to accept a bitmask
of controllers to use. Only create dirs in the controllers
that are requested. When creating cgroups for domains,
respect the active controller list from the parent cgroup
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use the qemu specific callback to fill this data in the qemu driver as
it's the only place where it was used and fix tests as the qemu test
capability object didn't configure the defaults for the tests.

This patch removes the defaultDiskDriverName from the virCaps
structure. This particular default value is used only in the qemu driver
so this patch uses the recently added callback to fill the driver name
if it's needed instead of propagating it through virCaps.

This patch implements the devices post parse callback and uses it to fill
the default qemu network card model into the XML if none is specified.

Libvirt assumes that the network card model for qemu is the "rtl8139".
Record this in the XML using the new callback to avoid user
confusion.

This patch adds instrumentation that will allow hypervisor drivers to
fill and validate domain and device definitions after parsed by the XML
parser.

With this patch, after the XML is parsed, a callback to the driver is
issued requesting to fill and validate driver specific details of the
configuration. This allows to use sensible defaults and checks on a per
driver basis at the time the XML is parsed.

Two callback pointers are stored in the new virDomainXMLConf object:
* virDomainDeviceDefPostParseCallback (devicesPostParseCallback)
  - called for a single device parsed and for every single device in a
    domain config. A virDomainDeviceDefPtr is passed along with the
    domain definition and virCaps.

* virDomainDefPostParseCallback, (domainPostParseCallback)
  - A callback that is meant to process the domain config after it's
  parsed.  A virDomainDefPtr is passed along with virCaps.

Both types of callbacks support arbitrary opaque data passed for the
callback functions.

Errors may be reported in those callbacks resulting in a XML parsing
failure.

This patch is the result of running:

for i in $(git ls-files | grep -v html | grep -v \.po$ ); do
  sed -i -e "s/virDomainXMLConf/virDomainXMLOption/g" -e "s/xmlconf/xmlopt/g" $i
done

and a few manual tweaks.

The virCaps structure gathered a ton of irrelevant data over time that.
The original reason is that it was propagated to the XML parser
functions.

This patch aims to create a new data structure virDomainXMLConf that
will contain immutable data that are used by the XML parser. This will
allow two things we need:

1) Get rid of the stuff from virCaps

2) Allow us to add callbacks to check and add driver specific stuff
after domain XML is parsed.

This first attempt removes pointers to private data allocation functions
to this new structure and update all callers and function that require
them.

At least one caller may call qemuSharedDiskEntryFree with NULL as the
first argument. Let's make the function similar to other *Free functions
and do nothing in such case.

Commit f506a4c1 changed virSetUIDGID() to be a noop
when uid/gid are -1, while it used to be a noop when
they are <= 0.

The changes in this commit broke creating new VMs in GNOME Boxes
as qemuDomainCheckDiskPresence gets called during domain creation/startup,
which in turn calls virFileAccessibleAs which fails after calling
virSetUIDGID(0, 0) (Boxes uses session libvirtd). virSetUIDGID is called with
(0, 0) as these are the default user/group values in virQEMUDriverConfig
for session libvirtd.

This commit changes virQEMUDriverConfigNew to use -1 as the unpriviledged
uid/gid. I've also looked at the various places where cfg->user is used,
and they all seem to handle -1 correctly.

There is a lock ordering problem in the QEMU close callback
APIs.

When starting a guest we have a lock on the VM. We then
set a autodestroy callback, which acquires a lock on the
close callbacks.

When running auto-destroy, we obtain a lock on the close
callbacks, then run each callbacks - which obtains a lock
on the VM.

This causes deadlock if anyone tries to start a VM, while
autodestroy is taking place.

The fix is to do autodestroy in 2 phases. First obtain
all the callbacks and remove them from the list under
the close callback lock. Then invoke each callback
from outside the close callback lock.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virQEMUCloseCallbacksRunOne method was passing a uuid string
to virDomainObjListFindByUUID, when it actually expected to get
a raw uuid buffer. This was not caught by the compiler because
the method was using a 'void *uuid' instead of first casting
it to the expected type.

This regression was accidentally caused by refactoring in

  commit 568a6cda
  Author: Jiri Denemark <jdenemar@redhat.com>
  Date:   Fri Feb 15 15:11:47 2013 +0100

    qemu: Avoid deadlock in autodestroy
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Change the error label to "error" and simplify some error paths.

Since closeCallbacks were turned into virObjectLockable, we can no
longer call virQEMUCloseCallbacks APIs from within a registered close
callback.

To avoid having to hold the qemu driver lock while iterating through
close callbacks and calling them. This fixes a real deadlock when a
domain which is being migrated from another host gets autodestoyed as a
result of broken connection to the other host.

Found by John Ferlan (coverity script)

The hash entry is changed from "ref" to {ref, @domains}. With this, the
caller can simply call qemuRemoveSharedDisk, without afraid of removing
the entry belongs to other domains. qemuProcessStart will obviously
benifit from it on error codepath (which calls qemuProcessStop to do
the cleanup).

Based on moving various checking into qemuAddSharedDisk, this
avoids the caller using it in wrong ways. Also this adds two
new checking for qemuCheckSharedDisk (disk device not 'lun'
and kernel doesn't support unpriv_sgio simply returns 0).

This moves the various checking into the helpers, to avoid the
callers missing the checking.

With the majority of fields in the virQEMUDriverPtr struct
now immutable or self-locking, there is no need for practically
any methods to be using the QEMU driver lock. Only a handful
of helper APIs in qemu_conf.c now need it

Currently the APIs for managing the shared disk list take
a virHashTablePtr as the primary argument. This is bad
because it requires the caller to deal with locking of
the QEMU driver. Switch the APIs to take the full
virQEMUDriverPtr instance
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The 'driver->caps' pointer can be changed on the fly. Accessing
it currently requires the global driver lock. Isolate this
access in a single helper, so a future patch can relax the
locking constraints.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use atomic ops to increment nextvmid and encapsulate it in a
method to prevent accidental non-atomic access

The APIs names for accessing the domain list object are
very inconsistent. Rename them all to have a standard
virDomainObjList prefix.

Currently the virQEMUDriverPtr struct contains an wide variety
of data with varying access needs. Move all the static config
data into a dedicated virQEMUDriverConfigPtr object. The only
locking requirement is to hold the driver lock, while obtaining
an instance of virQEMUDriverConfigPtr. Once a reference is held
on the config object, it can be used completely lockless since
it is immutable.

NB, not all APIs correctly hold the driver lock while getting
a reference to the config object in this patch. This is safe
for now since the config is never updated on the fly. Later
patches will address this fully.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The autodestroy callback code has the following function
called from a hash iterator

  qemuDriverCloseCallbackRun(void *payload,
                             const void *name,
                             void *opaque)
  {
    ...
    char *uuidstr = name
    ...

    dom = closeDef->cb(data->driver, dom, data->conn);
    if (dom)
        virObjectUnlock(dom);

    virHashRemoveEntry(data->driver->closeCallbacks, uuidstr);
  }

The closeDef->cb function may well cause the current callback
to be removed, if it shuts down 'dom'. As such the use of
'uuidstr' in virHashRemoveEntry is accessing free'd memory.
We must make a copy of the uuid str before invoking the
callback to be safe.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The QEMU driver default max port is 65535, but it then increments
this by 1 to 65536. This maps to 0 in an unsigned short :-( This
was apparently done so that for() loops could use "< max" instead
of "<= max". Remove this insanity and just make the loop do the
right thing.

The  virDomainObj, qemuAgent, qemuMonitor, lxcMonitor classes
all require a mutex, so can be switched to use virObjectLockable
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This introduces a hash table for qemu driver, to store the shared
disk's info as (@major:minor, @ref_count). @ref_count is the number
of domains which shares the disk.

Since we only care about if the disk support unprivileged SG_IO
commands, and the SG_IO commands only make sense for block disk,
this patch only manages (add/remove hash entry) the shared disk for
block disk.

* src/qemu/qemu_conf.h: (Add member 'sharedDisks' of type
                         virHashTablePtr; Declare helpers
                         qemuGetSharedDiskKey, qemuAddSharedDisk
                         and qemuRemoveSharedDisk)
* src/qemu/qemu_conf.c (Implement the 3 helpers)
* src/qemu/qemu_process.c (Update 'sharedDisks' when domain
                           starting and shutdown)
* src/qemu/qemu_driver.c (Update 'sharedDisks' when attaching
                          or detaching disk).