This commit adds the QEMU driver support for CCW addresses. The
current QEMU only allows virtio devices to be attached to the
CCW bus. We named the new capability indicating that support
QEMU_CAPS_VIRTIO_CCW accordingly.

The fact that CCW devices can only be assigned to domains with a
machine type of s390-ccw-virtio requires a few extra checks for
machine type in qemu_command.c on top of querying
QEMU_CAPS_VIRTIO_{CCW|S390}.

The majority of the new functions deals with CCW address generation
and management.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

With our recent patch (1715c83b) we thrive to get the correct
number of maximal VCPUs. However, we are using a constant from
linux/kvm.h which may be not defined in every distro. Hence, we
should guard usage of the constant with ifdef preprocessor
directive. This was introduced in kernel:

    commit 8c3ba334f8588e1d5099f8602cf01897720e0eca
    Author: Sasha Levin <levinsasha928@gmail.com>
    Date:   Mon Jul 18 17:17:15 2011 +0300

    KVM: x86: Raise the hard VCPU count limit

    The patch raises the hard limit of VCPU count to 254.

    This will allow developers to easily work on scalability
    and will allow users to test high VCPU setups easily without
    patching the kernel.

    To prevent possible issues with current setups, KVM_CAP_NR_VCPUS
    now returns the recommended VCPU limit (which is still 64) - this
    should be a safe value for everybody, while a new KVM_CAP_MAX_VCPUS
    returns the hard limit which is now 254.

$ git desc 8c3ba334f
v3.1-rc7-48-g8c3ba33

The virCaps structure gathered a ton of irrelevant data over time that.
The original reason is that it was propagated to the XML parser
functions.

This patch aims to create a new data structure virDomainXMLConf that
will contain immutable data that are used by the XML parser. This will
allow two things we need:

1) Get rid of the stuff from virCaps

2) Allow us to add callbacks to check and add driver specific stuff
after domain XML is parsed.

This first attempt removes pointers to private data allocation functions
to this new structure and update all callers and function that require
them.

At least one caller may call qemuSharedDiskEntryFree with NULL as the
first argument. Let's make the function similar to other *Free functions
and do nothing in such case.

The detection of the maximum number of cpus used incorrect ioctl
argument value. This flaw caused that on kvm hosts this returns always
"160" as the maximum. This is just a recommended maximum value. The real
value is higher than that.

This patch tweaks the detection function to behave as described by the
kernel docs:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/virtual/kvm/api.txt?id=refs/tags/v3.9-rc2#n199

When there are two concurrent threads, we may dereference a NULL
pointer, even though it has been checked before:

1. Thread1: starts executing qemuDomainBlockStatsFlags() with nparams != 0.
            It finds given disk and successfully pass check for disk->info.alias
            not being NULL.
2. Thread2: starts executing qemuDomainDetachDeviceFlags() on the very same
            disk as Thread1 is working on.
3. Thread1: gets to qemuDomainObjBeginJob() where it sets a job on a
            domain.
4. Thread2: also tries to set a job. However, we are not guaranteed which
            thread wins. So assume it's Thread2 who can continue.
5. Thread2: does the actual detach and frees disk->info.alias
6. Thread2: quits the job
7. Thread1: now successfully acquires the job, and accesses a NULL pointer.

The current QEMU code for skipping log messages only skips over
'debug' message, switch to virLogProbablyLogMessage to make sure
it skips over all of them

virsh subcommand memtune forgot updating domain live xml
after setting cgroup value.

19c6ad9a (qemu: Refactor qemuDomainSetMemoryParameters) introduced
a new macro, VIR_GET_LIMIT_PARAMETER(PARAM, VALUE). But if statement
in the macro is not correct and so set_XXXX flags are set to false
in the wrong. As a result, libvirt ignores all memtune parameters.
This patch fixes the conditional expression to work correctly.
Signed-off-by: NSatoru Moriya <satoru.moriya@hds.com>

At the start of the guest after the image is unlinked the state wasn't
touched up to match the state on disk.

Commit f506a4c1 changed virSetUIDGID() to be a noop
when uid/gid are -1, while it used to be a noop when
they are <= 0.

The changes in this commit broke creating new VMs in GNOME Boxes
as qemuDomainCheckDiskPresence gets called during domain creation/startup,
which in turn calls virFileAccessibleAs which fails after calling
virSetUIDGID(0, 0) (Boxes uses session libvirtd). virSetUIDGID is called with
(0, 0) as these are the default user/group values in virQEMUDriverConfig
for session libvirtd.

This commit changes virQEMUDriverConfigNew to use -1 as the unpriviledged
uid/gid. I've also looked at the various places where cfg->user is used,
and they all seem to handle -1 correctly.

This reverts the hack done in

commit 568a6cda
Author: Jiri Denemark <jdenemar@redhat.com>
Date:   Fri Feb 15 15:11:47 2013 +0100

    qemu: Avoid deadlock in autodestroy

since we now have a fix which avoids the deadlock scenario
entirely

There is a lock ordering problem in the QEMU close callback
APIs.

When starting a guest we have a lock on the VM. We then
set a autodestroy callback, which acquires a lock on the
close callbacks.

When running auto-destroy, we obtain a lock on the close
callbacks, then run each callbacks - which obtains a lock
on the VM.

This causes deadlock if anyone tries to start a VM, while
autodestroy is taking place.

The fix is to do autodestroy in 2 phases. First obtain
all the callbacks and remove them from the list under
the close callback lock. Then invoke each callback
from outside the close callback lock.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When the auto-destroy callback runs it is supposed to return
NULL if the virDomainObjPtr is no longer valid. It was not
doing this for transient guests, so we tried to virObjectUnlock
a mutex which had been freed. This often led to a crash.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

qemuProcessStart expects to be run with a job already set and every
caller except for qemuMigrationPrepareAny use it correctly. This bug can
be observed in libvirtd logs during incoming migration as

    warning : qemuDomainObjEnterMonitorInternal:979 : This thread seems
    to be the async job owner; entering monitor without asking for a
    nested job is dangerous

As pointed out in
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1034661
The sentence

"The function of PCI device addresses must less than 8"

does not quite make sense.  Update that to read

"The function of PCI device addresses must be less than 8"
Signed-off-by: NSerge Hallyn <serge.hallyn@ubuntu.com>

Currently, qemuDomainShutdownFlags() chooses the agent method of
shutdown whenever the agent is configured. However, this
assumption is not enough as the guest agent may be unresponsive
at the moment. So unless guest agent method has been explicitly
requested, we should fall back to the ACPI method.

The unitialized local variable qemuVersion can cause an random value
to be returned for the hypervisor version, observable with virsh version.
Introduced by commit b46f7f4aSigned-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

disk->src is still used for disks->hosts->name, do not free it.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

The QEMU driver has a list of devices nodes that are whitelisted
for all guests. The kernel has recently started returning an
error if you try to whitelist a device which does not exist.
This causes a warning in libvirt logs and an audit error for
any missing devices. eg

2013-02-27 16:08:26.515+0000: 29625: warning : virDomainAuditCgroup:451 : success=no virt=kvm resrc=cgroup reason=allow vm="vm031714" uuid=9d8f1de0-44f4-a0b1-7d50-e41ee6cd897b cgroup="/sys/fs/cgroup/devices/libvirt/qemu/vm031714/" class=path path=/dev/kqemu rdev=? acl=rw
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

s/VIR_QEMU_PROCESS_START_AUTODESROY/VIR_QEMU_PROCESS_START_AUTODESTROY/
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The code for putting the emulator threads in a separate cgroup
would spam the logs with warnings

2013-02-27 16:08:26.731+0000: 29624: warning : virCgroupMoveTask:887 : no vm cgroup in controller 3
2013-02-27 16:08:26.731+0000: 29624: warning : virCgroupMoveTask:887 : no vm cgroup in controller 4
2013-02-27 16:08:26.732+0000: 29624: warning : virCgroupMoveTask:887 : no vm cgroup in controller 6

This is because it has only created child cgroups for 3 of the
controllers, but was trying to move the processes from all the
controllers. The fix is to only try to move threads in the
controllers we actually created. Also remove the warning and
make it return a hard error to avoid such lazy callers in the
future.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virQEMUCloseCallbacksRunOne method was passing a uuid string
to virDomainObjListFindByUUID, when it actually expected to get
a raw uuid buffer. This was not caught by the compiler because
the method was using a 'void *uuid' instead of first casting
it to the expected type.

This regression was accidentally caused by refactoring in

  commit 568a6cda
  Author: Jiri Denemark <jdenemar@redhat.com>
  Date:   Fri Feb 15 15:11:47 2013 +0100

    qemu: Avoid deadlock in autodestroy
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=896092 mentions that
qemu 1.4 and earlier only accept a simple start-stop range for
the cpu=... argument of -numa.  Libvirt would attempt to use
-numa cpu=1,3 for a disjoint range, which did not work as intended.

Upstream qemu will be adding a new syntax for disjoint cpu ranges
in 1.5; but the design for that syntax is still under discussion
at the time of this patch.  So for libvirt 1.0.3, it is safest to
just reject attempts to build an invalid qemu command line; in the
future, we can add a capability bit and translate to the final
accepted design for selecting a disjoint cpu range in numa.

* src/qemu/qemu_command.c (qemuBuildNumaArgStr): Reject disjoint
ranges.

This change tried to fix a crash with changing CDROM media but
failed to actually do so

  commit d0172d2b
  Author: Osier Yang <jyang@redhat.com>
  Date:   Tue Feb 19 20:27:45 2013 +0800

    qemu: Remove the shared disk entry if the operation is ejecting or updating

It was still accessing disk->src, when the entire 'disk' object
has been free'd already. Even if it weren't free'd, accessing
the 'src' value of virDomainDiskDef is not allowed without
first validating disk->type is file or block. Just remove the
broken code entirely.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently we call virSetDeviceUnprivSGIO with val == 0 if a block device
has an sgio attribute.  But for sgio='filtered', we know that a
kernel with no unpriv_sgio support will always behave as the user
wanted.  In this case, there is no need to call the function and
report a (bogus) error.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

If virCondInit fails (okay, so that's unlikely), then we end up
attempting a virObjectUnlock() on the cleanup path, even though
we don't hold a lock.  This is not guaranteed to be safe.  While
at it, I noticed a couple places where we were referencing mon->fd
outside locks.

* src/qemu/qemu_monitor.c (qemuMonitorOpenInternal): Minimize lock
duration.  mon->watch doesn't need clean up on error.
(qemuMonitorGetBlockExtent, qemuMonitorBlockResize): Don't
dereference fd outside of lock.

I built without yajl support, and noticed a strange failure message
in qemumonitorjsontest:

2013-02-22 16:12:37.503+0000: 19812: error : virJSONValueToString:1119 : internal error No JSON parser implementation is available
2013-02-22 16:12:37.503+0000: 19812: error : qemuMonitorJSONCommandWithFd:253 : out of memory

While a later patch will fix the test to skip when json is not present,
this patch avoids overriding the more useful error message from
virJSONValueToString returning NULL.

* src/qemu/qemu_monitor_json.c (qemuMonitorJSONCommandWithFd):
Don't override message.
(qemuMonitorJSONCheckError): Don't print NULL.
* src/qemu/qemu_agent.c (qemuAgentCommand): Don't override message.
(qemuAgentCheckError): Don't print NULL.
(qemuAgentArbitraryCommand): Properly fail on OOM.

The new TypedParam helper APIs allow to simplify this function
significantly.

This patch integrates the fix in 75e5bec9
by correctly ordering the setting functions instead of reordering the
parameters.

This patch adds a new capability bit QEMU_CAPS_OBJECT_RNG_EGD and code
to support the egd backend for the VirtIO RNG device.

The device is added by 3 qemu command line options:
-chardev socket,id=charrng0,host=1.2.3.4,port=1234 (communication
                                                    backend)
-object rng-egd,chardev=charrng0,id=rng0 (RNG protocol client)
-device virtio-rng-pci,rng=rng0,bus=pci.0,addr=0x4 (the RNG device)

This patch implements support for the virtio-rng-pci device and the
rng-random backend in qemu.

Two capabilities bits are added to track support for those:

QEMU_CAPS_DEVICE_VIRTIO_RNG - for the device support and
QEMU_CAPS_OBJECT_RNG_RANDOM - for the backend support.

qemu is invoked with these additional parameters if the device is
enabled:

-object rng-random,id=rng0,filename=/test/phile (to add the backend)
-device virtio-rng-pci,rng=rng0,bus=pci.0,addr=0x4 (to add the device)

If a migration fails, we need to stop all block jobs running so
qemu doesn't try to send data to destination over and over again.

At the end of migration, it is important to stop NBD
server and thus release all allocated resources.

This function does the source part of NBD magic. It
invokes drive-mirror on each non shared and RW disk with
a source and wait till the mirroring process completes.
When it does we can proceed with migration.

Currently, an active waiting is done: every 500ms libvirt
asks qemu if block-job is finished or not.  However, once
the job finishes, qemu doesn't report its progress so we
can only assume if the job finished successfully or not.
The better solution would be to listen to the event which
is sent as soon as the job finishes. The event does
contain the result of job.

We need to start NBD server and feed it with all non-<shared/>,
RW and source-full disks. Moreover, with new virPortAllocator we
must ensure the borrowed port for NBD server will be returned if
either migration completes or qemu process is torn down.

This will be used after all migration work is done
to stop NBD server running on destination.  It
doesn't take any arguments, just issues a command.

This will be used with new migration scheme.
This patch creates basically just monitor stub
functions. Wiring them into something useful
is done in later patches.

This will be used with new migration scheme.
This patch creates basically just monitor stub
functions. Wiring them into something useful
is done in later patches.

This migration cookie is meant for two purposes. The first is to be sent
in begin phase from source to destination to let it know we support new
implementation of VIR_MIGRATE_NON_SHARED_{DISK,INC} so destination can
start NBD server. Then, the second purpose is, destination can let us
know, on which port the NBD server is running.

This just keeps track whether qemu knows nbd-server-* commands
so we can use it during migration or not.