While working with a pmsuspend vs. snapshot issue, I noticed that
the state file in /var/run/libvirt/qemu/dom.xml contained a rather
suspicious "(null)" string, which does not round-trip well through
a libvirtd restart.  Had I been on a platform other than glibc
where printf("%s",NULL) crashes instead of printing (null), we might
have noticed the problem much sooner.

And in fixing that problem, I also noticed that we had several
missing states, because we were #defining several *_LAST names
to a value _different_ than what they were already given as enums
in libvirt.h.  Yuck.  I got rid of default: labels in the case
statements, because they get in the way of gcc's -Wswitch helping
us ensure we cover all enum values.

* src/conf/domain_conf.c (virDomainStateReasonToString)
(virDomainStateReasonFromString): Fill in missing domain states;
rewrite case statement to let compiler enforce checking.
(VIR_DOMAIN_NOSTATE_LAST, VIR_DOMAIN_RUNNING_LAST)
(VIR_DOMAIN_BLOCKED_LAST, VIR_DOMAIN_PAUSED_LAST)
(VIR_DOMAIN_SHUTDOWN_LAST, VIR_DOMAIN_SHUTOFF_LAST)
(VIR_DOMAIN_CRASHED_LAST): Drop dead defines.
(VIR_DOMAIN_PMSUSPENDED_LAST): Drop dead define.
(virDomainPMSuspendedReason): Add missing enum function.
(virDomainRunningReason, virDomainPausedReason): Add missing enum
value.
* src/conf/domain_conf.h (virDomainPMSuspendedReason): Declare
missing functions.
* src/libvirt_private.syms (domain_conf.h): Export them.

I got bit by 'make check' complaining that the sort order I got
by emacs' sort-lines function differed from expectations.

* src/libvirt_private.syms: Add emacs trailer.
* src/libvirt_atomic.syms: Likewise.
* src/libvirt_daemon.syms: Likewise.
* src/libvirt_esx.syms: Likewise.
* src/libvirt_libssh2.syms: Likewise.
* src/libvirt_linux.syms: Likewise.
* src/libvirt_openvz.syms: Likewise.
* src/libvirt_sasl.syms: Likewise.
* src/libvirt_vmx.syms: Likewise.
* src/libvirt_xenxs.syms: Likewise.

With our code, we fail to query for tray-open attribute currently.
That's because in HMP it is 'tray-open' and in QMP it's 'tray_open'.
It always has been. However, we got it exactly the opposite.

A logic bug meant we reported KVM was possible for every
architecture, merely based on whether the query-kvm command
exists. We should instead have been doing it based on whether
the query-kvm command returns 'present: 1'
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently QEMU capabilities are initialized before the QEMU driver
sets ownership on its various directories. The upshot is that if
you change the user/group in the qemu.conf file, libvirtd will fail
to probe QEMU the first time it is run after the config change.
Moving QEMU capabilities initialization to after the chown() calls
fixes this
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This previous commit

  commit 1a50ba2c
  Author: Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
  Date:   Mon Nov 26 15:17:13 2012 +0100

    qemu: Fix QMP Capabability Probing Failure

which attempted to make sure the QEMU process used for probing
ran as the right user id, caused serious performance regression
and unreliability in probing. The -daemonize switch in QEMU
guarantees that the monitor socket is present before the parent
process exits. This means libvirtd is guaranteed to be able to
connect immediately. By switching from -daemonize to the
virCommandDaemonize API libvirtd was no longer synchronized with
QEMU's startup process. The result was that the QEMU monitor
failed to open and went into its 200ms sleep loop. This happened
for all 25 binaries resulting in 5 seconds worth of sleeping
at libvirtd startup. In addition sometimes when libvirt connected,
QEMU would be partially initialized and crash causing total
failure to probe that binary.

This commit reverts the previous change, ensuring we do use the
-daemonize flag to QEMU. Startup delay is cut from 7 seconds
to 2 seconds on my machine, which is on a par with what it was
prior to the capabilities rewrite.

To deal with the fact that QEMU needs to be able to create the
pidfile, we switch pidfile location fron runDir to libDir, which
QEMU is guaranteed to be able to write to.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently, there is no reason to hold qemu driver locked
throughout whole API execution. Moreover, we can use the
new qemuDomObjFromDomain() internal API to lookup domain then.

Linefeed is missed in the help of node-memory-tune.
This patch just adds '\n' to get a correct help message.
Signed-off-by: NSatoru Moriya <satoru.moriya@hds.com>

Hosts for rbd are ceph monitor daemons. These have fixed IP addresses,
so they are often referenced by IP rather than hostname for
convenience, or to avoid relying on DNS. Using IPv4 addresses as the
host name works already, but IPv6 addresses require rbd-specific
escaping because the colon is used as an option separator in the
string passed to qemu.

Escape these colons, and enclose the IPv6 address in square brackets
so it is distinguished from the port, which is currently mandatory.
Acked-by: NOsier Yang <jyang@redhat.com>
Signed-off-by: NJosh Durgin <josh.durgin@inktank.com>

https://bugzilla.redhat.com/show_bug.cgi?id=876829 complains that
if a guest is put into S3 state (such as via virsh dompmsuspend)
and then an external snapshot is taken, qemu forcefully transitions
the domain to paused, but libvirt doesn't reflect that change
internally.  Thus, a user has to use 'virsh suspend' to get libvirt
back in sync with qemu state, and if the user doesn't know this
trick, then the guest appears hung.

* src/qemu/qemu_driver.c (qemuDomainSnapshotCreateActiveExternal):
Track fact that qemu wakes up a suspended domain on migration.

https://bugzilla.redhat.com/show_bug.cgi?id=895882

virDomainSnapshot.getDomain() and virDomainSnapshot.getConnect()
wrappers around virDomainSnapshotGet{Domain,Connect} were not supposed
to be ever implemented. The class should contain proper domain() and
connect() accessors that fetch python objects stored internally within
the class. While domain() was already provided, connect() was missing.

This patch adds connect() method to virDomainSnapshot class and
reimplements getDomain() and getConnect() methods as aliases to domain()
and connect() for backward compatibility.

The previous fix to avoid leaking securityDriverNames forgot to
handle the case of securityDriverNames being NULL, leading to
a crash
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

As of python >= 2.2, it is recommended that all objects inherit
from the 'object' base class. We already require python >= 2.3
for libvirt for thread macro support, so we should follow this
best practice.

See also

  http://stackoverflow.com/questions/4015417/python-class-inherits-objectSigned-off-by: NDaniel P. Berrange <berrange@redhat.com>

When shutting down, the QEMU driver forgot to free the
securityDriverNames string list
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The autodestroy callback code has the following function
called from a hash iterator

  qemuDriverCloseCallbackRun(void *payload,
                             const void *name,
                             void *opaque)
  {
    ...
    char *uuidstr = name
    ...

    dom = closeDef->cb(data->driver, dom, data->conn);
    if (dom)
        virObjectUnlock(dom);

    virHashRemoveEntry(data->driver->closeCallbacks, uuidstr);
  }

The closeDef->cb function may well cause the current callback
to be removed, if it shuts down 'dom'. As such the use of
'uuidstr' in virHashRemoveEntry is accessing free'd memory.
We must make a copy of the uuid str before invoking the
callback to be safe.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When linuxNodeInfoCPUPopulate() method triggered use of an
uninitialize value, since it did not initialize the 'sockets'
field in the virNodeInfoPtr struct:

==30020== Conditional jump or move depends on uninitialised value(s)
==30020==    at 0x5125DBD: linuxNodeInfoCPUPopulate (nodeinfo.c:513)
==30020==    by 0x51261A0: nodeGetInfo (nodeinfo.c:884)
==30020==    by 0x149B9B10: qemuCapsInit (qemu_capabilities.c:846)
==30020==    by 0x14A11B25: qemuCreateCapabilities (qemu_driver.c:424)
==30020==    by 0x14A12426: qemuStartup (qemu_driver.c:874)
==30020==    by 0x512A7AF: virStateInitialize (libvirt.c:822)
==30020==    by 0x40DE04: daemonRunStateInit (libvirtd.c:877)
==30020==    by 0x50ADCE5: virThreadHelper (virthreadpthread.c:161)
==30020==    by 0x328CA07D14: start_thread (pthread_create.c:308)
==30020==    by 0x328C6F246C: clone (clone.S:114)
(happened twice)

        if (socks > nodeinfo->sockets)    <--- here
            nodeinfo->sockets = socks;

Rather than doing this for each field, just make the caller memset
the entire struct to zero.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

0eedb1d9 fixed the wrong variable

Commit 87b4c10c added code that may call
the virCapabilitiesClearHostNUMACellCPUTopology function with
uninitialized second argument. Although the value wouldn't be used some
compilers whine about that.

According to Eric Paris this is slightly more efficient because it
only loads the regular expressions in libselinux once.

Be sure to VIR_FREE(accel) and moved virDomainVideoDefFree() within no_memory
label to be consistent

Resolve resource leak in parallelsApplyIfaceParams() when the 'oldnet' is
allocated locally. Also virCommandFree(cmd) as necessary.

This patch adds demo processor topology information for the test driver.

This patch adds data gathering to the NUMA gathering files and adds
support for outputting the data. The test driver and xend driver need to
be adapted to fill sensible data to the structure in a future patch.

This will allow storing additional topology data in the NUMA topology
definition.

This patch changes the storage type and fixes fallout of the change
across the drivers using it.

This patch also changes semantics of adding new NUMA cell information.
Until now the data were re-allocated and copied to the topology
definition. This patch changes the addition function to steal the
pointer to a pre-allocated structure to simplify the code.

This patch adds RNG schemas for adding more information in the topology
output of the NUMA section in the capabilities XML.

The added elements are designed to provide more information about the
placement and topology of the processors in the system to management
applications.

A demonstration of supported XML added by this patch:
<capabilities>
  <host>
    <topology>
      <cells num='3'>
        <cell id='0'>
          <cpus num='4'> <!-- this is node with Hyperthreading -->
            <cpu id='0' socket_id='0' core_id='0' siblings='0-1'/>
            <cpu id='1' socket_id='0' core_id='0' siblings='0-1'/>
            <cpu id='2' socket_id='0' core_id='1' siblings='2-3'/>
            <cpu id='3' socket_id='0' core_id='1' siblings='2-3'/>
          </cpus>
        </cell>
        <cell id='1'>
          <cpus num='4'> <!-- this is node with modules (Bulldozer) -->
            <cpu id='4' socket_id='0' core_id='2' siblings='4-5'/>
            <cpu id='5' socket_id='0' core_id='3' siblings='4-5'/>
            <cpu id='6' socket_id='0' core_id='4' siblings='6-7'/>
            <cpu id='7' socket_id='0' core_id='5' siblings='6-7'/>
          </cpus>
         </cell>
        <cell id='2'>
          <cpus num='4'> <!-- this is a normal multi-core node -->
            <cpu id='8' socket_id='1' core_id='0' siblings='8'/>
            <cpu id='9' socket_id='1' core_id='1' siblings='9'/>
            <cpu id='10' socket_id='1' core_id='2' siblings='10'/>
            <cpu id='11' socket_id='1' core_id='3' siblings='11'/>
          </cpus>
         </cell>
      </cells>
    </topology>
  </host>
</capabilities>

The socket_id field represents identification of the physical socket the
CPU is plugged in. This ID may not be identical to the physical socket
ID reported by the kernel.

The core_id identifies a core within a socket. Also this field may not
accurately represent physical ID's.

The core_id is guaranteed to be unique within a cell and a socket. There
may be duplicates between sockets. Only cores sharing core_id within one
cell and one socket can be considered as threads. Cores sharing core_id
within sparate cells are distinct cores.

The siblings field is a list of CPU id's the cpu id's the CPU is sibling
with - thus a thread. The list is in the cpuset format.

The way in that memory balloon suppression was handled for S390
is flawed for a number or reasons.
1. Just preventing the default balloon to be created in the case
   of VIR_ARCH_S390[X] is not sufficient. An explicit memballoon
   element in the guest definition will still be honored, resulting
   both in a -balloon option and the allocation of a PCI bus address,
   neither being supported.
2. Prohibiting balloon for S390 altogether at a domain_conf level
   is no good solution either as there's work in progress on the QEMU
   side to implement a virtio-balloon device, although in
   conjunction with a new machine type. Suppressing the balloon
   should therefore be done at the QEMU driver level depending
   on the present capabilities.

Therefore we remove the conditional suppression of the default
balloon in domain_conf.c.
Further, we are claiming the memballoon device for virtio-s390
during device address assignment to prevent it from being considered
as a PCI device.
Finally, we suppress the generation of the balloon command line option
if this is a virtio-s390 machine.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Should have been done in commit 56fd5134 already, but was missed
due to oversight: qemuDomainSendKey didn't release the driver lock
in its cleanup section. This fixes an issue introduced by commit
8c5d2bad.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

virNetworkObjUpdateParseFile used ',' as the termination character for
virBitmapParse. This would break if an non-contiguous range would be
parsed.

This patch changes the name of the @sep argument to @terminator and
clarifies it's usage. This patch also explicitly documents that
whitespace can't be used as @terminator as it is skipped multiple times
in the implementation.

Just add the head line to let the editor know it's XML document.

https://bugzilla.redhat.com/show_bug.cgi?id=892079

One of my previous patches (f2a4e5f1) tried to fix crashing
libvirtd on domain detroy. However, we need to copy pattern from
qemuProcessHandleMonitorEOF() instead of decrementing reference
counter. The rationale for this is, if qemu process is dying due
to domain being destroyed, we obtain EOF on both the monitor and
agent sockets. However, if the exit is expected, qemuProcessStop
is called, which cleans both agent and monitor sockets up. We
want qemuAgentClose() to be called iff the EOF is not expected,
so we don't leak an FD and memory. Moreover, there could be race
with qemuProcessHandleMonitorEOF() which could have already
closed the agent socket, in which case we don't want to do
anything.

test1: Need to check for bitmap before using as well as free it properly
test2: need to check for bitsString2 before using it.