https://bugzilla.redhat.com/show_bug.cgi?id=869100 mentioned some
confusion about 'virsh snapshot-list' errors.  Clean up a
misleading error message, and add some documentation.

* tools/virsh-snapshot.c (cmdSnapshotList): Mention --current
rather than --from when appropriate.
* tools/virsh.pod (snapshot-list): Mention that the named starting
point is NOT part of the list except under --tree.

I did a 'git add .', then realized that it ended up trying to
add the emacs lock file for a corresponding file that I had not
yet saved all my edits; thankfully I noticed it in time.  Since
we already exclude other temporary files, this makes the most
sense for preventing such a mistake from actually hitting upstream.

* .gitignore: Add .#* to the exclude list.

This commit changes the behavior of LIBVIRT_DEBUG=1 libvirtd:

$ git show 7022b091
commit 7022b091
Author: Daniel P. Berrange <berrange@redhat.com>
Date:   Thu Sep 27 13:13:09 2012 +0100

    Automatically enable systemd journal logging

    Probe to see if the systemd journal is accessible, and if
    so enable logging to the journal by default, rather than
    stderr (current default under systemd).

Previously  'LIBVIRT_DEBUG=1 /usr/sbin/libvirtd' would show all debug
output to stderr, now it send debug output to the journal.

Only use the journal by default if running in daemon mode, or
if stdin is _not_ a tty. This should make libvirtd launched from
systemd use the journal, but preserve the old behavior in most
situations.

This was found during testing of the fix for:

   https://bugzilla.redhat.com/show_bug.cgi?id=868483

networkValidate was supposed to check for the existence of multiple
portgroups and report an error if this was encountered. It did, but
there were two problems:

1) even though it logged an error, it still returned success, allowing
the operation to continue.

2) It could exit the portgroup checking loop early (or possibly not
even do it once) if a vlan tag was supplied in the base network config
or one of the portgroups.

This patch fixes networkValidate to return failure in addition to
logging the error, and also changes it to not exit the portgroup
checking loop early. The logic was a bit off in the checking for vlan
anyway, and it's intertwined with fixing the early loop exit, so I
fixed that as well. Now it correctly checks for combinations where a
<virtualport> is specified in the base network def and <vlan> is given
in a portgroup, as well as the opposite (<vlan> in base network def
and <virtualport> in portgroup), and ignores the case of a disallowed
vlan when using *no* portgroup if there is a default portgroup (since
in that case there is no way to not use any portgroup).

Noticed during the review of the previous patch.

* python/libvirt-override.c: Use space between 'if ('.

Added a method getCPUMap to virConnect.
It can be used as follows:

import libvirt
import sys
import os

conn = libvirt.openReadOnly(None)
if conn == None:
    print 'Failed to open connection to the hypervisor'
    sys.exit(1)

try:
    (cpus, cpumap, online) = conn.getCPUMap(0)
except:
    print 'Failed to extract the node cpu map information'
    sys.exit(1)

print 'CPUs total %d, online %d' % (cpus, online)
print 'CPU map %s' % str(cpumap)

del conn
print "OK"

sys.exit(0)
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Driver support added for:
- test: pretending 8 host CPUS, 3 being online
- qemu, lxc, openvz, uml: using nodeGetCPUMap
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

Added an implemention of virNodeGetCPUMap to nodeinfo.c,
(nodeGetCPUMap) which can be used by all drivers for a Linux
hypervisor host.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

Callers should not need to know what the name of the file to
be read in the Linux-specific version of nodeGetCPUmap;
furthermore, qemu cares about online cpus, not present cpus,
when determining which cpus to skip.

While at it, I fixed the fact that we were computing the maximum
online cpu id by doing a slow iteration, when what we really want
to know is the max available cpu.

* src/nodeinfo.h (nodeGetCPUmap): Rename...
(nodeGetCPUBitmap): ...and simplify signature.
* src/nodeinfo.c (linuxParseCPUmax): New function.
(linuxParseCPUmap): Simplify and alter signature.
(nodeGetCPUBitmap): Change implementation.
* src/libvirt_private.syms (nodeinfo.h): Reflect rename.
* src/qemu/qemu_driver.c (qemuDomainGetPercpuStats): Update
caller.

Sometimes it's handy to know how many bits are set.

* src/util/bitmap.h (virBitmapCountBits): New prototype.
(virBitmapNextSetBit): Use correct type.
* src/util/bitmap.c (virBitmapNextSetBit): Likewise.
(virBitmapSetAll): Maintain invariant of clear tail bits.
(virBitmapCountBits): New function.
* src/libvirt_private.syms (bitmap.h): Export it.
* tests/virbitmaptest.c (test2): Test it.

Recent storage patches changed signature of virStorageFileGetMetadata
and replaced chain with backingChain in virDomainDiskDef.

Also remove warnings for upcoming versions. There hadn't been any
compatibility problems with new ESX version over the whole lifetime
of the ESX driver, so I don't expect any in the future.

Update documentation to mention vSphere 5.x support.

Qemu has added some new feature flags. This patch adds them to libvirt.

The new features are for the cpuid function 0x7 that takes an argument
in the ecx register. Currently only 0x0 is used as the argument so I was
lazy and I just clear the registers to 0 before calling cpuid. In future
when there maybe will be some other possible arguments, we will need to
improve the cpu detection code to take this into account.

On one hand, numad probably will manage the affinity of domain process
dynamically in future. On the other hand, even numad won't manage it,
it still could confusion. Let's make things simpler enough to avoid
the lair for now.

When the cpu placement model is "auto", it sets the affinity for
domain process with the advisory nodeset from numad, however,
creating cgroup for the domain process (called emulator thread
in some contexts) later overrides that with pinning it to all
available pCPUs.

How to reproduce:

  * Configure the domain with "auto" placement for <vcpu>, e.g.
    <vcpu placement='auto'>4</vcpu>
  * % virsh start dom
  * % cat /proc/$dompid/status

Though the emulator cgroup cause conflicts, but we can't simply
prohibit creating it, as other tunables are still useful, such
as "emulator_period", which is used by API
virDomainSetSchedulerParameter. So this patch doesn't prohibit
creating the emulator cgroup, but inherit the nodeset from numad,
and reset the affinity for domain process.

* src/qemu/qemu_cgroup.h: Modify definition of qemuSetupCgroupForEmulator
                          to accept the passed nodenet
* src/qemu/qemu_cgroup.c: Set the affinity with the passed nodeset

Abstract the codes to prepare cpumap into a helper a function,
which can be used later.

* src/qemu/qemu_process.h: Declare qemuPrepareCpumap
* src/qemu/qemu_process.c: Implement qemuPrepareCpumap, and use it.

- Added a new host command nodecpumap
- Added documentation

Example:
$ virsh nodecpumap
CPUs present:  8
CPUs online:   3
CPU map:       y-y-y---
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

New macro VIR_CPU_USED added to facilitate the interpretation of
cpu maps.
Further, hardened the other cpumap macros against invocations
like VIR_CPU_USE(cpumap + 1, cpu)
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

- Defined the wire protocol format for virNodeGetCPUMap and its
  arguments
- Implemented remote method invocation (remoteNodeGetCPUMap)
- Implemented method dispatcher (remoteDispatchNodeGetCPUMap)
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Adding a new API to obtain information about the
host node's present, online and offline CPUs.

int virNodeGetCPUMap(virConnectPtr conn,
                     unsigned char **cpumap,
                     unsigned int *online,
                     unsigned int flags);

The function will return the number of CPUs present on the host
or -1 on failure;
If cpumap is non-NULL virNodeGetCPUMap will allocate an array
containing a bit map representation of the online CPUs. It's
the callers responsibility to deallocate cpumap using free().
If online is non-NULL, the variable pointed to will contain
the number of online host node CPUs.
The variable flags has been added to support future extensions
and must be set to 0.

Extend the driver structure by nodeGetCPUMap entry in support of the
new API virNodeGetCPUMap.
Added implementation of virNodeGetCPUMap to libvirt.c
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit 12ad7435 added new functions (virNodeGetMemoryParameters,
virNodeSetMemoryParameters) into the section of the file reserved
for deprecated names.  Fix this by moving things earlier; split
into two patches to make git diff easier to read.

* include/libvirt/libvirt.h.in: Move virNodeGetMemoryParameters
and friends earlier, add a note to prevent relapse.

Commit 12ad7435 added new functions (virNodeGetMemoryParameters,
virNodeSetMemoryParameters) into the section of the file reserved
for deprecated names.  Fix this by moving things earlier; split
into two patches to make git diff easier to read.

* include/libvirt/libvirt.h.in: Move virTypedParameter earlier.

Based on this coreutils bug report, since coreutils copied what
libvirt had done:
https://lists.gnu.org/archive/html/bug-coreutils/2012-10/msg00121.html

* m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Allow
for user to pre-define _FORTIFY_SOURCE.

Transport Open vSwitch per-port data during live
migration by using the utility functions
virNetDevOpenvswitchGetMigrateData() and
virNetDevOpenvswitchSetMigrateData().
Signed-off-by: NKyle Mestery <kmestery@cisco.com>

Add utility functions for Open vSwitch to both save
per-port data before a live migration, and restore the
per-port data after a live migration.
Signed-off-by: NKyle Mestery <kmestery@cisco.com>

Add the ability for the Qemu V3 migration protocol to
include transporting network configuration. A generic
framework is proposed with this patch to allow for the
transfer of opaque data.
Signed-off-by: NKyle Mestery <kmestery@cisco.com>
Signed-off-by: NLaine Stump <laine@laine.org>

In commit 371ddc98, I mistakenly added the check for sysctl
version 9 after setting the hypercall version to 1, which will
fail with

error : xenHypervisorDoV1Op:967 : Unable to issue hypervisor
ioctl 3166208: Function not implemented

This check should be included along with the others that use
hypercall version 2.

When restoring selinux labels after a VM is stopped, any non-standard
path that doesn't have a default selinux label causes the process
to stop and exit early. This isn't really an error condition IMO.

Of course the selinux API could be erroring for some other reason
but hopefully that's rare enough to not need explicit handling.

Common example here is storing disk images in a non-standard location
like under /mnt.

Reported by Michal Privoznik.

* src/security/security_dac.c (virSecurityDACGenLabel): Use
correct format.

We put a comment containing "virsh edit <domain_name>" at the start of
the XML. W3C recommendation forbids the use of "--" in comments [1] and
libvirt can't parse it either. This patch omits the domain name if it
contains a double hyphen.

[1] http://www.w3.org/TR/REC-xml/#sec-comments

Rename the 'wait' parameter to 'loop'.
This silences the warning:
storage/storage_backend.c:1348:34: error: declaration of 'wait' shadows
a global declaration [-Werror=shadow]
and fixes the build with -Werror.
--
Note: loop is pool backwards.

When using --without-$name --without-secdriver-$name with $name being
selinux or apparmor, configure will fail saying that AppArmor/SELinux
development package must be installed.
This is caused by a small bug in --with-secdriver-$name handling in
configure.ac which treats --without-secdriver-$name when $name as if the
user had requested to enable $name when $name couldn't be detected on
the system.

This commit also makes sure the detection checks for disabled
secdrivers do not needlessly get run, especially as this could
cause an error as well in --with-$name --without-secdriver-$name
situations.

* .gnulib: Update to latest, prior to freeze.

The snapshot code when reusing an existing file had hard-to-read
logic, as well as a missing sanity check: REUSE_EXT should require
the destination to already be present.

* src/qemu/qemu_driver.c (qemuDomainSnapshotDiskPrepare): Require
destination on REUSE_EXT, rename variable for legibility.

Fixes a build failure on cygwin:
cc1: warnings being treated as errors
security/security_dac.c: In function 'virSecurityDACSetProcessLabel':
security/security_dac.c:862:5: error: format '%u' expects type 'unsigned int', but argument 7 has type 'uid_t' [-Wformat]
security/security_dac.c:862:5: error: format '%u' expects type 'unsigned int', but argument 8 has type 'gid_t' [-Wformat]

* src/security/security_dac.c (virSecurityDACSetProcessLabel)
(virSecurityDACGenLabel): Use proper casts.

virStorageVolLookupByPath is an API call that virt-manager uses
quite a bit when dealing with storage. This call use BackendStablePath
which has several usleep() heuristics that can be tripped up
and hang virt-manager for a while.

Current example: an empty mpath pool pointing to /dev/mapper makes
_any_ calls to virStorageVolLookupByPath take 5 seconds.

The sleep heuristics are actually only needed in certain cases
when we are waiting for new storage to appear, so let's skip the
timeout steps when calling from LookupByPath.

If building on a 64bit host, rename the affected tapsets to <name>-64.stp.
This is similar to what the python package does in fedora.

https://bugzilla.redhat.com/show_bug.cgi?id=831425

We were just installing them in the top level html directory, which
broke navigation and overwrote other pages.

https://bugzilla.redhat.com/show_bug.cgi?id=837825

Since the option doesn't exist. Fixes booting with
cpu mode='host-model' and qemu 1.2.0