Add virKeepAliveTimeout and virKeepAliveTrigger APIs that can be used to
set poll timeouts and trigger keepalive timer. virKeepAliveTrigger
checks if it is called to early and does nothing in that case.
(cherry picked from commit 28c75382)

The code that needs to be run every keepalive interval of inactivity was
only called from a timer and thus from the main event loop. We will need
to call the code directly from another place.
(cherry picked from commit a2ba8686)

As we never drop non-blocking calls, the return value that used to
indicate a call was dropped is no longer needed.
(cherry picked from commit ca9b13e3)

As non-blocking calls are no longer dropped, we don't really need to
care that much about their fate and wait for the thread with the buck
to process them. If another thread has the buck, we can just push a
non-blocking call to the queue and be done with it.
(cherry picked from commit ef392614)

So far, we were dropping non-blocking calls whenever sending them would
block. In case a client is sending lots of stream calls (which are not
supposed to generate any reply), the assumption that having other calls
in a queue is sufficient to get a reply from the server doesn't work. I
tried to fix this in b1e374a7 but
failed and reverted that commit.

With this patch, non-blocking calls are never dropped (unless the
connection is being closed) and will always be sent.
(cherry picked from commit 78602c4e)

Normally, when every call has a thread associated with it, the thread
may get the buck and be in charge of sending all calls until its own
call is done. When we introduced non-blocking calls, we had to add
special handling of new non-blocking calls. This patch uses event loop
to send data if there is no thread to get the buck so that any
non-blocking calls left in the queue are properly sent without having to
handle them specially. It also avoids adding even more cruft to client
IO loop in the following patches.

With this change in, non-blocking calls may see unpredictable delays in
delivery when the client has no event loop registered. However, the only
non-blocking calls we have are keepalives and we already require event
loop for them, which makes this a non-issue until someone introduces new
non-blocking calls.
(cherry picked from commit 9e747e5c)

When analyzing our debug log, I'm always confused about what each of the
pointers mean. Let's be explicit.
(cherry picked from commit 71689f95)

The docs for virConnectSetKeepAlive() advertise that this function
should be able to disable keepalives on negative or zero interval time.

This patch removes the check that prohibited this and adds code to
disable keepalives on negative/zero interval.

* src/libvirt.c: virConnectSetKeepAlive(): - remove check for negative
                                             values
* src/rpc/virnetclient.c
* src/rpc/virnetclient.h: - add virNetClientKeepAliveStop() to disable
                            keepalive messages
* src/remote/remote_driver.c: remoteSetKeepAlive(): -add ability to
                                                     disable keepalives
(cherry picked from commit 6446a9e2)

Commit ba226d33 tried to fix crash of
the daemon when a domain with an open console was destroyed. The fix was
wrong as it tried to remove the callback also when the stream was
aborted, where at that point the fd stream driver was already freed and
removed.

This patch clears the callbacks with a helper right before the hash is
freed, so that it doesn't interfere with other codepaths where the
stream object is freed.
(cherry picked from commit 45edefc7)

(cherry picked from commit 7de158cf)

The stream object wasn't set to NULL after freeing causing a double free
attempt on the cleanup path.
(cherry picked from commit e3b8808b)

Pushed without ack, under the trivial rule.
(cherry picked from commit 2beed2da)

Detected by Coverity.

* src/qemu/qemu_hotplug.c (qemuDomainAttachHostDevice): Avoid
double free of usb on failure.
(cherry picked from commit 665c8cde)

When a domain has a active console connection and is destroyed the
callback is called on private data that no longer exist causing a
segfault.
(cherry picked from commit ba226d33)

If a domain is explicitly configured with <seclabel type="none"/> we
correctly ensure that no labeling will be done by setting
norelabel=true. However, if no seclabel element is present in domain XML
and hypervisor is configured not to confine domains by default, we only
set type to "none" without turning off relabeling. Thus if such a domain
is being started, security driver wants to relabel resources with
default label, which doesn't make any sense.

Moreover, with SELinux security driver, the generated image label lacks
"s0" sensitivity, which causes setfilecon() fail with EINVAL in
enforcing mode.
(cherry picked from commit ce53382b)

No check for conn->uri being NULL in virAuthGetConfigFilePath (valid
state) made the client segfault. This happens for example with these
settings:
 - no virtualbox driver installed (modifies conn->uri)
 - no default URI set (VIRSH_DEFAULT_CONNECT_URI="",
   LIBVIRT_DEFAULT_URI="", uri_default="")
 - auth_sock_rw="sasl"
 - virsh run as root

That are unfortunately the settings with fresh Fedora 17 installation
with VDSM.

The check ought to be enough as conn->uri being NULL is valid in later
code and is handled properly.
(cherry picked from commit 5eef7432)

some system call and signal will interrupt poll,
making event loop stops and fails to react events and keepalive message
from libvirt.
adding handling EINTR to poll to make it more robust
Signed-off-by: NRoyce Lv <lvroyce@linux.vnet.ibm.com>
(cherry picked from commit 5e62ba34)

The time keeping example was missing quotes which resulted in an error
if you copied and pasted the example into a domain's XML. Additionally
the rest of the examples use single quotes (') instead of double quotes
(") so standardized that.
(cherry picked from commit d57e17d5)

When the system doesn't have IPv6 available (e.g. not built into the
kernel or the module isn't loaded), you can not create an IPv6 socket.
The test determines earlier on that IPv6 isn't available then goes and
creates a socket. This makes socket creation conditional on IPv6
availability.
(cherry picked from commit faffe269)

As per the comment, the Xen hypervisor driver is considered to
be mandatory when running privileged. When it fails to open,
we should thus return an error, not ignore it.
(cherry picked from commit 489900e3)

When sending SIGHUP to libvirtd, it will trigger the virStateDriver
reload operation. This is intended to reload the configuration files
for guests. For unknown historical reasons this is also triggering
autostart of all guests. Autostart is generally expected to be
something that happens on OS startup. Starting VMs on SIGHUP will
violate that expectation and potentially cause dangerous scenarios
if the admin has explicitly shutdown a misbehaving VM that has
been marked as autostart
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit 95c2c19f)

Detected on Cygwin.  Broken in commit 387117ad.

* src/util/virnetdev.c (virNetDevValidateConfig)
(virNetDevReplaceNetConfig): Fix prototypes.
* src/util/virnetlink.c (virNetlinkEventAddClient)
(virNetlinkEventRemoveClient): Likewise.
(cherry picked from commit 68a97bd8)

Remote driver needs to make sure the driver lock is released before
entering client IO loop as that may block indefinitely in poll(). As a
direct consequence of not following this in stream APIs, tunneled
migration to a destination host which becomes non-responding may block
qemu driver. Luckily, if keepalive is turned for p2p migrations, both
remote and qemu drivers will get automagically unblocked after keepalive
timeout.
(cherry picked from commit 17f3be07)

When host CPU could not be properly detected, virConnectCompareCPU will
just report that any CPU is incompatible with host CPU instead of
failing.
(cherry picked from commit 87c86231)

When --direct is used when migrating a domain running on a hypervisor
that does not support direct migration (such as QEMU), the caller would
get the following error message:

    this function is not supported by the connection driver:
    virDomainMigrateToURI2

which is a complete nonsense since qemu driver implements
virDomainMigrateToURI2. This patch would emit a more sensible error in
this case:

    Requested operation is not valid: direct migration is not supported
    by the connection driver
(cherry picked from commit 3189dfb1)

Commit 32a9aac2 switched libvirt to use the XDG base directories
to locate most of its data/config. In particular, the per-user socket
for qemu:///session is now stored in the XDG runtime directory.
This directory is located by looking at the XDG_RUNTIME_DIR environment
variable, with a fallback to ~/.cache/libvirt if this variable is not
set.

When the daemon is autospawned because a client application wants
to use qemu:///session, the daemon is ran in a clean environment
which does not contain XDG_RUNTIME_DIR. It will create its socket
in ~/.cache/libvirt. If the client application has XDG_RUNTIME_DIR
set, it will not look for the socket in the fallback place, and will
fail to connect to the autospawned daemon.

This patch adds XDG_RUNTIME_DIR to the daemon environment before
auto-starting it. I've done this in virNetSocketForkDaemon rather
than in virCommandAddEnvPassCommon as I wasn't sure we want to pass
these variables to other commands libvirt spawns. XDG_CACHE_HOME
and XDG_CONFIG_HOME are also added to the daemon env as it makes use
of those as well.
(cherry picked from commit efe6c802)

This makes the driver fail with a clear error message in case of UUID
collisions (for example if somebody copied a container configuration
without updating the UUID) and also raises an error on other hash map
failures.

OpenVZ itself doesn't complain about duplicate UUIDs since this
parameter is only used by libvirt.
(cherry picked from commit 31351c31)

Commit 5e6ce1 moved down detection of the ACPI feature in
qemuParseCommandLine. However, when ACPI is detected, it clears
all feature flags in def->features to only set ACPI. This used to
be fine because this was the first place were def->features was set,
but after the move this is no longer necessarily true because this
block comes before the ACPI check:

if (strstr(def->emulator, "kvm")) {
    def->virtType = VIR_DOMAIN_VIRT_KVM;
    def->features |= (1 << VIR_DOMAIN_FEATURE_PAE);
}

Since def is allocated in qemuParseCommandLine using VIR_ALLOC, we
can always use |= when modifying def->features
(cherry picked from commit 626dd518)

Domains configured with autostart may fail to start if the host
network stack has not been started.  E.g. when using bridged
networking autostarting a domain can fail with

libvirtd[1403]: 2012-06-20 13:23:49.833+0000: 1485: error :
qemuAutostartDomain:177 : Failed to autostart VM 'test': Cannot get
interface MTU on 'br0': No such device
(cherry picked from commit 4036aa91)

Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
(cherry picked from commit 097da1ab)

Clarify the docs to make more clear what this command does and that it
requires a guest agent running in the guest.
(cherry picked from commit e16d434d)

As the storage pool sources are stored in a list of structs, the pointer
returned by virStoragePoolSourceListNewSource() shouldn't be freed as it
points in the middle of a memory block. This combined with a regression
that takes the error path every time on caused a double-free abort on
the src struct in question.
(cherry picked from commit ab9c72ae)

The extra data need to be added to one test case
(cherry picked from commit c47a8aec)

Recently the Ceph project defaulted auth_supported from 'none' to 'cephx'.

When no auth information was set for Ceph disks this would lead to librados defaulting to
'cephx', but there would be no additional authorization information.

We now explicitly set auth_supported to none when passing down arguments to Qemu.
Signed-off-by: NWido den Hollander <wido@widodh.nl>
(cherry picked from commit ccb94785)

(crobinso: Add Wido to AUTHORS)

QEMU (and librbd) flush the cache on the source before the
destination starts, and the destination does not read any
changeable data before that, so live migration with rbd caching
is safe.

This makes 'virsh migrate' work with rbd and caching without the
--unsafe flag.
Reported-by: NVladimir Bashkirtsev <vladimir@bashkirtsev.com>
Signed-off-by: NJosh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 78290b16)

* .mailmap: Add a name alias.
(cherry picked from commit 30a30a7a)

virDomainBlockStatsFlags can't collect total_time_ns for read/write/flush
because of key typo when retriveing from qemu cmd result
Signed-off-by: Nlvroyce <lvroyce@linux.vnet.ibm.com>
(cherry picked from commit 811cea18)

Fixed up virsh -V output by removing invalid WITH_PROXY & WITH_ONE
checks, adding several missing checks, and fixing the DTrace check.
Signed-off-by: NDoug Goldstein <cardoe@cardoe.com>
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 9faaaba4)

Below patch fixes this coverity report:

/libvirt/src/conf/nwfilter_conf.c:382:
leaked_storage: Variable "varAccess" going out of scope leaks the storage it points to.

(cherry picked from commit b1675bac)

Hello,

This is a patch to fix vm's outbound traffic control problem.

Currently, vm's outbound traffic control by libvirt doesn't go well.
This problem was previously discussed at libvir-list ML, however
it seems that there isn't still any answer to the problem.
http://www.redhat.com/archives/libvir-list/2011-August/msg00333.html

I measured Guest(with virtio-net) to Host TCP throughput with the
command "netperf -H".
Here are the outbound QoS parameters and the results.

outbound average rate[kilobytes/s] : Guest to Host throughput[Mbit/s]
======================================================================
1024  (8Mbit/s)                    : 4.56
2048  (16Mbit/s)                   : 3.29
4096  (32Mbit/s)                   : 3.35
8192  (64Mbit/s)                   : 3.95
16384 (128Mbit/s)                  : 4.08
32768 (256Mbit/s)                  : 3.94
65536 (512Mbit/s)                  : 3.23

The outbound traffic goes down unreasonably and is even not controled.

The cause of this problem is too large mtu value in "tc filter" command run by
libvirt. The command uses burst value to set mtu and the burst is equal to
average rate value if it's not set. This value is too large. For example
if the average rate is set to 1024 kilobytes/s, the mtu value is set to 1024
kilobytes. That's too large compared to the size of network packets.
Here libvirt applies tc ingress filter to Host's vnet(tun) device.
Tc ingress filter is implemented with TBF(Token Buckets Filter) algorithm. TBF
uses mtu value to calculate the amount of token consumed by each packet. With too
large mtu value, the token consumption rate is set too large. This leads to
token starvation and deterioration of TCP throughput.

Then, should we use the default mtu value 2 kilobytes?
The anser is No, because Guest with virtio-net device uses 65536 bytes
as mtu to transmit packets to Host, and the tc filter with the default mtu
value 2k drops packets whose size is larger than 2k. So, the most packets
is droped and again leads to deterioration of TCP throughput.

The appropriate mtu value is 65536 bytes which is equal to the maximum value
of network interface device defined in <linux/netdevice.h>. The value is
not so large that it causes token starvation and not so small that it
drops most packets.
Therefore this patch set the mtu value to 64kb(== 65535 bytes).

Again, here are the outbound QoS parameters and the TCP throughput with
the libvirt patched.

outbound average rate[kilobytes/s] : Guest to Host throughput[Mbit/s]
======================================================================
1024  (8Mbit/s)                    : 8.22
2048  (16Mbit/s)                   : 16.42
4096  (32Mbit/s)                   : 32.93
8192  (64Mbit/s)                   : 66.85
16384 (128Mbit/s)                  : 133.88
32768 (256Mbit/s)                  : 271.01
65536 (512Mbit/s)                  : 547.32

The outbound traffic conforms to the given limit.

Thank you,
Signed-off-by: NEiichi Tsukata <eiichi.tsukata.xh@hitachi.com>
(cherry picked from commit 0ac3baee)

Conflicts:

	AUTHORS