Move error reporting out of the callers, into virURIParse
and virURIFormat, to get consistency.

* include/libvirt/virterror.h, src/util/virterror.c: Add VIR_FROM_URI
* src/util/viruri.c, src/util/viruri.h: Add error reporting
* src/esx/esx_driver.c, src/libvirt.c, src/libxl/libxl_driver.c,
  src/lxc/lxc_driver.c, src/openvz/openvz_driver.c,
  src/qemu/qemu_driver.c, src/qemu/qemu_migration.c,
  src/remote/remote_driver.c, src/uml/uml_driver.c,
  src/vbox/vbox_tmpl.c, src/vmx/vmx.c, src/xen/xen_driver.c,
  src/xen/xend_internal.c, tests/viruritest.c: Remove error
  reporting
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The parameter in the virURIFormat impl mistakenly used the
xmlURIPtr type, instead of virURIPtr. Since they will soon
cease to be identical, this needs fixing
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Since we defined a custom virURIPtr type, we should use a
virURIFree method instead of assuming it will always be
a typedef for xmlURIPtr

* src/util/viruri.c, src/util/viruri.h, src/libvirt_private.syms:
  Add a virURIFree method
* src/datatypes.c, src/esx/esx_driver.c, src/libvirt.c,
  src/qemu/qemu_migration.c, src/vmx/vmx.c, src/xen/xend_internal.c,
  tests/viruritest.c: s/xmlFreeURI/virURIFree/
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When a client which started non-p2p migration dies in a bad time, the
source libvirtd never clears the migration job and almost nothing can be
done with the domain without restarting the daemon. This patch makes use
of connection close callbacks and ensures that migration job is properly
discarded when the client disconnects.

Add support for registering arbitrary callback to be called for a domain
when a connection gets closed.

Destination daemon should not rely on the client or source daemon
(depending on the type of migration) to call Finish when migration
fails, because the client may crash before it can do so. The domain
prepared for incoming migration is set to be destroyed (and migration
job cleaned up) when connection with the client closes but this is not
enough. If the associated qemu process crashes after Prepare step and
the domain is cleaned up before the connection gets closed, autodestroy
is not called for the domain and migration jobs remains set. In case the
domain is defined on destination host (i.e., it is not completely
removed once destroyed) we keep the job set for ever. To fix this, we
register a cleanup callback which is responsible to clean migration-in
job when a domain dies anywhere between Prepare and Finish steps. Note
that we can't blindly clean any job when spotting EOF on monitor since
normally an API is running at that time.

Add support for registering cleanup callbacks to be run when a domain
transitions to shutoff state.

This reverts commit 61f2b6ba and most of
commit d8916dc8, which effectively
brings back commit ef1065cf written by
Jim Fehlig:

The qemu migration speed default is 32MiB/s as defined in migration.c

/* Migration speed throttling */
static int64_t max_throttle = (32 << 20);

There's no need to throttle migration when targeting a file, so set
migration speed to unlimited prior to migration, and restore to libvirt
default value after migration.

Default units is MB for migrate_set_speed monitor command, so
(INT64_MAX / (1024 * 1024)) is used for unlimited migration speed.

This was reverted because migration to file could not be canceled and
even monitored since qemu was not processing any monitor commands until
the migration finished. This is now different as we make sure the
file descriptor we pass to qemu is able to properly report EAGAIN.
Recent qemu changes might have helped as well.

I tested managedsave with this patch in and indeed, it is 10x faster
while I can still monitor its progress.

A few times libvirt users manually setting mac addresses have
complained of a networking failure that ends up being due to a multicast
mac address being used for a guest interface. This patch prevents that
by logging an error and failing if a multicast mac address is
encountered in each of the three following cases:

1) domain xml <interface> mac address.
2) network xml bridge mac address.
3) network xml dhcp/host mac address.

There are several other places where a mac address can be input that
aren't controlled in this manner because failure to do so has no
consequences (e.g., if the address will be used to search through
existing interfaces for a match).

The RNG has been updated to add multiMacAddr and uniMacAddr along with
the existing macAddr, and macAddr was switched to uniMacAddr where
appropriate.

If an error was encountered parsing a dhcp host entry mac address or
name, parsing would continue and log a less descriptive error that
might make it more difficult to notice the true nature of the problem.

This patch returns immediately on logging the first error.

This patch is in response to:

  https://bugzilla.redhat.com/show_bug.cgi?id=798467

If a guest's tap device is created using the same MAC address the
guest uses for its own network card (which connects to the tap
device), the Linux kernel will log the following message and traffic
will not pass:

 kernel: vnet9: received packet with own address as source address

This patch disallows MAC addresses with a first byte of 0xFE, but only in
the case that the MAC address is used for a guest interface that's
connected by way of a standard tap device. (In other words, the
validation is done at runtime at the same place the MAC address is
modified for the tap device, rather than when mac address is parsed,
the idea being that it is then we know for sure the address will be
problematic.)

Using inheritance, this patch cleans up the cpu_map.xml file and also
sorts all CPU features according to the feature and registry
values. Model features are sorted the same way as foeatures in the
specification.
Also few models that are related were organized together and parts of
the XML are marked with comments

If a guest is paused, we were silently ignoring the quiesce flag,
which results in unclean snapshots, contrary to the intent of the
flag.  Since we can't quiesce without guest agent support, we should
instead fail if the guest is not running.

Meanwhile, if we attempt a quiesce command, but the guest agent
doesn't respond, and we time out, we may have left the command
pending on the guest's queue, and when the guest resumes parsing
commands, it will freeze even though our command is no longer
around to issue a thaw.  To be safe, we must _always_ pair every
quiesce call with a counterpart thaw, even if the quiesce call
failed due to a timeout, so that if a guest wakes up and starts
processing a command backlog, it will not get stuck in a frozen
state.

* src/qemu/qemu_driver.c (qemuDomainSnapshotCreateDiskActive):
Always issue thaw after a quiesce, even if quiesce failed.
(qemuDomainSnapshotFSThaw): Add a parameter.

This patch fixes a NULL pointer check that was causing SegFault on
some specific configurations. It also reverts commit 59d0c980
that was checking for this value in one place.

A common coding pattern for changing blkio parameters is

  1. virDomainGetBlkioParameters

  2. change one or more params

  3. virDomainSetBlkioParameters

For this to work, it must be possible to roundtrip through
the methods without error. Unfortunately virDomainGetBlkioParameters
will return "" for the deviceWeight parameter for guests by default,
which virDomainSetBlkioParameters will then reject as invalid.

This fixes the handling of "" to be a no-op, and also improves the
error message to tell you what was invalid

How to reproduce:

% valgrind -v --leak-check=full virsh migrate mig \
  qemu+ssh://$dest/system --unsafe

== 8 bytes in 1 blocks are definitely lost in loss record 1 of 28
==    at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==    by 0x3EB7115FB8: xdr_reference (in /lib64/libc-2.12.so)
==    by 0x3EB7115F10: xdr_pointer (in /lib64/libc-2.12.so)
==    by 0x4D1EA84: xdr_remote_string (remote_protocol.c:40)
==    by 0x4D1EAD8: xdr_remote_domain_migrate_prepare3_ret (remote_protocol.c:4772)
==    by 0x4D2FFD2: virNetMessageDecodePayload (virnetmessage.c:382)
==    by 0x4D2789C: virNetClientProgramCall (virnetclientprogram.c:382)
==    by 0x4D0707D: callWithFD (remote_driver.c:4549)
==    by 0x4D070FB: call (remote_driver.c:4570)
==    by 0x4D12AEE: remoteDomainMigratePrepare3 (remote_driver.c:4138)
==    by 0x4CF7BE9: virDomainMigrateVersion3 (libvirt.c:4815)
==    by 0x4CF9432: virDomainMigrate2 (libvirt.c:5454)
==
== LEAK SUMMARY:
==    definitely lost: 8 bytes in 1 blocks
==    indirectly lost: 0 bytes in 0 blocks
==      possibly lost: 0 bytes in 0 blocks
==    still reachable: 126,995 bytes in 1,343 blocks
==         suppressed: 0 bytes in 0 blocks

This patch also fixes the leaks in remoteDomainMigratePrepare and
remoteDomainMigratePrepare2.

* src/libvirt.c (virStorageVolResize): correct comment typo according to
  virStorageVolResizeFlags enum definition.
Signed-off-by: NAlex Jia <ajia@redhat.com>

If no <interface> elements are included in an LXC guest XML
description, then the LXC guest will just see the host's
network interfaces. It is desirable to be able to hide the
host interfaces, without having to define any guest interfaces.

This patch introduces a new feature flag <privnet/> to allow
forcing of a private network namespace for LXC. In the future
I also anticipate that we will add <privuser/> to force a
private user ID namespace.

* src/conf/domain_conf.c, src/conf/domain_conf.h: Add support
  for <privnet/> feature. Auto-set <privnet> if any <interface>
  devices are defined
* src/lxc/lxc_container.c: Honour request for private network
  namespace

Commit e457d5ef adds ability to pass the
default URI using the client configuration file. If the file is not
present, it still accesses the NULL config object causing a segfault.

Caught running "make check".

This function potentially allocates new virCgroup but never
frees it.

Wire up the domain graphics event notifications for SPICE. Adapted
from a RHEL-only patch written by Dan Berrange that used custom
__com.redhat_SPICE events - equivalent events are now available in
upstream QEMU (including a SPICE_CONNECTED event, which was missing in
the __COM.redhat_SPICE version).

* src/qemu/qemu_monitor_json.c: Wire up SPICE graphics events

src/libvirt_private.syms:
  s/virDomainCpuPlacement/virDomainCpuPlacementMode/
src/qemu/qemu_process.c
  def->mem.cur_balloon expects "llu"
--
pushed under build-breaker rule

Systemd has declared that all container virtualization technologies
should set 'container_uuid' to identify themselves.

http://cgit.freedesktop.org/systemd/systemd/commit/?id=09b967eaa51a39dabb7f238927f67bd682466dbc

Currently if the URI passed to virConnectOpen* is NULL, then we

 - Look for LIBVIRT_DEFAULT_URI env var
 - Probe for drivers

This changes it so that

 - Look for LIBVIRT_DEFAULT_URI env var
 - Look for 'uri_default' in $HOME/.libvirt/libvirt.conf
 - Probe for drivers

numad is an user-level daemon that monitors NUMA topology and
processes resource consumption to facilitate good NUMA resource
alignment of applications/virtual machines to improve performance
and minimize cost of remote memory latencies. It provides a
pre-placement advisory interface, so significant processes can
be pre-bound to nodes with sufficient available resources.

More details: http://fedoraproject.org/wiki/Features/numad

"numad -w ncpus:memory_amount" is the advisory interface numad
provides currently.

This patch add the support by introducing a new XML attribute
for <vcpu>. e.g.

  <vcpu placement="auto">4</vcpu>
  <vcpu placement="static" cpuset="1-10^6">4</vcpu>

The returned advisory nodeset from numad will be printed
in domain's dumped XML. e.g.
  <vcpu placement="auto" cpuset="1-10^6">4</vcpu>

If placement is "auto", the number of vcpus and the current
memory amount specified in domain XML will be used for numad
command line (numad uses MB for memory amount):
  numad -w $num_of_vcpus:$current_memory_amount / 1024

The advisory nodeset returned from numad will be used to set
domain process CPU affinity then. (e.g. qemuProcessInitCpuAffinity).

If the user specifies both CPU affinity policy (e.g.
(<vcpu cpuset="1-10,^7,^8">4</vcpu>) and placement == "auto"
the specified CPU affinity will be overridden.

Only QEMU/KVM drivers support it now.

See docs update in patch for more details.

And don't allow to hotplug a usb disk with "device == lun". This
is the missed pieces in previous virtio-scsi patchset:

http://www.redhat.com/archives/libvir-list/2012-February/msg01052.html

With current code, we pass true iff domain is cold booting. However,
if disk is inaccessible and startupPolicy for that disk is set to
'requisite' we have to fail iff cold booting.

This patch adds support for the new tsc-deadline feature flag
and a new model to the supported model list describing the
Intel Sandy Bridge platform.

AMD Bulldozer (or Opteron_G4 as called in QEMU) was added to the list
of cpu models, flags were taken from upstream qemu cpu specifications
and should be sorted by bit values (or first occurence in the feature
specification part of cpu_map.xml).

Based on QEMU upstream commit 885bb0369a4f0abe2c0185178f3cb347cb02cdf1.

Even though we say in documentation setting (tls-)port to -1 is legacy
compat style for enabling autoport, we're roughly doing this for VNC.
However, in case of SPICE auto enable autoport iff both port & tlsPort
are equal -1 as documentation says autoport plays with both.

In qemuDomainDetachNetDevice, detach was being used before it had been
validated. If no matching device was found, this resulted in a
dereference of a NULL pointer.

This behavior was a regression introduced in commit
cf90342b, so it has not been a part of
any official libvirt release.

When host-model and host-passthrouh CPU modes were introduced, qemu
driver was properly modify to update guest CPU definition during
migration so that we use the right CPU at the destination. However,
similar treatment is needed for (managed)save and snapshots since they
need to save the exact CPU so that a domain can be properly restored.
To avoid repetition of such situation, all places that need live XML
share the code which generates it.

As a side effect, this patch fixes error reporting from
qemuDomainSnapshotWriteMetadata().

Thanks to cgroups, providing user vs. system time of the overall
guest is easy to add to our existing API.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_CPU_STATS_USERTIME)
(VIR_DOMAIN_CPU_STATS_SYSTEMTIME): New constants.
* src/util/virtypedparam.h (virTypedParameterArrayValidate)
(virTypedParameterAssign): Enforce checking the result.
* src/qemu/qemu_driver.c (qemuDomainGetPercpuStats): Fix offender.
(qemuDomainGetTotalcpuStats): Implement new parameters.
* tools/virsh.c (cmdCPUStats): Tweak output accordingly.

As documented in linux.git/Documentation/cgroups/cpuacct.txt,
cpuacct.stat returns user and system time in ticks (the same
unit used in times(2)).  It would be a bit nicer if it were like
getrusage(2) and reported timeval contents, or like cpuacct.usage
and in nanoseconds, but we can't be picky.

* src/util/cgroup.h (virCgroupGetCpuacctStat): New function.
* src/util/cgroup.c (virCgroupGetCpuacctStat): Implement it.
(virCgroupGetValueStr): Allow for multi-line files.
* src/libvirt_private.syms (cgroup.h): Export it.

If there is a disk file with a comma in the name, QEmu expects a double
comma instead of a single one (e.g., the file "virtual,disk.img" needs
to be specified as "virtual,,disk.img" in QEmu's command line). This
patch fixes libvirt to work with that feature. Fix RHBZ #801036.

Based on an initial patch by Crístian Viana.

* src/util/buf.h (virBufferEscape): Alter signature.
* src/util/buf.c (virBufferEscape): Add parameter.
(virBufferEscapeSexpr): Fix caller.
* src/qemu/qemu_command.c (qemuBuildRBDString): Likewise.  Also
escape commas in file names.
(qemuBuildDriveStr): Escape commas in file names.
* docs/schemas/basictypes.rng (absFilePath): Relax RNG to allow
commas in input file names.
* tests/qemuxml2argvdata/*-disk-drive-network-sheepdog.*: Update
test.
Signed-off-by: NEric Blake <eblake@redhat.com>

this was raised by our hindi localization team
chandan kumar <chandankumar.093047@gmail.com>

We found few more AMD-specific features in cpu64-rhel* models that
made it impossible to start qemu guest on Intel host (with this
setting) even though qemu itself starts correctly with them.
This impacts one test, thus the fix in tests/cputestdata/.

virNetworkDNSHostsDefParseXML was calling VIR_ALLOC(def->hosts) if
def->hosts was NULL. This is a waste of time, though, since
VIR_REALLOC_N is called a few lines further down, prior to any use of
def->hosts. (initializing def->nhosts to 0 is also redundant, because
the newly allocated memory will always be cleared to all 0's anyway).

If user hasn't supplied any tlsPort we default to setting it
to zero in our internal structure. However, when building command
line we test it against -1 which is obviously wrong.