In preparation for code re-organization, rename the Macvtap
management APIs to have the following patterns

  virNetDevMacVLanXXXXX     - macvlan/macvtap interface management
  virNetDevVPortProfileXXXX - virtual port profile management

* src/util/macvtap.c, src/util/macvtap.h: Rename APIs
* src/conf/domain_conf.c, src/network/bridge_driver.c,
  src/qemu/qemu_command.c, src/qemu/qemu_command.h,
  src/qemu/qemu_driver.c, src/qemu/qemu_hotplug.c,
  src/qemu/qemu_migration.c, src/qemu/qemu_process.c,
  src/qemu/qemu_process.h: Update for renamed APIs

Add routines to generate -numa QEMU command line option based on
<numa> ... </numa> XML specifications.
Signed-off-by: NBharata B Rao <bharata@linux.vnet.ibm.com>

This patch adds XML definitions for guest NUMA specification and contains
routines to parse the same. The guest NUMA specification looks like this:

<cpu>
        ...
        <topology sockets='2' cores='4' threads='2'/>
        <numa>
                <cell cpus='0-7' memory='512000'/>
                <cell cpus='8-15' memory='512000'/>
        </numa>
        ...
</cpu>
Signed-off-by: NBharata B Rao <bharata@linux.vnet.ibm.com>

For whatever reason, the kernel allows you to create a regular
file named /dev/sdc.12345; although this file will disappear the
next time devtmpfs is remounted.  If you let libvirt generate
the name of the external snapshot for a disk image originally
using the block device /dev/sdc, then the domain will be rendered
unbootable once the qcow2 file is lost on the next devtmpfs
remount.  In this case, the user should have used 'virsh
snapshot-create --xmlfile' or 'virsh snapshot-create-as --diskspec'
to specify the name for the qcow2 file in a sane location, rather
than relying on libvirt generating a name that is most likely to
be wrong.  We can help avoid naive mistakes by enforcing that
the user provide the external name for any backing file that is
not a regular file.

* src/conf/domain_conf.c (virDomainSnapshotAlignDisks): Only
generate names if backing file exists as regular file.
Reported by MATSUDA Daiki.

I missed adding virNetServerGetDBusConn() to libvirtd_private.syms
in commit b8adfcc6, which didn't cause a problem in 0.9.6 but
results in this build error in 0.9.7

libvirtd-remote.o: In function `remoteDispatchAuthPolkit':
remote.c:(.text+0x188dd): undefined reference to `virNetServerGetDBusConn'

Due to the asynchronous nature of streams, we might continue to
receive some stream packets from the server even after we have
shutdown the stream on the client side. These should be discarded
silently, rather than raising an error in the RPC layer.

* src/rpc/virnetclient.c: Discard stream data silently

Add a new virNetClientSendNonBlock which returns 2 on
full send, 1 on partial send, 0 on no send, -1 on error

If a partial send occurs, then a subsequent call to any
of the virNetClientSend* APIs will finish any outstanding
I/O.

TODO: the virNetClientEvent event handler could be used
to speed up completion of partial sends if an event loop
is present.

* src/rpc/virnetsocket.h, src/rpc/virnetsocket.c: Add new
  virNetSocketHasPendingData() API to test for cached
  data pending send.
* src/rpc/virnetclient.c, src/rpc/virnetclient.h: Add new
  virNetClientSendNonBlock() API to send non-blocking API

* src/rpc/virnetclient.c: Add helper for setting I/O callback events

Stop multiplexing virNetClientSend for two different purposes,
instead add virNetClientSendWithReply and virNetClientSendNoReply

* src/rpc/virnetclient.c, src/rpc/virnetclient.h: Replace
  virNetClientSend with virNetClientSendWithReply and
  virNetClientSendNoReply
* src/rpc/virnetclientprogram.c, src/rpc/virnetclientstream.c:
  Update for new API names

Remove some duplication by pulling the code for passing the
buck out into a helper method

* src/rpc/virnetclient.c: Introduce virNetClientIOEventLoopPassTheBuck

Instead of inferring whether the buck is held from the waitDispatch
pointer, use an explicit 'bool haveTheBuck' field

* src/rpc/virnetclient.c: Explicitly track the buck

Directly messing around with the linked list is potentially
dangerous. Introduce some helper APIs to deal with list
manipulating the list

* src/rpc/virnetclient.c: Create linked list handlers

Signed-off-by: NEli Qiao <taget@linux.vnet.ibm.com>

This improves the support for qemu rbd devices by adding support for a few
key features (e.g., authentication) and cleaning up the way in which
rbd configuration options are passed to qemu.

An <auth> member of the disk source xml specifies how librbd should
authenticate. The username attribute is the Ceph/RBD user to authenticate as.
The usage or uuid attributes specify which secret to use. Usage is an
arbitrary identifier local to libvirt.

The old RBD support relied on setting an environment variable to
communicate information to qemu/librbd.  Instead, pass those options
explicitly to qemu.  Update the qemu argument parsing and tests
accordingly.
Signed-off-by: NSage Weil <sage@newdream.net>
Signed-off-by: NJosh Durgin <josh.durgin@dreamhost.com>

Replacing the strchr call with two variables through a strstr call.
Calling strchr with two variables triggers a gcc 4.3/4.4
bug when used in combination with -Wlogical-op and at least -O1.

include stdint.h to fix the build

The ifaceSetMac and ifaceGetMac APIs duplicate the functionality
of the virNetDevSetMAC and virNetDevGetMAC APIs, but returning
errno's instead of raising errors.

* src/util/interface.c, src/util/interface.h: Remove
  ifaceSetMac and ifaceGetMac APIs, adjusting callers
  for new error behaviour

The ifaceUp, ifaceDown, ifaceCtrl & ifaceIsUp APIs can be replaced
with calls to virNetDevSetOnline and virNetDevIsOnline

* src/util/interface.c, src/util/interface.h: Delete ifaceUp,
  ifaceDown, ifaceCtrl & ifaceIsUp
* src/nwfilter/nwfilter_gentech_driver.c, src/util/macvtap.c:
  Update to use virNetDevSetOnline and virNetDevIsOnline

Move the virNetDevSetName and virNetDevSetNamespace APIs out
of LXC's veth.c and into virnetdev.c.

Move the remaining content of the file to src/util/virnetdevveth.c

* src/lxc/veth.c: Rename to src/util/virnetdevveth.c
* src/lxc/veth.h: Rename to src/util/virnetdevveth.h
* src/util/virnetdev.c, src/util/virnetdev.h: Add
  virNetDevSetName and virNetDevSetNamespace
* src/lxc/lxc_container.c, src/lxc/lxc_controller.c,
  src/lxc/lxc_driver.c: Update include paths

The src/lxc/veth.c file contains APIs for managing veth devices,
but some of the APIs duplicate stuff from src/util/virnetdev.h.
Delete thed duplicate APIs and rename the remaining ones to
follow virNetDevVethXXXX

* src/lxc/veth.c, src/lxc/veth.h: Rename APIs & delete duplicates
* src/lxc/lxc_container.c, src/lxc/lxc_controller.c,
  src/lxc/lxc_driver.c: Update for API renaming

The src/util/network.c file is a dumping ground for many different
APIs. Split it up into 5 pieces, along functional lines

 - src/util/virnetdevbandwidth.c: virNetDevBandwidth type & helper APIs
 - src/util/virnetdevvportprofile.c: virNetDevVPortProfile type & helper APIs
 - src/util/virsocketaddr.c: virSocketAddr and APIs
 - src/conf/netdev_bandwidth_conf.c: XML parsing / formatting
   for virNetDevBandwidth
 - src/conf/netdev_vport_profile_conf.c: XML parsing / formatting
   for virNetDevVPortProfile

* src/util/network.c, src/util/network.h: Split into 5 pieces
* src/conf/netdev_bandwidth_conf.c, src/conf/netdev_bandwidth_conf.h,
  src/conf/netdev_vport_profile_conf.c, src/conf/netdev_vport_profile_conf.h,
  src/util/virnetdevbandwidth.c, src/util/virnetdevbandwidth.h,
  src/util/virnetdevvportprofile.c, src/util/virnetdevvportprofile.h,
  src/util/virsocketaddr.c, src/util/virsocketaddr.h: New pieces
* daemon/libvirtd.h, daemon/remote.c, src/conf/domain_conf.c,
  src/conf/domain_conf.h, src/conf/network_conf.c,
  src/conf/network_conf.h, src/conf/nwfilter_conf.h,
  src/esx/esx_util.h, src/network/bridge_driver.c,
  src/qemu/qemu_conf.c, src/rpc/virnetsocket.c,
  src/rpc/virnetsocket.h, src/util/dnsmasq.h, src/util/interface.h,
  src/util/iptables.h, src/util/macvtap.c, src/util/macvtap.h,
  src/util/virnetdev.h, src/util/virnetdevtap.c,
  tools/virsh.c: Update include files

The virtual port profile parsing/formatting APIs do not
correctly handle unknown profile type strings/numbers.
They behave as a no-op, instead of raising an error

* src/util/network.c, src/util/network.h: Fix error
  handling of port profile APIs
* src/conf/domain_conf.c, src/conf/network_conf.c: Update
  for API changes

Rename the virVirtualPortProfileParams struct to be
virNetDevVPortProfile, and rename the APIs to match
this prefix.

* src/util/network.c, src/util/network.h: Rename port profile
  APIs
* src/conf/domain_conf.c, src/conf/domain_conf.h,
  src/conf/network_conf.c, src/conf/network_conf.h,
  src/network/bridge_driver.c, src/qemu/qemu_hotplug.c,
  src/util/macvtap.c, src/util/macvtap.h: Update for
  renamed APIs/structs

Hi

Commit c31d23a7 removed the "conn"
parameter from qemuPhysIfaceConnect(), but it's still used if
WITH_MACVTAP is false.  Also, it's still mentioned in the comment
above the function:

/**
 * qemuPhysIfaceConnect:
 * @def: the definition of the VM (needed by 802.1Qbh and audit)
 * @conn: pointer to virConnect object
 * @driver: pointer to the qemud_driver
 * @net: pointer to he VM's interface description with direct device type
 * @qemuCaps: flags for qemu
 *
 * Returns a filedescriptor on success or -1 in case of error.
 */
int
qemuPhysIfaceConnect(virDomainDefPtr def,
                     struct qemud_driver *driver,
                     virDomainNetDefPtr net,
                     virBitmapPtr qemuCaps,
                     enum virVMOperationType vmop)
{
    int rc;
#if WITH_MACVTAP
[...]
#else
    (void)def;
    (void)conn;
    (void)net;
    (void)qemuCaps;
    (void)driver;
    (void)vmop;
    qemuReportError(VIR_ERR_INTERNAL_ERROR,
                    "%s", _("No support for macvtap device"));
    rc = -1;
#endif
    return rc;
}

--
Michael Wood <esiotrot@gmail.com>

From f4fc43b4111a4c099395c55902e497b8965e2b53 Mon Sep 17 00:00:00 2001
From: Michael Wood <esiotrot@gmail.com>
Date: Sat, 12 Nov 2011 13:37:53 +0200
Subject: [PATCH] Fix build without MACVTAP.

which would blow away all volumes. Honor VIR_STORAGE_POOL_BUILD_OVERWRITE
to force a rebuild.

This was caught by libvirt-tck's storage/110-disk-pool.t.

Qemu will be the first driver to make use of a typed string in the
next round of additions.  Separate out the trivial addition.

* src/qemu/qemu_driver.c (qemudSupportsFeature): Advertise feature.
(qemuDomainGetBlkioParameters, qemuDomainGetMemoryParameters)
(qemuGetSchedulerParametersFlags, qemudDomainBlockStatsFlags):
Allow typed strings flag where trivially supported.

Send and receive string typed parameters across RPC.  This also
completes the back-compat mentioned in the previous patch - the
only time we have an older client talking to a newer server is
if RPC is in use, so filtering out strings during RPC prevents
returning an unknown type to the older client.

* src/remote/remote_protocol.x (remote_typed_param_value): Add
another union value.
* daemon/remote.c (remoteDeserializeTypedParameters): Handle
strings on rpc.
(remoteSerializeTypedParameters): Likewise; plus filter out
strings when replying to older clients.  Adjust callers.
* src/remote/remote_driver.c (remoteFreeTypedParameters)
(remoteSerializeTypedParameters)
(remoteDeserializeTypedParameters): Handle strings on rpc.
* src/rpc/gendispatch.pl: Properly clean up typed arrays.
* src/remote_protocol-structs: Update.
Based on an initial patch by Hu Tao, with feedback from
Daniel P. Berrange.
Signed-off-by: NEric Blake <eblake@redhat.com>

This allows strings to be transported between client and server
in the context of name-type-value virTypedParameter functions.
For compatibility,

    o new clients will not send strings to old servers, based on
      a feature check
    o new servers will not send strings to old clients without the
      flag VIR_TYPED_PARAM_STRING_OKAY; this will be enforced at
      the RPC layer in the next patch, so that drivers need not
      worry about it in general.  The one exception is that
      virDomainGetSchedulerParameters lacks a flags argument, so
      it must not return a string; drivers that forward that
      function on to virDomainGetSchedulerParametersFlags will
      have to pay attention to the flag.
    o the flag VIR_TYPED_PARAM_STRING_OKAY is set automatically,
      based on a feature check (so far, no driver implements it),
      so clients do not have to worry about it

Future patches can then enable the feature on a per-driver basis.

This patch also ensures that drivers can blindly strdup() field
names (previously, a malicious client could stuff 80 non-NUL bytes
into field and cause a read overrun).

* src/libvirt_internal.h (VIR_DRV_FEATURE_TYPED_PARAM_STRING): New
driver feature.
* src/libvirt.c (virTypedParameterValidateSet)
(virTypedParameterSanitizeGet): New helper functions.
(virDomainSetMemoryParameters, virDomainSetBlkioParameters)
(virDomainSetSchedulerParameters)
(virDomainSetSchedulerParametersFlags)
(virDomainGetMemoryParameters, virDomainGetBlkioParameters)
(virDomainGetSchedulerParameters)
(virDomainGetSchedulerParametersFlags, virDomainBlockStatsFlags):
Use them.
* src/util/util.h (virTypedParameterArrayClear): New helper
function.
* src/util/util.c (virTypedParameterArrayClear): Implement it.
* src/libvirt_private.syms (util.h): Export it.
Based on an initial patch by Hu Tao, with feedback from
Daniel P. Berrange.
Signed-off-by: NEric Blake <eblake@redhat.com>

Add virnetdev.h,virnetdevbridge.h,virnetdevtap.h to private symbols,
since debian linker no longer allows transitive link resolution
Signed-off-by: NEli Qiao <taget@linux.vnet.ibm.com>

This reverts commit ef1065cf; see also this bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=751900

In qemu 0.15.1 and earlier, during migration to file, the
qemu_savevm_state_begin and qemu_savevm_state_iterate methods
will both process as much migration data as possible until either

  1. The file descriptor returns EAGAIN
  2. The bandwidth rate limit is reached

If we set the rate limit to ULONG_MAX, test 2 never becomes true. We're
passing a plain file descriptor to QEMU and POSIX does not support EAGAIN on
regular files / block devices, so test 1 never becomes true either.

In the 'virsh save --bypass-cache' case, we pass a pipe instead of a
regular fd, but using a pipe adds I/O overhead, so always passing a
pipe just so qemu can see EAGAIN doesn't seem nice.

The ultimate fix needs to come from qemu - background migration must
respect asynchronous abort requests, or else periodically return
control to the main handling loop without an EAGAIN and without
waiting to hit an insanely large amount of data.  But until a
version of qemu is fixed to support "unlimited" data rates while
still allowing cancellation, the best we can do is avoid the
automatic use of unlimited rates from within libvirt (users can
still explicitly change the migration rates, if they are aware that
they are giving up the ability to cancel a job).

Reverting the lone use of QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX is
the simplest patch; this slows migration back down to a default
32M/sec cap, but also ensures that the main qemu processing loop
will still be responsive to cancellation requests.  Hopefully
upstream qemu will provide us a means of safely using unlimited
speed, including a runtime probe of that capability.

* src/qemu/qemu_migration.c (qemuMigrationToFile): Revert attempt
to use unlimited migration bandwidth when migrating to file.
Signed-off-by: NDaniel Veillard <veillard@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

steps to reproduce:

1. having a network xml file(named default.xml) like this one:

<network>
  <name>default</name>
  <uuid>c5322c4c-81d0-4985-a363-ad6389780d89</uuid>
  <bridge name="virbr0" />
  <forward/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254" />
    </dhcp>
  </ip>
</network>

in /etc/libvirt/qemu/networks/, and mark it as autostart:

$ ls -l /etc/libvirt/qemu/networks/autostart
total 0
lrwxrwxrwx 1 root root 14 Oct 12 14:02 default.xml -> ../default.xml

2. start libvirtd and the device virbr0 is not automatically up.

The reason is that the function virNetDevExists is now returns 1 if
the device exists, comparing to the former one returns 0 if the device
exists. But with only this fix will cause a segmentation fault(the same
steps as above)  that is fixed by the second chunk of code.

* .gnulib: Update to latest, for improved syntax-check.
* src/lxc/lxc_container.c (includes): Drop unused include.
* src/network/bridge_driver.c: Likewise.
* src/node_device/node_device_linux_sysfs.c: Likewise.
* src/openvz/openvz_driver.c: Likewise.
* src/qemu/qemu_conf.c: Likewise.
* src/storage/storage_backend_iscsi.c: Likewise.
* src/storage/storage_backend_mpath.c: Likewise.
* src/uml/uml_conf.c: Likewise.
* src/uml/uml_driver.c: Likewise.

  CC     libvirt_driver_xenapi_la-xenapi_driver.lo
xenapi/xenapi_driver.c: In function 'xenapiDomainGetVcpus':
xenapi/xenapi_driver.c:1209:21: error: variable 'cpus' set but not used [-Werror=unused-but-set-variable]

* src/xenapi/xenapi_driver.c (xenapiDomainGetVcpus): Silence
compiler warning.

Remove the code that instantiates network filters on direct type
of interfaces. The parser already does not accept it.

It's not worth even worrying about a temporary file, unless we
ever expect the script to exceed maximum command-line argument
length limits.

* src/nwfilter/nwfilter_ebiptables_driver.c (ebiptablesExecCLI):
Run the commands as an argument to /bin/sh, rather than worrying
about a temporary file.
(ebiptablesWriteToTempFile): Delete unused function.

If /tmp is mounted with the noexec flag (common on security-conscious
systems), then nwfilter will fail to initialize, because we cannot
run any temporary script via virRun("/tmp/script"); but we _can_
use "/bin/sh /tmp/script".  For that matter, using /tmp risks collisions
with other unrelated programs; we already have /var/run/libvirt as a
dedicated temporary directory for use by libvirt.

* src/nwfilter/nwfilter_ebiptables_driver.c
(ebiptablesWriteToTempFile): Use internal directory, not /tmp;
drop attempts to make script executable; and detect close error.
(ebiptablesExecCLI): Switch to virCommand, and invoke the shell to
read the script, rather than requiring an executable script.

Rename virBandwidth to virNetDevBandwidth, and virRate to
virNetDevBandwidthRate.

* src/util/network.c, src/util/network.h: Rename bandwidth
  structs and APIs
* src/conf/domain_conf.c, src/conf/domain_conf.h,
  src/conf/network_conf.c, src/conf/network_conf.h,
  src/lxc/lxc_driver.c, src/network/bridge_driver.c,
  src/qemu/qemu_command.c, src/util/macvtap.c,
  src/util/macvtap.h, tools/virsh.c: Update for API changes.

The socket address APIs in src/util/network.h either take the
form  virSocketAddrXXX, virSocketXXX or virSocketXXXAddr.

Sanitize this so everything is virSocketAddrXXXX, and ensure
that the virSocketAddr parameter is always the first one.

* src/util/network.c, src/util/network.h: Santize socket
  address API naming
* src/conf/domain_conf.c, src/conf/network_conf.c,
  src/conf/nwfilter_conf.c, src/network/bridge_driver.c,
  src/nwfilter/nwfilter_ebiptables_driver.c,
  src/nwfilter/nwfilter_learnipaddr.c,
  src/qemu/qemu_command.c, src/rpc/virnetsocket.c,
  src/util/dnsmasq.c, src/util/iptables.c,
  src/util/virnetdev.c, src/vbox/vbox_tmpl.c: Update for
  API renaming

Following the renaming of the bridge management APIs, we can now
split the source file into 3 corresponding pieces

 * src/util/virnetdev.c: APIs for any type of network interface
 * src/util/virnetdevbridge.c: APIs for bridge interfaces
 * src/util/virnetdevtap.c: APIs for TAP interfaces

* src/util/virnetdev.c, src/util/virnetdev.h,
  src/util/virnetdevbridge.c, src/util/virnetdevbridge.h,
  src/util/virnetdevtap.c, src/util/virnetdevtap.h: Copied
  from bridge.{c,h}
* src/util/bridge.c, src/util/bridge.h: Split into 3 pieces
* src/lxc/lxc_driver.c, src/network/bridge_driver.c,
  src/openvz/openvz_driver.c, src/qemu/qemu_command.c,
  src/qemu/qemu_conf.h, src/uml/uml_conf.c, src/uml/uml_conf.h,
  src/uml/uml_driver.c: Update #include directives