https://bugzilla.redhat.com/show_bug.cgi?id=589465

Some guests (eg with badly configured grub, or Windows' installation cd)
require quick response from the console user. That's why we have a
"launchPaused" option in vdsm.

To implement it via libvirt, we need to ask libvirt not to call
qemuMonitorStartCPUs() after starting qemu.  Calling virDomainStop
immediately after the domain is up is inherently raceful.

* src/qemu/qemu_driver.c (qemudStartVMDaemon): Add new parameter;
all callers adjusted.
(qemudDomainCreate): Implement support for new flag.

* include/libvirt/libvirt.h.in (virDomainCreateFlags): Add
VIR_DOMAIN_START_PAUSED.
* src/libvirt.c (virDomainCreateXML): Update documentation.
* src/lxc/lxc_driver.c (lxcDomainCreateAndStart): Reject new flag
as unimplemented.
* src/opennebula/one_driver.c (oneDomainCreateAndStart):
Likewise.
* src/openvz/openvz_driver.c (openvzDomainCreateXML): Likewise.
* src/phyp/phyp_driver.c (phypDomainCreateAndStart): Likewise.
* src/qemu/qemu_driver.c (qemudDomainCreate): Likewise.
* src/test/test_driver.c (testDomainCreateXML): Likewise.
* src/uml/uml_driver.c (umlDomainCreate): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainCreateXML): Likewise.
* src/xen/xend_internal.c (xenDaemonCreateXML): Likewise.
* src/xenapi/xenapi_driver.c (xenapiDomainCreateXML): Likewise.

* This patch is a modification of a patch submitted by Nigel Jones.
  It fixes several memory leaks on device addition/removal:

1. Free the virNodeDeviceDefPtr in udevAddOneDevice if the return
   value is non-zero

2. Always release the node device reference after the device has been
   processed.

* Refactored for better readability per the suggestion of clalance

* Fix broken rng schema
* Add test input & output files

* Use virBufferEscapeString() per Dan B.

The virtio serial changes broke the test suite because they forgot
to add the new address attribute to the domain XML schema. The
xml2xml test also broke because the XML no longer roundtrips. This
is due to testing of auto-addition of <controller> elements. Split
that test case off into a separate XML file to avoid breakage

* docs/schemas/domain.rng: Allow port number for virtio serial addresses
* tests/qemuxml2argvdata/qemuxml2argv-channel-virtio.args,
  tests/qemuxml2argvdata/qemuxml2argv-channel-virtio.xml: Revert to
  a simple config to avoid breaking xml2xml test
* tests/qemuxml2argvdata/qemuxml2argv-channel-virtio-auto.xml,
  tests/qemuxml2argvdata/qemuxml2argv-channel-virtio-auto.args: Add
  complex test case for auto-controller addition for xml2argv test
* tests/qemuxml2argvtest.c: Add channel-virtio-auto test

A look at the QEMU source revealed the missing bits of info about
the VPC file format, so we can enable this now

* src/util/storage_file.c: Enable VPC format, providing version
  and disk size offset fields

When an attempt to hotplug a PCI device to a guest fails,
the device was left attached to pci-stub. It is neccessary
to reset the device and then attach it to the host driver
again.

* src/qemu/qemu_driver.c: Reattach PCI device to host if
  hotadd fails

The restore code is done in places where errors cannot be
raised, since they will overwrite over pre-existing errors.

* src/security/security_selinux.c: Only warn about failures
  in label restore, don't report errors

Any output at all from device_add indicates an error in the
command execution. Thus it needs to check for reply != ""

* src/qemu/qemu_monitor_text.c: Fix reply check for errors
  to treat any output as an error

HAL is deprecated and UDEV is the future. Thus if both
options are compiled, we should prefer use of UDEV over
HAL

* src/node_device/node_device_driver.c: Switch init
  order to try UDEV first, then HAL

When SELinux is running in MLS mode, libvirtd will have a
different security level to the VMs. For libvirtd to be
able to connect to the monitor console, the client end of
the UNIX domain socket needs a different label. This adds
infrastructure to set the socket label via the security
driver framework

* src/qemu/qemu_driver.c: Call out to socket label APIs in
  security driver
* src/qemu/qemu_security_stacked.c: Wire up socket label
  drivers
* src/security/security_driver.h: Define security driver
  entry points for socket labelling
* src/security/security_selinux.c: Set socket label based on
  VM label

The network driver is not doing correct checking for
duplicate UUID/name values. This introduces a new method
virNetworkObjIsDuplicate, based on the previously
written virDomainObjIsDuplicate.

* src/conf/network_conf.c, src/conf/network_conf.c,
  src/libvirt_private.syms: Add virNetworkObjIsDuplicate,
* src/network/bridge_driver.c: Call virNetworkObjIsDuplicate
  for checking uniqueness of uuid/names

The storage pool driver is mistakenly using the error code
VIR_ERR_INVALID_STORAGE_POOL which is for diagnosing invalid
pointers. This patch switches it to use VIR_ERR_NO_STORAGE_POOL
which is the correct code for cases where the storage pool does
not exist

* src/storage/storage_driver.c: Replace VIR_ERR_INVALID_STORAGE_POOL
  with VIR_ERR_NO_STORAGE_POOL

The storage pool driver is not doing correct checking for
duplicate UUID/name values. This introduces a new method
virStoragePoolObjIsDuplicate, based on the previously
written virDomainObjIsDuplicate.

* src/conf/storage_conf.c, src/conf/storage_conf.c,
  src/libvirt_private.syms: Add virStoragePoolObjIsDuplicate,
* src/storage/storage_driver.c: Call virStoragePoolObjIsDuplicate
  for checking uniqueness of uuid/names

The domain parsing code would auto-add a virtio serial controller
if it saw any virtio serial channel defined. Unfortunately it
always added a controller with index=0, even if the channel address
specified an index != 0. It only added one controller, even if
multiple controllers were referenced by channels. Finally, it let
the ports+vectors parameters initialize to zero instead of -1, which
prevented the controllers accepting any ports.

* src/conf/domain_conf.c: Initialize ports+vectors when adding
  virtio serial controllers. Add all neccessary virtio serial
  controllers, instead of hardcoding controller 0
* qemuxml2argvdata/qemuxml2argv-channel-virtio.args,
  qemuxml2argvdata/qemuxml2argv-channel-virtio.xml: Expand to
  test controller auto-add behaviour

To ensure that the device addressing scheme is stable across
hotplug/unplug, all virtio serial channels needs to have an
associated port number in their address. This is then specified
to QEMU using the nr=NNN parameter

* src/conf/domain_conf.c, src/conf/domain_conf.h: Parsing
  for port number in vioserial address types.
* src/qemu/qemu_conf.c: Set 'nr=NNN' parameter with virtio
  serial port number
* tests/qemuxml2argvdata/qemuxml2argv-channel-virtio.args,
  tests/qemuxml2argvdata/qemuxml2argv-channel-virtio.xml: Expand
  data set to ensure coverage of port addressing

QEMU upstream decided against adding a 'reason' field to
the block IO event in QMP. Disable this code to remove a
annoying warning message. It will be renabled when the
error string reason is re-introduced in QEMU

When libvirtd exits it is leaving UNIX domain sockets on
the filesystem. These need to be removed.

The qemudInitPaths() method has signficant code churn to
switch from using a pre-allocated buffer on the stack, to
dynamically allocating on the heap.

* daemon/libvirtd.c, daemon/libvirtd.h: Store a reference
  to the UNIX domain socket path and unlink it on shutdown

Signed-off-by: NEric Blake <eblake@redhat.com>

Refactor to update AppArmor security driver to adjust profile for
save/restore. This addresses the following bugs:

 https://bugzilla.redhat.com/show_bug.cgi?id=529363
 https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/457716

Adjust args to qemudStartVMDaemon() to also specify path to stdin_fd,
so this can be passed to the AppArmor driver via SetSecurityAllLabel().

This updates all calls to qemudStartVMDaemon() as well as setting up
the non-AppArmor security driver *SetSecurityAllLabel() declarations
for the above. This is required for the following
"apparmor-fix-save-restore" patch since AppArmor resolves the passed
file descriptor to the pathname given to open().

* .mailmap: Map a stray commit-author email address to the canonical one.
Reported by Justin Clift.

See https://bugzilla.redhat.com/show_bug.cgi?id=599091

Saving a paused 512MB domain took 3m47s with the old block size of 512
bytes. Changing the block size to 1024*1024 decreased the time to 56
seconds. (Doubling again to 2048*1024 yielded 0 improvement; lowering
to 512k increased the save time to 1m10s, about 20%)

The pointer to the xml describing the domain is saved into an object
prior to calling VIR_REALLOC_N() to make the size of the memory it
points to a multiple of QEMU_MONITOR_MIGRATE_TO_FILE_BS. If that
operation needs to allocate new memory, the pointer that was saved is
no longer valid.

To avoid this situation, adjust the size *before* saving the pointer.

(This showed up when experimenting with very large values of
QEMU_MONITOR_MIGRATE_TO_FILE_BS).

Fixes for issues in commit 211dd1e9 noted by by Jim Meyering.

1. Allocate content buffer of size content_length + 1 to ensure
   NUL-termination.
2. Limit content buffer size to 64k
3. Fix whitespace issue

V2:
  - Add comment to clarify allocation of content buffer
  - Add ATTRIBUTE_NONNULL where appropriate
  - User NULLSTR macro

* autobuild.sh: Replace 'set -o pipefail' with POSIX alternative.
Reported by Matthias Bolte.

There are cases when a response from xend can exceed 4096 bytes, in
which case anything beyond 4096 is ignored. This patch changes the
current fixed-size, stack-allocated buffer to a dynamically allocated
buffer based on Content-Length in HTTP header.

* It appears that the udev event for HBA creation arrives before the
  associated sysfs data is fully populated, resulting in bogus data
  for the nodedev entry until the entry is refreshed.  This problem is
  particularly troublesome when creating NPIV vHBAs because it results
  in libvirt failing to find the newly created adapter and waiting for
  the full timeout period before erroneously failing the create
  operation.  This patch forces an update before any attempt to use
  any scsi_host nodedev entry.

* Set return value in error cases
* Clarify error message when parent device is not vport capable

This patch that adds support for configuring 802.1Qbg and 802.1Qbh
switches. The 802.1Qbh part has been successfully tested with real
hardware. The 802.1Qbg part has only been tested with a (dummy)
server that 'behaves' similarly to how we expect lldpad to 'behave'.

The following changes were made during the development of this patch:

 - Merging Scott's v13-pre1 patch
 - Fixing endptr related bug while using virStrToLong_ui() pointed out
   by Jim Meyering
 - Addressing Jim Meyering's comments to v11
 - requiring mac address to the vpDisassociateProfileId() function to
   pass it further to the 802.1Qbg disassociate part (802.1Qbh untouched)
 - determining pid of lldpad daemon by reading it from /var/run/libvirt.pid
   (hardcode as is hardcode alson in lldpad sources)
 - merging netlink send code for kernel target and user space target
   (lldpad) using one function nlComm() to send the messages
 - adding a select() after the sending and before the reading of the
   netlink response in case lldpad doesn't respond and so we don't hang
 - when reading the port status, in case of 802.1Qbg, no status may be
   received while things are 'in progress' and only at the end a status
   will be there.
 - when reading the port status, use the given instanceId and vf to pick
   the right IFLA_VF_PORT among those nested under IFLA_VF_PORTS.
 - never sending nor parsing IFLA_PORT_SELF type of messages in the
   802.1Qbg case
 - iterating over the elements in a IFLA_VF_PORTS to pick the right
   IFLA_VF_PORT by either IFLA_PORT_PROFILE and given profileId
   (802.1Qbh) or IFLA_PORT_INSTANCE_UUID and given instanceId (802.1Qbg)
   and reading the current status in IFLA_PORT_RESPONSE.
 - recycling a previous patch that adds functionality to interface.c to
   - get the vlan identifier on an interface
   - get the flags of an interface and some convenience function to
     check whether an interface is 'up' or not (not currently used here)
 - adding function to determine the root physical interface of an
   interface. For example if a macvtap is linked to eth0.100, it will
   find eth0. Also adding a function that finds the vlan on the 'way to
   the root physical interface'
 - conveying the root physical interface name and index in case of 802.1Qbg
 - conveying mac address of macvlan device and vlan identifier in
   IFLA_VFINFO_LIST[ IFLA_VF_INFO[ IFLA_VF_MAC(mac), IFLA_VF_VLAN(vlan) ] ]
   to (future) lldpad via netlink
  - To enable build with --without-macvtap rename the
    [dis|]associatePortProfileId functions, prepend 'vp' before their
    name and make them non-static functions.
  - Renaming variable multicast to nltarget_kernel and inverting
    the logic
  - Addressing Jim Meyering's comments; this also touches existing
    code for example for correcting indentation of break statements or
    simplification of switch statements.
  - Renamed occurrencvirVirtualPortProfileDef to virVirtualPortProfileParamses
  - 802.1Qbg part prepared for sending a RTM_SETLINK and getting
    processing status back plus a subsequent RTM_GETLINK to
    get IFLA_PORT_RESPONSE.
    Note: This interface for 802.1Qbg may still change
  - [David Allan] move getPhysfn inside IFLA_VF_PORT_MAX to avoid
compiler
    warning when latest if_link.h isn't available
  - move from Stefan's 802.1Qb{g|h} XML v8 to v9
  - move hostuuid and vf index calcs to inside doPortProfileOp8021Qbh
  - remove debug fprintfs
  - use virGetHostUUID (thanks Stefan!)
  - fix compile issue when latest if_link.h isn't available
  - change poll timeout to 10s, at 1/8 intervals
     - if polling times out, log msg and return -ETIMEDOUT
  - Add Stefan's code for getPortProfileStatus
  - Poll for up to 2 secs for port-profile status, at 1/8 sec intervals:
     - if status indicates error, abort openMacvtapTap
     - if status indicates success, exit polling
     - if status is "in-progress" after 2 secs of polling, exit
       polling loop silently, without error

My patch finishes out the 802.1Qbh parts, which Stefan had mostly complete.
I've tested using the recent kernel updates for VF_PORT netlink msgs and
enic for Cisco's 10G Ethernet NIC.  I tested many VMs, each with several
direct interfaces, each configured with a port-profile per the XML.  VM-to-VM,
and VM-to-external work as expected.  VM-to-VM on same host (using same NIC)
works same as VM-to-VM where VMs are on diff hosts.  I'm able to change
settings on the port-profile while the VM is running to change the virtual
port behaviour.  For example, adjusting a QoS setting like rate limit.  All
VMs with interfaces using that port-profile immediatly see the effect of the
change to the port-profile.

I don't have a SR-IOV device to test so source dev is a non-SR-IOV device,
but most of the code paths include support for specifing the source dev and
VF index.  We'll need to complete this by discovering the PF given the VF
linkdev.  Once we have the PF, we'll also have the VF index.  All this info-
mation is available from sysfs.

* src/util/bitmap.c (virBitmapAlloc): Tighten sanity check.

Fedora bug https://bugzilla.redhat.com/show_bug.cgi?id=598272

Some files under /sys/bus/usb/devices/ have the format 'usbX', where
X is the USB bus number. Use STRPREFIX to correctly parse the bus numbers.

This is the corresponding patch for the virsh man page (virsh.pod).

When creating pools from dedicated disks, the existing pool-define-as
and pool-create-as commands are a bit non-optimal.

Ideally, a person would be able to specify all of the required options
directly on the command line instead of having to edit the XML.

At the moment, there is no way to specify the format type (ie gpt) so it
gets included in the XML the pool is constructed with.

Please find attached a simple (tested) patch to add an optional
"--source-format 'type'" to virsh.  This is patched against current git
master and will apply cleanly.

Also created a Red Hat BZ ticket for this (#597790) for tracking.

This is just a trivial patch to virsh.pod (from git master). It adds the
following pieces to the virsh man page:

  + Shows the --inactive and --all optional parameters for the list
    command.

    Closes Bugzilla #575512, reported by Renich Bon Ciric
    https://bugzilla.redhat.com/show_bug.cgi?id=575512

  + Corrects the existing description of the list command, to now say
    that only running domains are listed if no domains are specified.

    The man page up until this point has said all domains are listed if
    no domains are specified, which is incorrect.

  + Adds the "shut off" state to the list of states for the list
    command.

  + Adds a missing =back around line 755, that pod2man was complaining
    was missing.

* cfg.mk (sc_check_author_list): Look in correct location.

Otherwise, VPATH builds fail with:

make[1]: *** No rule to make target `libvirt-guests.init', needed by `all'.

Regression introduced in commit 482e08a9.

* daemon/Makefile.am (%.init): Look in correct place for
config.status.

Currently there is no way to opt out of libvirt dropping POSIX
capabilities for qemu. This at least is a useful debugging tool, but
is also wanted by users (and distributors):

https://bugzilla.redhat.com/show_bug.cgi?id=559154
https://bugzilla.redhat.com/show_bug.cgi?id=573850

v2: Clarify qemu.conf comment, warn about security implications

v3: Add .aug changes