https://bugzilla.redhat.com/show_bug.cgi?id=589465

Some guests (eg with badly configured grub, or Windows' installation cd)
require quick response from the console user. That's why we have a
"launchPaused" option in vdsm.

To implement it via libvirt, we need to ask libvirt not to call
qemuMonitorStartCPUs() after starting qemu.  Calling virDomainStop
immediately after the domain is up is inherently raceful.

* src/qemu/qemu_driver.c (qemudStartVMDaemon): Add new parameter;
all callers adjusted.
(qemudDomainCreate): Implement support for new flag.

* include/libvirt/libvirt.h.in (virDomainCreateFlags): Add
VIR_DOMAIN_START_PAUSED.
* src/libvirt.c (virDomainCreateXML): Update documentation.
* src/lxc/lxc_driver.c (lxcDomainCreateAndStart): Reject new flag
as unimplemented.
* src/opennebula/one_driver.c (oneDomainCreateAndStart):
Likewise.
* src/openvz/openvz_driver.c (openvzDomainCreateXML): Likewise.
* src/phyp/phyp_driver.c (phypDomainCreateAndStart): Likewise.
* src/qemu/qemu_driver.c (qemudDomainCreate): Likewise.
* src/test/test_driver.c (testDomainCreateXML): Likewise.
* src/uml/uml_driver.c (umlDomainCreate): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainCreateXML): Likewise.
* src/xen/xend_internal.c (xenDaemonCreateXML): Likewise.
* src/xenapi/xenapi_driver.c (xenapiDomainCreateXML): Likewise.

When an attempt to hotplug a PCI device to a guest fails,
the device was left attached to pci-stub. It is neccessary
to reset the device and then attach it to the host driver
again.

* src/qemu/qemu_driver.c: Reattach PCI device to host if
  hotadd fails

When SELinux is running in MLS mode, libvirtd will have a
different security level to the VMs. For libvirtd to be
able to connect to the monitor console, the client end of
the UNIX domain socket needs a different label. This adds
infrastructure to set the socket label via the security
driver framework

* src/qemu/qemu_driver.c: Call out to socket label APIs in
  security driver
* src/qemu/qemu_security_stacked.c: Wire up socket label
  drivers
* src/security/security_driver.h: Define security driver
  entry points for socket labelling
* src/security/security_selinux.c: Set socket label based on
  VM label

Adjust args to qemudStartVMDaemon() to also specify path to stdin_fd,
so this can be passed to the AppArmor driver via SetSecurityAllLabel().

This updates all calls to qemudStartVMDaemon() as well as setting up
the non-AppArmor security driver *SetSecurityAllLabel() declarations
for the above. This is required for the following
"apparmor-fix-save-restore" patch since AppArmor resolves the passed
file descriptor to the pathname given to open().

See https://bugzilla.redhat.com/show_bug.cgi?id=599091

Saving a paused 512MB domain took 3m47s with the old block size of 512
bytes. Changing the block size to 1024*1024 decreased the time to 56
seconds. (Doubling again to 2048*1024 yielded 0 improvement; lowering
to 512k increased the save time to 1m10s, about 20%)

The pointer to the xml describing the domain is saved into an object
prior to calling VIR_REALLOC_N() to make the size of the memory it
points to a multiple of QEMU_MONITOR_MIGRATE_TO_FILE_BS. If that
operation needs to allocate new memory, the pointer that was saved is
no longer valid.

To avoid this situation, adjust the size *before* saving the pointer.

(This showed up when experimenting with very large values of
QEMU_MONITOR_MIGRATE_TO_FILE_BS).

This patch that adds support for configuring 802.1Qbg and 802.1Qbh
switches. The 802.1Qbh part has been successfully tested with real
hardware. The 802.1Qbg part has only been tested with a (dummy)
server that 'behaves' similarly to how we expect lldpad to 'behave'.

The following changes were made during the development of this patch:

 - Merging Scott's v13-pre1 patch
 - Fixing endptr related bug while using virStrToLong_ui() pointed out
   by Jim Meyering
 - Addressing Jim Meyering's comments to v11
 - requiring mac address to the vpDisassociateProfileId() function to
   pass it further to the 802.1Qbg disassociate part (802.1Qbh untouched)
 - determining pid of lldpad daemon by reading it from /var/run/libvirt.pid
   (hardcode as is hardcode alson in lldpad sources)
 - merging netlink send code for kernel target and user space target
   (lldpad) using one function nlComm() to send the messages
 - adding a select() after the sending and before the reading of the
   netlink response in case lldpad doesn't respond and so we don't hang
 - when reading the port status, in case of 802.1Qbg, no status may be
   received while things are 'in progress' and only at the end a status
   will be there.
 - when reading the port status, use the given instanceId and vf to pick
   the right IFLA_VF_PORT among those nested under IFLA_VF_PORTS.
 - never sending nor parsing IFLA_PORT_SELF type of messages in the
   802.1Qbg case
 - iterating over the elements in a IFLA_VF_PORTS to pick the right
   IFLA_VF_PORT by either IFLA_PORT_PROFILE and given profileId
   (802.1Qbh) or IFLA_PORT_INSTANCE_UUID and given instanceId (802.1Qbg)
   and reading the current status in IFLA_PORT_RESPONSE.
 - recycling a previous patch that adds functionality to interface.c to
   - get the vlan identifier on an interface
   - get the flags of an interface and some convenience function to
     check whether an interface is 'up' or not (not currently used here)
 - adding function to determine the root physical interface of an
   interface. For example if a macvtap is linked to eth0.100, it will
   find eth0. Also adding a function that finds the vlan on the 'way to
   the root physical interface'
 - conveying the root physical interface name and index in case of 802.1Qbg
 - conveying mac address of macvlan device and vlan identifier in
   IFLA_VFINFO_LIST[ IFLA_VF_INFO[ IFLA_VF_MAC(mac), IFLA_VF_VLAN(vlan) ] ]
   to (future) lldpad via netlink
  - To enable build with --without-macvtap rename the
    [dis|]associatePortProfileId functions, prepend 'vp' before their
    name and make them non-static functions.
  - Renaming variable multicast to nltarget_kernel and inverting
    the logic
  - Addressing Jim Meyering's comments; this also touches existing
    code for example for correcting indentation of break statements or
    simplification of switch statements.
  - Renamed occurrencvirVirtualPortProfileDef to virVirtualPortProfileParamses
  - 802.1Qbg part prepared for sending a RTM_SETLINK and getting
    processing status back plus a subsequent RTM_GETLINK to
    get IFLA_PORT_RESPONSE.
    Note: This interface for 802.1Qbg may still change
  - [David Allan] move getPhysfn inside IFLA_VF_PORT_MAX to avoid
compiler
    warning when latest if_link.h isn't available
  - move from Stefan's 802.1Qb{g|h} XML v8 to v9
  - move hostuuid and vf index calcs to inside doPortProfileOp8021Qbh
  - remove debug fprintfs
  - use virGetHostUUID (thanks Stefan!)
  - fix compile issue when latest if_link.h isn't available
  - change poll timeout to 10s, at 1/8 intervals
     - if polling times out, log msg and return -ETIMEDOUT
  - Add Stefan's code for getPortProfileStatus
  - Poll for up to 2 secs for port-profile status, at 1/8 sec intervals:
     - if status indicates error, abort openMacvtapTap
     - if status indicates success, exit polling
     - if status is "in-progress" after 2 secs of polling, exit
       polling loop silently, without error

My patch finishes out the 802.1Qbh parts, which Stefan had mostly complete.
I've tested using the recent kernel updates for VF_PORT netlink msgs and
enic for Cisco's 10G Ethernet NIC.  I tested many VMs, each with several
direct interfaces, each configured with a port-profile per the XML.  VM-to-VM,
and VM-to-external work as expected.  VM-to-VM on same host (using same NIC)
works same as VM-to-VM where VMs are on diff hosts.  I'm able to change
settings on the port-profile while the VM is running to change the virtual
port behaviour.  For example, adjusting a QoS setting like rate limit.  All
VMs with interfaces using that port-profile immediatly see the effect of the
change to the port-profile.

I don't have a SR-IOV device to test so source dev is a non-SR-IOV device,
but most of the code paths include support for specifing the source dev and
VF index.  We'll need to complete this by discovering the PF given the VF
linkdev.  Once we have the PF, we'll also have the VF index.  All this info-
mation is available from sysfs.

Currently there is no way to opt out of libvirt dropping POSIX
capabilities for qemu. This at least is a useful debugging tool, but
is also wanted by users (and distributors):

https://bugzilla.redhat.com/show_bug.cgi?id=559154
https://bugzilla.redhat.com/show_bug.cgi?id=573850

v2: Clarify qemu.conf comment, warn about security implications

v3: Add .aug changes

Since the macvtap device needs active tear-down and the teardown logic
is based on the interface name, it can happen that if for example 1 out
of 3 interfaces was successfully created, that during the failure path
the macvtap's target device name is used to tear down an interface that
is doesn't own (owned by another VM).

So, in this patch, the target interface name is reset so that there is
no target interface name and the interface name is always cleared after
a tear down.

The hotplug methods still had the qemuCmdFlags variable declared
as an int, instead of unsigned long long. This caused flag checks
to be incorrect for flags > 31

* src/qemu/qemu_driver.c: Fix integer overflow in hotplug

This allows libvirt to open the PCI device sysfs config file prior
to dropping privileges so qemu can access the full config space.
Without this, a de-privileged qemu can only access the first 64
bytes of config space.

* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h: Detect support
  for pci-assign.configfd option. Use this option when formatting
  PCI device string if possible
* src/qemu/qemu_driver.c: Pre-open PCI sysfs config file and pass
  to QEMU

We've been running into a lot of situations where
virGetHostname() is returning "localhost", where a plain
gethostname() would have returned the correct thing.  This
is because virGetHostname() is *always* trying to canonicalize
the name returned from gethostname(), even when it doesn't
have to.

This patch changes virGetHostname so that if the value returned
from gethostname() is already FQDN or localhost, it returns
that string directly.  If the value returned from gethostname()
is a shortened hostname, then we try to canonicalize it.  If
that succeeds, we returned the canonicalized hostname.  If
that fails, and/or returns "localhost", then we just return
the original string we got from gethostname() and hope for
the best.

Note that after this patch it is up to clients to check whether
"localhost" is an allowed return value.  The only place
where it's currently not is in qemu migration.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

This patch parses the following two XML descriptions, one for
802.1Qbg and one for 802.1Qbh, and stores the data internally.
The actual triggering of the switch setup protocol has not been
implemented here but the relevant code to do that should go into
the functions associatePortProfileId() and disassociatePortProfileId().

   <interface type='direct'>
      <source dev='eth0.100' mode='vepa'/>
      <model type='virtio'/>
      <virtualport type='802.1Qbg'>
        <parameters managerid='12' typeid='0x123456' typeidversion='1'
         instanceid='fa9b7fff-b0a0-4893-8e0e-beef4ff18f8f'/>
      </virtualport>
      <filterref filter='clean-traffic'/>
    </interface>

    <interface type='direct'>
      <source dev='eth0.100' mode='vepa'/>
      <model type='virtio'/>
      <virtualport type='802.1Qbh'>
        <parameters profileid='my_profile'/>
      </virtualport>
    </interface>

I'd suggest to use this patch as a base for triggering the setup
protocol with the 802.1Qb{g|h} switch.

Several rounds of changes were made to this patch. The
following is a list of these changes.
- Renamed structure virVirtualPortProfileDef to virVirtualPortProfileParams
  as per Daniel Berrange's request
- Addressing Daniel Berrange's comments:
 - removing macvtap.h's dependency on domain_conf.h by
   moving the virVirtualPortProfileDef structure into macvtap.h
   and not passing virtDomainNetDefPtr to any functions in
   macvtap.c
- Addressed most of Chris Wright's comments:
  - indicating error in case virtualport XML node cannot be parsed
    properly
  - parsing hex and decimal numbers using virStrToLong_ui() with
    parameter '0' for base
  - tgifname (target interface name) variable wasn't necessary
    to pass to openMacvtapTap function anymore
- assigning the virtual port data structure to the virDomainNetDef
  only if it was previously parsed
- make sure that the error code returned by openMacvtapTap() is a negative n
  in case the associatePortProfileId() function failed.
- renaming vsi in the XML to virtualport
- replace all occurrences of vsi in the source as well
- removing mode and MAC address parameters from the functions that
  will communicate with the hareware diretctly or indirectly
- moving the associate and disassociate functions to the end of the
  file for subsequent patches to easier make them generally available
  for export
- passing the macvtap interface name rather than the link device since
  this otherwise gives funny side effects when using netlink messages
  where IFLA_IFNAME and IFLA_ADDRESS are specified and the link dev
  all of a sudden gets the MAC address of the macvtap interface.
- Removing rc = -1 error indications in the case of 802.1Qbg|h setup in case
  we wanted to use hook scripts for the setup and so the setup doesn't fail
  here.
- if instance ID UUID is not supplied it will automatically be generated
  - adapted schema to make instance ID UUID optional
  - added test case
- parser and XML generator have been separated into their own
  functions so they can be re-used elsewhere (passthrough case
  for example)
- Adapted XML parser and generator support the above shown type
  (802.1Qbg, 802.1Qbh).
- Adapted schema to above XML
- Adapted test XML to above XML
- Passing through the VM's UUID which seems to be necessary for
  802.1Qbh -- sorry no host UUID
- adding virtual function ID to association function, in case it's
  necessary to use (for SR-IOV)

Allow for a host UUID in the capabilities XML. Local drivers
will initialize this from the SMBIOS data. If a sanity check
shows SMBIOS uuid is invalid, allow an override from the
libvirtd.conf configuration file

* daemon/libvirtd.c, daemon/libvirtd.conf: Support a host_uuid
  configuration option
* docs/schemas/capability.rng: Add optional host uuid field
* src/conf/capabilities.c, src/conf/capabilities.h: Include
  host UUID in XML
* src/libvirt_private.syms: Export new uuid.h functions
* src/lxc/lxc_conf.c, src/qemu/qemu_driver.c,
  src/uml/uml_conf.c: Set host UUID in capabilities
* src/util/uuid.c, src/util/uuid.h: Support for host UUIDs
* src/node_device/node_device_udev.c: Use the host UUID functions
* tests/confdata/libvirtd.conf, tests/confdata/libvirtd.out: Add
  new host_uuid config option to test

The cgroups ACL code was only allowing the primary disk image.
It is possible to chain images together, so we need to search
for backing stores and add them to the ACL too. Since the ACL
only handles block devices, we ignore the EINVAL we get from
plain files. In addition it was missing code to teardown the
cgroup when hot-unplugging a disk

* src/qemu/qemu_driver.c: Allow backing stores in cgroup ACLs
  and add missing teardown code in unplug path

Basic live migration was broken by the commit that added
non-shared block support in two ways:

1)  It added a virCheckFlags() to doNativeMigrate().  Besides
the fact that typical usage of virCheckFlags() is in driver
entry points, and doNativeMigrate() is not an entry point,
it was missing important flags like VIR_MIGRATE_LIVE.  Move
the virCheckFlags to the top-level qemuDomainMigratePrepare2
and friends.

2)  It also added a memory leak in qemuMonitorTextMigrate()
by not freeing the memory used by virBufferContentAndReset().
This is fixed by storing the pointer in a temporary variable
and freeing it at the end.

With this patch in place, normal live migration works again.

v3: Instead of the churn for virCheckFlagsUI and UL, instead
always promote flags to an unsigned long and always use %lx
for the fprintf.
v2: Add back flags check, which required adding virCheckFlagsUI
and virCheckFlagsUL
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>

The device path doesn't make use of guestAddr, so the memcpy corrupts
the guest info struct.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>

The virDomainGetBlockInfo API allows query physical block
extent and allocated block extent. These are normally the
same value unless storing a special format like qcow2
inside a block device. In this scenario we can query QEMU
to get the actual allocated extent.

Since last time:

 - Return fatal error in text monitor
 - Only invoke monitor command for block devices
 - Fix error handling JSON code

* src/qemu/qemu_driver.c: Fill in block aloction extent when VM
  is running
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  API to query the highest block extent via info blockstats

Signed-off-by: NChris Lalancette <clalance@redhat.com>

The event is already generated and sent by qemudDomainObjStart, no need
to do anything about here.

The qemu driver contains a subtle race in the logic to find next
available vnc port.  Currently it iterates through all available ports
and returns the first for which bind(2) succeeds.  However it is possible
that a previously issued port has not yet been bound by qemu, resulting
in the same port used for a subsequent domain.

This patch addresses the race by using a simple bitmap to "reserve" the
ports allocated by libvirt.

V2:
  - Put port bitmap in struct qemud_driver
  - Initialize bitmap in qemudStartup

V3:
  - Check for failure of virBitmapGetBit
  - Additional check for port != -1 before calling virbitmapClearBit

V4:
  - Check for failure of virBitmap{Set,Clear}Bit

We need a common internal function for starting managed domains to be
used during autostart. This patch factors out relevant code from
qemudDomainStart into qemudDomainObjStart and makes it use the
refactored code for domain restore instead of calling qemudDomainRestore
API directly.

We need to be able to restore a domain which we already locked and
started a job for it without undoing these steps. This patch factors
out internals of qemudDomainRestore into separate functions which work
for locked objects.

These changes avoid false-positive syntax-check failure,
and also make the resulting diagnostics more comprehensible.

Run this:
  git grep -l 'VIR_ERROR\s*("'|xargs perl -pi -e \
    's/(VIR_ERROR)\s*\((".*?"),/$1(_($2),/'

Run this:
  git grep -l 'VIR_ERROR0\s*("'|xargs perl -pi -e \
    's/(VIR_ERROR0)\s*\((".*?")\)/$1(_($2))/'

Change VIR_ERROR("%s", "..."
to     VIR_ERROR0("..."

and

Change VIR_ERROR("%s", _("...")
to     VIR_ERROR0(_("...")

Use this command:
  git grep -E -l 'VIR_ERROR\("%s", (_\()?"'|xargs perl -pi -e \
  's/VIR_ERROR\("%s", (_\()?"/VIR_ERROR0($1"/'

The current cleanup: in StartVMDaemon path is a poor duplication.
qemuShutdownVMDaemon can handle teardown for inactive VMs, so let's use it.

v2: Remove old abort: label, only use cleanup:

Clients that require this already seem to do so. Calling this function
with pid < 1 also should not cause problems.

This only exists for a certain cleanup path in StartVMDaemon, but is
unneeded since domain_conf.c handles this for us automatically.

Everything after hostdev setup needs to jump to cleanup on error.

There doesn't seem to be anything specific to tap devices for this
array of file descriptors which need to stay open of the guest to use.
Rename then for others to make use of.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* src/qemu/qemu_driver.c (qemudDomainMigrateFinish2): Don't ignore
virDomainSaveStatus failure.
* src/conf/domain_conf.h (virDomainSaveStatus): Use
ATTRIBUTE_RETURN_CHECK, so this doesn't happen again.

Do not require each caller of virStorageFileGetMetadata and
virStorageFileGetMetadataFromFD to first clear the storage of the
"meta" buffer.  Instead, initialize that storage in
virStorageFileGetMetadataFromFD.
* src/util/storage_file.c (virStorageFileGetMetadataFromFD): Clear
"meta" here, not before each of the following callers.
* src/qemu/qemu_driver.c (qemuSetupDiskCgroup): Don't clear "meta" here.
(qemuTeardownDiskCgroup): Likewise.
* src/qemu/qemu_security_dac.c (qemuSecurityDACSetSecurityImageLabel):
Likewise.
* src/security/security_selinux.c (SELinuxSetSecurityImageLabel):
Likewise.
* src/security/virt-aa-helper.c (get_files): Likewise.

* src/qemu/qemu_driver.c (qemuTeardownDiskCgroup): Remove
bogus empty-body while-loop.
(qemuSetupDiskCgroup): Likewise.

Approximately 60 messages were marked.  Since these diagnostics are
intended solely for developers and maintainers, encouraging translation
is deemed to be counterproductive:
http://thread.gmane.org/gmane.comp.emulators.libvirt/25050/focus=25052

Run this command:
  git grep -l VIR_WARN|xargs perl -pi -e \
    's/(VIR_WARN0?)\s*\(_\((".*?")\)/$1($2/'