The current way of assigning names to the host network backend and
NIC device in QEMU was over complicated, by varying naming scheme
based on the NIC model and backend type. This simplifies the naming
to simply be 'net0' and 'hostnet0', allowing code to easily determine
the host network name and vlan based off the primary device alias
name 'net0'. This in turn allows removal of alot of QEMU specific
code from the XML parser, and makes it easier to assign new unique
names for NICs that are hotplugged

* src/conf/domain_conf.c, src/conf/domain_conf.h: Remove hostnet_name
  and vlan fields from virNetworkDefPtr
* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h, src/qemu/qemu_driver.c:
  Use a single network alias naming scheme regardless of NIC type
  or backend type. Determine VLANs from the alias name.
* tests/qemuxml2argvdata/qemuxml2argv-net-eth-names.args,
  tests/qemuxml2argvdata/qemuxml2argv-net-virtio-device.args,
  tests/qemuxml2argvdata/qemuxml2argv-net-virtio-netdev.args: Update
  for new simpler naming scheme

PCI disk, disk controllers, net devices and host devices need to
have PCI addresses assigned before they are hot-plugged

* src/qemu/qemu_conf.c: Add APIs for ensuring a device has an
  address and releasing unused addresses
* src/qemu/qemu_driver.c: Ensure all devices have addresses
  when hotplugging.

The current QEMU code allocates PCI addresses incrementally starting
at 4. This is not satisfactory because the user may have given some
addresses in their XML config, which need to be skipped over when
allocating addresses to remaining devices.

It is thus neccessary to maintain a list of already allocated PCI
addresses and then only allocate ones that remain unused. This is
also required for domain device hotplug to work properly later.

* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h: Add APIs for creating
  list of existing PCI addresses, and allocating new addresses.
  Refactor address assignment to use this code
* src/qemu/qemu_driver.c: Pull PCI address assignment up into the
  qemuStartVMDaemon() method, as a prelude to moving it into the
  'define' method. Update list of allocated addresses when connecting
  to a running VM at daemon startup.
* tests/qemuxml2argvtest.c, tests/qemuargv2xmltest.c,
  tests/qemuxml2xmltest.c: Remove USB product test since all
  passthrough is done based on address
* tests/qemuxml2argvdata/qemuxml2argv-hostdev-usb-product.args,
  tests/qemuxml2argvdata/qemuxml2argv-hostdev-usb-product.xml: Kil
  unused data files

Since QEMU startup uses the new -device argument, the hotplug
code needs todo the same. This converts disk, network and
host device hotplug to use the device_add command

* src/qemu/qemu_driver.c: Use new device_add monitor APIs
  whereever possible

All the helper functions for building command line arguments
now return a 'char *', instead of acepting a 'char **' or
virBufferPtr argument

* qemu/qemu_conf.c: Standardize syntax for building args
* qemu/qemu_conf.h: Export all functions for building args
* qemu/qemu_driver.c: Update for changed syntax for building
  NIC/hostnet args

* src/qemu/qemu_driver.c (qemudLogReadFD): Don't pass a negative
offset (from a preceding failed attempt to seek to EOF) to this use
of lseek.

Similar to the race fixed by
be34c3c7, make sure
to wait around for KVM to release the resources from
a hot-detached PCI device before attempting to
rebind that device to the host driver.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

If you shutdown libvirtd while a domain with PCI
devices is running, then try to restart libvirtd,
libvirtd will crash.

This happens because qemuUpdateActivePciHostdevs() is calling
pciDeviceListSteal() with a dev of 0x0 (NULL), and then trying
to dereference it.  This patch fixes it up so that
qemuUpdateActivePciHostdevs() steals the devices after first
Get()'ting them, avoiding the crash.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Certain hypervisors (like qemu/kvm) map the PCI bar(s) on
the host when doing device passthrough.  This can lead to a race
condition where the hypervisor is still cleaning up the device while
libvirt is trying to re-attach it to the host device driver.  To avoid
this situation, we look through /proc/iomem, and if the hypervisor is
still holding onto the bar (denoted by the string in the matcher variable),
then we can wait around a bit for that to clear up.

v2: Thanks to review by DV, make sure we wait the full timeout per-device
Signed-off-by: NChris Lalancette <clalance@redhat.com>

The loop looking for the controller associated with a SCI drive had
an off by one, causing it to miss the last controller.

* src/qemu/qemu_driver.c: Fix off-by-1 in searching for SCSI
  drive hotplug

The hotplug code in QEMU was leaking memory because although the
inner device object was being moved into the main virDomainDefPtr
config object, the outer container virDomainDeviceDefPtr was not.

 * src/qemu/qemu_driver.c: Clarify code to show that the inner
   device object is owned by the main domain config upon
   successfull attach.

The hotplug code was not correctly invoking the security driver
in error paths. If a hotplug attempt failed, the device would
be left with VM permissions applied, rather than restored to the
original permissions. Also, a CDROM media that is ejected was
not restored to original permissions. Finally there was a bogus
call to set hostdev permissions in the hostdev unplug code

* qemu/qemu_driver.c: Fix security driver usage in hotplug/unplug

If there is a problem with VM startup, PCI devices may be left
assigned to pci-stub / pci-back. Adding a call to reattach
host devices in the cleanup path is required.

* qemu/qemu_driver.c: qemuDomainReAttachHostDevices() when
  VM startup fails

Remove all the QEMU driver calls for setting file ownership and
process uid/gid. Instead wire in the QEMU DAC security driver,
stacking it ontop of the primary SELinux/AppArmour driver.

* qemu/qemu_driver.c: Switch over to new DAC security driver

Pulling the disk labelling code out of the exec hook, and into
libvirtd will allow it to access shared state in the daemon. It
will also make debugging & error reporting easier / more reliable.

* qemu/qemu_driver.c: Move initial disk labelling calls up into
  libvirtd. Add cleanup of disk labels upon failure

If a VM fails to start, we can't simply free the security label
strings, we must call the domainReleaseSecurityLabel() method
otherwise the reserved 'mcs' level will be leaked in SElinux

* src/qemu/qemu_driver.c: Invoke domainReleaseSecurityLabel()
  when domain fails to start

The current security driver architecture has the following
split of logic

 * domainGenSecurityLabel

    Allocate the unique label for the domain about to be started

 * domainGetSecurityLabel

    Retrieve the current live security label for a process

 * domainSetSecurityLabel

    Apply the previously allocated label to the current process
    Setup all disk image / device labelling

 * domainRestoreSecurityLabel

    Restore the original disk image / device labelling.
    Release the unique label for the domain

The 'domainSetSecurityLabel' method is special because it runs
in the context of the child process between the fork + exec.

This is require in order to set the process label. It is not
required in order to label disks/devices though. Having the
disk labelling code run in the child process limits what it
can do.

In particularly libvirtd would like to remember the current
disk image label, and only change shared image labels for the
first VM to start. This requires use & update of global state
in the libvirtd daemon, and thus cannot run in the child
process context.

The solution is to split domainSetSecurityLabel into two parts,
one applies process label, and the other handles disk image
labelling. At the same time domainRestoreSecurityLabel is
similarly split, just so that it matches the style. Thus the
previous 4 methods are replaced by the following 6 new methods

 * domainGenSecurityLabel

    Allocate the unique label for the domain about to be started
    No actual change here.

 * domainReleaseSecurityLabel

   Release the unique label for the domain

 * domainGetSecurityProcessLabel

   Retrieve the current live security label for a process
   Merely renamed for clarity.

 * domainSetSecurityProcessLabel

   Apply the previously allocated label to the current process

 * domainRestoreSecurityAllLabel

    Restore the original disk image / device labelling.

 * domainSetSecurityAllLabel

    Setup all disk image / device labelling

The SELinux and AppArmour drivers are then updated to comply with
this new spec. Notice that the AppArmour driver was actually a
little different. It was creating its profile for the disk image
and device labels in the 'domainGenSecurityLabel' method, where as
the SELinux driver did it in 'domainSetSecurityLabel'. With the
new method split, we can have consistency, with both drivers doing
that in the domainSetSecurityAllLabel method.

NB, the AppArmour changes here haven't been compiled so may not
build.

The QEMU driver is doing 90% of the calls to check for static vs
dynamic labelling. Except it is forgetting todo so in many places,
in particular hotplug is mistakenly assigning disk labels. Move
all this logic into the security drivers themselves, so the HV
drivers don't have to think about it.

* src/security/security_driver.h: Add virDomainObjPtr parameter
  to virSecurityDomainRestoreHostdevLabel and to
  virSecurityDomainRestoreSavedStateLabel
* src/security/security_selinux.c, src/security/security_apparmor.c:
  Add explicit checks for VIR_DOMAIN_SECLABEL_STATIC and skip all
  chcon() code in those cases
* src/qemu/qemu_driver.c: Remove all checks for VIR_DOMAIN_SECLABEL_STATIC
  or VIR_DOMAIN_SECLABEL_DYNAMIC. Add missing checks for possibly NULL
  driver entry points.

* src/lxc/lxc_container.c src/lxc/lxc_controller.c src/lxc/lxc_driver.c
  src/network/bridge_driver.c src/qemu/qemu_driver.c
  src/uml/uml_driver.c: virFileMakePath returns 0 for success, or the
  value of errno on failure, so error checking should be to test
  if non-zero, not if lower than 0

Invoking the virConnectGetCapabilities() method causes the QEMU
driver to rebuild its internal capabilities object. Unfortunately
it was forgetting to register the custom domain status XML hooks
again.

To avoid this kind of error in the future, the code which builds
capabilities is refactored into one single method, which can be
called from all locations, ensuring reliable rebuilds.

* src/qemu/qemu_driver.c: Fix rebuilding of capabilities XML and
  guarentee it is always consistent

I noticed some debug messages are printed with an empty lines after
them. This patch removes these empty lines from all invocations of the
following macros:
    VIR_DEBUG
    VIR_DEBUG0
    VIR_ERROR
    VIR_ERROR0
    VIR_INFO
    VIR_WARN
    VIR_WARN0
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Instead of relying on QEMU to assign PCI addresses and then querying
them with 'info pci', manually assign all PCI addresses before starting
the guest.  These addresses are not stable across reboots. That will
come in a later patch

NB, the PIIX3 (IDE, FDC, ISA-Bridge) will always have slot 1 and
VGA will always have slot 2. We declare the Virtio Balloon gets
slot 3, and then all remaining slots are for configured devices.

* src/qemu/qemu_conf.c: If -device is supported, then assign all PCI
  addresses when building the command line
* src/qemu/qemu_driver.c: Don't query monitor for PCI addresses if
  they have already been assigned
* tests/qemuxml2argvdata/qemuxml2argv-hostdev-pci-address-device.args,
  tests/qemuxml2argvdata/qemuxml2argv-net-virtio-device.args,
  tests/qemuxml2argvdata/qemuxml2argv-sound-device.args,
  tests/qemuxml2argvdata/qemuxml2argv-watchdog-device.args: Update
  to include PCI slot/bus information

The current character device syntax uses either

  -serial tty,path=/dev/ttyS2

Or

  -chardev tty,id=serial0,path=/dev/ttyS2 -serial chardev:serial0

With the new -device support, we now prefer

  -chardev file,id=serial0,path=/tmp/serial.log -device isa-serial,chardev=serial0

This patch changes the existing -chardev syntax to use this new
scheme, and fallbacks to the old plain -serial syntax for old
QEMU.

The monitor device changes to

  -chardev socket,id=monitor,path=/tmp/test-monitor,server,nowait -mon chardev=monitor

In addition, this patch adds --nodefaults, which kills off the
default serial, parallel, vga and nic devices. THis avoids the
need for us to explicitly turn each off

When starting a guest, give every device a unique alias. This will
be used for the 'id' parameter in -device args in later patches.
It can also be used to uniquely identify devices in the monitor

For old QEMU without -device, assign disk names based on QEMU's
historical naming scheme.

* src/qemu/qemu_conf.c: Assign unique device aliases
* src/qemu/qemu_driver.c: Remove obsolete qemudDiskDeviceName
  and use the device alias in eject & blockstats commands

This patch introduces the support for giving all devices a short,
unique name, henceforth known as a 'device alias'.  These aliases
are not set by the end user, instead being assigned by the hypervisor
if it decides it want to support this concept.

The QEMU driver sets them whenever using the -device arg syntax
and uses them for improved hotplug/hotunplug. it is the intent
that other APIs (block / interface stats & device hotplug) be
able to accept device alias names in the future.

The XML syntax is

   <alias name="video0"/>

This may appear in any type of device that supports device info.

* src/conf/domain_conf.c, src/conf/domain_conf.h: Add a 'alias'
  field to virDomainDeviceInfo struct & parse/format it in XML
* src/libvirt_private.syms: Export virDomainDefClearDeviceAliases
* src/qemu/qemu_conf.c: Replace use of "nic_name" field with the
  standard device alias
* src/qemu/qemu_driver.c: Clear device aliases at shutdown

The PCI device addresses are only valid while the VM is running,
since they are auto-assigned by QEMU. After shutdown they must
all be cleared. Future QEMU driver enhancement will allow for
persistent PCI address assignment

* src/conf/domain_conf.h, src/conf/domain_conf.c, src/libvirt_private.syms
  Add virDomainDefClearPCIAddresses() method for wiping out auto assigned
  PCI addresses
* src/qemu/qemu_driver.c: Clear PCI addresses at VM shutdown

Hotunplug of devices requires that we know their PCI address. Even
hotplug of SCSI drives, required that we know the PCI address of
the SCSI controller to attach the drive to. We can find this out
by running 'info pci' and then correlating the vendor/product IDs
with the devices we booted with.

Although this approach is somewhat fragile, it is the only viable
option with QEMU < 0.12, since there is no way for libvirto set
explicit PCI addresses when creating devices in the first place.
For QEMU > 0.12, this code will not be used.

* src/qemu/qemu_driver.c: Assign all dynamic PCI addresses on
  startup of QEMU VM, matching vendor/product IDs
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  API for fetching PCI device address mapping

The current SCSI hotplug support attaches a brand new SCSI controller
for every disk. This is broken because the semantics differ from those
used when starting the VM initially. In the latter case, each SCSI
controller is filled before a new one is added.

If the user specifies an high drive index (sdazz) then at initial
startup, many intermediate SCSI controllers may be added with no
drives.

This patch changes SCSI hotplug so that it exactly matches the
behaviour of initial startup. First the SCSI controller number is
determined for the drive to be hotplugged. If any controller upto
and including that controller number is not yet present, it is
attached. Then finally the drive is attached to the last controller.

NB, this breaks SCSI hotunplug, because there is no 'drive_del'
command in current QEMU. Previous SCSI hotunplug was broken in
any case because it was unplugging the entire controller, not
just the drive in question.

A future QEMU will allow proper SCSI hotunplug of a drive.

This patch is derived from work done by Wolfgang Mauerer on disk
controllers.

* src/qemu/qemu_driver.c: Fix SCSI hotplug to add a drive to
 the correct controller, instead of just attaching a new
  controller.
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  support for 'drive_add' command

This patch allows for explicit hotplug/unplug of SCSI controllers.
Ordinarily this is not required, since QEMU/libvirt will attach
a new SCSI controller whenever one is required. Allowing explicit
hotplug of controllers though, enables the caller to specify a
static PCI address, instead of auto-assigning the next available
PCI slot. Or it will when we have static PCI addressing.

This patch is derived from Wolfgang Mauerer's disk controller
patch series.

* src/qemu/qemu_driver.c: Support hotplug & unplug of SCSI
  controllers
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  new API for attaching PCI SCSI controllers

qemudFindCharDevicePTYsMonitor reports an error if 'info chardev' didn't
provide information for a requested device, even if the log output parsing
had found the pty path for that device. This makes pty assignment fail for
older QEMU/KVM versions. For example KVM 72 on Debian doesn't support
'info chardev', so qemuMonitorTextGetPtyPaths cannot parse any useful
information and the hash for device-id-to-pty-path mapping stays empty.

Make qemudFindCharDevicePTYsMonitor report an error only if the log output
parsing and the 'info chardev' parsing failed to provide the pty path.

Convert the QEMU monitor APIs over to use virDomainDeviceAddress
structs for passing addresses in/out, instead of individual bits.
This makes the number of parameters smaller & easier to deal with.
No functional change

* src/qemu/qemu_driver.c, src/qemu/qemu_monitor.c,
  src/qemu/qemu_monitor.h, src/qemu/qemu_monitor_text.c,
  src/qemu/qemu_monitor_text.h: Change monitor hotplug APIs to
  take an explicit address ptr for all host/guest addresses

All guest devices now use a common device address structure
summarized by:

  enum virDomainDeviceAddressType {
    VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE,
    VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI,
  };

  struct _virDomainDevicePCIAddress {
    unsigned int domain;
    unsigned int bus;
    unsigned int slot;
    unsigned int function;
  };

  struct _virDomainDeviceInfo {
    int type;
    union {
        virDomainDevicePCIAddress pci;
    } addr;
  };

This replaces the anonymous structs in Disk/Net/Hostdev data
structures. Where available, the address is *always* printed
in the XML file, instead of being hidden in the internal state
file.

  <address type='pci' domain='0x0000' bus='0x1e' slot='0x07' function='0x0'/>

The structure definition is based on Wolfgang Mauerer's disk
controller patch series.

* docs/schemas/domain.rng: Define the <address> syntax and
  associate it with disk/net/hostdev devices
* src/conf/domain_conf.h, src/conf/domain_conf.c,
  src/libvirt_private.syms: APIs for parsing/formatting address
  information. Also remove the QEMU specific 'pci_addr' attributes
* src/qemu/qemu_driver.c: Replace use of 'pci_addr' attrs with
  new standardized format.

Based off how QEMU does it, look through /sys/bus/usb/devices/* for
matching vendor:product info, and if found, use info from the surrounding
files to build the device's /dev/bus/usb path.

This fixes USB device assignment by vendor:product when running qemu
as non-root (well, it should, but for some reason I couldn't reproduce
the failure people are seeing in [1], but it appears to work properly)

[1] https://bugzilla.redhat.com/show_bug.cgi?id=542450

This way we won't squash the original error report in
MigratePerform, as is done for P2P and Tunneled migration.

qemudWaitForMonitor calls qemudReadLogOutput with qemudFindCharDevicePTYs
as callback. qemudFindCharDevicePTYs calls qemudExtractTTYPath to assign
a string to chr->data.file.path. Afterwards qemudWaitForMonitor may call
qemudFindCharDevicePTYsMonitor that overwrites chr->data.file.path without
freeing the old value. This results in leaking the memory allocated by
qemudExtractTTYPath.

Report an OOM error if the strdup in qemudFindCharDevicePTYsMonitor fails.

* src/qemu/qemu_driver.c (qemudDomainMigratePrepare2): Remove useless
test of always-non-NULL uri_out parameter.  Use ATTRIBUTE_NONNULL to
inform tools.

* src/qemu/qemu_driver.c (qemudDomainAttachDevice): Call
virCgroupDenyDevicePath only if cgroup is non-NULL.

The XML XPath for detecting JSON in the running VM statefile was
wrong causing all VMs to get JSON mode enabled at libvirtd restart.

In addition if a VM was running a JSON enabled QEMU once, and then
altered to point to a non-JSON enabled QEMU later the 'monJSON'
flag would not get reset to 0.

* src/qemu/qemu_driver.c: Fix setting/detection of JSON mode