src/qemu/qemu_hostdev.c:
refactor qemuPrepareHostdevUSBDevices function, make it focus on
adding usb device to activeUsbHostdevs after check. After that,
the usb hotplug function qemuDomainAttachHostDevice also could use
it.
expand qemuPrepareHostUSBDevices to perform the usb search,
rollback on failure.

src/qemu/qemu_hotplug.c:
If there are multiple usb devices available with same vendorID and productID,
but with different value of "bus, device", we give an error to let user
use <address> to specify the desired one.

usbFindDevice():get usb device according to
                idVendor, idProduct, bus, device
                it is the exact match of the four parameters

usbFindDeviceByBus():get usb device according to bus, device
                  it returns only one usb device same as usbFindDevice

usbFindDeviceByVendor():get usb device according to idVendor,idProduct
                     it probably returns multiple usb devices.

usbDeviceSearch(): a helper function to do the actual search

If the daemon is restarted it will lose list of active
USB devices assigned to active domains. Therefore we need
to rebuild this list on qemuProcessReconnect().

To prevent assigning one USB device to two domains,
we keep a list of assigned USB devices. On domain
startup - qemuProcessStart() - we insert devices
used by domain into the list but remove them only
on detach-device. Devices are, however, released
on qemuProcessStop() as well.

and add debug message when adding USB device
to the list of active devices.

These changes are applied only if the hostdev has a parent net device
(i.e. if it was defined as "<interface type='hostdev'>" rather than
just "<hostdev>").  If the parent netdevice has virtual port
information, the original virtualport associate functions are called
(these set and restore both mac and port profile on an
interface). Otherwise, only mac address is set on the device.

Note that This is only supported for SR-IOV Virtual Functions (not for
standard PCI or USB netdevs), and virtualport association is only
supported for 802.1Qbh. For all other types of cards and types of
virtualport, a "Config Unsupported" error is returned and the
operation fails.
Signed-off-by: NRoopa Prabhu <roprabhu@cisco.com>

pciTrySecondaryBusReset checks if there is active device on the
same bus, however, qemu driver doesn't maintain an effective
list for the inactive devices, and it passes meaningless argument
for parameter "inactiveDevs". e.g. (qemuPrepareHostdevPCIDevices)

if (!(pcidevs = qemuGetPciHostDeviceList(hostdevs, nhostdevs)))
    return -1;

..skipped...

if (pciResetDevice(dev, driver->activePciHostdevs, pcidevs) < 0)
    goto reattachdevs;

NB, the "pcidevs" used above are extracted from domain def, and
thus one won't be able to attach a device of which bus has other
device even detached from host (nodedev-detach). To see more
details of the problem:

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=773667

This patch is to resolve the problem by introducing an inactive
PCI device list (just like qemu_driver->activePciHostdevs), and
the whole logic is:

  * Add the device to inactive list during nodedev-dettach
  * Remove the device from inactive list during nodedev-reattach
  * Remove the device from inactive list during attach-device
    (for non-managed device)
  * Add the device to inactive list after detach-device, only
    if the device is not managed

With the above, we have a sufficient inactive PCI device list, and thus
we can use it for pciResetDevice. e.g.(qemuPrepareHostdevPCIDevices)

if (pciResetDevice(dev, driver->activePciHostdevs,
                   driver->inactivePciHostdevs) < 0)
    goto reattachdevs;

In order to avoid situation where a USB device is
in use by two domains, we must keep a list of already
attached devices like we do for PCI.

* src/qemu/qemu_hostdev.c (qemuDomainReAttachHostdevDevices):
pciDeviceListFree(pcidevs) in the end free()s the device even if
it's in use by other domain, which can cause a race.

How to reproduce:

<script>

virsh nodedev-dettach pci_0000_00_19_0
virsh start test
virsh attach-device test hostdev.xml
virsh start test2

for i in {1..5}; do
        echo "[ -- ${i}th time --]"
        virsh nodedev-reattach pci_0000_00_19_0
done

echo "clean up"
virsh destroy test
virsh nodedev-reattach pci_0000_00_19_0
</script>

Device pci_0000_00_19_0 dettached

Domain test started

Device attached successfully

error: Failed to start domain test2
error: Requested operation is not valid: PCI device 0000:00:19.0 is in use by domain test

[ -- 1th time --]
Device pci_0000_00_19_0 re-attached

[ -- 2th time --]
Device pci_0000_00_19_0 re-attached

[ -- 3th time --]
Device pci_0000_00_19_0 re-attached

[ -- 4th time --]
Device pci_0000_00_19_0 re-attached

[ -- 5th time --]
Device pci_0000_00_19_0 re-attached

clean up
Domain test destroyed

Device pci_0000_00_19_0 re-attached

The patch also fixes another problem, there won't be error like
"qemuDomainReAttachHostdevDevices: Not reattaching active
device 0000:00:19.0" in daemon log if some device is in active.
As pciResetDevice and pciReattachDevice won't be called for
the device anymore. This is sensible as we already reported
error when preparing the device if it's active. Blindly trying
to pciResetDevice & pciReattachDevice on the device and getting
an error is just redundant.

To support "managed" mode of host PCI device, we record the original
states (unbind_from_stub, remove_slot, and reprobe) so that could
reattach the device to host with original driver. But there is no XML
for theses attrs, and thus after daemon is restarted, we lose the
original states. It's easy to reproduce:

    1) virsh start domain
    2) virsh attach-device dom hostpci.xml (in 'managed' mode)
    3) service libvirtd restart
    4) virsh destroy domain

    You will see the device won't be bound to the original driver
if there was one.

This patch is to solve the problem by introducing internal XML
(won't be dumped to user, only dumped to status XML). The XML is:
    <origstates>
      <unbind/>
      <remove_slot/>
      <reprobe/>
    </origstates>

Which will be child node of <hostdev><source>...</souce></hostdev>.
(only for PCI device).

A new struct "virDomainHostdevOrigStates" is introduced for the XML,
and the according members are updated when preparing the PCI device.
And function "qemuUpdateActivePciHostdevs" is modified to honor
the original states. Use of qemuGetPciHostDeviceList is removed
in function "qemuUpdateActivePciHostdevs", and the "managed" value of
the device config is honored by the change. This fixes another problem
alongside:

    qemuGetPciHostDeviceList set the device as "managed" force
    regardless of whether the device is configured as "managed='yes'"
    or not in XML, which is not right.

Waiting for qemu-kvm cleaning up the PCI bar(s) mapping with long time
while the device is not managed is just waste of time.

BZ# https://bugzilla.redhat.com/show_bug.cgi?id=736214

The problem is caused by the original info of domain's PCI dev is
maintained by qemu_driver->activePciHostdevs list, (E.g. dev->reprobe,
which stands for whether need to reprobe driver for the dev when do
reattachment). The fields (dev->reprobe, dev->unbind_from_stub, and
dev->remove_slot) are initialized properly when preparing the PCI
device for managed attachment. However, when do reattachment, it
construct a complete new "pciDevice" without honoring the original
dev info, and thus the dev won't get the original driver or can get
other problem.

This patch is to fix the problem by get the devs from list
driver->activePciHostdevs.

Tested with following 3 scenarios:
  * the PCI was bound to some driver not pci-stub before attaching

    result: the device will be bound to the original driver

  * the PCI was bound to pci-stub before attaching

    result: no driver reprobing, and still bound to pci-stub

  * The PCI was not bound to any driver

    result: no driver reprobing, and still not bound to any driver.

When failing on starting a domain, it tries to reattach all the PCI
devices defined in the domain conf, regardless of whether the devices
are still used by other domain. This will cause the devices to be deleted
from the list qemu_driver->activePciHostdevs, thus the devices will be
thought as usable even if it's not true. And following commands
nodedev-{reattach,reset} will be successful.

How to reproduce:
  1) Define two domains with same PCI device defined in the confs.
  2) # virsh start domain1
  3) # virsh start domain2
  4) # virsh nodedev-reattach $pci_device

You will see the device will be reattached to host successfully.
As pciDeviceReattach just check if the device is still used by
other domain via checking if the device is in list driver->activePciHostdevs,
however, the device is deleted from the list by step 2).

This patch is to prohibit the bug by:
  1) Prohibit a domain starting or device attachment right at
     preparation period (qemuPrepareHostdevPCIDevices) if the
     device is in list driver->activePciHostdevs, which means
     it's used by other domain.

  2) Introduces a new field for struct _pciDevice, (const char *used_by),
     it will be set as the domain name at preparation period,
     (qemuPrepareHostdevPCIDevices). Thus we can prohibit deleting
     the device from driver->activePciHostdevs if it's still used by
     other domain when stopping the domain process.

* src/pci.h (define two internal functions, pciDeviceSetUsedBy and
    pciDevceGetUsedBy)
* src/pci.c (new field "const char *used_by" for struct _pciDevice,
    implementations for the two new functions)
* src/libvirt_private.syms (Add the two new internal functions)
* src/qemu_hostdev.h (Modify the definition of functions
    qemuPrepareHostdevPCIDevices, and qemuDomainReAttachHostdevDevices)
* src/qemu_hostdev.c (Prohibit preparation and don't delete the
    device from activePciHostdevs list if it's still used by other domain)
* src/qemu_hotplug.c (Update function usage, as the definitions are
    changed)
Signed-off-by: NEric Blake <eblake@redhat.com>

pciDeviceListSteal(pcidevs, dev) removes dev from pcidevs reducing
the length of pcidevs, so moving onto what was the next dev is wrong.

Instead callers should pop entry 0 repeatedly until pcidevs is empty.
Signed-off-by: NSteve Hodgson <shodgson@solarflare.com>
Signed-off-by: NShradha Shah <sshah@solarflare.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Reattach all pci devices that we detached when qemuPrepareHostdevPCIDevices()
failed.

We should not mark pci devices as active when qemuPrepareHostdevPCIDevices()
failed.

The QEMU driver file is far too large. Move all the hostdev
helper code out into a separate file. No functional change.

* src/qemu/qemu_hostdev.c, src/qemu/qemu_hostdev.h,
  src/Makefile.am: Add hostdev helper file
* src/qemu/qemu_driver.c: Delete hostdev code