Add an attribute named 'removable' to the 'target' element of disks,
which controls the removable flag. For instance, on a Linux guest it
controls the value of /sys/block/$dev/removable. This option is only
valid for USB disks (i.e. bus='usb'), and its default value is 'off',
which is the same behaviour as before.

To achieve this, 'removable=on' (or 'off') is appended to the '-device
usb-storage' parameter sent to qemu when adding a USB disk via
'-disk'. A capability flag QEMU_CAPS_USB_STORAGE_REMOVABLE was added
to keep track if this option is supported by the qemu version used.

Bug: https://bugzilla.redhat.com/show_bug.cgi?id=922495Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

Allow use of the usb-storage device only if the new capability flag
QEMU_CAPS_DEVICE_USB_STORAGE is set, which it is for qemu(-kvm)
versions >= 0.12.1.2-rhel62-beta.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

Remove unused 'cgroup' variable in qemuDomainAttachDeviceDiskLive() to
resolve coverity DEADCODE complaint

vhost only works in KVM mode at the moment, and is infact compiled
out if the emulator is built for non-native architecture. While it
may work at some point in the future for plain qemu, for now it's
just noise on the command line (and which contributes to arm cli
breakage).

When using a <interface type="network"> that points to a network with
hostdev forwarding mode a hostdev alias is created for the network. This
allias is inserted into the hostdev list, but is backed with a part of
the network object that it is connected to.

When a VM is being stopped qemuProcessStop() calls
networkReleaseActualDevice() which eventually frees the memory for the
hostdev object. Afterwards when the domain definition is being freed by
virDomainDefFree() an invalid pointer is accessed by
virDomainHostdevDefFree() and may cause a crash of the daemon.

This patch removes the entry in the hostdev list before freeing the
depending memory to avoid this issue.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1000973

QEMU commit 3984890 introduced the "pci-hole64-size" property,
to i440FX-pcihost and q35-pcihost with a default setting of 2 GB.

Translate <pcihole64>x<pcihole64/> to:
-global q35-pcihost.pci-hole64-size=x for q35 machines and
-global i440FX-pcihost.pci-hole64-size=x for i440FX-based machines.

Error out on other machine types or if the size was specified
but the pcihost device lacks 'pci-hole64-size' property.

https://bugzilla.redhat.com/show_bug.cgi?id=990418

The ftp protocol is already recognized by qemu/KVM so add this support to
libvirt as well.
The xml should be as following:

     <disk type='network' device='cdrom'>
       <source protocol='ftp' name='/url/path'>
         <host name='host.name' port='21'/>
       </source>
     </disk>
Signed-off-by: NAline Manera <alinefm@br.ibm.com>

QEMU/KVM already allows a HTTP URL for the cdrom ISO image so add this support
to libvirt as well.
The xml should be as following:

    <disk type='network' device='cdrom'>
      <source protocol='http' name='/url/path'>
        <host name='host.name' port='80'/>
      </source>
    </disk>
Signed-off-by: NAline Manera <alinefm@br.ibm.com>

If there's no hard_limit set and domain uses VFIO we still must lock
the guest memory (prerequisite from qemu). Hence, we should compute
the amount to be locked from max_balloon.

We don't want tests to wait 5 seconds for an event which we know will
never come.

We will have to pass a mock-up of the driver when testing monitor
events.

Otherwise defining variables that hold callbacks pointers is ugly and
several places have to be changed when new parameters are added.

When cpu hotplug fails without reporting an error, we would fail the
command but update the count of vCPUs anyways.

Commit 761fc481 fixed the case when CPU
hot-unplug failed silently, but forgot to fix up the value in this case.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1000357

The virDomainOpenGraphics method accepts a UNIX socket FD from
the client app. It must set the label on this FD otherwise QEMU
will be prevented from receiving it with recvmsg.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If user requested multiqueue networking, beside multiple /dev/tap and
/dev/vhost-net openings, we forgot to pass mq=on onto the -device
virtio-net-pci command line. This is advised at:

  http://www.linux-kvm.org/page/Multiqueue#Enable_MQ_feature

Re-arrange the code so that the returned bitmap is always initialized to
NULL even on early failures and return an error message as some callers
are already expecting it. Fix up the rest not to shadow the error.

https://bugzilla.redhat.com/show_bug.cgi?id=822052

When doing a live migration, if the destination fails for any
reason after the point in which files should be labeled, then
the cleanup of the destination would restore the labels to their
defaults, even though the source is still trying to continue
running with the image open.  Bug 822052 mentioned one source
of live migration failure - a mismatch in SELinux virt_use_nfs
settings (on for source, off for destination); but I found other
situations that would also trigger it (for example, having a
graphics device tied to port 5999 on the source, and a different
domain on the destination already using that port, so that the
destination cannot reuse the port).

In short, just as cleanup of the source on a successful migration
must not relabel files (because the destination would be crippled
by the relabel), cleanup of the destination on a failed migration
must not relabel files (because the source would be crippled).

* src/qemu/qemu_process.c (qemuProcessStart): Set flag to avoid
label restoration when cleaning up on failed migration.
Signed-off-by: NEric Blake <eblake@redhat.com>

Each of the modules handled reporting error messages from the secret fetching
slightly differently with respect to the error. Provide a similar message
for each error case and provide as much data as possible.

Following XML would fail :

    <disk type='network' device='lun'>
      <driver name='qemu' type='raw'/>
      <source protocol='iscsi' name='iqn.2013-07.com.example:iscsi/1'>
        <host name='example.com' port='3260'/>
      </source>
      <target dev='sda' bus='scsi'/>
    </disk>

With the message:

error: Failed to start domain iscsilun
error: Unable to get device ID 'iqn.2013-07.com.example:iscsi/1': No such fi

Cause was commit id '1f49b05a' which added 'virDomainDiskSourceIsBlockType'

If there's no hard_limit set and domain uses VFIO we still must lock the
guest memory (prerequisite from qemu). Hence, we should compute the
amount to be locked from max_balloon.

Since 16bcb3 we have a regression. The hard_limit is set
unconditionally. By default the limit is zero. Hence, if user hasn't
configured any, we set the zero in cgroup subsystem making the kernel
kill the corresponding qemu process immediately. The proper fix is to
set hard_limit iff user has configured any.

This function is to guess the correct limit for maximal memory
usage by qemu for given domain. This can never be guessed
correctly, not to mention all the pains and sleepless nights this
code has caused. Once somebody discovers algorithm to solve the
Halting Problem, we can compute the limit algorithmically. But
till then, this code should never see the light of the release
again.

Currently the virConnectBaselineCPU API does not expose the CPU features
that are part of the CPU's model.  This patch adds a new flag,
VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES, that causes the API to explicitly
list all features that are part of that model.
Signed-off-by: NDon Dugger <donald.d.dugger@intel.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

This restores the error message when QMP probing is not used.

https://bugzilla.redhat.com/show_bug.cgi?id=991334

qemuDomainAttachVirtioDiskDevice passes NULL as domainDef which is later
referenced in qemuDomainAttachVirtioDiskDevice:

 Program terminated with signal 11, Segmentation fault.
 #0  qemuBuildDeviceAddressStr (buf=buf@entry=0xb646de78, info=info@entry=0xb0a02360, qemuCaps=qemuCaps@entry=0xb8fdfdc8,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>) at qemu/qemu_command.c:2869
 2869            for (i = 0; i < domainDef->ncontrollers; i++) {
 (gdb) bt
 #0  qemuBuildDeviceAddressStr (buf=buf@entry=0xb646de78, info=info@entry=0xb0a02360, qemuCaps=qemuCaps@entry=0xb8fdfdc8,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>) at qemu/qemu_command.c:2869
 #1  0xb18ad6f8 in qemuBuildDriveDevStr (def=def@entry=0x0, disk=disk@entry=0xb0a02288, bootindex=bootindex@entry=0, qemuCaps=0xb8fdfdc8)
     at qemu/qemu_command.c:4316
 #2  0xb18d097f in qemuDomainAttachVirtioDiskDevice (conn=conn@entry=0xb90129a8, driver=driver@entry=0xb8fe29b8, vm=vm@entry=0xb8fe0c40,
     disk=disk@entry=0xb0a02288) at qemu/qemu_hotplug.c:278
 #3  0xb193f7ba in qemuDomainAttachDeviceDiskLive (dev=0xb0a35308, vm=0xb8fe0c40, driver=0xb8fe29b8, conn=0xb90129a8) at qemu/qemu_driver.c:6356
 #4  qemuDomainAttachDeviceLive (dev=0xb0a35308, vm=0xb8fe0c40, dom=<optimized out>) at qemu/qemu_driver.c:6418
 #5  qemuDomainAttachDeviceFlags (dom=dom@entry=0xb0a020b8,
     xml=xml@entry=0xb90953f0 "<disk type='file' device='disk'>\n  <source file='/var/lib/jenkins/jobs/libvirt-tck-build/workspace/scratchdir/200-disk-hotplug/extra.img'/>\n  <target dev='vdb' bus='virtio'/>\n</disk>\n", flags=3103664568, flags@entry=1) at qemu/qemu_driver.c:7079
 #6  0xb193f9cb in qemuDomainAttachDevice (dom=0xb0a020b8,
     xml=0xb90953f0 "<disk type='file' device='disk'>\n  <source file='/var/lib/jenkins/jobs/libvirt-tck-build/workspace/scratchdir/200-disk-hotplug/extra.img'/>\n  <target dev='vdb' bus='virtio'/>\n</disk>\n") at qemu/qemu_driver.c:7120
 #7  0xb7244827 in virDomainAttachDevice (domain=domain@entry=0xb0a020b8,
     xml=0xb90953f0 "<disk type='file' device='disk'>\n  <source file='/var/lib/jenkins/jobs/libvirt-tck-build/workspace/scratchdir/200-disk-hotplug/extra.img'/>\n  <target dev='vdb' bus='virtio'/>\n</disk>\n") at libvirt.c:10912
 #8  0xb7765ddb in remoteDispatchDomainAttachDevice (args=0xb9094ef0, rerr=0xb646e1f0, client=<optimized out>, server=<optimized out>,
     msg=<optimized out>) at remote_dispatch.h:2296
 #9  remoteDispatchDomainAttachDeviceHelper (server=0xb8fba0e8, client=0xb0a00730, msg=0xb0a350b8, rerr=0xb646e1f0, args=0xb9094ef0, ret=0xb9094dc8)
     at remote_dispatch.h:2274
 #10 0xb72b1013 in virNetServerProgramDispatchCall (msg=0xb0a350b8, client=0xb0a00730, server=0xb8fba0e8, prog=0xb8fc21c8)
     at rpc/virnetserverprogram.c:435
 #11 virNetServerProgramDispatch (prog=0xb8fc21c8, server=server@entry=0xb8fba0e8, client=0xb0a00730, msg=0xb0a350b8) at rpc/virnetserverprogram.c:305
 #12 0xb72aa167 in virNetServerProcessMsg (msg=<optimized out>, prog=<optimized out>, client=<optimized out>, srv=0xb8fba0e8)
     at rpc/virnetserver.c:165
 #13 virNetServerHandleJob (jobOpaque=0xb0a0a850, opaque=0xb8fba0e8) at rpc/virnetserver.c:186
 #14 0xb7189108 in virThreadPoolWorker (opaque=opaque@entry=0xb8fa3250) at util/virthreadpool.c:144
 #15 0xb71885e5 in virThreadHelper (data=0xb8fa32a8) at util/virthreadpthread.c:161
 #16 0xb70d6954 in start_thread (arg=0xb646eb70) at pthread_create.c:304
 #17 0xb704e95e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130

This was found by libvirtt-tck:

     http://honk.sigxcpu.org:8001/job/libvirt-tck-debian-wheezy-qemu-session/1311/console

Hotplugging a single SCSI device works, but adding additional ones
result in an error from QEMU:

[root@gpok197 ~]# virsh attach-device guest01 blah.xml
Device attached successfully
[root@gpok197 ~]# virsh attach-device guest01 blah2.xml
error: Failed to attach device from blah2.xml
error: internal error unable to execute QEMU command 'device_add': Duplicate ID 'hostdev0' for device

The hostdev ID that is created is always set to zero, regardless
of the contents of the XML.  Changing the index in the hotplug case
to a negative one so the next available index is used.
Signed-off-by: NEric Farman <farman@linux.vnet.ibm.com>
Reviewed-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

Go through disks of guest, if one disk doesn't exist or its backing
chain is broken, with 'optional' startupPolicy, for CDROM and Floppy
we only discard its source path definition in xml, for disks we drop
it from disk list and free it.

This patch addresses two concerns with the error reporting when an
incompatible PCI address is specified for a device:

1) It wasn't always apparent which device had the problem. With this
patch applied, any error about an incompatible address will always
contain the full address as given in the config, so it will be easier
to determine which device's config aused the problem.

2) In some cases when the problem came from bad config, the error
message was erroneously classified as VIR_ERR_INTERNAL_ERROR. With
this patch applied, the same error message will be changed to indicate
either "internal" or "xml" error depending on whether the address came
from the config, or was automatically generated by libvirt.

Note that in the case of "internal" (due to bad auto-generation)
errors, the PCI address won't be of much use in finding the location
in config to change (because it was automatically generated). Of
course that makes perfect sense, but still the address could provide a
clue about a bug in libvirt attempting to use a type of pci bus that
doesn't have its flags set correctly (or something similar). In other
words, it's not perfect, but it is definitely better.

q35 machines have an implicit ahci (sata) controller at 00:1F.2 which
has no "id" associated with it. For this reason, we can't refer to it
as "ahci0". Instead, we don't give an id on the commandline, which
qemu interprets as "use the first ahci controller". We then need to
specify the unit with "unit=%d" rather than adding it onto the bus
arg.

https://bugzilla.redhat.com/show_bug.cgi?id=979477

Since 1.0.3 we are using the new way to copy non shared storage during
migration (the NBD way). However, whether the new or old way is used is
not controllable by user but unconditionally turned on if both sides of
migration support it. Moreover, the implementation is not complete: the
combination for VIR_MIGRATE_TUNNELLED flag is missing (as we need to
open new port on the destination) in which case we just error out. This
is a deadly combination: not letting users choose their destiny and
erroring out. We should not do that but VIR_WARN and turn the NBD off
instead.

We had been setting the device alias in the devinceinfo for pci
controllers to "pci%u", but then hardcoding "pci.%u" when creating the
device address for other devices using that pci bus. This all worked
just fine until we encountered the built-in "pcie.0" bus (the PCIe
root complex) in Q35 machines.

In order to create the correct commandline for this one case, this
patch:

1) sets the alias for PCI controllers correctly, to "pci.%u" (or
"pcie.%u" for the pcie-root controller)

2) eliminates the hardcoded "pci.%u" for pci controllers when
generatuing device address strings, and instead uses the controller's
alias.

3) plumbs a pointer to the virDomainDef all the way down to
qemuBuildDeviceAddressStr. This was necessary in order to make the
aliase of the controller *used by a device* available (previously
qemuBuildDeviceAddressStr only had the deviceinfo of the device
itself, *not* of the controller it was connecting to). This made for a
larger than desired diff, but at least in the future we won't have to
do it again, since all the information we could possibly ever need for
future enhancements is in the virDomainDef. (right?)

This should be done for *all* controllers, but for now we just do it
in the case of PCI controllers, to reduce the likelyhood of
regression.

This patch adds in special handling for a few devices that need to be
treated differently for q35 domains:

usb - there is no implicit/default usb controller for the q35
machinetype. This is done because normally the default usb controller
is added to a domain by just adding "-usb" to the qemu commandline,
and it's assumed that this will add a single piix3 usb1 controller at
slot 1 function 2. That's not what happens when the machinetype is
q35, though. Instead, adding -usb to the commandline adds 3 usb
(version 2) controllers to the domain at slot 0x1D.{1,2,7}. Rather
than having

  <controller type='usb' index='0'/>

translate into 3 separate devices on the PCI bus, it's cleaner to not
automatically add a default usb device; one can always be added
explicitly if desired. Or we may decide that on q35 machines, 3 usb
controllers will be automatically added when none is given. But for
this initial commit, at least we aren't locking ourselves into
something we later won't want.

video - qemu always initializes the primary video device immediately
after any integrated devices for the machinetype. Unless instructed
otherwise (by using "-device vga..." instead of "-vga" which libvirt
uses in many cases to work around deficiencies and bugs in various
qemu versions) qemu will always pick the first unused slot. In the
case of the "pc" machinetype and its derivatives, this is always slot
2, but on q35 machinetypes, the first free slot is slot 1 (since the
q35's integrated peripheral devices are placed in other slots,
e.g. slot 0x1f). In order to make the PCI address of the video device
predictable, that slot (1 or 2, depending on machinetype) is reserved
even when no video device has been specified.

sata - a q35 machine always has a sata controller implicitly added at
slot 0x1F, function 2. There is no way to avoid this controller, so we
always add it. Note that the xml2xml tests for the pcie-root and q35
cases were changed to use DO_TEST_DIFFERENT() so that we can check for
the sata controller being automatically added. This is especially
important because we can't check for it in the xml2argv output (it has
no effect on that output since it's an implicit device).

ide - q35 has no ide controllers.

isa and smbus controllers - these two are always present in a q35 (at
slot 0x1F functions 0 and 3) but we have no way of modelling them in
our config. We do need to reserve those functions so that the user
doesn't attempt to put anything else there though. (note that the "pc"
machine type also has an ISA controller, which we also ignore).

This PCI controller, named "dmi-to-pci-bridge" in the libvirt config,
and implemented with qemu's "i82801b11-bridge" device, connects to a
PCI Express slot (e.g. one of the slots provided by the pcie-root
controller, aka "pcie.0" on the qemu commandline), and provides 31
*non-hot-pluggable* PCI (*not* PCIe) slots, numbered 1-31.

Any time a machine is defined which has a pcie-root controller
(i.e. any q35-based machinetype), libvirt will automatically add a
dmi-to-pci-bridge controller if one doesn't exist, and also add a
pci-bridge controller. The reasoning here is that any useful domain
will have either an immediate (startup time) or eventual (subsequent
hot-plug) need for a standard PCI slot; since the pcie-root controller
only provides PCIe slots, we need to connect a dmi-to-pci-bridge
controller to it in order to get a non-hot-plug PCI slot that we can
then use to connect a pci-bridge - the slots provided by the
pci-bridge will be both standard PCI and hot-pluggable.

Since pci-bridge devices themselves can not be hot-plugged into a
running system (although you can hot-plug other devices into a
pci-bridge's slots), any new pci-bridge controller that is added can
(and will) be plugged into the dmi-to-pci-bridge as long as it has
empty slots available.

This patch is also changing the qemuxml2xml-pcie test from a "DO_TEST"
to a "DO_DIFFERENT_TEST". This is so that the "before" xml can omit
the automatically added dmi-to-pci-bridge and pci-bridge devices, and
the "after" xml can include it - this way we are testing if libvirt is
properly adding these devices.

This controller is implicit on q35 machinetypes. It provides 31 PCIe
(*not* PCI) slots as controller 0.

Currently there are no devices that can connect to pcie-root, and no
implicit pci controller on a q35 machine, so q35 is still
unusable. For a usable q35 system, we need to add a
"dmi-to-pci-bridge" pci controller, which can connect to pcie-root,
and provides standard pci slots that can be used to connect other
devices.

Previous refactoring of the guest PCI address reservation/allocation
code allowed for slot types other than basic PCI (e.g. PCI express,
non-hotpluggable slots, etc) but would not auto-allocate a slot for a
device that required any type other than a basic hot-pluggable
PCI slot.

This patch refactors the code to be aware of different slot types
during auto-allocation of addresses as well - as long as there is an
empty slot of the required type, it will be found and used.

The piece that *wasn't* added is that we don't auto-create a new PCI
bus when needed for anything except basic PCI devices. This is because
there are multiple different types of controllers that can provide,
for example, a PCI express slot (in addition to the pcie-root
controller, these can also be found on a "root-port" or on a
"downstream-switch-port"). Since we currently don't support any PCIe
devices (except pending support for dmi-to-pci-bridge), we can defer
any decision on what to do about this.