Set the VIR_PCI_CONNECT_AGGREGATE_SLOT flag for pcie-root-ports so
that they will be assigned to all the functions on a slot.

Some qemu test case outputs had to be adjusted due to the
pcie-root-ports now being put on multiple functions.

If there are multiple devices assigned to the different functions of a
single PCI slot, they will not work properly if the device at function
0 doesn't have its "multi" attribute turned on, so it makes sense for
libvirt to turn it on during PCI address assignment. Setting multi
then assures that the new setting is stored in the config (so it will
be used next time the domain is started), preventing any potential
problems in the case that a future change in the configuration
eliminates the devices on all non-0 functions (multi will still be set
for function 0 even though it is the only function in use on the slot,
which has no useful purpose, but also doesn't cause any problems).

(NB: If we were to instead just decide on the setting for
multifunction at runtime, a later removal of the non-0 functions of a
slot would result in a silent change in the guest ABI for the
remaining device on function 0 (although it may seem like an
inconsequential guest ABI change, it *is* a guest ABI change to turn
off the multi bit).)

virtio-pci is the way forward for aarch64 guests: it's faster
and less alien to people coming from other architectures.
Now that guest support is finally getting there (Fedora 24,
CentOS 7.3, Ubuntu 16.04 and Debian testing all support
virtio-pci out of the box), we'd like to start using it by
default instead of virtio-mmio.

Users and applications can already opt-in by explicitly using

  <address type='pci'/>

inside the relevant elements, but that's kind of cumbersome and
requires all users and management applications to adapt, which
we'd really like to avoid.

What we can do instead is use virtio-mmio only if the guest
already has at least one virtio-mmio device, and use virtio-pci
in all other situations.

That means existing virtio-mmio guests will keep using the old
addressing scheme, and new guests will automatically be created
using virtio-pci instead. Users can still override the default
in either direction.

Existing tests such as aarch64-aavmf-virtio-mmio and
aarch64-virtio-pci-default already cover all possible
scenarios, so no additions to the test suites are necessary.

Modify _virDomainBlockIoTuneInfo and rng schema to support the group_name
option for iotune throttling. Document the new value.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

With the QEMU components in place, provide the XML parsing to
invoke that code when given the following XML snippet:

    <hostdev mode='subsystem' type='scsi_host'>
      <source protocol='vhost' wwpn='naa.501234567890abcd'/>
    </hostdev>

An optional address element can be specified within the hostdev
(pick CCW or PCI as necessary):

    <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0625'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

Add basic vhost-scsi tests which were cloned from hostdev-scsi-virtio-scsi
in both xml2argv and xml2xml. Added ones for both vhost-scsi-ccw and
vhost-scsi-pci since the syntaxes are slightly different between them.

Also adjusted the docs to describe the changes.
Signed-off-by: NEric Farman <farman@linux.vnet.ibm.com>
Reviewed-by: NBoris Fiuczynski <fiuczy@linux.vnet.ibm.com>

For machinetypes with a pci-root bus (all legacy PCI), libvirt will
make a "fake" reservation for one extra slot prior to assigning
addresses to unaddressed PCI endpoint devices in the domain. This will
trigger auto-adding of a pci-bridge for the final device to be
assigned an address *if that device would have otherwise instead been
the last device on the last available pci-bridge*; thus it assures
that there will always be at least one slot left open in the domain's
bus topology for expansion (which is important both for hotplug (since
a new pci-bridge can't be added while the guest is running) as well as
for offline additions to the config (since adding a new device might
otherwise in some cases require re-addressing existing devices, which
we want to avoid)).

It's important to note that for the above case (legacy PCI), we must
check for the special case of all slots on all buses being occupied
*prior to assigning any addresses*, and avoid attempting to reserve
the extra address in that case, because there is no free address in
the existing topology, so no place to auto-add a pci-bridge for
expansion (i.e. it would always fail anyway). Since that condition can
only be reached by manual intervention, this is acceptable.

For machinetypes with pcie-root (Q35, aarch64 virt), libvirt's
methodology for automatically expanding the bus topology is different
- pcie-root-ports are plugged into slots (soon to be functions) of
pcie-root as needed, and the new endpoint devices are assigned to the
single slot in each pcie-root-port. This is done so that the devices
are, by default, hotpluggable (the slots of pcie-root don't support
hotplug, but the single slot of the pcie-root-port does). Since
pcie-root-ports can only be plugged into pcie-root, and we don't
auto-assign endpoint devices to the pcie-root slots, this means
topology expansion doesn't compete with endpoint devices for slots, so
we don't need to worry about checking for all "useful" slots being
free *prior* to assigning addresses to new endpoint devices - as a
matter of fact, if we attempt to reserve the open slots before the
used slots, it can lead to errors.

Instead this patch just reserves one slot for a "future potential"
PCIe device after doing the assignment for actual devices, but only
if the only PCI controller defined prior to starting address
assignment was pcie-root, and only if we auto-added at least one PCI
controller during address assignment. This assures two things:

1) that reserving the open slots will only be done when the domain is
   initially defined, never at any time after, and

2) that if the user understands enough about PCI controllers that they
   are adding them manually, that we don't mess up their plan by
   adding extras - if they know enough to add one pcie-root-port, or
   to manually assign addresses such that no pcie-root-ports are
   needed, they know enough to add extra pcie-root-ports if they want
   them (this could be called the "libguestfs clause", since
   libguestfs needs to be able to create domains with as few
   devices/controllers as possible).

This is set to reserve a single free port for now, but could be
increased in the future if public sentiment goes in that direction
(it's easy to increase later, but essentially impossible to decrease)

Real Q35 hardware has an ICH9 chip that includes several integrated
devices at particular addresses (see the file docs/q35-chipset.cfg in
the qemu source). libvirt already attempts to put the first two sets
of ich9 USB2 controllers it finds at 00:1D.* and 00:1A.* to match the
real hardware. This patch does the same for the ich9 "HD audio"
device.

The main inspiration for this patch is that currently the *only*
device in a reasonable "workstation" type virtual machine config that
requires a legacy PCI slot is the audio device, Without this patch,
the standard Q35 machine created by virt-manager will have a
dmi-to-pci-bridge and a pci-bridge just for the sound device; with the
patch (and if you change the sound device model from the default
"ich6" to "ich9"), the machine definition constructed by virt-manager
has absolutely no legacy PCI controllers - any legacy PCI devices
(e.g. video and sound) are on pcie-root as integrated devices.

Previously we added a set of EHCI+UHCI controllers to Q35 machines to
mimic real hardware as closely as possible, but recent discussions
have pointed out that the nec-usb-xhci (USB3) controller is much more
virtualization-friendly (uses less CPU), so this patch switches the
default for Q35 machinetypes to add an XHCI instead (if it's
supported, which it of course *will* be).

Since none of the existing test cases left out USB controllers in the
input XML, a new Q35 test case was added which has *no* devices, so
ends up with only the defaults always put in by qemu, plus those added
by libvirt.

Previously libvirt would only add pci-bridge devices automatically
when an address was requested for a device that required a legacy PCI
slot and none was available. This patch expands that support to
dmi-to-pci-bridge (which is needed in order to add a pci-bridge on a
machine with a pcie-root), and pcie-root-port (which is needed to add
a hotpluggable PCIe device). It does *not* automatically add
pcie-switch-upstream-ports or pcie-switch-downstream-ports (and
currently there are no plans for that).

Given the existing code to auto-add pci-bridge devices, automatically
adding pcie-root-ports is fairly straightforward. The
dmi-to-pci-bridge support is a bit tricky though, for a few reasons:

1) Although the only reason to add a dmi-to-pci-bridge is so that
   there is a reasonable place to plug in a pci-bridge controller,
   most of the time it's not the presence of a pci-bridge *in the
   config* that triggers the requirement to add a dmi-to-pci-bridge.
   Rather, it is the presence of a legacy-PCI device in the config,
   which triggers auto-add of a pci-bridge, which triggers auto-add of
   a dmi-to-pci-bridge (this is handled in
   virDomainPCIAddressSetGrow() - if there's a request to add a
   pci-bridge we'll check if there is a suitable bus to plug it into;
   if not, we first add a dmi-to-pci-bridge).

2) Once there is already a single dmi-to-pci-bridge on the system,
   there won't be a need for any more, even if it's full, as long as
   there is a pci-bridge with an open slot - you can also plug
   pci-bridges into existing pci-bridges. So we have to make sure we
   don't add a dmi-to-pci-bridge unless there aren't any
   dmi-to-pci-bridges *or* any pci-bridges.

3) Although it is strongly discouraged, it is legal for a pci-bridge
   to be directly plugged into pcie-root, and we don't want to
   auto-add a dmi-to-pci-bridge if there is already a pci-bridge
   that's been forced directly into pcie-root.

Although libvirt will now automatically create a dmi-to-pci-bridge
when it's needed, the code still remains for now that forces a
dmi-to-pci-bridge on all domains with pcie-root (in
qemuDomainDefAddDefaultDevices()). That will be removed in a future
patch.

For now, the pcie-root-ports are added one to a slot, which is a bit
wasteful and means it will fail after 31 total PCIe devices (30 if
there are also some PCI devices), but helps keep the changeset down
for this patch. A future patch will have 8 pcie-root-ports sharing the
functions on a single slot.

The nec-usb-xhci device (which is a USB3 controller) has always
presented itself as a PCI device when plugged into a legacy PCI slot,
and a PCIe device when plugged into a PCIe slot, but libvirt has
always auto-assigned it to a legacy PCI slot.

This patch changes that behavior to auto-assign to a PCIe slot on
systems that have pcie-root (e.g. Q35 and aarch64/virt).

Since we don't yet auto-create pcie-*-port controllers on demand, this
means a config with an nec-xhci USB controller that has no PCI address
assigned will also need to have an otherwise-unused pcie-*-port
controller specified:

   <controller type='pci' model='pcie-root-port'/>
   <controller type='usb' model='nec-xhci'/>

(this assumes there is an otherwise-unused slot on pcie-root to accept
the pcie-root-port)

libvirt previously assigned nearly all devices to a "hotpluggable"
legacy PCI slot even on machines with a PCIe root bus (and even though
most such machines don't even support hotplug on legacy PCI slots!)
Forcing all devices onto legacy PCI slots means that the domain will
need a dmi-to-pci-bridge (to convert from PCIe to legacy PCI) and a
pci-bridge (to provide hotpluggable legacy PCI slots which, again,
usually aren't hotpluggable anyway).

To help reduce the need for these legacy controllers, this patch tries
to assign virtio-1.0-capable devices to PCIe slots whenever possible,
by setting appropriate connectFlags in
virDomainCalculateDevicePCIConnectFlags(). Happily, when that function
was written (just a few commits ago) it was created with a
"virtioFlags" argument, set by both of its callers, which is the
proper connectFlags to set for any virtio-*-pci device - depending on
the arch/machinetype of the domain, and whether or not the qemu binary
supports virtio-1.0, that flag will have either been set to PCI or
PCIe. This patch merely enables the functionality by setting the flags
for the device to whatever is in virtioFlags if the device is a
virtio-*-pci device.

NB: the first virtio video device will be placed directly on bus 0
slot 1 rather than on a pcie-root-port due to the override for primary
video devices in qemuDomainValidateDevicePCISlotsQ35(). Whether or not
to change that is a topic of discussion, but this patch doesn't change
that particular behavior.

NB2: since the slot must be hotpluggable, and pcie-root (the PCIe root
complex) does *not* support hotplug, this means that suitable
controllers must also be in the config (i.e. either pcie-root-port, or
pcie-downstream-port). For now, libvirt doesn't add those
automatically, so if you put virtio devices in a config for a qemu
that has PCIe-capable virtio devices, you'll need to add extra
pcie-root-ports yourself. That requirement will be eliminated in a
future patch, but for now, it's simple to do this:

   <controller type='pci' model='pcie-root-port'/>
   <controller type='pci' model='pcie-root-port'/>
   <controller type='pci' model='pcie-root-port'/>
   ...

Partially Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1330024

The old ivshmem is deprecated in QEMU, so let's use the better
ivshmem-{plain,doorbell} variants instead.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Modify _virDomainBlockIoTuneInfo and rng schema to support the _length
options for bps/iops throttling values. Document the new values.

Add an optional "tls='yes|no'" attribute for a TCP chardev.

For QEMU, this will allow for disabling the host config setting of the
'chardev_tls' for a domain chardev channel by setting the value to "no" or
to attempt to use a host TLS environment when setting the value to "yes"
when the host config 'chardev_tls' setting is disabled, but a TLS environment
is configured via either the host config 'chardev_tls_x509_cert_dir' or
'default_tls_x509_cert_dir'
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

The code is entirely correct, but it still managed to trip me
up when I first ran into it because I did not realize right away
that VIR_PCI_CONNECT_TYPES_ENDPOINT was not a single flag, but
rather a mask including both VIR_PCI_CONNECT_TYPE_PCI_DEVICE and
VIR_PCI_CONNECT_TYPE_PCIE_DEVICE.

In order to save the next distracted traveler in PCI Address Land
some time, document this fact with a comment. Add a test case for
the behavior as well.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1369633Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

If QEMU in question supports QMP, this capability is set if
QEMU_CAPS_DEVICE_QXL was set based on existence of "-device qxl". If
libvirt needs to parse *help*, because there is no QMP support, it
checks for existence of "-vga qxl", but it also parses output of
"-device ?" and sets QEMU_CAPS_DEVICE_QXL too.

Now that libvirt supports only QEMU that has "-device" implemented it's
safe to drop this capability and stop using it.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

This patch simplifies QEMU capabilities for QXL video device.  QEMU
exposes this device as *qxl-vga* and *qxl* and they are both the same
device with the same set of parameters, the only difference is that
*qxl-vga* includes VGA compatibility.

Based on QEMU code they are tied together so it's safe to check only for
presence of only one of them.

This patch also removes an invalid test case "video-qxl-sec-nodevice"
where there is only *qxl-vga* device and *qxl* device is not present.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

If one of QEMU_CAPS_DEVICE_QXL_VGA or QEMU_CAPS_DEVICE_QXL is set the
other one will always be set as well because both devices are tied
together in QEMU.

The change of args files is caused by the presence of capability
QEMU_CAPS_DEVICE_VIDEO_PRIMARY which means it's safe to use
"-device qxl-vga" instead of "-vga qxl", see commit (e3f2686b) and
by the fact that if QEMU_CAPS_VGA_QXL is set QEMU_CAPS_DEVICE_QXL_VGA
and QEMU_CAPS_DEVICE_QXL would be set too (since we support only qemu
with "-device" option).
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

It was missing... Also since I'm using the soft link from qemuxml2xmloutdata
to the qemuxml2argvdata file, modify the output file to have the necessary
<address> elements plus the mouse and keyboard.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Most of QEMU's PCI display device models, such as:

  libvirt video/model/@type  QEMU -device
  -------------------------  ------------
  cirrus                     cirrus-vga
  vga                        VGA
  qxl                        qxl-vga
  virtio                     virtio-vga

come with a linear framebuffer (sometimes called "VGA compatibility
framebuffer"). This linear framebuffer lives in one of the PCI device's
MMIO BARs, and allows guest code (primarily: firmware drivers, and
non-accelerated OS drivers) to display graphics with direct memory access.

Due to architectural reasons on aarch64/KVM hosts, this kind of
framebuffer doesn't / can't work in

  qemu-system-(arm|aarch64) -M virt

machines. Cache coherency issues guarantee a corrupted / unusable display.
The problem has been researched by several people, including kvm-arm
maintainers, and it's been decided that the best way (practically the only
way) to have boot time graphics for such guests is to consolidate on
QEMU's "virtio-gpu-pci" device.

>From <https://bugzilla.redhat.com/show_bug.cgi?id=1195176>, libvirt
supports

  <devices>
    <video>
      <model type='virtio'/>
    </video>
  </devices>

but libvirt unconditionally maps @type='virtio' to QEMU's "virtio-vga"
device model. (See the qemuBuildDeviceVideoStr() function and the
"qemuDeviceVideo" enum impl.)

According to the above, this is not right for the "virt" machine type; the
qemu-system-(arm|aarch64) binaries don't even recognize the "virtio-vga"
device model (justifiedly). Whereas "virtio-gpu-pci", which is a pure
virtio device without a compatibility framebuffer, is available, and works
fine.

(The ArmVirtQemu ("AAVMF") platform of edk2 -- that is, the UEFI firmware
for "virt" -- supports "virtio-gpu-pci", as of upstream commit
3ef3209d3028. See
<https://tianocore.acgmultimedia.com/show_bug.cgi?id=66>.)

Override the default mapping of "virtio", from "virtio-vga" to
"virtio-gpu-pci", if qemuDomainMachineIsVirt() evaluates to true.

Cc: Andrea Bolognani <abologna@redhat.com>
Cc: Drew Jones <drjones@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Martin Kletzander <mkletzan@redhat.com>
Suggested-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1372901Signed-off-by: NLaszlo Ersek <lersek@redhat.com>
Acked-by: NMartin Kletzander <mkletzan@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1366989

QEMU added another virtio-net tunable [1]. It basically allows
users to set the size of RX virtio ring. But because virtio-net
uses two separate ring buffers to pass data from/to guest they
named it explicitly rx_queue_size. We should expose it in our XML
too.

1: http://lists.nongnu.org/archive/html/qemu-devel/2016-08/msg02029.htmlSigned-off-by: NMichal Privoznik <mprivozn@redhat.com>

Add a new TLS X.509 certificate type - "chardev". This will handle the
creation of a TLS certificate capability (and possibly repository) for
properly configured character device TCP backends.

Unlike the vnc and spice there is no "listen" or "passwd" associated. The
credentials eventually will be handled via a libvirt secret provided to
a specific backend.

Make use of the default verify option as well.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

All these configurations are already covered for qemuxml2argv,
but there were no equivalent tests for qemuxml2xml.

Now that DO_TEST() can be passed capabilities, there is little
need to use DO_TEST_FULL() instead of DO_TEST().

This will allow us to remove most DO_TEST_FULL() usages. For the
time being, just add the extra argument to all DO_TEST() calls.

A bunch of cases were only being tested for WHEN_ACTIVE or
WHEN_INACTIVE. Use WHEN_BOTH for all except the very few that
actually require the existing setup.

Instead of testing it twice using WHEN_ACTIVE and WHEN_INACTIVE
separately, just use WHEN_BOTH.

Failure to parse a XML that was not supposed to fail would result into a
crash in the test suite as the vcpu bitmap would not be filled prior to
the active XML->XML test.

Skip formatting of the vcpu snippet in the fake status XML formatter in
such case to avoid the crash. The test would fail anyways.

We were requiring a USB port path in the schema, but not enforcing it.
Omitting the USB port would lead to libvirt formatting it as (null).
Such domain cannot be started and will disappear after libvirtd restart
(since it cannot parse back the XML).

Only format the port if it has been specified and mark it as optional
in the XML schema.

Commit id's '9bbf0d7e' and '2552fec2' added some XML parsing tests
for a LUKS volume to use a 'passphrase' secret format. After commit,
this was deemed to be incorrect, so covert the various tests to use
the volume usage format where the 'usage' is the path to the volume
rather than a user defined name string.

Also, removed the qemuxml2argv-luks-disk-cipher.xml since it was
just a duplicate of qemuxml2argv-luks-disks.xml.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

A device with an attribute 'model', with just one model
so far:

<devices>
  ...
  <iommu model='intel'/>
</devices>

https://bugzilla.redhat.com/show_bug.cgi?id=1235580

Note the vcpu ID so that once we allow non-contiguous vCPU topologies it
will be possible to pair thread id's with the vcpus.

Status XML tests were done by prepending a constant string to an
existing XML. With the planned changes the header will depend on data
present in the definition rather than just on the data that was parsed.

The first dynamic element in the header will be the vcpu thread list.
Reuse and rename qemuXML2XMLPreFormatCallback for gathering the relevant
data when checking the active XML parsing and formating and pass the
bitmap to a newly crated header generator.

This is preferrable to -nographic which (in addition to disabling
graphics output) redirects the serial port to stdio and on OpenBIOS
enables the firmware's serial console.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

For type='ethernet' interfaces only.

(This patch had been pushed earlier in
commit 0b4645a7, but was reverted in
commit 84d47a3c because it had been
accidentally pushed during the freeze for release 2.0.0)

For a luks device, allow the configuration of a specific cipher to be
used for encrypting the volume.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Add parse and format of the luks/passphrase secret including tests for
volume XML parsing.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

In order to use more common code and set up for a future type, modify the
encryption secret to allow the "usage" attribute or the "uuid" attribute
to define the secret. The "usage" in the case of a volume secret would be
the path to the volume as dictated by the backwards compatibility brought
on by virStorageGenerateQcowEncryption where it set up the usage field as
the vol->target.path and didn't allow someone to provide it. This carries
into virSecretObjListFindByUsageLocked which takes the secret usage attribute
value from from the domain disk definition and compares it against the
usage type from the secret definition. Since none of the code dealing
with qcow/qcow2 encryption secrets uses usage for lookup, it's a mostly
cosmetic change. The real usage comes in a future path where the encryption
is expanded to be a luks volume and the secret will allow definition of
the usage field.

This code will make use of the virSecretLookup{Parse|Format}Secret common code.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

This reverts commit 0b4645a7.

This feature was accidentally pushed in the feature freeze.