Since PCI bridges, PCIe bridges, PCIe switches, and PCIe root ports
all share the same namespace, they are all defined as controllers of
type='pci' in libvirt (but with a differing model attribute). Each of
these controllers has a certain connection type upstream, allows
certain connection types downstream, and each can either allow a
single downstream connection at slot 0, or connections from slot 1 -
31.

Right now, we only support the pci-root and pci-bridge devices, both
of which only allow PCI devices to connect, and both which have usable
slots 1 - 31. In preparation for adding other types of controllers
that have different capabilities, this patch 1) adds info to the
qemuDomainPCIAddressBus object to indicate the capabilities, 2) sets
those capabilities appropriately for pci-root and pci-bridge devices,
and 3) validates that the controller being connected to is the proper
type when allocating slots or validating that a user-selected slot is
appropriate for a device..

Having this infrastructure in place will make it much easier to add
support for the other PCI controller types.

While it would be possible to do all the necessary checking by just
storing the controller model in the qemyuDomainPCIAddressBus, it
greatly simplifies all the validation code to also keep a "flags",
"minSlot" and "maxSlot" for each - that way we can just check those
attributes rather than requiring a nearly identical switch statement
everywhere we need to validate compatibility.

You may notice many places where the flags are seemingly hard-coded to

  QEMU_PCI_CONNECT_HOTPLUGGABLE | QEMU_PCI_CONNECT_TYPE_PCI

This is currently the correct value for all PCI devices, and in the
future will be the default, with small bits of code added to change to
the flags for the few devices which are the exceptions to this rule.

Finally, there are a few places with "FIXME" comments. Note that these
aren't indicating places that are broken according to the currently
supported devices, they are places that will need fixing when support
for new PCI controller models is added.

To assure that there was no regression in the auto-allocation of PCI
addresses or auto-creation of integrated pci-root, ide, and usb
controllers, a new test case (pci-bridge-many-disks) has been added to
both the qemuxml2argv and qemuxml2xml tests. This new test defines a
domain with several dozen virtio disks but no pci-root or
pci-bridges. The .args file of the new test case was created using
libvirt sources from before this patch, and the test still passes
after this patch has been applied.

There are two ways to use a iSCSI LUN as disk source for qemu.

 * The LUN's path as it shows up on host, e.g.
   /dev/disk/by-path/ip-$ip:3260-iscsi-$iqn-fc18:iscsi.iscsi0-lun-1

 * The libiscsi URI from the storage pool source element host attribute, e.g.
   iscsi://demo.org:6000/iqn.1992-01.com.example/1

For a "volume" type disk, if the specified "pool" is of iscsi
type, we should support to use the LUN in either of above 2 ways.
That's why to introduce a new XML tag "mode" for the disk source
(libvirt should support iscsi pool with libiscsi, but it's another
new feature, which should be done later).

The "mode" can be either of "host" or "direct". Use "host" to indicate
use of the LUN with the path as it shows up on host. Use "direct" to
indicate to use it with the source pool host URI (future patches may support
to use network type libvirt storage too, e.g. Ceph)

Actually, I'm turning this function into a macro as filename,
function name and line number needs to be passed. The new
function virAsprintfInternal is introduced with the extended set
of arguments.

<hyperv>
  <spinlocks state='off'/>
</hyperv>

results in:
error: XML error: missing HyperV spinlock retry count

Don't require retries when state is off and use virXPathUInt
instead of virXPathString to simplify parsing.

https://bugzilla.redhat.com/show_bug.cgi?id=784836#c19

For s390 the default console target type is virtio. This also requires
that an implicit virtio-serial controller is instantiated.
This testcase verifies that the target type of virtio is correctly set
in the generated XML if no target element was given and that the
corresponding virtio-serial element is generated too.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

With unknown good reasons, the attribute "bus" of scsi device
address is always set to 0, same for attribute "target". (See
virDomainDiskDefAssignAddress).

Though we might need to change the algorithm to honor "bus"
and "target" too, that's a different issue. The address generator
for scsi host device in this patch just follows the unknown
good reasons, only considering the "controller" and "unit".
It walks through all scsi controllers and their units, to see
if the address $controller:0:0:$unit can be used (if not used
by any disk or scsi host device yet), if found one, it sits on
it, otherwise, it creates a new controller (actually the controller
is implicitly created by someone else), and sits on
$new_controller:0:0:0 instead.

This attribute is going to represent number of queues for
multique vhost network interface. This commit implements XML
extension part of the feature and add one test as well. For now,
we can only do xml2xml test as qemu command line generation code
is not adapted yet.

"sgio" is only valid for scsi host device.

Unlike disk device, the scsi-generic always writethrough the data,
so no need to introduce a "cache" tag, and set "cache=off".

The reason for it's not exposed for such long time is that the
enums for VirtioEventIdx and CopyOnReadType have same enum values
and Correspondingstrings. This fixes the bug and adds test.

QEMU introduced "discard" option for drive since commit a9384aff53,

<...>
@var{discard} is one of "ignore" (or "off") or "unmap" (or "on") and
controls whether @dfn{discard} (also known as @dfn{trim} or @dfn{unmap})
requests are ignored or passed to the filesystem.  Some machine types
may not support discard requests.
</...>

This patch exposes the support in libvirt.

QEMU supported "discard" for "-drive" since v1.5.0-rc0:

% git tag --contains a9384aff53
contains
v1.5.0-rc0
v1.5.0-rc1

So this only detects the capability bit using virQEMUCapsProbeQMPCommandLine.

Adding a VNC WebSocket support for QEMU driver.  This functionality is
in upstream qemu from commit described as v1.3.0-982-g7536ee4, so the
capability is being recognized based on QEMU version for now.

QEMU introduced command line "-mem-merge=on|off" (defaults to on) to
enable/disable the memory merge (KSM) at guest startup. This exposes
it by new XML:
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>

The XML tag is same with what we used internally for old RHEL.

Since it's generic enough to be used by other types in future, I
put it in <hostdev> as sub-element, though now it's only used by
scsi host device.

Except the scsi host device's controller is "lsilogic", mapping
between the libvirt attributes and scsi-generic properties is:

  libvirt     qemu
-----------------------------------------
  controller  bus ($libvirt_controller.0)
  bus         channel
  target      scsi-id
  unit        lun

For scsi host device with "lsilogic" controller, the mapping is:
('target (libvirt)' must be 0, as it's not used; 'unit (libvirt)
must <= 7).

  libvirt            qemu
----------------------------------------------------------
  controller && bus  bus ($libvirt_controller.$libvirt_bus)
  unit               scsi-id

It's not good to hardcode/hard-check limits of these attributes,
and even worse, these limits are not documented, one has to find
out by either testing or reading the qemu code, I'm looking forward
to qemu expose limits like these one day). For example, exposing
"max_target", "max_lun" for megasas:

static const struct SCSIBusInfo megasas_scsi_info = {
    .tcq = true,
    .max_target = MFI_MAX_LD,
    .max_lun = 255,

    .transfer_data = megasas_xfer_complete,
    .get_sg_list = megasas_get_sg_list,
    .complete = megasas_command_complete,
    .cancel = megasas_command_cancel,
};

Example of the qemu command line (lsilogic controller):

  -drive file=/dev/sg2,if=none,id=drive-hostdev-scsi_host7-0-0-0 \
  -device scsi-generic,bus=scsi0.0,scsi-id=8,\
  drive=drive-hostdev-scsi_host7-0-0-0,id=hostdev-scsi_host7-0-0-0

Example of the qemu command line (virtio-scsi controller):

  -drive file=/dev/sg2,if=none,id=drive-hostdev-scsi_host7-0-0-0 \
  -device scsi-generic,bus=scsi0.0,channel=0,scsi-id=128,lun=128,\
  drive=drive-hostdev-scsi_host7-0-0-0,id=hostdev-scsi_host7-0-0-0
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat.com>

An example of the scsi hostdev XML:

    <hostdev mode='subsystem' type='scsi'>
      <source>
        <adapter name='scsi_host0'/>
        <address bus='0' target='0' unit='0'/>
      </source>
      <address type='drive' controller='0' bus='0' target='4' unit='8'/>
    </hostdev>

Controller is implicitly added for scsi hostdev, though the scsi
controller's model defaults to "lsilogic", which might be not what
the user wants (same problem exists for virtio-scsi disk). It's
the existing problem, will be addressed later.

The device address must be specified manually. Later patch will let
libvirt generate it automatically.

This only introduces the generic XMLs for scsi hostdev, later patches
will add other elements, e.g. <readonly>, <shareable>.
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat.com>

The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.

The device option for vfio-pci is nearly identical to that for
pci-assign - only the configfd parameter isn't supported (or needed).

Checking for presence of the bootindex parameter is done separately
from constructing the commandline, similar to how it is done for
pci-assign.

This patch contains tests to check for proper commandline
construction. It also includes tests for parser-formatter-parser
roundtrips (xml2xml), because those tests use the same data files, and
would have failed had they been included before now.

qemu: xml/args tests for VFIO hostdev and <interface type='hostdev'/>

These should be squashed in with the patch that adds commandline
handling of vfio (they would fail at any earlier time).

Add a "dry run" address allocation to figure out how many bridges
will be needed for all the devices without explicit addresses.

Auto-add just enough bridges to put all the devices on, or up to the
bridge with the largest specified index.

This patch is to add command line builder and parser
for NVRAM device, and add test cases.
Signed-off-by: NLi Zhang <zhlcindy@linux.vnet.ibm.com>

No reason to include it in both "if" and "else" branches.

Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: NCorey Bryant <coreyb@linux.vnet.ibm.com>
Tested-by: NCorey Bryant <coreyb@linux.vnet.ibm.com>

With this patch, one can specify the disk source using libvirt
storage like:

  <disk type='volume' device='disk'>
    <driver name='qemu' type='raw' cache='none'/>
    <source pool='default' volume='fc18.img'/>
    <target dev='vdb' bus='virtio'/>
  </disk>

"seclabels" and "startupPolicy" are not supported for this new
disk type ("volume"). They will be supported in later patches.

docs/formatdomain.html.in:
  * Add documents for new XMLs
docs/schemas/domaincommon.rng:
  * Add rng for new XMLs;
src/conf/domain_conf.h:
  * New struct for 'volume' type disk source (virDomainDiskSourcePoolDef)
  * Add VIR_DOMAIN_DISK_TYPE_VOLUME for enum virDomainDiskType
src/conf/domain_conf.c:
  * New helper virDomainDiskSourcePoolDefParse to parse the 'volume'
    type disk source.
  * New helper virDomainDiskSourcePoolDefFree to free the source def
    if 'volume' type disk.
tests/qemuxml2argvdata/qemuxml2argv-disk-source-pool.xml:
tests/qemuxml2xmltest.c:
  * New test

This introduce a new attribute "num_queues" (same with the good name
QEMU uses) for virtio-scsi controller. An example of the XML:

<controller type='scsi' index='0' model='virtio-scsi' num_queues='8'/>

The corresponding QEMU command line:

-device virtio-scsi-pci,id=scsi0,num_queues=8,bus=pci.0,addr=0x3 \

This patch removes the defaultDiskDriverName from the virCaps
structure. This particular default value is used only in the qemu driver
so this patch uses the recently added callback to fill the driver name
if it's needed instead of propagating it through virCaps.

Move the xmlopt and caps arguments to the end of the argument list.

This patch is the result of running:

for i in $(git ls-files | grep -v html | grep -v \.po$ ); do
  sed -i -e "s/virDomainXMLConf/virDomainXMLOption/g" -e "s/xmlconf/xmlopt/g" $i
done

and a few manual tweaks.

This does nothing more than adding the new device and capability.
The device is present since QEMU 1.2.0.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This plumbs in the XML description of iSCSI shares.  The next patches
will add support for the libiscsi userspace initiator.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

QEMU 1.3 and newer support an alternative URI-based syntax to specify
the location of an NBD server.  Libvirt can keep on using the old
syntax in general, but only the URI syntax supports IPv6 addresses.

The URI syntax also supports relative paths to Unix sockets.  These
should never be used but aren't explicitly blocked either by the parser,
so support it just in case.

The URI syntax is intentionally compatible with Gluster's, and the
code can be reused.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

This reuses the XML format that was introduced for Gluster.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

These are supported by nbd-server and by the NBD server that QEMU
embeds for live image access.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Enable more testing of NBD parsing, to ensure rewrites work.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

The virCaps structure gathered a ton of irrelevant data over time that.
The original reason is that it was propagated to the XML parser
functions.

This patch aims to create a new data structure virDomainXMLConf that
will contain immutable data that are used by the XML parser. This will
allow two things we need:

1) Get rid of the stuff from virCaps

2) Allow us to add callbacks to check and add driver specific stuff
after domain XML is parsed.

This first attempt removes pointers to private data allocation functions
to this new structure and update all callers and function that require
them.

Adds XML parsing and qemu commandline tests for the VirtIO RNG device
support.

To enable virCapabilities instances to be reference counted,
turn it into a virObject. All cases of virCapabilitiesFree
turn into virObjectUnref
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Like "rawio", "sgio" is only allowed for block disk of device
type "lun".

It doesn't default disk->sgio to "filtered" when parsing, as
it won't be able to distinguish explicitly requested "filtered"
and a default "filtered" in driver then. We have to error out for
explicit request when the kernel doesn't support the new sysfs
knob "unpriv_sgio", however, for defaulted "filtered", we can
just ignore it if the kernel doesn't support "unpriv_sgio".

QEMU supports setting vendor and product strings for disk since
1.2.0 (only scsi-disk, scsi-hd, scsi-cd support it), this patch
exposes it with new XML elements <vendor> and <product> of disk
device.

Remove the obsolete 'qemud' naming prefix and underscore
based type name. Introduce virQEMUDriverPtr as the replacement,
in common with LXC driver naming style