This patch adds support for the QEMU vhost-user feature to libvirt.
vhost-user enables the communication between a QEMU virtual machine
and other userspace process using the Virtio transport protocol.
It uses a char dev (e.g. Unix socket) for the control plane,
while the data plane based on shared memory.

The XML looks like:

<interface type='vhostuser'>
    <mac address='52:54:00:3b:83:1a'/>
    <source type='unix' path='/tmp/vhost.sock' mode='server'/>
    <model type='virtio'/>
</interface>
Signed-off-by: NMichele Paolino <m.paolino@virtualopensystems.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This introduces two new attributes "cmd_per_lun" and "max_sectors" same
with the names QEMU uses for virtio-scsi. An example of the XML:

<controller type='scsi' index='0' model='virtio-scsi' cmd_per_lun='50'
max_sectors='512'/>

The corresponding QEMU command line:

-device virtio-scsi-pci,id=scsi0,cmd_per_lun=50,max_sectors=512,
bus=pci.0,addr=0x3
Signed-off-by: NMike Perez <thingee@gmail.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Commit 7c6fc394 introduced a regression in the XML produced for older
clients.  The argument at the time was that clients shouldn't be
depending on output-only data for something that is only going to
be triggered for a transient guest; but John Ferlan reported that
the automated testsuite was such a client.  It's better to be safe
than sorry by guaranteeing back-compat cruft.  Note that later
patches will be using <mirror> for active block commit, but there
we don't have to worry about back-compat.

* src/conf/domain_conf.c (virDomainDiskDefFormat): Restore old
style output when necessary.
* docs/schemas/domaincommon.rng: Validate back-compat style.
* docs/formatdomain.html.in: Update the documentation.
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-mirror-old.xml:
Update tests.
* tests/qemuxml2argvdata/qemuxml2argv-disk-mirror.xml: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Now that we track a disk mirror as a virStorageSource, we might
as well update the XML to theoretically allow any type of
mirroring destination (not just a local file).  A later patch
will also be reusing <mirror> to track the block commit of the
top layer of a chain, which is another case where libvirt needs
to update the backing chain after the job is finally pivoted,
and since backing chains can have network backing files as the
destination to commit into, it makes more sense to display that
in the XML.

This patch changes output-only XML; it was already documented
that <mirror> does not affect a domain definition at this point
(because qemu doesn't provide persistent bitmaps yet).  Any
application that was starting a block copy job with older libvirt
and then relying on the domain XML to determine if it was
complete will no longer be able to access the file= and format=
attributes of mirror that were previously used.  However, this is
not going to be a problem in practice: the only time a block copy
job works is on a transient domain, and any app that is managing
a transient domain probably already does enough of its own
bookkeeping to know which file it is mirroring into without
having to re-read it from the libvirt XML.  The one thing that
was likely to be used in a mirroring job was the ready=
attribute, which is unchanged.  Meanwhile, I made sure the schema
and parser still accept the old format, even if we no longer
output it, so that upgrading from an older version of libvirt is
seamless.

* docs/schemas/domaincommon.rng (diskMirror): Alter definition.
* src/conf/domain_conf.c (virDomainDiskDefParseXML): Parse two
styles of mirror elements.
(virDomainDiskDefFormat): Output new style.
* tests/qemuxml2argvdata/qemuxml2argv-disk-mirror-old.xml: New
file, copied from...
* tests/qemuxml2argvdata/qemuxml2argv-disk-mirror.xml: ...here
before modernizing.
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-mirror-old*: New
files.
* tests/qemuxml2xmltest.c (mymain): Test both styles.
Signed-off-by: NEric Blake <eblake@redhat.com>

ppce500v2 is not machine supported by official release of QEMU.
It should be replaced by ppce500.
Signed-off-by: NJán Tomko <jtomko@redhat.com>

The XML for quite a longish backing chain is shown below:

  <disk type='network' device='disk'>
    <driver name='qemu' type='qcow2'/>
    <source protocol='nbd' name='bar'>
      <host transport='unix' socket='/var/run/nbdsock'/>
    </source>
    <backingStore type='block' index='1'>
      <format type='qcow2'/>
      <source dev='/dev/HostVG/QEMUGuest1'/>
      <backingStore type='file' index='2'>
        <format type='qcow2'/>
        <source file='/tmp/image2.qcow'/>
        <backingStore type='file' index='3'>
          <format type='qcow2'/>
          <source file='/tmp/image3.qcow'/>
          <backingStore type='file' index='4'>
            <format type='qcow2'/>
            <source file='/tmp/image4.qcow'/>
            <backingStore type='file' index='5'>
              <format type='qcow2'/>
              <source file='/tmp/image5.qcow'/>
              <backingStore type='file' index='6'>
                <format type='raw'/>
                <source file='/tmp/Fedora-17-x86_64-Live-KDE.iso'/>
                <backingStore/>
              </backingStore>
            </backingStore>
          </backingStore>
        </backingStore>
      </backingStore>
    </backingStore>
    <target dev='vdb' bus='virtio'/>
  </disk>

Various disk types and formats can be mixed in one chain. The
<backingStore/> empty element marks the end of the backing chain and it
is there mostly for future support of parsing the chain provided by a
user. If it's missing, we are supposed to probe for the rest of the
chain ourselves, otherwise complete chain was provided by the user. The
index attributes of backingStore elements can be used to unambiguously
identify a specific part of the image chain.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

I noticed that depending on the <driver> attributes the user passed
in, the output may omit the <driver> element altogether.  For example,
the rerror_policy has had this problem since commit 4bb4109f in Oct
2011.  But in adding testsuite coverage to expose it, I found another
problem: the C code is just fine without a driver name, but the
XML validator required either a name or a cache mode.

* src/conf/domain_conf.c (virDomainDiskDefFormat): Update
conditional.
* docs/schemas/domaincommon.rng (diskDriver): Simplify.
* tests/qemuxml2argvdata/qemuxml2argv-disk-drive-copy-on-read.xml:
* tests/qemuxml2argvdata/qemuxml2argv-disk-drive-copy-on-read.args:
New files.
* tests/qemuxml2argvdata/qemuxml2argv-disk-drive-discard.xml:
Enhance test.
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-discard.xml:
Likewise.
* tests/qemuxml2argvtest.c (mymain): New test.
* tests/qemuxml2xmltest.c (mymain): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

To make <disk> schema more maintainable and to allow for moving the
pieces to a common file in the future. It relies on the ability to
override definitions as part of an include, set up in the previous
patch.

The diff is a bit hard to read, because it mixes reindentation
with refactoring; 'git diff -b --patience' may help.

* docs/schemas/domaincommon.rng (disk): Refactor into pieces.
(diskSource, diskSourceFile, diskSourceBlock, diskSourceDir)
(diskSourceVolume: New defines.
(diskSourceNetwork): Revise scope.
* docs/schemas/domainsnapshot.rng (disksnapshot): Adjust.
* tests/domainsnapshotxml2xmlin/disk-seclabel-invalid.xml,
tests/domainsnapshotxml2xmlin/disk-network-seclabel-invalid.xml: New
tests to check seclabel is forbidden in domain snapshot by schema.
Signed-off-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

This patch is my first experience playing with nested grammars,
as documented in http://relaxng.org/tutorial-20011203.html#IDA3PZR.
I plan on doing more overrides in order to make the RelaxNG
grammar mirror the C code refactoring into a common
virStorageSource, but where different clients of that source do
not support the same subset of functionality.  By starting with
something fairly easy to validate, I can make sure my later
patches will be possible.

This patch adds a use of the no-op <ref
name='sourceStartupPolicy'/> to the disksnapshot definition, so
that the snapshot version of a type='file' <source> more closely
resembles the version in domaincommon.  A future patch will merge
the two files into using a common define, but this patch is
sufficient for testing that adding <source
startupPolicy='optional'/> in any of the
tests/domainsnapshotxml2xmlin/*.xml files still gets rejected
unless it occurs within the <domain> subelement, because the
definition of startupPolicy is empty outside of domain.rng.

* docs/schemas/storagecommon.rng (storageStartupPolicy)
(storageSourceExtra): Create no-op defaults.
* docs/schemas/domainsnapshot.rng (domain): Use nested grammar
to avoid restricting <domain>.
(storageSourceExtra): Create new override.
(disksnapshot): Access overrides through common names.
* docs/schemas/domaincommon.rng (disk): Access overrides through
common names.
* docs/schemas/domain.rng (storageStartupPolicy)
(storageSourceExtra): Create new overrides.
Signed-off-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

We had incomplete RelaxNG support for storage formats listed
in virstoragefile.h: commit 027bf2ea added 'vdi' but forgot
to update the <volume> and <domain> xml lists; the <volume>
list was also missing 'fat' and 'vhd'.  Maintaining two lists
is a recipe for them getting out of sync, so make the list
common so that both contexts benefit the next time we add a
format in a single location.

* docs/schemas/domaincommon.rng (storageFormat): Move...
* docs/schemas/storagecommon.rng: ...here, and add vdi.
* docs/schemas/storagevol.rng (formatfile): Use common list.
Signed-off-by: NEric Blake <eblake@redhat.com>

In general, we try to make virt-xml-validate tolerant of input
elements in any order when possible.  However, as written, the
RNG grammar did not permit <source> unless there was an explicit
type= attribute (even though the C code manages just fine by
defaulting to type='file').  After making the attribute optional
on the 'file' branch, I noticed that the use of diskspec was now
redundant with the branch when no <source> was supplied.

View this patch with 'git diff -b' for a better picture of the
schema change.

* docs/schemas/domaincommon.rng (disk): Hoist 'diskspec' out of
choice, make type='file' default, and still preserve interleave.
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-source-pool.xml:
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-discard.xml:
New files.
* tests/qemuxml2argvdata/qemuxml2argv-disk-source-pool.xml:
* tests/qemuxml2argvdata/qemuxml2argv-disk-drive-discard.xml:
Reorder XML.
* tests/qemuxml2xmltest.c (mymain): Cover new files.
Signed-off-by: NEric Blake <eblake@redhat.com>

Having two tiny files with a couple definitions didn't make
as much sense as one common file, especially since I plan to
add more definitions and use it in more places.

* docs/schemas/storageencryption.rng: Merge this...
* docs/schemas/storagefilefeatures.rng: ...and this, into...
* docs/schemas/storagecommon.rng: ...this new file.
* docs/schemas/Makefile.am (schema_DATA): Reflect renames.
* docs/schemas/storagevol.rng: Likewise.
* docs/schemas/domaincommon.rng: Likewise.
* libvirt.spec.in: Likewise.
* mingw-libvirt.spec.in: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Add VIR_DOMAIN_HOSTDEV_PCI_BACKEND_XEN. For legacy xen, it will use "pciback" as
stub driver.

There is no keyboard support currently in libvirt.

For some platforms (PPC64 QEMU) this makes graphics unusable,
since the keyboard is not implicit and it can't be added via libvirt.
Signed-off-by: NLi Zhang <zhlcindy@linux.vnet.ibm.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Add a new character device backend called 'spiceport' that uses
spice's channel for communications and apart from spicevmc can be used
as a backend for any character device from libvirt's point of view.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Add a new <timer> for the HyperV reference time counter enlightenment
and the iTSC reference page for Windows guests.

This feature provides a paravirtual approach to track timer events for
the guest (similar to kvmclock) with the option to use real hardware
clock on systems with a iTSC with compensation across various hosts.

According to the documentation describing various tunables for domain
timers not all the fields are supported by all the driver types. Express
these in the RNG:

- rtc, platform: Only these support the "track" attribute.
- tsc: only one to support "frequency" and "mode" attributes
- hpet, pit: tickpolicy/catchup attribute/element
- kvmclock: no extra attributes are supported

Additionally the attributes of the <catchup> element for
tickpolicy='catchup' are optional according to the parsing code. Express
this in the XML and fix a spurious space added while formatting the
<catchup> element and add tests for it.

spice-server offers an API to disable file transfer messages
on the agent channel between the client and the guest.
This is supported in qemu through the disable-agent-file-xfer option.

This patch exposes this option to libvirt.
Adds a new element 'filetransfer', with one property,
'enable', which accepts a boolean.
Default is enabled, for backward compatibility.

Depends on the capability exported in the first patch of the series.
Signed-off-by: NFrancesco Romani <fromani@redhat.com>

This patch introduces new xml elements under <blkiotune>,
we use these new elements to setup the throttle blkio
cgroup for domain. The new blkiotune node looks like this:

<blkiotune>
  <device>
    <path>/path/to/block</path>
    <weight>1000</weight>
    <read_iops_sec>10000</read_iops_sec>
    <write_iops_sec>10000</write_iops_sec>
    <read_bytes_sec>1000000</read_bytes_sec>
    <write_bytes_sec>1000000</write_bytes_sec>
  </device>
</blkiotune>
Signed-off-by: NGuan Qiang <hzguanqiang@corp.netease.com>
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

When idmap was added to LXC, we forgot to cover it in the testsuite.
The schema was missing an <element> layer, and as a result,
virt-xml-validate was failing on valid dumpxml output.

Reported by Eduard - Gabriel Munteanu on IRC.

* docs/schemas/domaincommon.rng (idmap): Include <idmap> element,
and support interleaves.
* tests/lxcxml2xmldata/lxc-idmap.xml: New file.
* tests/lxcxml2xmltest.c (mymain): Test it.
Signed-off-by: NEric Blake <eblake@redhat.com>

AArch64 qemu has similar behavior as armv7l, like use of mmio etc.
This patch adds similar bypass checks what we have for armv7l to aarch64.
E.g. we are enabling mmio transport for Nicdev.
Making addDefaultUSB and addDefaultMemballoon to false etc.

V3:
- Adding missing domain rng schema for aarcg64 and test case in
  testutilsqemu.c which was causing test suite failure
  while running make check.

V2:
- Added testcase to qemuxml2argvtest as suggested
  during review comments of V1.

V1:
- Initial patch.
Signed-off-by: NAnup Patel <anup.patel@linaro.org>
Signed-off-by: NPranavkumar Sawargaonkar <pranavkumar@linaro.org>

panic device is a device that enables libvirt to receive notification
of guest panic event.
Signed-off-by: NEric Blake <eblake@redhat.com>

For example:
<address type='isa' iobase='0x505' irq='0x1'/>
Signed-off-by: NEric Blake <eblake@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1035118

When outputting the XML for the RNG device, the code didn't format the
PCI address info. Additionally the schema wasn't expecting the info
although it was being parsed and used internally. Fix those mistakes and
add test for the PCI info section.

The linux kernel recently added support for paravirtual spinlock
handling to avoid performance regressions on overcomitted hosts. This
feature needs to be turned in the hypervisor so that the guest OS is
notified about the possible support.

This patch adds a new feature "paravirt-spinlock" to the XML and
supporting code to enable the "kvm_pv_unhalt" pseudo CPU feature in
qemu.

https://bugzilla.redhat.com/show_bug.cgi?id=1008989

Use absDirPath instead of absFilePath.

https://bugzilla.redhat.com/show_bug.cgi?id=1028107

Change the RelaxNG schema option name so that it can be reused for
non-hyperv feature flags.

Since 76b644c3 when the support for RAM filesystems was introduced,
libvirt accepted the following XML:
<source usage='1024' unit='KiB'/>

This was parsed correctly and internally stored in bytes, but it
was formatted as (with an extra 's'):
<source usage='1024' units='KiB'/>
When read again, this was treated as if the units were missing,
meaning libvirt was unable to parse its own XML correctly.

The usage attribute was documented as being in KiB, but it was not
scaled if the unit was missing. Transient domains still worked,
because this was balanced by an extra 'k' in the mount options.

This patch:
Changes the parser to use 'units' instead of 'unit', as the latter
was never documented (fixing persistent domains) and some programs
(libvirt-glib, libvirt-sandbox) already parse the 'units' attribute.

Removes the extra 'k' from the tmpfs mount options, which is needed
because now we parse our own XML correctly.

Changes the default input unit to KiB to match documentation, fixing:
https://bugzilla.redhat.com/show_bug.cgi?id=1015689

This resolves one of the issues in:

   https://bugzilla.redhat.com/show_bug.cgi?id=1003983

This device is identical to qemu's "intel-hda" device (known as "ich6"
in libvirt), but has a different PCI device ID (which matches the ID
of the hda audio built into the ich9 chipset, of course). It's not
supported in earlier versions of qemu, so it requires a capability
bit.

Currently the XML parser already allows the following syntax:
  <disk type='block' device='cdrom'>
    <source startupPolicy='optional'/>
    <target dev='hda' bus='ide'/>
    <address type='drive' controller='0' bus='0' target='0' unit='0'/>
  </disk>

But it if the dev value is NULL then it would not have the leading
"<source ", resulting in invalid XML.

qemu/KVM also supports a tftp URL while specifying the cdrom ISO image.

The xml should be as following:

    <disk type='network' device='cdrom'>
      <source protocol='tftp' name='/url/path'>
        <host name='host.name' port='69'/>
      </source>
    </disk>
Signed-off-by: NAline Manera <alinefm@br.ibm.com>

The ftps protocol is another protocol supported by qemu/KVM while specifying
the cdrom ISO image.

The xml should be as following:

    <disk type='network' device='cdrom'>
      <source protocol='ftps' name='/url/path'>
        <host name='host.name' port='990'/>
      </source>
    </disk>
Signed-off-by: NAline Manera <alinefm@br.ibm.com>

The https protocol is also accepted by qemu/KVM when specifying the cdrom ISO
image.

The xml should be as following:

    <disk type='network' device='cdrom'>
      <source protocol='https' name='/url/path'>
        <host name='host.name' port='443'/>
      </source>
    </disk>
Signed-off-by: NAline Manera <alinefm@br.ibm.com>

This corresponds to '-sd' and '-drive if=sd' on the qemu command line.
Needed for many ARM boards which don't provide any other way to
pass in storage.

And add test cases for a basic working ARM guest.

Add an attribute named 'removable' to the 'target' element of disks,
which controls the removable flag. For instance, on a Linux guest it
controls the value of /sys/block/$dev/removable. This option is only
valid for USB disks (i.e. bus='usb'), and its default value is 'off',
which is the same behaviour as before.

To achieve this, 'removable=on' (or 'off') is appended to the '-device
usb-storage' parameter sent to qemu when adding a USB disk via
'-disk'. A capability flag QEMU_CAPS_USB_STORAGE_REMOVABLE was added
to keep track if this option is supported by the qemu version used.

Bug: https://bugzilla.redhat.com/show_bug.cgi?id=922495Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

<controller type='pci' index='0' model='pci-root'>
  <pcihole64 unit='KiB'>1048576</pcihole64>
</controller>

It can be used to adjust (or disable) the size of the 64-bit
PCI hole. The size attribute is in kilobytes (different unit
can be specified on input), but it gets rounded up to
the nearest GB by QEMU.

Disabling it will be needed for guests that crash with the
64-bit PCI hole (like Windows XP), see:
https://bugzilla.redhat.com/show_bug.cgi?id=990418

The ftp protocol is already recognized by qemu/KVM so add this support to
libvirt as well.
The xml should be as following:

     <disk type='network' device='cdrom'>
       <source protocol='ftp' name='/url/path'>
         <host name='host.name' port='21'/>
       </source>
     </disk>
Signed-off-by: NAline Manera <alinefm@br.ibm.com>

QEMU/KVM already allows a HTTP URL for the cdrom ISO image so add this support
to libvirt as well.
The xml should be as following:

    <disk type='network' device='cdrom'>
      <source protocol='http' name='/url/path'>
        <host name='host.name' port='80'/>
      </source>
    </disk>
Signed-off-by: NAline Manera <alinefm@br.ibm.com>

Commit 01b88127 changed aliases for PCI controller devices to "pcie.0" or
"pci.%u". Thus device aliases may now contain dots.