This patch introduces a function that will allow us to resolve a
relative difference between two elements of a disk backing chain. This
function will be used to allow relative block commit and block pull
where we need to specify the new relative name of the image to qemu.

This patch also adds unit tests for the function to verify that it works
correctly.

Commit 2cff94cc fixed the shadowed 'link' added by commit 975f0e2e,
but forgot the 'link' added by commit 08aa22ec.

Introduce a common function that will take a callback to resolve links
that will be used to canonicalize paths on various storage systems and
add extensive tests.

To free string lists with some strings stolen from the middle we need to
walk the complete array. Introduce a new helper that takes the string
list size to free such string lists.

There are no options to parse here other than the name of the device,
and all three possible device names have the same prefix
("virtio-balloon" with "-ccw", "-pci", or "-device" appended), so the
code is fairly simple. It has been implemented such that it will be
easier to add handling for other -device entries that aren't otherwise
recognized - just add another "else if (STRPREFIX(opts, ....)" clause.

qemuParseCommandLineString() previously would always add a <memballoon
model='virtio'/> to every result (the comments erroneously say that it
is adding a <memballoon model='none'/>) This has been changed to add
model='none', and 84 test case xml's updated accordingly (so that
qemuxml2argvtest won't fail).

Now that the memballoon device is properly parsed, we can safely add a
test for properly ignoring -nodefconfig and -nodefaults. Rather than
adding an entire new test case for this (and memballoon), we just
randomly pick the clock-utc test and modify it slightly to fulfill the
purpose.

makefile_conditionals
tests/Makefile.am:293:endif
maint.mk: match "if FOO" with "endif FOO" in Makefiles
make: *** [sc_makefile_conditionals] Error 1

'virstoragetest' accesses backing chains of files on local storage with
the help of the storage driver. Disable the test on builds without the
storage driver as the test is crashing otherwise.

Reported by: Roman Bogorodskiy

The virNodeParseSocket() function tries to get socked ID from
'topology/physical_package_id' file. However, on some architectures
the file contains the -1 constant which makes in turn libvirt think
the info extraction was unsuccessful. If that's the case, we need to
overwrite the obtained integer with zero like we are doing for other
architectures.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Currently, we are opening the cpuinfo file via fopen() which if fails
doesn't print any error message. We should do that instead.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

So far, we are doing compile time decisions on which architecture is
used. However, for testing purposes it's much easier if we pass host
architecture as parameter and then let the function decide which code
snippet for extracting host CPU info will be used.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This modifies the formatting function of virInterface to be a proper
mirror of the parse function, including the addition of a
"parentIfType" arg so that we can decide whether or not it is
appropriate to emit the elements that are only in toplevel interfaces,
as well as the <link> element (which isn't allowed for bridge
interfaces).

Since the restructuring of the code necessarily changes the order of
some of the elements, some test case data had to be updated.

The interface state for bonds and vlans does seem to reflect the state
of the underlying physical devices, at least in some cases, so it
makes sense to allow reporting it (netcf now does).

The link state/speed for bridge devices is meaningless though, so we
don't even look for it.

There are two places where you'll find info on page sizes. The first
one is under <cpu/> element, where all supported pages sizes are
listed. Then the second one is under each <cell/> element which refers
to concrete NUMA node. At this place, the size of page's pool is
reported. So the capabilities XML looks something like this:

<capabilities>

  <host>
    <uuid>01281cda-f352-cb11-a9db-e905fe22010c</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Westmere</model>
      <vendor>Intel</vendor>
      <topology sockets='1' cores='1' threads='1'/>
      ...
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>
    ...
    <topology>
      <cells num='4'>
        <cell id='0'>
          <memory unit='KiB'>4054408</memory>
          <pages unit='KiB' size='4'>1013602</pages>
          <pages unit='KiB' size='2048'>3</pages>
          <pages unit='KiB' size='1048576'>1</pages>
          <distances/>
          <cpus num='1'>
            <cpu id='0' socket_id='0' core_id='0' siblings='0'/>
          </cpus>
        </cell>
        <cell id='1'>
          <memory unit='KiB'>4071072</memory>
          <pages unit='KiB' size='4'>1017768</pages>
          <pages unit='KiB' size='2048'>3</pages>
          <pages unit='KiB' size='1048576'>1</pages>
          <distances/>
          <cpus num='1'>
            <cpu id='1' socket_id='0' core_id='0' siblings='1'/>
          </cpus>
        </cell>
        ...
      </cells>
    </topology>
    ...
  </host>

  <guest/>

</capabilities>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

On RHEL6 the vboxsnapshotxmltest fails because of wrong xml that
is generated by libvirt. However the core issue is in the xml data
itself with the wrong indentation.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

Coverity checks for patterns of handling return values of functions.
Some recent addition must have tripped a threshold where coverity now
complains that we usually check the return value of virUUIDGenerate but
don't do it in one place. Add a check to make coverity happy.

Commit 7c6fc394 introduced a regression in the XML produced for older
clients.  The argument at the time was that clients shouldn't be
depending on output-only data for something that is only going to
be triggered for a transient guest; but John Ferlan reported that
the automated testsuite was such a client.  It's better to be safe
than sorry by guaranteeing back-compat cruft.  Note that later
patches will be using <mirror> for active block commit, but there
we don't have to worry about back-compat.

* src/conf/domain_conf.c (virDomainDiskDefFormat): Restore old
style output when necessary.
* docs/schemas/domaincommon.rng: Validate back-compat style.
* docs/formatdomain.html.in: Update the documentation.
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-mirror-old.xml:
Update tests.
* tests/qemuxml2argvdata/qemuxml2argv-disk-mirror.xml: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

This new element is there to represent PCI-Express capabilities
of a PCI devices, like link speed, number of lanes, etc.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The block commit code looks for an explicit base file relative
to the discovered top file; so for a chain of:
  base <- snap1 <- snap2 <- snap3
and a command of:
  virsh blockcommit $dom vda --base snap2 --top snap1
we got a sane message (here from libvirt 1.0.5):
error: invalid argument: could not find base 'snap2' below 'snap1' in chain for 'vda'

Meanwhile, recent refactoring has slightly reduced the quality of the
libvirt error messages, by losing the phrase 'below xyz':
error: invalid argument: could not find image 'snap2' in chain for 'snap3'

But we had a one-off, where we were not excluding the top file
itself in searching for the base; thankfully qemu still reports
the error, but the quality is worse:
  virsh blockcommit $dom vda --base snap2 --top snap2
error: internal error unable to execute QEMU command 'block-commit': Base '/snap2' not found

Fix the one-off in blockcommit by changing the semantics of name
lookup - if a starting point is specified, then the result must
be below that point, rather than including that point.  The only
other call to chain lookup was blockpull code, which was already
forcing the lookup to omit the active layer and only needs a
tweak to use the new semantics.

This also fixes the bug exposed in the testsuite, where when doing
a lookup pinned to an intermediate point in the chain, we were
unable to return the name of the parent also in the chain.

* src/util/virstoragefile.c (virStorageFileChainLookup): Change
semantics for non-NULL startFrom.
* src/qemu/qemu_driver.c (qemuDomainBlockJobImpl): Adjust caller,
to keep existing semantics.
* tests/virstoragetest.c (mymain): Adjust to expose new semantics.
Signed-off-by: NEric Blake <eblake@redhat.com>

Add some more tests of what happens when we restrict a lookup
to begin at a point in the middle of a chain.  In particular,
we want to ensure that a parent is not found when starting at
the child.  This commit also demonstrates that we have a slight
difference in behavior on what parent we report when filtering
is in effect; as the determination of the parent affects the
code in block commit, exposing this in the testsuite will help
justify changes in future patches that tweak the semantics of
what lookups are allowed.

* tests/virstoragetest.c (testStorageLookup): Test user input.
(TEST_LOOKUP_TARGET): Add parameter.
(mymain): Add lookup tests.
Signed-off-by: NEric Blake <eblake@redhat.com>

The next patch will be adding tests, including adding a parameter
for testing more conditions.  For ease of review of that patch, I
want to create common context lines that don't change when the new
tests are added (it's easier to visually review additions than it
is to review an entire chunk of tests rewritten into another
larger chunk of tests).

* tests/virstoragetest.c (mymain): Add a parameter and renumber
the lookup tests.
Signed-off-by: NEric Blake <eblake@redhat.com>

Typing chain->backingStore->backingStore gets old after a while;
introduce some alias variables to make the test more compact.

* tests/virstoragetest.c (mymain): Introduce some shorthand.
Signed-off-by: NEric Blake <eblake@redhat.com>

Automatically allocate PCI addresses for devices instead
of hardcoding them in the driver code. The current
allocation schema is to dedicate an entire slot for each devices.

Also, allow having arbitrary number of devices.

When the test is failing but the debug output isn't enabled the
resulting line would look ugly like and would not contain the actual
difference.

TEST: virstoragetest
      .................chain member 1!chain member 1!chain member 1!

Store the member index in the actual checked string to hide this problem

The vcpupin command allowed specifying a negative number for the --vcpu
argument. This would the overflow when the underlying virDomainPinVcpu
API was called.

 $ virsh vcpupin r7 -1 0
 error: numerical overflow: input too large: 4294967295

Switch the vCPU variable to a unsigned int and parse it using the
corresponding function.

Also improve the vcpupin test to cover all the defects.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1101059Signed-off-by: NJincheng Miao <jmiao@redhat.com>
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

Since commit d69415d4, vmware version is parsed from both stdout and
stderr. This patch makes version parsing work even if there is garbage
(libvirt debug messages for example) in the command output.

Add test data for this case.

Currently it is not possible to determine the speed of an interface
and whether a link is actually detected from the API. Orchestrating
platforms want to be able to determine when the link has failed and
where multiple speeds may be available which one the interface is
actually connected at. This commit introduces an extension to our
interface XML (without implementation to interface driver backends):

  <interface type='ethernet' name='eth0'>
    <start mode='none'/>
    <mac address='aa:bb:cc:dd:ee:ff'/>
    <link speed='1000' state='up'/>
    <mtu size='1492'/>
    ...
  </interface>

Where @speed is negotiated link speed in Mbits per second, and state
is the current NIC state (can be one of the following:  "unknown",
"notpresent", "down", "lowerlayerdown","testing", "dormant", "up").
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This structure contains the data to be saved in the VirtualBox XML file
and can be manipulated with severals exposed functions.
The structure is created by vboxSnapshotLoadVboxFile taking the
machine XML file.
It also can rewrite the XML by using vboxSnapshotSaveVboxFile.

qmeuargv2xmltest.c would fail any test that logged anything during
qemuParseCommandline(), but then discard the log message, even with
VIR_TEST_DEBUG=2. This patch outputs the log messages with
fprintf(stderr,...) when debug logging is on.

In the process of modifying that logic, the testInfo data was made
more similar to that of qemuxml2argvtest.c - rather than turning
info->extraFlags into a bool, an enum of flags is defined, the info
struct is given an "unsigned int flags", and FLAG_EXPECT_WARNING is
saved into info->flags, to be checked during the test; this will make
it easier to add other FLAG_EXPECT_* items in the future.

Now that we track a disk mirror as a virStorageSource, we might
as well update the XML to theoretically allow any type of
mirroring destination (not just a local file).  A later patch
will also be reusing <mirror> to track the block commit of the
top layer of a chain, which is another case where libvirt needs
to update the backing chain after the job is finally pivoted,
and since backing chains can have network backing files as the
destination to commit into, it makes more sense to display that
in the XML.

This patch changes output-only XML; it was already documented
that <mirror> does not affect a domain definition at this point
(because qemu doesn't provide persistent bitmaps yet).  Any
application that was starting a block copy job with older libvirt
and then relying on the domain XML to determine if it was
complete will no longer be able to access the file= and format=
attributes of mirror that were previously used.  However, this is
not going to be a problem in practice: the only time a block copy
job works is on a transient domain, and any app that is managing
a transient domain probably already does enough of its own
bookkeeping to know which file it is mirroring into without
having to re-read it from the libvirt XML.  The one thing that
was likely to be used in a mirroring job was the ready=
attribute, which is unchanged.  Meanwhile, I made sure the schema
and parser still accept the old format, even if we no longer
output it, so that upgrading from an older version of libvirt is
seamless.

* docs/schemas/domaincommon.rng (diskMirror): Alter definition.
* src/conf/domain_conf.c (virDomainDiskDefParseXML): Parse two
styles of mirror elements.
(virDomainDiskDefFormat): Output new style.
* tests/qemuxml2argvdata/qemuxml2argv-disk-mirror-old.xml: New
file, copied from...
* tests/qemuxml2argvdata/qemuxml2argv-disk-mirror.xml: ...here
before modernizing.
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-mirror-old*: New
files.
* tests/qemuxml2xmltest.c (mymain): Test both styles.
Signed-off-by: NEric Blake <eblake@redhat.com>

As part of the work on backing chains, I'm finding that it would
be easier to directly manipulate chains of pointers (adding a
snapshot merely adjusts pointers to form the correct list) rather
than copy data from one struct to another.  This patch converts
domain disk source to be a pointer.

In this patch, the pointer is ALWAYS allocated (thanks in part to
the previous patch forwarding all disk def allocation through a
common point), and all other changse are just mechanical fallout of
the new type; there should be no functional change.  It is possible
that we may want to leave the pointer NULL for a cdrom with no
medium in a later patch, but as that requires a closer audit of the
source to ensure we don't fault on a null dereference, I didn't do
it here.

* src/conf/domain_conf.h (_virDomainDiskDef): Change type of src.
* src/conf/domain_conf.c: Adjust all clients.
* src/security/security_selinux.c: Likewise.
* src/qemu/qemu_domain.c: Likewise.
* src/qemu/qemu_command.c: Likewise.
* src/qemu/qemu_conf.c: Likewise.
* src/qemu/qemu_process.c: Likewise.
* src/qemu/qemu_migration.c: Likewise.
* src/qemu/qemu_driver.c: Likewise.
* src/lxc/lxc_driver.c: Likewise.
* src/lxc/lxc_controller.c: Likewise.
* tests/securityselinuxlabeltest.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

My commit 7d8afc47 was passing the incorrect size to
virBitmapDataToString in the newly added test.

A PCI device can be associated with a specific NUMA node. Later, when
a guest is pinned to one NUMA node the PCI device can be assigned on
different NUMA node. This makes DMA transfers travel across nodes and
thus results in suboptimal performance. We should expose the NUMA node
locality for PCI devices so management applications can make better
decisions.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

For converting bitmap data to human-readable strings.

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

For now only one test is introduced. It's purpose in life
is to check we don't break NUMA host distances XML format.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

If user or management application wants to create a guest,
it may be useful to know the cost of internode latencies
before the guest resources are pinned. For example:

<capabilities>

  <host>
    ...
    <topology>
      <cells num='2'>
        <cell id='0'>
          <memory unit='KiB'>4004132</memory>
          <distances>
            <sibling id='0' value='10'/>
            <sibling id='1' value='20'/>
          </distances>
          <cpus num='2'>
            <cpu id='0' socket_id='0' core_id='0' siblings='0'/>
            <cpu id='2' socket_id='0' core_id='2' siblings='2'/>
          </cpus>
        </cell>
        <cell id='1'>
          <memory unit='KiB'>4030064</memory>
          <distances>
            <sibling id='0' value='20'/>
            <sibling id='1' value='10'/>
          </distances>
          <cpus num='2'>
            <cpu id='1' socket_id='0' core_id='0' siblings='1'/>
            <cpu id='3' socket_id='0' core_id='2' siblings='3'/>
          </cpus>
        </cell>
      </cells>
    </topology>
    ...
  </host>
  ...
</capabilities>

We can see the distance from node1 to node0 is 20 and within nodes 10.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The qemu JSON monitor test allows to test also expected command
arguments. As the error from the monitor simulator is returned as a
simulated qemu error (in JSON) all other JSON contained in the error
message needs to be escaped. This will happen if the monitor command
under test receives a JSON array as an argument.

This will improve the error message from:
libvirt:  error : internal error: cannot parse json { "error":  { "desc":
"Invalid value of argument 'keys' of command 'send-key': expected 'ble'
got '[{"type":"number","data":43},{"type":"number","data":26},
{"type":"number","data":46},{"type":"number","data":32}]'",
"class": "UnexpectedCommand" } }: lexical error: invalid string in json text.

To:
libvirt: QEMU Driver error : internal error: unable to execute QEMU
command 'send-key': Invalid value of argument 'keys' of command
'send-key': expected 'ble' got '[{"type":"number","data":43},
{"type":"number","data":26},{"type":"number","data":46},
{"type":"number","data":32}]'

This improvement will not have any effect on tests executing as
expected, but it will help test development.

virstoragetest now requires parts of the storage driver to be built.
Without this change the test can't be compiled on platforms that don't
build the storage driver (mingw).

make[2]: *** No rule to make target `../src/libvirt_driver_storage_impl.la', needed by `virstoragetest.exe'.  Stop.

Broken by commit 713cc3b0

qemuMonitorJSONSendKey declares the "holdtime" argument as unsigned int
while the command was constructed in qemuMonitorJSONMakeCommand using
the "P" modifier which took a unsigned long from the variable
arguments which then made it possible to access uninitialized memory.

This broke the qemumonitorjsontest on 32bit fedora 20:
64) qemuMonitorJSONSendKey
... libvirt: QEMU Driver error : internal error: unsupported data type 'W' for arg 'WVSƒì ‹D$0è‘wÿÿÃAå' FAILED

Uncovered by upstream commit f744b831.

Additionally add test for the hold-time option.

All the fields crammed into two lines weren't easy to parse by human
eyes. Split up the format string into lines and put it into a central
variable so that changes in two places aren't necessary.