The docs about remote URIs in uri.html are somewhat sparse with the full
docs being in remote.html. Move all the URI content from remote.html
into uri.html so the user only needs to look in one place for URI info.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The generation and deployment of x509 certificates for TLS is complex
and verbose and thus deserves its own standalone page.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The script needs two files to open:

  $(builddir)/test_file_access.txt, and
  $(srcdir)/file_access_whitelist.txt.

However, the script is opening the files from the $CWD which
won't work for a VPATH build. Make the script accept paths to the
files through @ARGV and tune the Makefile.am to pass them.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

As part of a goal to eliminate Perl from libvirt build tools,
rewrite the test-wrap-argv.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

As part of a goal to eliminate Perl from libvirt build tools,
rewrite the pdwtags processing script in Python.

The original inline shell and perl code was completely
unintelligible. The new python code is a manual conversion
that attempts todo basically the same thing.
Tested-by: NCole Robinson <crobinso@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

python2 will be end of life by the time of the next
libvirt release. All our supported build targets, including
CentOS7, have a python3 build available.
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Since the max unit of virtio scsi disk is 16383, update the range of
driveUnit to it.
Signed-off-by: NHan Han <hhan@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Signed-off-by: NHan Han <hhan@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

Introduced in c8007fdc, it should use 'greater than max' instead of
'equal or greater than max' for the condition of checking invalid scsi
unit.
Signed-off-by: NHan Han <hhan@redhat.com>
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>

If we static link to libvirt_util.la then we can't override functions in
this file by simply implementing them in the test code. Any tests should
dynamic link to the main libvirt.la and ensure symbols are exported.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The python array slice syntax expects the first and last indexes,
not the first length and element count.
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Reviewed-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

The hard dependancy between the virt drivers and the network
or storage drivers was removed quite a while back now, so
the tests no longer need to link to these drivers.
Reviewed-by: NLaine Stump <laine@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

As of 0985a959 we no longer distribute generated files. mingw
needs to call genprotocol.pl itself, which means it needs rpcgen
and cpp installed.
Signed-off-by: NCole Robinson <crobinso@redhat.com>

We clear some capabilities here so the lockouts need to be
re-evaluated.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

Blockdev is required to do incremental backups properly. Add a helper
function for locking out capabilities and export it to allow re-doing
the processing if a different code path modifies capabilities.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

Checking whether a qemu capability set right before clearing it without
any other logic doesn't make sense.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

Do all post-processing of capabilities in qemuProcessPrepareQEMUCaps.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

Move the post-processing of the QEMU_CAPS_CHARDEV_FD_PASS flag to the
new function.

The clearing of the capability is based on the presence of
VIR_QEMU_PROCESS_START_STANDALONE so we must also pass in the process
start flags.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NCole Robinson <crobinso@redhat.com>

Start aggregating all capability post-processing code in one place.

The comment was modified while moving it as it was mentioning floppies
which are no longer clearing the blockdev capability.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The function is now used only in qemu_process.c so move it there and
name it 'qemuProcessPrepareQEMUCaps' which is more appropriate to what
it's doing.

The reworded comment now mentions that it will also post-process the
caps for VM startup.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

The redetection was originally added in 43c01d38 as a way to recover
from libvirtd upgrade from the time when we didn't persist the qemu
capabilities in the status XML. Also this the oldest supported qemu by
more than two years.

Even if somebody would have a running VM running at least qemu 1.5 with
such an old libvirt we certainly wouldn't do the right thing by
redetecting the capabilities and then trying to communicate with qemu.

For now it will be the best to just stop considering this scenario any
more and error out for such VM.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

Commit c90fb5a8 added explicit use of the private copy of the qemu
capabilities to various places. The change to qemuProcessInit was bogus
though as at the point where we re-initiate the post parse callbacks
priv->qemuCaps is still NULL as we clear it after shutdown of the VM and
don't initiate it until a later point.

Using the value from priv->qemuCaps might mislead readers of the code
into thinking that something useful is being passed at that point so go
with an explicit NULL instead.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NJán Tomko <jtomko@redhat.com>

qemuDomainGetJobInfo didn't always reset the return data in @info.
Thankfully this wouldn't be a problem as the RPC layer does it but we
should do it anyways.

Since we reset the struct we don't have to set the type to
VIR_DOMAIN_JOB_NONE as the value is 0.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The statistics fields are used in two places:

1) virDomainGetJobStats where the job type which ultimately holds
whether the job was successful or not is returned via a different
argument.

2) The virConnectDomainEventJobCompleted event where we report just the
statistics via typed parameters.

Since it might be useful to report the event also for jobs which
completed unsuccessfully and we don't have the means to transport the
state via a different variable with the event let's add a new field
which will hold the success state.

Since this is meant primarily for completed jobs a plain boolean is
sufficient to convey whether the job was successful or not.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Introduce --rawstats which prints all statistics fields from the new API
similarly to how the virsh event handler prints them.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Introduce the --anystats flag which does not skip the printing of the
stats if the job was unsuccessful.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Printing that a job failed is rather unhelpful. Print at least the
operation which failed.

Achieve this by moving the check whether to print stats later but
replace it with a check which will skip printing of the operation if
there's no job.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

virDomainGetJobStats destroys the completed statistics on the first
read. Give the user possibility to keep them around if they wish so.

Add a flag VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED which will read the stats
without destroying them.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

To simplify the stats printer code we convert the new statistics from
the typed parameter list into the old stats structure.

Extract this code since it takes a lot of space.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The 32-bit x86 binary is called qemu-system-i386, not
qemu-system-i686. This mistake across many test XML files was
not noticed because the mistake was also made in testutilsqemu.c
when mocking the capabilities.
Reviewed-by: NErik Skultety <eskultet@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

For managed save we can choose between various compression
methods. I randomly tested the 'xz' program on a 8 GB guest
and was surprised to have to wait > 50 minutes for it to
finish compressing, with 'xz' burning 100% cpu for the
entire time. Despite the impressive compression, this is
completely useless in the real world as it is far too long
to wait to save the VM.

The 'xz' binary defaults to '-6' optimization level which
aims for high compression, with moderate memory usage,
at the expense of speed.

This change switches it to use the '-3' optimization level
which is documented as being the one that optimizes speed
at expense of compression. Even with this, it will still
outperform all the other options in terms of compression
level. It is a little less than x4 faster than '-6' which
means it starts to be a viable choice to use 'xz' for
people who really want best compression.

The test results on a 1 GB, fairly freshly booted VM are
as follows

  format | save  | restore  size
  =======+=======+=============
  raw    |   05s |    1s  | 428 MB
  lzop   |   05s |    3s  | 160 MB
  gzip   |   29s |    5s  | 118 MB
  bz2    |   54s |   22s  | 114 MB
  xz     | 4m37s |   13s  |  86 MB
  xz -3  | 1m20s |   12s  |  95 MB

Based on this we can say

 * For moderate compression with no noticable loss in speed

       => use lzop

 * For high compression with moderate loss in speed

       => use gzip

 * For best compression with significant loss in speed

       => use xz
Reviewed-by: NMichal Privoznik <mprivozn@redhat.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

My commit 3bbe1020 forgot
to update the news.
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Commit v5.7.0-248-g03449e25 removed "cd tests" without updating the
patch to test-suite.log.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>
Reviewed-by: NPavel Hrdina <phrdina@redhat.com>

Signed-off-by: NJán Tomko <jtomko@redhat.com>