Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Both 030 and 041 use create_image().  Move it to iotests.py.

Also drop ImageStreamingTestCase since the class now has no methods.
Suggested-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The iotests.compare_images() function returns True if two image files
have the identical data.  Previously this was implemented by converting
images to raw and then comparing their contents using Python.  Since
"qemu-img compare" is now available and is more efficient, switch to it.

This function will be reused by the 'drive-backup' test case.
Suggested-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The cancel_and_wait() function has been duplicated in 030 and 041.  Move
it into iotests.py and let it return the event so tests can perform
additional asserts.

Note that 041's cancel_and_wait(wait_ready=True) is replaced by
wait_ready_and_cancel(), which uses the new wait_ready() and
cancel_and_wait() underneath.
Suggested-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Tests 030 and 041 both use query-block-jobs to check whether any block
jobs are active.  Make this code common so that 'drive-backup' and other
new feature tests will be able to reuse it.
Suggested-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Commit f3f4d2c0 added a hint to increase
the cluster size when a large image cannot be created.  Test 054 now has
outdated output and fails because the golden output does not match.

This patch updates the 054 golden output.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

glibc wipes malloc(3) memory when the MALLOC_PERTURB_ environment
variable is set.  The value of the environment variable determines the
bit pattern used to wipe memory.  For more information, see
http://udrepper.livejournal.com/11429.html.

Set MALLOC_PERTURB_ for gtester and qemu-iotests.  Note we pick a random
value from 1 to 255 to expose more bugs.  If you need to reproduce a
crash use 'show environment' in gdb to extract the MALLOC_PERTURB_
value from a core dump.

Both make check and qemu-iotests pass with MALLOC_PERTURB_ enabled.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1369661331-28041-1-git-send-email-stefanha@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

It's supposed to fail gracefully instead of segfaulting.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>

017 and 018 use /bin/mv to move base img from t.IMGFMG to t.IMGFMT.base
after filling data, this is not enough for vmdk, when t.IMGFMT is only a
description text file who points to t-{flat,s001,f001,...}.IMGFMT as
data extent, so testing such subformats alway fails on them.

This patch use the trick of temprorily changing TEST_IMG to avoid using
/bin/mv.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

043 tests recursive backing file by changing backing file. VMDK has not
implemented this yet, and qcow1 probably never will.
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Zero sized disk is not supported by qemu vmdk driver, exclude vmdk from
the test script.

As tested on vmware-vdiskmanager and vmware workstation, zero sized disk
is not supported by vmware, either.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Filter out vmdk creation option 'adapter_type' for vmdk. So that tests
with an explicit './check -o adapter_type=XXX' will not fail.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Cover new image creation options for vmdk, so we can use '-o
zeroed_grain=XXX' and '-o subformat=XXX' to run the tests successfully.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

We don't want to commit to the API yet before everything is worked out.
Disable it for the 1.5 release. This commit is meant to be reverted
after the 1.5 release.

The disabling of the driver-specific options is achieved by applying the
old checks while parsing the command line.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Test that qemu-img convert -c works when input image length is not a
multiple of the cluster size.

Previously an error message would be produced:

  qemu-img: error while compressing sector 0: Input/output error

Now that qcow2 and qcow handle this case the test passes successfully.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Specifying the wrong driver could fail an assertion:

$ qemu-system-x86_64 -drive file.driver=qcow2,file=x
qemu-system-x86_64: block.c:721: bdrv_open_common: Assertion `file !=
((void *)0)' failed.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

$QEMU_PROG happens to be 'qemu' in my setup, so this sed command
replaces a bit too much. Restrict it to the start of the line and to
when it's followed by a colon, i.e. the form used by error messages.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

Note in order to run these tests on ssh, you must be running a local
ssh daemon, and that daemon must accept loopback connections, and
ssh-agent has to be set up to allow logins on the local daemon.  In
other words, the following command should just work without demanding
any passphrase:

 ssh localhost
Signed-off-by: NRichard W.M. Jones <rjones@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Filter the name of the QEMU executable so the output can be diffed no
matter what QEMU_PROG is (e.g. qemu-system-x86_64).
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Instead of just checking once in exactly this order if there are
dependendies, non-COW clusters and new allocation, this starts looping
around these. This way we can, for example, gather non-COW clusters after
new allocations as long as the host cluster offsets stay contiguous.

Once handle_dependencies() is extended so that COW areas of in-flight
allocations can be overwritten, this allows to continue with gathering
other clusters (we wouldn't be able to do that without this change
because we would have missed a possible second dependency in one of the
next clusters).

This means that in the typical sequential write case, we can combine the
COW overwrite of one cluster with the allocation of the next cluster as
soon as something like Delayed COW gets actually implemented. It is only
by avoiding splitting requests this way that Delayed COW actually starts
improving performance noticably.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The old code detected an overlapping allocation even when the
allocations didn't actually overlap, but were only adjacent.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This should be based on the virtual disk size, not on the size of the
image.

Interesting observation: With some VM state stored in the image file,
percentages higher than 100% are possible, even though snapshots
themselves are ignored. This is a qcow2 bug to be fixed another day: The
VM state should be discarded in the active L2 tables after completing
the snapshot creation.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Check that writes to an image opened with BDRV_O_SNAPSHOT do not modify
the underlying image file.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

A comment explains that -nographic hangs test case 007.  This is no
longer the case so add -nographic.  This makes the test suite faster and
more pleasant to run since no windows pop up.

I am not sure exactly when -nographic starting working for this case but
there is no fundamental reason why graphics are needed here.  Make sure
the serial port is not on stdio, it would conflict with the monitor.

Also remove unnecessary trailing whitespace on these lines.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Tested-by: NKevin Wolf <kwolf@redhat.com>

If zero clusters are erroneously treated as unallocated, "qemu-img rebase"
will copy the backing file's contents onto the cluster.

The bug existed also in image streaming, but since the root cause was in
qcow2's is_allocated implementation it is enough to test it with qemu-img.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Just create lots of images and try out each of the creation options that
qcow2 provides (except backing_file/fmt for now)

I'm not totally happy with the behaviour of qemu-img in each of the
cases, but let's be explicit and update the test when we do change
things later.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Simple test for qemu-img compare to check it's working correctly.
Signed-off-by: NMiroslav Rezanina <mrezanin@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Show how many clusters are compressed.  This can be used to monitor how
many compressed clusters remain and whether to recompress the image.
Suggested-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This patch adds the support for reporting the image end offset (in
bytes). This is particularly useful after a conversion (or a rebase)
where the destination is a block device in order to find the first
unused byte at the end of the image.
Signed-off-by: NFederico Simoncelli <fsimonce@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This test verifies two mirroring issues are fixed with resized images:

 * sync='top' creates an image that is the proper size
 * sync='full' doesn't cause an assertion failure and crash qemu
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

It turned out that the change in b7ab0fea was actually a real qcow2
corruption fix. This is a reproducer for the bug.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This makes sense when the next commit starts using the extra buffer space
to perform many I/O operations asynchronously.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

When mirroring runs, the backing files for the target may not yet be
ready.  However, this means that a copy-on-write operation on the target
would fill the missing sectors with zeros.  Copy-on-write only happens
if the granularity of the dirty bitmap is smaller than the cluster size
(and only for clusters that are allocated in the source after the job
has started copying).  So far, the granularity was fixed to 1MB; to avoid
the problem we detected the situation and required the backing files to
be available in that case only.

However, we want to lower the granularity for efficiency, so we need
a better solution.  The solution is to always copy a whole cluster the
first time it is touched.  The code keeps a bitmap of clusters that
have already been allocated by the mirroring job, and only does "manual"
copy-on-write if the chunk being copied is zero in the bitmap.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This adds some first tests for qcow2's dependency handling when two
parallel write requests access the same cluster.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NCorey Bryant <coreyb@linux.vnet.ibm.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

These spelling bugs were found by codespell:

supressing -> suppressing
transfered -> transferred
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>