we currently do not check if a sector is allocated during convert.
This means if a sector is unallocated that we allocate a bounce
buffer of zeroes, find out its zero later and do not write it
in the best case. In the worst case this can lead to reading
blocks from a raw device (like iSCSI) altough we could easily
know via get_block_status that they are zero and simply skip them.

This patch also fixes the progress output not being at 100% after
a successful conversion.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Now qemu-img convert have similar options as qemu-nbd for internal
snapshot.
Signed-off-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This case can't run when IMGPROTO=nbd, since it needs to create some
internal snapshot which would fail for EOF write request, even when
TEST_IMG is exported with "-f raw" in common.rc, so set _supported_proto
to file.

_require_command() is changed to tip what util is missing, instead
of printing a blank.
Signed-off-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Now it is possible to directly export an internal snapshot, which
can be used to probe the snapshot's contents without qemu-img
convert.
Signed-off-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Since later this function will be used so improve it. The only caller of it
now is qemu-img, and it is not impacted by introduce function
bdrv_snapshot_load_tmp_by_id_or_name() that call bdrv_snapshot_load_tmp()
twice to keep old search logic. bdrv_snapshot_load_tmp_by_id_or_name() return
int to let caller know the errno, and errno will be used later.
Also fix a typo in comments of bdrv_snapshot_delete().
Signed-off-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Format "raw" doesn't always work on certain file systems (e.g. tmpfs).
Use qcow2 to make the allocation status explicit and split into a new
case.

[Resolved merge conflict due to "qemu-io> " prompt filter, added 074 to
group file, and fixed up s/048/074/ copy-paste mistake.
--Stefan]
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Whitespace changes to align columns.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NBenoit Canet <benoit@irqsave.net>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

So that the tests can run faster.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This replaces _unsupported_qemu_io_options and check for support of
current cache mode, and allow to provide a default if user didn't
specify.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This will allow overriding cache mode from the "-c mode" option.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The option sets cache mode used in the tests. "-nocache" is changed to
an alias to "-c none", and internally passes "-t none" to qemu-io.

Python scripts will make use of option this in the next commit.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Strictly speaking, this is only required for has_zero_init() == false,
but it's easy enough to just do a cluster-aligned write that is padded
with zeros after the header.

This fixes that after 'qemu-img create' header extensions are attempted
to be parsed that are really just random leftover data.

Cc: qemu-stable@nongnu.org
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Leaving the backing file open although it is not needed anymore can
cause problems if it is opened through a block driver which allows
exclusive access only and if the create function of the block driver
used for the top image (the one being created) tries to close and reopen
the image file (which will include opening the backing file a second
time).

In particular, this will happen with a backing file opened through
qemu-nbd and using qcow2 as the top image file format (which reopens the
image to flush it to disk).

In addition, the BlockDriverState in bdrv_img_create() is used for the
backing file only; it should therefore be made local to the respective
block.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Fetch the data to be written from the input buffer.  If it is all zeroes,
we can use the write_zeroes call (possibly with the new MAY_UNMAP flag).
Otherwise, do as many write cycles as needed, writing 512k at a time.

Strictly speaking, this is still incorrect because a zero cluster should
only be written if the MAY_UNMAP flag is set.  But this is a bug in qcow2
and the other formats, not in the SCSI code.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Since we report ANC_SUP==0 in VPD page B2h, we need to return
an error (ILLEGAL REQUEST/INVALID FIELD IN CDB) for all WRITE SAME
requests with ANCHOR==1.

Inspired by a similar patch to the LIO in-kernel target.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This is the same that is already done for WRITE SAME.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The code is similar to the implementation of discard and write_zeroes
with UNMAP.  However, failure must be propagated up to block.c.

The stale page cache problem can be reproduced as follows:

    # modprobe scsi-debug lbpws=1 lbprz=1
    # ./qemu-io /dev/sdXX
    qemu-io> write -P 0xcc 0 2M
    qemu-io> write -z 0 1M
    qemu-io> read -P 0x00 0 512
    Pattern verification failed at offset 0, 512 bytes
    qemu-io> read -v 0 512
    00000000:  cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc  ................
    ...

    # ./qemu-io --cache=none /dev/sdXX
    qemu-io> write -P 0xcc 0 2M
    qemu-io> write -z 0 1M
    qemu-io> read -P 0x00 0 512
    qemu-io> read -v 0 512
    00000000:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    ...

And similarly with discard instead of "write -z".
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

See the next commit for the description of the Linux kernel problem
that is worked around in raw_open_common.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Writing zeroes to a file can be done by punching a hole if
MAY_UNMAP is set.

Note that in this case ENOTSUP is not ignored, but makes
the block layer fall back to the generic implementation.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The current check is right for MAY_UNMAP=1.  For MAY_UNMAP=0, just
try and fall back to regular writes as soon as a WRITE SAME command
fails.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

added myself to reflect recent work on the iscsi block driver.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

since commit 3ac21627 the default value changed to 0.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This will let misaligned but large requests use zero clusters.  This
is important because the cluster size is not guest visible.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Right now, bdrv_co_do_write_zeroes will only try to align the
beginning of the request.  However, it is simpler for many
formats to expect the block layer to separate both the head *and*
the tail.  This makes sure that the format's bdrv_co_write_zeroes
function will be called with aligned sector_num and nb_sectors for
the bulk of the request.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Similar to write_zeroes, let the generic code receive a ENOTSUP for
discard operations.  Since bdrv_discard has advisory semantics,
we can just swallow the error.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This will be used by the SCSI layer.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This lets bdrv_co_do_rw receive flags, so that it can be used for
zero writes.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

bdrv_co_discard is only covering drivers which have a .bdrv_co_discard()
implementation, but not those with .bdrv_aio_discard(). Not very nice,
and easy to avoid.
Suggested-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The buffer for description file was 4096 which only covers a few
hundred of extents. This changes the buffer to dynamic allocated with
g_strdup_printf in order to support bigger cases.
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The AioContext ctx field is apparently unused in qemu codebase since
02ffb504.
Signed-off-by: NMarc-André Lureau <marcandre.lureau@gmail.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

qemu_co_queue_wait_insert_head() is unused in qemu code base now.
Signed-off-by: NMarc-André Lureau <marcandre.lureau@gmail.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This removes "qemu-io> " prompt from qemu-io output in _filter_qemu_io,
and updates all the output files with the following command:

 cd tests/qemu-iotests && sed -i "s/qemu-io> //g" *.out
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>