Now that we have a separate aio pool structure we can remove those
aio pool details from BlockDriver.

Every driver supporting AIO now needs to declare a static AIOPool
with the aiocb size and the cancellation method.  This cleans up the
current code considerably and will make it cleaner and more obvious
to support two different aio implementations behind a single
BlockDriver.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Don't write each single changed refcount block entry to the disk after it is
written, but update all entries of the block and write all of them at once.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

This is a preparation patch with no functional changes. It moves the allocation
of new refcounts block to a new function and makes update_cluster_refcount (for
one cluster) call update_refcount (for multiple clusters) instead the other way
round.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

There is only one (internal) user left and it can be switched to the normal
emulation provided in block.c
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Add an option to specify the cluster size of a newly created qcow2 image.
Default is 4k which is the same value that was hard-coded before.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Now we can make use of the newly introduced option structures. Instead of
having bdrv_create carry more and more parameters (which are format specific in
most cases), just pass a option structure as defined by the driver itself.

bdrv_create2() contains an emulation of the old interface to simplify the
transition.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

This was caught by a7d27b53 which
aborted on this attempt, thanks to Alex Ivanov for report.
Signed-off-by: Nmalc <av1474@comtv.ru>

Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Include assert.h from qemu-common.h and remove other direct uses.
cpu-all.h still need to include it because of the dyngen-exec.h hacks
Signed-off-by: NPaul Brook <paul@codesourcery.com>

Need to drop QCOW_OFLAG_COPIED from a cluster pointer before freeing it.

Add an explanation how thing meant to work.
Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

this patch adds a buffer_alignment field to BlockDriverState and
implements a qemu_blockalign function that uses that field to allocate a
memory aligned buffer to be used by the block driver.
buffer_alignment is initialized to 512 but each block driver can set
a different value (at the moment none of them do).
This patch modifies ide.c, block-qcow.c, block-qcow2.c and block.c to
use qemu_blockalign instead of qemu_memalign.
There is only one place left that still uses qemu_memalign to allocate
buffers used by block drivers that is posix-aio-compat:handle_aiocb_rw
because it is not possible to get the BlockDriverState from that
function. However I think it is not important because posix-aio-compat
already deals with driver specific code so it is supposed to know its
own needs.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7229 c046a42c-6fe2-441c-8c8c-71466251a162

From: Kevin Wolf <kwolf@redhat.com>

All L1 and L2 entries must point at the start of a cluster. If there is some
offset into the cluster, the entry is corrupted.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7217 c046a42c-6fe2-441c-8c8c-71466251a162

This is purely cosmetical changes to make the code easier to read. Move L2
table processing from a deeply nested block to its own function, add some
comments.

Patch v2: Fix misplaced bracket causing false positives
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7216 c046a42c-6fe2-441c-8c8c-71466251a162

From: Kevin Wolf <kwolf@redhat.com>

Introduce a new bdrv_check function pointer for block drivers. Modify qcow2 to
return an error status in check_refcounts(), so it can implement bdrv_check.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7214 c046a42c-6fe2-441c-8c8c-71466251a162

From: Kevin Wolf <kwolf@redhat.com>

This code is currently only compiled when DEBUG_ALLOC is defined, so you
usually don't see compiler warnings on it. This patch series wants to enable
the code, so fix the format string warnings first.

While we're at it, let's print error messages to stderr.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7213 c046a42c-6fe2-441c-8c8c-71466251a162

This patch fixes a qcow2 corruption bug introduced in SVN Rev 5861. L2 tables
are big endian, so entries must be converted before being passed to functions.

This bug is easy to trigger. The following script will create and destroy a
qcow2 image (the header is gone after three loop iterations):

    #!/bin/bash
    qemu-img create -f qcow2 test.qcow 1M
    for i in $(seq 1 10); do
    qemu-system-x86_64 -hda test.qcow -monitor stdio > /dev/null 2>&1 <<EOF
    savevm test-$i
    quit
    EOF
    done
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7165 c046a42c-6fe2-441c-8c8c-71466251a162

git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7103 c046a42c-6fe2-441c-8c8c-71466251a162

Make all AIO requests vectored and defer linearization until the actual
I/O thread.  This prepares for using native preadv/pwritev.

Also enables asynchronous direct I/O by handling that case in the I/O thread.

Qcow and qcow2 propably want to be adopted to directly deal with multi-segment
requests, but that can be implemented later.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7020 c046a42c-6fe2-441c-8c8c-71466251a162

Always use the vectored APIs to reduce code churn once we switch the BlockDriver
API to be vectored.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7019 c046a42c-6fe2-441c-8c8c-71466251a162

We now enforce that you cannot write beyond the end of a non-growable file.
qcow2 files are not growable but we rely on them being growable to do
savevm/loadvm.  Temporarily allow them to be growable by introducing a new
API specifically for savevm read/write operations.

Reported-by: malc
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6994 c046a42c-6fe2-441c-8c8c-71466251a162

When creating large disk images w/ qcow2 format, qcow2_create is hard
coded to creating a single refcount block.  This is insufficient for
large images, and will cause qemu-img to segfault as it walks off the
end of the refcount block.  Keep track of the space needed during image
create and create proper number of refcount blocks accordingly.

https://bugzilla.redhat.com/show_bug.cgi?id=491943Signed-off-by: NChris Wright <chrisw@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6982 c046a42c-6fe2-441c-8c8c-71466251a162

qcow2's get_cluster_offset() scans forward in the l2 table to find other
clusters that have the same allocation status as the first cluster.
This is used by (among others) qcow_is_allocated().

Unfortunately, it was not checking to be sure that it didn't fall off
the end of the l2 table.  This patch adds that check.

The symptom that motivated me to look into this was that
bdrv_is_allocated() was returning false when there was in fact data
there.  This is one of many ways this bug could lead to data corruption.

I checked the other place that scans for consecutive unallocated blocks
(alloc_cluster_offset()) and it appears to be OK:
    nb_clusters = MIN(nb_clusters, s->l2_size - l2_index);
appears to prevent the same problem from occurring.

Signed-off-by: Nolan Leake <nolan <at> sigbus.net>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6977 c046a42c-6fe2-441c-8c8c-71466251a162

Recent patches added two compiler warnings about the format string
usage in qcow_read_extensions.  One is printing a uint64_t using
%lu which is incorrect on many platforms as it can be a unsigned
long long, the second one is printing the result of sizeof as
%lu, but it is a size_t so it needs to be printed using %zu.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6944 c046a42c-6fe2-441c-8c8c-71466251a162

Use a qcow2 extension to keep the backing file format.

By keeping the backing file format, we can:
1. Provide a way to know the backing file format without probing
   it (setting the format at creation time).
2. Enable using qcow2 format over host block devices.
   (only if the user specifically asks for it, by providing the format
   at creation time).

Also fixes a security flaw found by Daniel P. Berrange on [1]
which summarizes: "Autoprobing: just say no."

[1] http://lists.gnu.org/archive/html/qemu-devel/2008-12/msg01083.htmlSigned-off-by: NUri Lublin <uril@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6909 c046a42c-6fe2-441c-8c8c-71466251a162

Qcow2 extensions are build of magic (id) len (in bytes) and data.
They reside right after the qcow2 header.
If a backing filename exists it follows the qcow2 extension (if exist)

Qcow2 extensions are read upon image open.
Qcow2 extensions are identified by their magic.
Unknown qcow2 extensions (unknown magic) are skipped.
A Special magic of 0 means end-of-qcow2-extensions.

In this patchset, to be used to keep backing file format.

Based on a work done by Shahar Frank <sfrank@redhat.com>.
Signed-off-by: NUri Lublin <uril@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6907 c046a42c-6fe2-441c-8c8c-71466251a162

This series is broken by design as it requires expensive IO operations at
open time causing very long delays when starting a virtual machine for the
first time.
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6816 c046a42c-6fe2-441c-8c8c-71466251a162

This series is broken by design as it requires expensive IO operations at
open time causing very long delays when starting a virtual machine for the
first time.
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6815 c046a42c-6fe2-441c-8c8c-71466251a162

This series is broken by design as it requires expensive IO operations at
open time causing very long delays when starting a virtual machine for the
first time.
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6813 c046a42c-6fe2-441c-8c8c-71466251a162

Consistently use the C99 named initializer format for the BlockDriver
methods to make the method table more readable and more easily
extensible.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAurelien Jarno <aurelien@aurel32.net>

git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6768 c046a42c-6fe2-441c-8c8c-71466251a162

Signed-off-by: NAvi Kivity <avi@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6527 c046a42c-6fe2-441c-8c8c-71466251a162

'num_free_bytes' is the number of non-allocated bytes below highest-allocation.
It's useful, together with the highest-allocation, to figure out how
fragmented the image is, and how likely it will run out-of-space soon.

For example when the highest allocation is high (almost end-of-disk), but 
many bytes (clusters) are free, and can be re-allocated when neeeded, than
we know it's probably not going to reach end-of-disk-space soon.

Added bookkeeping to block-qcow2.c
Export it using BlockDeviceInfo
Show it upon 'info blockstats' if BlockDeviceInfo exists
Signed-off-by: NUri Lublin <uril@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6407 c046a42c-6fe2-441c-8c8c-71466251a162

Signed-off-by: NUri Lublin <uril@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6405 c046a42c-6fe2-441c-8c8c-71466251a162

We want to know the highest written offset for qcow2 images.
This gives a pretty good (and easy to calculate) estimation to how
much more allocation can be done for the block device.

It can be usefull for allocating more diskspace for that image
(if possible, e.g. lvm) before we run out-of-disk-space
Signed-off-by: NUri Lublin <uril@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6404 c046a42c-6fe2-441c-8c8c-71466251a162

Currently qemu_mallocz calls malloc and handling of zero by malloc is
implementation defined behaviour:
http://www.opengroup.org/onlinepubs/7990989775/xsh/malloc.html

malloc(0) on AIX returns NULL[1] and qcow2 images without snapshots
are thus unusable

[1] Unless special Linux compatibility define is used when compiling

git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6359 c046a42c-6fe2-441c-8c8c-71466251a162

Don't leak memory

Rebased for qemu tree.

Signed-off-by: Uri Lublin <uril@redhat.com> 
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>



git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6245 c046a42c-6fe2-441c-8c8c-71466251a162

When allocating multiple clusters at once, the qcow2 implementation
tries to find as many physically contiguous clusters as possible to
allow larger writes. This search includes allocated clusters which are
in the right place and still free clusters. If the range to allocate
spans clusters in patterns like "10 allocated, then 10 free, then again
10 allocated" it is only checked that the chunks of allocated clusters
are contiguous for themselves.

However, what is actually needed is to have _all_ allocated clusters
contiguous, starting at the first cluster of the allocation and spanning
multiple such chunks. This patch changes the check so that each offset
is not compared to the offset of the first cluster in its own chunk but
to the first cluster in the whole allocation.

I haven't seen it happen, but without this fix data corruption on qcow2
images is possible.
Signed-off-by: NKevin Wolf <kwolf@suse.de>
Acked-by: NGleb Natapov <gleb@redhat.com>



git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6213 c046a42c-6fe2-441c-8c8c-71466251a162

Correctly calculate number of contiguous clusters.
Acked-by: NKevin Wolf <kwolf@suse.de>
Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>



git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6212 c046a42c-6fe2-441c-8c8c-71466251a162

qcow2 writes a cluster reference count on every cluster update.  This causes
performance to crater when using anything but cache=writeback.  This is most
noticeable when using savevm.  Right now, qcow2 isn't a reliable format
regardless of the type of cache your using because metadata is not updated in
the correct order.  Considering this, I think it's somewhat reasonable to use
writeback caching by default with qcow2 files.

It at least avoids the massive performance regression for users until we sort
out the issues in qcow2. 
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>



git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@5879 c046a42c-6fe2-441c-8c8c-71466251a162

Change order of metadata update to prevent loosing guest data because of unexpected exit (Gleb Natapov)

Currently the order is this (during cow since it's the interesting case):
1. Decrement refcount of old clusters
2. Increment refcount for newly allocated clusters
3. Copy content of old sectors that will not be rewritten
4. Update L2 table with pointers to new clusters
5. Write guest data into new clusters (asynchronously)

There are several problems with this order. The first one is that if qemu
crashes (or killed or host reboots) after new clusters are linked into L2
table but before user data is written there, then on the next reboot guest
will find neither old data nor new one in those sectors and this is not
what gust expects even when journaling file system is in use.  The other
problem is that if qemu is killed between steps 1 and 4 then refcount
of old cluster will be incorrect and may cause snapshot corruption.

The patch change the order to be like this:
1. Increment refcount for newly allocated clusters
2. Write guest data into new clusters (asynchronously)
3. Copy content of old sectors that were not rewritten
4. Update L2 table with pointers to new clusters
5. Decrement refcount of old clusters

Unexpected crash may cause cluster leakage, but guest data should be safe.
Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>



git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@5861 c046a42c-6fe2-441c-8c8c-71466251a162