The wrong version of the preallocation patch has been applied, so this is the
remaining diff.

We can't use truncate to grow the image file to the right size because we don't
know if metadata has been written after the last data cluster. In this case
truncate would shrink the file and destroy its metadata. Write a zero sector at
the end of the virtual disk instead to ensure that the file is big enough.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

If nb_sectors is 0, cluster_offset will not be initialized.
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

This introduces a qemu-img create option for qcow2 which allows the metadata to
be preallocated, i.e. clusters are reserved in the refcount table and L1/L2
tables, but no data is written to them. Metadata is quite small, so this
happens in almost no time.

Especially with qcow2 on virtio this helps to gain a bit of performance during
the initial writes. However, as soon as create a snapshot, we're back to the
normal slow speed, obviously. So this isn't the real fix, but kind of a cheat
while we're still having trouble with qcow2 on virtio.

Note that the option is disabled by default and needs to be specified
explicitly using qemu-img create -f qcow2 -o preallocation=metadata.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The VM state offset is a concept internal to the image format.  Replace
the old bdrv_{get,put}_buffer method that require an index into the
image file that is constructed from the VM state offset and an offset
into the vmstate with the bdrv_{load,save}_vmstate that just take an
offset into the VM state.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Contrary to what one could expect, the size of L1 tables is not cluster
aligned. So as we're writing whole sectors now instead of single entries,
we need to ensure that the L1 table in memory is large enough; otherwise
write would access memory after the end of the L1 table.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The performance of qcow2 has improved meanwhile, so we don't need to
special-case it any more. Switch the default to write-through caching
like all other block drivers.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Updated to use C99 comments.
Signed-off-by: NFilip Navara <filip.navara@gmail.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The qcow2 source is now split into several more manageable files. During the
conversion quite some functions that were static before needed to be changed to
be global to make the source compile again.

We were lucky enough not to get name conflicts with these additional global
names, but they are not nice. This patch adds a qcow2_ prefix to all of the
global functions in qcow2.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

qcow2-snapshot.c contains the code related to snapshotting.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

qcow2-cluster.c contains all functions related to the management of guest
clusters, i.e. what the guest sees on its virtual disk. This code is about
mapping these guest clusters to host clusters in the image file using the
two-level lookup tables.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

qcow2-refcount.c contains all functions which are related to cluster
allocation and management in the image file. A large part of this is the
reference counting of these clusters.

Also a header file qcow2.h is introduced which will contain the interface of
the split qcow2 modules.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Larger cluster sizes mean less metadata. This has been discussion a few times,
let's do it now. This turns 64k clusters on by default for new images.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

This patch adds a small help text to each of the options in the block drivers
which can be displayed by using qemu-img create -f fmt -o ?
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Now that we have a separate aio pool structure we can remove those
aio pool details from BlockDriver.

Every driver supporting AIO now needs to declare a static AIOPool
with the aiocb size and the cancellation method.  This cleans up the
current code considerably and will make it cleaner and more obvious
to support two different aio implementations behind a single
BlockDriver.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Don't write each single changed refcount block entry to the disk after it is
written, but update all entries of the block and write all of them at once.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

This is a preparation patch with no functional changes. It moves the allocation
of new refcounts block to a new function and makes update_cluster_refcount (for
one cluster) call update_refcount (for multiple clusters) instead the other way
round.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

There is only one (internal) user left and it can be switched to the normal
emulation provided in block.c
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Add an option to specify the cluster size of a newly created qcow2 image.
Default is 4k which is the same value that was hard-coded before.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Now we can make use of the newly introduced option structures. Instead of
having bdrv_create carry more and more parameters (which are format specific in
most cases), just pass a option structure as defined by the driver itself.

bdrv_create2() contains an emulation of the old interface to simplify the
transition.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

This was caught by a7d27b53 which
aborted on this attempt, thanks to Alex Ivanov for report.
Signed-off-by: Nmalc <av1474@comtv.ru>

Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Include assert.h from qemu-common.h and remove other direct uses.
cpu-all.h still need to include it because of the dyngen-exec.h hacks
Signed-off-by: NPaul Brook <paul@codesourcery.com>

Need to drop QCOW_OFLAG_COPIED from a cluster pointer before freeing it.

Add an explanation how thing meant to work.
Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

this patch adds a buffer_alignment field to BlockDriverState and
implements a qemu_blockalign function that uses that field to allocate a
memory aligned buffer to be used by the block driver.
buffer_alignment is initialized to 512 but each block driver can set
a different value (at the moment none of them do).
This patch modifies ide.c, block-qcow.c, block-qcow2.c and block.c to
use qemu_blockalign instead of qemu_memalign.
There is only one place left that still uses qemu_memalign to allocate
buffers used by block drivers that is posix-aio-compat:handle_aiocb_rw
because it is not possible to get the BlockDriverState from that
function. However I think it is not important because posix-aio-compat
already deals with driver specific code so it is supposed to know its
own needs.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7229 c046a42c-6fe2-441c-8c8c-71466251a162

From: Kevin Wolf <kwolf@redhat.com>

All L1 and L2 entries must point at the start of a cluster. If there is some
offset into the cluster, the entry is corrupted.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7217 c046a42c-6fe2-441c-8c8c-71466251a162

This is purely cosmetical changes to make the code easier to read. Move L2
table processing from a deeply nested block to its own function, add some
comments.

Patch v2: Fix misplaced bracket causing false positives
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7216 c046a42c-6fe2-441c-8c8c-71466251a162

From: Kevin Wolf <kwolf@redhat.com>

Introduce a new bdrv_check function pointer for block drivers. Modify qcow2 to
return an error status in check_refcounts(), so it can implement bdrv_check.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7214 c046a42c-6fe2-441c-8c8c-71466251a162

From: Kevin Wolf <kwolf@redhat.com>

This code is currently only compiled when DEBUG_ALLOC is defined, so you
usually don't see compiler warnings on it. This patch series wants to enable
the code, so fix the format string warnings first.

While we're at it, let's print error messages to stderr.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7213 c046a42c-6fe2-441c-8c8c-71466251a162

This patch fixes a qcow2 corruption bug introduced in SVN Rev 5861. L2 tables
are big endian, so entries must be converted before being passed to functions.

This bug is easy to trigger. The following script will create and destroy a
qcow2 image (the header is gone after three loop iterations):

    #!/bin/bash
    qemu-img create -f qcow2 test.qcow 1M
    for i in $(seq 1 10); do
    qemu-system-x86_64 -hda test.qcow -monitor stdio > /dev/null 2>&1 <<EOF
    savevm test-$i
    quit
    EOF
    done
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7165 c046a42c-6fe2-441c-8c8c-71466251a162

git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7103 c046a42c-6fe2-441c-8c8c-71466251a162

Make all AIO requests vectored and defer linearization until the actual
I/O thread.  This prepares for using native preadv/pwritev.

Also enables asynchronous direct I/O by handling that case in the I/O thread.

Qcow and qcow2 propably want to be adopted to directly deal with multi-segment
requests, but that can be implemented later.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7020 c046a42c-6fe2-441c-8c8c-71466251a162

Always use the vectored APIs to reduce code churn once we switch the BlockDriver
API to be vectored.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7019 c046a42c-6fe2-441c-8c8c-71466251a162

We now enforce that you cannot write beyond the end of a non-growable file.
qcow2 files are not growable but we rely on them being growable to do
savevm/loadvm.  Temporarily allow them to be growable by introducing a new
API specifically for savevm read/write operations.

Reported-by: malc
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6994 c046a42c-6fe2-441c-8c8c-71466251a162

When creating large disk images w/ qcow2 format, qcow2_create is hard
coded to creating a single refcount block.  This is insufficient for
large images, and will cause qemu-img to segfault as it walks off the
end of the refcount block.  Keep track of the space needed during image
create and create proper number of refcount blocks accordingly.

https://bugzilla.redhat.com/show_bug.cgi?id=491943Signed-off-by: NChris Wright <chrisw@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6982 c046a42c-6fe2-441c-8c8c-71466251a162

qcow2's get_cluster_offset() scans forward in the l2 table to find other
clusters that have the same allocation status as the first cluster.
This is used by (among others) qcow_is_allocated().

Unfortunately, it was not checking to be sure that it didn't fall off
the end of the l2 table.  This patch adds that check.

The symptom that motivated me to look into this was that
bdrv_is_allocated() was returning false when there was in fact data
there.  This is one of many ways this bug could lead to data corruption.

I checked the other place that scans for consecutive unallocated blocks
(alloc_cluster_offset()) and it appears to be OK:
    nb_clusters = MIN(nb_clusters, s->l2_size - l2_index);
appears to prevent the same problem from occurring.

Signed-off-by: Nolan Leake <nolan <at> sigbus.net>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6977 c046a42c-6fe2-441c-8c8c-71466251a162

Recent patches added two compiler warnings about the format string
usage in qcow_read_extensions.  One is printing a uint64_t using
%lu which is incorrect on many platforms as it can be a unsigned
long long, the second one is printing the result of sizeof as
%lu, but it is a size_t so it needs to be printed using %zu.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6944 c046a42c-6fe2-441c-8c8c-71466251a162

Use a qcow2 extension to keep the backing file format.

By keeping the backing file format, we can:
1. Provide a way to know the backing file format without probing
   it (setting the format at creation time).
2. Enable using qcow2 format over host block devices.
   (only if the user specifically asks for it, by providing the format
   at creation time).

Also fixes a security flaw found by Daniel P. Berrange on [1]
which summarizes: "Autoprobing: just say no."

[1] http://lists.gnu.org/archive/html/qemu-devel/2008-12/msg01083.htmlSigned-off-by: NUri Lublin <uril@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6909 c046a42c-6fe2-441c-8c8c-71466251a162

Qcow2 extensions are build of magic (id) len (in bytes) and data.
They reside right after the qcow2 header.
If a backing filename exists it follows the qcow2 extension (if exist)

Qcow2 extensions are read upon image open.
Qcow2 extensions are identified by their magic.
Unknown qcow2 extensions (unknown magic) are skipped.
A Special magic of 0 means end-of-qcow2-extensions.

In this patchset, to be used to keep backing file format.

Based on a work done by Shahar Frank <sfrank@redhat.com>.
Signed-off-by: NUri Lublin <uril@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6907 c046a42c-6fe2-441c-8c8c-71466251a162

This series is broken by design as it requires expensive IO operations at
open time causing very long delays when starting a virtual machine for the
first time.
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6816 c046a42c-6fe2-441c-8c8c-71466251a162