This patch combines the lseek+read/write calls to use pread/pwrite
instead.  This will result in fewer system calls and is already used by
AIO.

Thanks to Jan Kiszka <jan.kiszka@siemens.com> for identifying excessive
lseek and Christoph Hellwig <hch@lst.de> for confirming that this
approach should work.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Format drivers shouldn't need to bother with things like file names, but rather
just get an open BlockDriverState for the underlying protocol. This patch
introduces this behaviour for bdrv_open implementation. For protocols which
need to access the filename to open their file/device/connection/... a new
callback bdrv_file_open is introduced which doesn't get an underlying file
opened.

For now, also some of the more obscure formats use bdrv_file_open because they
open() the file themselves instead of using the block.c functions. They need to
be fixed in later patches.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

We're running into various problems because the "raw" file access, which
is used internally by the various image formats is entangled with the
"raw" image format, which maps the VM view 1:1 to a file system.

This patch renames the raw file backends to the file protocol which
is treated like other protocols (e.g. nbd and http) and adds a new
"raw" image format which is just a wrapper around calls to the underlying
protocol.

The patch is surprisingly simple, besides changing the probing logical
in block.c to only look for image formats when using bdrv_open and
renaming of the old raw protocols to file there's almost nothing in there.

For creating images, a new bdrv_create_file is introduced which guesses the
protocol to use. This allows using qemu-img create -f raw (or just using the
default) for both files and host devices. Converting the other format drivers
to use this function to create their images is left for later patches.

The only issues still open are in the handling of the host devices.
Firstly in current qemu we can specifiy the host* format names
on various command line acceping images, but the new code can't
do that without adding some translation.  Second the layering breaks
the no_zero_init flag in the BlockDriver used by qemu-img.  I'm not
happy how this is done per-driver instead of per-state so I'll
prepare a separate patch to clean this up.

There's some more cleanup opportunity after this patch, e.g. using
separate lists and registration functions for image formats vs
protocols and maybe even host drivers, but this can be done at a
later stage.

Also there's a check for protocol in bdrv_open for the BDRV_O_SNAPSHOT
case that I don't quite understand, but which I fear won't work as
expected - possibly even before this patch.

Note that this patch requires various recent block patches from Kevin
and me, which should all be in his block queue.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Split up the raw_getlength into separate generic, solaris and BSD
versions to reduce the ifdef maze a bit.  The BSD variant still
is a complete maze, but to clean it up properly we'd need some
people using the BSD variants to figure out what code is used
for what variant.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

bdrv_open already takes care of this for us.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAurelien Jarno <aurelien@aurel32.net>

Now that we output an error message according to the returned error code in
qemu-img, let's return the real error codes. "Input/output error" for
everything isn't helpful.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAurelien Jarno <aurelien@aurel32.net>

This shouldn't happen under any normal circumstances. However, it looks like
it's possible to achieve this with corrupted images. Without this patch
raw_pread is hanging in an endless loop in such cases.

The patch is not affecting growable files, for which such reads happen in
normal use cases. raw_pread_aligned already handles these cases and won't
return zero in the first place.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Win32 suffers from a very big memory leak when dealing with SCSI devices.
Each read/write request allocates memory with qemu_memalign (ie
VirtualAlloc) but frees it with qemu_free (ie free).
Pair all qemu_memalign() calls with qemu_vfree() to prevent such leaks.
Signed-off-by: NHerve Poussineau <hpoussin@reactos.org>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The BDRV_O_CREAT option is unused inside qemu and partially duplicates
the bdrv_create method.  Remove it, and the -C option to qemu-io which
isn't used in qemu-iotests anyway.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Clean-up a little bit the RW related bits of BDRV_O_FLAGS. BDRV_O_RDONLY gone (and so is BDRV_O_ACCESS). Default value for bdrv_flags (0/zero) is READ-ONLY. Need to explicitly request READ-WRITE.

Instead of using the field 'readonly' of the BlockDriverState struct for passing the request,
pass the request in the flags parameter to the function.
Signed-off-by: NNaphtali Sprei <nsprei@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Current legacy floppy detection is hardcoded based on source file
name. Make this smarter on linux by attempting a floppy specific
ioctl.

v2:
    Give ioctl check higher priority than filename check
    s/IDE/legacy/

v3:
    Actually initialize 'prio' variable
    Check for ioctl success rather than absence of specific failure

v4:
    Explicitly mention that change is linux specific.
Signed-off-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Current CDROM detection is hardcoded based on source file name.
Make this smarter on linux by attempting a CDROM specific ioctl.

This makes '-cdrom /dev/sr0' succeed with no media present.

v2:
    Give ioctl check higher priority than filename check.

v3:
    Actually initialize 'prio' variable.
    Check for ioctl success rather than absence of specific failure.

v4:
    Explicitly mention that change is linux specific.
Signed-off-by: NCole Robinson <crobinso@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

We're leaking file descriptors to child processes. Set FD_CLOEXEC on file
descriptors that don't need to be passed to children to stop this misbehaviour.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

I haven't heard yet of anyone using qemu-img to copy an image to a real floppy,
but it's a valid use case.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NAurelien Jarno <aurelien@aurel32.net>

The context parameter in paio_submit isn't used anyway, so there is no reason
why block drivers should need to remember it. This also avoids passing a Linux
AIO context to paio_submit (which doesn't do any harm as long as the parameter
is unused, but it is highly confusing).
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

When using Linux AIO raw still falls back to POSIX AIO sometimes, so we should
initialize it.

Not initializing it happens to work if POSIX AIO is used by another drive, or
if the format is not specified (probing the format uses POSIX AIO) or by pure
luck (e.g. it doesn't seem to happen any more with qcow2 since we have re-added
synchronous qcow2 functions).
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Today host_devices have a create function, so they also need a create_options
field to prevent qemu-img from complaining.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Instead stalling the VCPU while serving a cache flush try to do it
asynchronously.  Use our good old helper thread pool to issue an
asynchronous fdatasync for raw-posix.  Note that while Linux AIO
implements a fdatasync operation it is not useful for us because
it isn't actually implement in asynchronous fashion.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

If we are flushing the caches for our image files we only care about the
data (including the metadata required for accessing it) but not things
like timestamp updates.  So try to use fdatasync instead of fsync to
implement the flush operations.

Unfortunately many operating systems still do not support fdatasync,
so we add a qemu_fdatasync wrapper that uses fdatasync if available
as per the _POSIX_SYNCHRONIZED_IO feature macro or fsync otherwise.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

This patch fixes linker errors when building QEMU without Linux AIO support.

It is based on suggestions from malc and Kevin Wolf.
Signed-off-by: NStefan Weil <weil@mail.berlios.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Now that do have a nicer interface to work against we can add Linux native
AIO support.  It's an extremly thing layer just setting up an iocb for
the io_submit system call in the submission path, and registering an
eventfd with the qemu poll handler to do complete the iocbs directly
from there.

This started out based on Anthony's earlier AIO patch, but after
estimated 42,000 rewrites and just as many build system changes
there's not much left of it.

To enable native kernel aio use the aio=native sub-command on the
drive command line.  I have also added an option to qemu-io to
test the aio support without needing a guest.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Currently the raw-posix.c code contains a lot of knowledge about the
asynchronous I/O scheme that is mostly implemented in posix-aio-compat.c.
All this code does not really belong here and is getting a bit in the
way of implementing native AIO on Linux.

So instead move all the guts of the AIO implementation into
posix-aio-compat.c (which might need a better name, btw).

There's now a very small interface between the AIO providers and raw-posix.c:

 - an init routine is called from raw_open_common to return an AIO context
   for this drive.  An AIO implementation may either re-use one context
   for all drives, or use a different one for each as the Linux native
   AIO support will do.
 - an submit routine is called from the aio_reav/writev methods to submit
   an AIO request

There are no indirect calls involved in this interface as we need to
decide which one to call manually.  We will only call the Linux AIO native
init function if we were requested to by vl.c, and we will only call
the native submit function if we are asked to and the request is properly
aligned.  That's also the reason why the alignment check actually does
the inverse move and now goes into raw-posix.c.

The old posix-aio-compat.h headers is removed now that most of it's
content is private to posix-aio-compat.c, and instead we add a new
block/raw-posix-aio.h headers is created containing only the tiny interface
between raw-posix.c and the AIO implementation.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

As requested by Anthony make pthreads mandatory.  This means we will always
have AIO available on posix hosts, and it will also allow enabling the I/O
thread unconditionally once it's ready.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

I used the following command to enable debugging:
perl -p -i -e 's/^\/\/#define DEBUG/#define DEBUG/g' * */* */*/*
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

In qemu-iotests, some large images are created using qemu-img.

Without checks for errors, qemu-img will just create an
empty image, and later read / write tests will fail.

With the patch, failures during image creation are detected
and reported.
Signed-off-by: NStefan Weil <weil@mail.berlios.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NJohn Arbuckle <programmingkidx@gmail.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

This reverts commit 707c0dbc.
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

When using O_DIRECT, qcow2 snapshots didn't work any more for me. In the
process of creating the snapshot, qcow2 tries to pwrite some new information
(e.g. new L1 table) which will often end up being after the old end of the
image file. Now pwrite tries to align things and reads the old contents of the
file, read returns 0 because there is nothing to read after the end of file and
pwrite is stuck in an endless loop.

This patch allows to pread beyond the end of an image file. Whenever the
given offset is after the end of the image file, the read succeeds and fills
the buffer with zeros.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Problem: It is impossible to feed filenames with the character colon because
qemu interprets such names as a protocol. For example filename scsi:0, is
interpreted as a protocol by name "scsi".

This patch allows user to espace colon characters. For example the above
filename can now be expressed either as 'scsi\:0' or as file:scsi:0

anything following the "file:" tag is interpreted verbatin. However if "file:"
tag is omitted then any colon characters in the string must be escaped using
backslash.

Here are couple of examples:

scsi\:0\:abc is a local file scsi:0:abc
http\://myweb is a local file by name http://myweb
file:scsi:0:abc is a local file scsi:0:abc
file:http://myweb is a local file by name http://mywebSigned-off-by: NRam Pai <linuxram@us.ibm.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

When we open a file, we first attempt to open it read-write, then fall back
to read-only.  Unfortunately we reuse the flags from the previous attempt,
so both attempts try to open the file with write permissions, and fail.

Fix by clearing the O_RDWR flag from the previous attempt.
Signed-off-by: NAvi Kivity <avi@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The flags argument to raw_common_open() contain bits defined by the BDRV_O_*
namespace, not the posix O_* namespace.

Adjust to use the correct constants.
Signed-off-by: NAvi Kivity <avi@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Rename raw_ioctl and raw_aio_ioctl to hdev_ioctl and hdev_aio_ioctl as they
are only used for the host device.  Also only add them to the method table
for the cases where we need them (generic hdev if linux and linux CDROM)
instead of declaring stubs and always add them.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Add a bdrv_probe_device method to all BlockDriver instances implementing
host devices to move matching of host device types into the actual drivers.
For now we keep exacly the old matching behaviour based on the devices names,
although we really should have better detetion methods based on device
information in the future.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Instead of declaring one BlockDriver for all host devices declared one
for each type:  a generic one for normal disk devices, a Linux floppy
driver and a CDROM driver for Linux and FreeBSD.  This gets rid of a lot
of messy ifdefs and switching based on the type in the various removal
device methods.

block.c grows a new method to find the correct host device driver based
on OS-sepcific criteria, which will later into the actual drivers in a
later patch in this series.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

raw_open and hdev_open contain the same basic logic.  Add a new
raw_open_common helper containing the guts of the open routine
and call it from raw_open and hdev_open.

We use the new open_flags field in BDRVRawState to allow passing
additional open flags to raw_open_common from both.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Both the Linux floppy and the FreeBSD CDROM host device need to store
the open flags so that they can re-open the device later.  Store the
open flags unconditionally to remove the ifdef mess and simply the
calling conventions for the later patches in the series.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

This patch adds a small help text to each of the options in the block drivers
which can be displayed by using qemu-img create -f fmt -o ?
Signed-off-by: NKevin Wolf <kwolf@redhat.com>