If two Linux AIO request completions are fetched in the same
io_getevents() call, QEMU will deadlock if request A's callback waits
for request B to complete using an aio_poll() loop.  This was reported
to happen with the mirror blockjob.

This patch moves completion processing into a BH and makes it resumable.
Nested event loops can resume completion processing so that request B
will complete and the deadlock will not occur.

Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Ming Lei <ming.lei@canonical.com>
Cc: Marcin Gibuła <m.gibula@beyond.pl>
Reported-by: NMarcin Gibuła <m.gibula@beyond.pl>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Tested-by: NMarcin Gibuła <m.gibula@beyond.pl>

when hotplug virtio-scsi disks using laio, the aio_nr will
increase in laio_init() by io_setup(), we can see the number by
  # cat /proc/sys/fs/aio-nr
  128
if the aio_nr attach the maxnum, which found from
  # cat /proc/sys/fs/aio-max-nr
  65536
the hotplug process will fail because of aio context leak.

Fix it by io_destroy in laio_cleanup().
Reported-by: Ndaifulai <daifulai@huawei.com>
Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This patch implements .bdrv_io_plug, .bdrv_io_unplug and
.bdrv_flush_io_queue callbacks for linux-aio Block Drivers,
so that submitting I/O as a batch can be supported on linux-aio.

[Unprocessed requests are completed with -EIO instead of a bogus ret
value.
--Stefan]
Signed-off-by: NMing Lei <ming.lei@canonical.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Hot unplugging -drive aio=native,file=test.img,format=raw images leaves
the Linux AIO event notifier and struct qemu_laio_state allocated.
Luckily nothing will use the event notifier after the BlockDriverState
has been closed so the handler function is never called.

It's still worth fixing this resource leak.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Drop the assumption that we're using the main AioContext for Linux AIO.
Convert the Linux AIO event notifier to use aio_set_event_notifier().

The .bdrv_detach/attach_aio_context() interfaces also need to be
implemented to move the event notifier handler from the old to the new
AioContext.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The .io_flush() handler no longer exists and has no users.  Drop the
io_flush argument to aio_set_fd_handler() and related functions.

The AioFlushEventNotifierHandler and AioFlushHandler typedefs are no
longer used and are dropped too.
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

.io_flush() is no longer called so drop qemu_laio_completion_cb().  It
turns out that count is now unused so drop that too.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Now that AIOPool no longer keeps a freelist, it isn't really a "pool"
anymore.  Rename it to AIOCBInfo and make it const since it no longer
needs to be modified.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Since linux-aio already uses an eventfd, converting it to use the
EventNotifier-based API simplifies the code even though it is not
meant to be portable.
Reviewed-by: NAnthony Liguori <anthony@codemonkey.ws>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Start introducing AioContext, which will let us remove globals from
aio.c/async.c, and introduce multiple I/O threads.

The bottom half functions now take an additional AioContext argument.
A bottom half is created with a specific AioContext that remains the
same throughout the lifetime.  qemu_bh_new is just a wrapper that
uses a global context.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This patch combines two functions into one, and replaces
the implementation with already existing iov_memset() from
iov.c.

The new prototype of qemu_iovec_memset():
  size_t qemu_iovec_memset(qiov, size_t offset, int fillc, size_t bytes)
It is different from former qemu_iovec_memset_skip(), and
I want to make other functions to be consistent with it
too: first how much to skip, second what, and 3rd how many
of it.  It also returns actual number of bytes filled in,
which may be less than the requested `bytes' if qiov is
smaller than offset+bytes, in the same way iov_memset()
does.

While at it, use utility function iov_memset() from
iov.h in posix-aio-compat.c, where qiov was used.
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Both unused after the previous patch.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

And remove several block_int.h inclusions that should not be there.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Initially done with the following semantic patch:

@ rule1 @
expression E;
statement S;
@@
  E = qemu_aio_get (...);
(
- if (E == NULL) { ... }
|
- if (E)
    { <... S ...> }
)

which however missed occurrences in linux-aio.c and posix-aio-compat.c.
Those were done by hand.

The change in vdi_aio_setup's caller was also done by hand.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This is the linux-aio version of commits 22afa7b5 (raw-posix, synchronous) and
ba1d1afd (posix-aio-compat). Reads now produce zeros after the end of file
instead of failing or resulting in short reads, making linux-aio compatible
with the behaviour of synchronous raw-posix requests and posix-aio-compat.

The problem can be reproduced like this:

dd if=/dev/zero of=/tmp/test.raw bs=1 count=1234
./qemu-io -k -n -g -c 'read -p 1024 512' /tmp/test.raw

Previously, the result of this was 'read failed: Invalid argument', now the
read completes successfully.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>

The error handling order was in the wrong order, so that either the ACB would
be leaked or the counter would be decremented when it shouldn't.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Add some notes about Linux AIO explaining why we don't use AIO in
some situations.
Signed-off-by: NFrediano Ziglio <freddy77@gmail.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

qemu_malloc/qemu_free no longer exist after this commit.
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The purpose of AsyncContexts was to protect qcow and qcow2 against reentrancy
during an emulated bdrv_read/write (which includes a qemu_aio_wait() call and
can run AIO callbacks of different requests if it weren't for AsyncContexts).

Now both qcow and qcow2 are protected by CoMutexes and AsyncContexts can be
removed.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Also for Linux AIO, don't call callbacks that don't belong to the active
AsyncContext.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

We'll leave some AIO completions unhandled when we can't call the callback.
qemu_aio_process_queue() is used later to run any callbacks that are left and
can be run then.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Now that do have a nicer interface to work against we can add Linux native
AIO support.  It's an extremly thing layer just setting up an iocb for
the io_submit system call in the submission path, and registering an
eventfd with the qemu poll handler to do complete the iocbs directly
from there.

This started out based on Anthony's earlier AIO patch, but after
estimated 42,000 rewrites and just as many build system changes
there's not much left of it.

To enable native kernel aio use the aio=native sub-command on the
drive command line.  I have also added an option to qemu-io to
test the aio support without needing a guest.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>