More sync, aio, and coroutine unification.  Make bdrv_aio_writev() go
through coroutine request processing.

Remove the dirty block callback mechanism which was needed only for aio
processing and can be done more naturally in coroutine context.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The aio write operation marks blocks dirty when the write operation
completes.  The coroutine write operation marks blocks dirty before
issuing the write operation.

It seems safest to mark the block dirty when the operation completes so
that anything tracking dirty blocks will not act before the change has
been made to the image file.

Make the coroutine write operation dirty blocks on write completion.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

More sync, aio, and coroutine unification.  Make bdrv_aio_readv() go
through coroutine request processing.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The bdrv_read()/bdrv_write() functions call .bdrv_read()/.bdrv_write().
They should go through bdrv_co_do_readv() and bdrv_co_do_writev()
instead in order to unify request processing code across sync, aio, and
coroutine interfaces.  This is also an important step towards removing
BlockDriverState .bdrv_read()/.bdrv_write() in the future.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The public interface for I/O in coroutine context is bdrv_co_readv() and
bdrv_co_writev().  Split out the request processing code into
bdrv_co_do_readv() and bdrv_co_writev() so that it can be called
internally when we refactor all request processing to use coroutines.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The emulation functions which supply default BlockDriver .bdrv_*()
functions given another implemented .bdrv_*() function should not use
public bdrv_*() interfaces.  This patch ensures they invoke .bdrv_*()
directly to avoid adding an extra layer of coroutine request processing
and possibly entering an infinite loop.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

We will unify block layer request processing across sync, aio, and
coroutines and this means a .bdrv_co_*() emulation function should not
call back into the public interface.  There's no need here, just call
.bdrv_aio_*() directly.

The gory details: bdrv_co_io_em() cannot call back into the public
bdrv_aio_*() interface since that will be handled using coroutines,
which causes us to call into bdrv_co_io_em() again in an infinite loop
:).
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Contains the I/O status for the given device. The key is only present
if the device supports it and the VM is configured to stop on errors.

Please, check the documentation being added in this commit for more
information.
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This commit adds support to the BlockDriverState type to keep track
of devices' I/O status.

There are three possible status: BDRV_IOS_OK (no error), BDRV_IOS_ENOSPC
(no space error) and BDRV_IOS_FAILED (any other error). The distinction
between no space and other errors is important because a management
application may want to watch for no space in order to extend the
space assigned to the VM and put it to run again.

Qemu devices supporting the I/O status feature have to enable it
explicitly by calling bdrv_iostatus_enable() _and_ have to be
configured to stop the VM on errors (ie. werror=stop|enospc or
rerror=stop).

In case of multiple errors being triggered in sequence only the first
one is stored. The I/O status is always reset to BDRV_IOS_OK when the
'cont' command is issued.

Next commits will add support to some devices and extend the
query-block/info block commands to return the I/O status information.
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

It is useful to know the BlockDriverState as well as the
sector_num/nb_sectors of an emulated .bdrv_co_*() request.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>

bdrv_open_common() is a useful point to trace since it reveals the
filename and block driver for a given BlockDriverState.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>

To let device models distinguish between eject and load.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

BlockDriverState member buffer_alignment is initially 512.  The device
model may set them, with bdrv_set_buffer_alignment().  If the device
model gets detached (hot unplug), the device's alignment is left
behind.  Only okay because device hot unplug automatically destroys
the BlockDriverState.  But that's a questionable feature, best not to
rely on it.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Device models should be able to set it without an unclean include of
block_int.h.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Need to ask the device, so this requires new BlockDevOps member
is_tray_open().
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

It's a confused mess (see previous commit).  No users remain.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

BlockDriverState member removable is a confused mess.  It is true when
an ide-cd, scsi-cd or floppy qdev is attached, or when the
BlockDriverState was created with -drive if={floppy,sd} or -drive
if={ide,scsi,xen,none},media=cdrom ("created removable"), except when
an ide-hd, scsi-hd, scsi-generic or virtio-blk qdev is attached.

Three users remain:

1. eject_device(), via bdrv_is_removable() uses it to determine
   whether a block device can eject media.

2. bdrv_info() is monitor command "info block".  QMP documentation
   says "true if the device is removable, false otherwise".  From the
   monitor user's point of view, the only sensible interpretation of
   "is removable" is "can eject media with monitor commands eject and
   change".

A block device can eject media unless a device is attached that
doesn't support it.  Switch the two users over to new
bdrv_dev_has_removable_media() that returns exactly that.

3. bdrv_getlength() uses to suppress its length cache when media can
   change (see commit 46a4e4e6).  Media change is either monitor
   command change (updates the length cache), monitor command eject
   (doesn't update the length cache, easily fixable), or physical
   media change (invalidates length cache, not so easily fixable).

I'm refraining from improving anything here, because this series is
long enough already.  Instead, I simply switch it over to
bdrv_dev_has_removable_media() as well.

This changes the behavior of the length cache and of monitor commands
eject and change in two cases:

a. drive not created removable, no device attached

   The commit makes the drive removable, and defeats the length cache.

   Example: -drive if=none

b. drive created removable, but the attached drive is non-removable,
   and doesn't call bdrv_set_removable(..., 0) (most devices don't)

   The commit makes the drive non-removable, and enables the length
   cache.

   Example: -drive if=xen,media=cdrom -M xenpv

   The other non-removable devices that don't call
   bdrv_set_removable() can't currently use a drive created removable,
   either because they aren't qdevified, or because they lack a drive
   property.  Won't stay that way.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

While there, make the locked parameter bool.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Requires new BlockDevOps member is_medium_locked().  Implement for IDE
and SCSI CD-ROMs.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The device model knows best when to accept the guest's eject command.
No need to detour through the block layer.

bdrv_eject() can't fail anymore.  Make it void.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Commit 4be9762a is now completely redone.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Commit 4be9762a changed bdrv_is_inserted() to fail when the tray is
open.  Unfortunately, there are two different kinds of users, with
conflicting needs.

1. Device models using bdrv_eject(), currently ide-cd and scsi-cd.
They expect bdrv_is_inserted() to reflect the tray status.  Commit
4be9762a makes them happy.

2. Code that wants to know whether a BlockDriverState has media, such
as find_image_format(), bdrv_flush_all().  Commit 4be9762a makes them
unhappy.  In particular, it breaks flush on VM stop for media ejected
by the guest.

Revert the change to bdrv_is_inserted().  Check the tray status in the
device models instead.

Note on IDE: Since only ATAPI devices have a tray, and they don't
accept ATA commands since the recent commit "ide: Reject ATA commands
specific to drive kinds", checking in atapi.c suffices.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

savevm and loadvm silently ignore block devices with removable media,
such as floppies and SD cards.  Rolling back a VM to a previous
checkpoint will *not* roll back writes to block devices with removable
media.

Moreover, bdrv_is_removable() is a confused mess, and wrong in at
least one case: it considers "-drive if=xen,media=cdrom -M xenpv"
removable.  It'll be cleaned up later in this series.

Read-only block devices are also ignored, but that's okay.

Fix by ignoring only read-only block devices and empty block devices.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Change (!bdrv_is_removable(bs) || bdrv_is_inserted(bs)) to just
bdrv_is_inserted().  Rationale:

    The value of bdrv_is_removable(bs) matters only when
    bdrv_is_inserted(bs) is false.

    bdrv_is_inserted(bs) is true when bs is open (bs->drv != NULL) and
    not an empty host drive (CD-ROM or floppy).

    Therefore, bdrv_is_removable(bs) matters only when:

    1. bs is not open
       old: may call bdrv_flush(bs), which does nothing
       new: won't call

    2. bs is an empty host drive
       old: may call bdrv_flush(bs), which calls driver method
            raw_flush(), which calls fdatasync() or equivalent, which
            can't do anything useful while the drive is empty
       new: won't call

Result is bs->drv && !bdrv_is_read_only(bs) && bdrv_is_inserted(bs).
bdrv_is_inserted(bs) implies bs->drv.  Drop the redundant test.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

hw/fdc.c is the only one that cares.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Multiplexing callbacks complicates matters needlessly.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

So we can more easily add device model callbacks.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

For now, this just protects against programming errors like having the
same drive back multiple non-qdev devices, or untimely bdrv_delete().
Later commits will add other interesting uses.

While there, rename BlockDriverState member peer to dev, bdrv_attach()
to bdrv_attach_dev(), bdrv_detach() to bdrv_detach_dev(), and
bdrv_get_attached() to bdrv_get_attached_dev().
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Most changes were made using these commands:

git grep -la '__attribute__((packed))'|xargs perl -pi -e 's/__attribute__\(\(packed\)\)/QEMU_PACKED/'
git grep -la '__attribute__ ((packed))'|xargs perl -pi -e 's/__attribute__ \(\(packed\)\)/QEMU_PACKED/'
git grep -la '__attribute__((__packed__))'|xargs perl -pi -e 's/__attribute__\(\(__packed__\)\)/QEMU_PACKED/'
git grep -la '__attribute__ ((__packed__))'|xargs perl -pi -e 's/__attribute__ \(\(__packed__\)\)/QEMU_PACKED/'
git grep -la '__attribute((packed))'|xargs perl -pi -e 's/__attribute\(\(packed\)\)/QEMU_PACKED/'

Whitespace in linux-user/syscall_defs.h was fixed manually
to avoid warnings from scripts/checkpatch.pl.

Manual changes were also applied to hw/pc.c.

I did not fix indentation with tabs in block/vvfat.c.
The patch will show 4 errors with scripts/checkpatch.pl.
Signed-off-by: NStefan Weil <weil@mail.berlios.de>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

Account the total latency for read/write/flush requests.  This allows
management tools to average it based on a snapshot of the nr ops
counters and allow checking for SLAs or provide statistics.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Decouple the I/O accounting from bdrv_aio_readv/writev/flush and
make the hardware models call directly into the accounting helpers.

This means:
 - we do not count internal requests from image formats in addition
   to guest originating I/O
 - we do not double count I/O ops if the device model handles it
   chunk wise
 - we only account I/O once it actuall is done
 - can extent I/O accounting to synchronous or coroutine I/O easily
 - implement I/O latency tracking easily (see the next patch)

I've conveted the existing device model callers to the new model,
device models that are using synchronous I/O and weren't accounted
before haven't been updated yet.  Also scsi hasn't been converted
to the end-to-end accounting as I want to defer that after the pending
scsi layer overhaul.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch adds -drive cache=directsync for O_DIRECT | O_SYNC host file
I/O with no disk write cache presented to the guest.

This mode is useful when guests may not be sending flushes when
appropriate and therefore leave data at risk in case of power failure.
When cache=directsync is used, write operations are only completed to
the guest when data is safely on disk.

This new mode is like cache=writethrough but it bypasses the host page
cache.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch introduces bdrv_parse_cache_flags() which sets open flags
given a cache mode.  Previously this was duplicated in blockdev.c and
qemu-img.c.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Fix code format to make checkpatch.pl happy.
Signed-off-by: NRobert Wang <wdongxu@linux.vnet.ibm.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

qemu_malloc/qemu_free no longer exist after this commit.
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

If we're already in a coroutine, there is no reason to use the synchronous
version of block layer functions when a coroutine one exists. This makes
bdrv_read/write/flush use bdrv_co_* when used inside a coroutine.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The purpose of AsyncContexts was to protect qcow and qcow2 against reentrancy
during an emulated bdrv_read/write (which includes a qemu_aio_wait() call and
can run AIO callbacks of different requests if it weren't for AsyncContexts).

Now both qcow and qcow2 are protected by CoMutexes and AsyncContexts can be
removed.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

In order to be able to call bdrv_co_readv/writev for drivers that don't
implement the functions natively, add an emulation that uses the AIO functions
to implement them.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>