This adds the wr_highest_sector blockstat which implements what is generally
known as the high watermark. It is the highest offset of a sector written to
the respective BlockDriverState since it has been opened.

The query-blockstat QMP command is extended to add this value to the result,
and also to add the statistics of the underlying protocol in a new "parent"
field. Note that to get the "high watermark" of a qcow2 image, you need to look
into the wr_highest_sector field of the parent (which can be a file, a
host_device, ...). The wr_highest_sector of the qcow2 BlockDriverState itself
is the highest offset on the _virtual_ disk that the guest has written to.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch adds the ability to grow qcow2 images in-place using
bdrv_truncate().  This enables qemu-img resize command support for
qcow2.

Snapshots are not supported and bdrv_truncate() will return -ENOTSUP.
The notion of resizing an image with snapshots could lead to confusion:
users may expect snapshots to remain unchanged, but this is not possible
with the current qcow2 on-disk format where the header.size field is
global instead of per-snapshot.  Others may expect snapshots to change
size along with the current image data.  I think it is safest to not
support snapshots and perhaps add behavior later if there is a
consensus.

Backing images continue to work.  If the image is now larger than its
backing image, zeroes are read when accessing beyond the end of the
backing image.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

While it's true that during regular operation free_clusters failure would be a
bug, an I/O error can always happen. There's no need to kill the VM, the worst
thing that can happen (and it will) is that we leak some clusters.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch adds a 'resize' command to grow/shrink disk images.  This
allows changing the size of disk images without copying to a new image
file.  Currently only raw files support resize.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The BlockDriver bdrv_getlength function is called from the I/O code path
when checking that the request falls within the device.  Unfortunately
this involves an lseek system call in the raw protocol; every read or
write request will incur this lseek cost.

Jan Kiszka <jan.kiszka@siemens.com> identified this issue and its
latency overhead.  This patch caches device length in the existing
total_sectors variable so lseek calls can be avoided for fixed size
devices.

Growable devices fall back to the full bdrv_getlength code path because
I have not added logic to detect extending the size of the device in a
write.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch combines the lseek+read/write calls to use pread/pwrite
instead.  This will result in fewer system calls and is already used by
AIO.

Thanks to Jan Kiszka <jan.kiszka@siemens.com> for identifying excessive
lseek and Christoph Hellwig <hch@lst.de> for confirming that this
approach should work.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The i loop iterator is shadowed by the next free cluster index.  Both
using the variable name 'i' makes the code harder to read.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

It is safer to set backing_hd to NULL after deleting it so that any use
after deletion is obvious during development.  Happy segfaulting!

This patch should be applied after Kevin Wolf's "vmdk: Convert to
bdrv_open" so that vmdk does not segfault on close.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

It's a format driver, so implement bdrv_open instead of bdrv_file_open.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

VMDK is doing interesting things when it needs to open a backing file. This
patch changes that part to look more like in other drivers. The nice side
effect is that the file name isn't needed any more in the open function.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

When trying to do COW, VMDK wrote the data back to the backing file. This
problem was revealed by the patch that made backing files read-only. This patch
does not only fix the problem, but also simplifies the VMDK code a bit.

This fixes the backing file qemu-iotests cases for VMDK.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This fixes the problem that qemu-img's use of no_zero_init only considered the
no_zero_init flag of the format driver, but not of the underlying protocols.

Between the raw/file split and this fix, converting to host devices is broken.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Format drivers shouldn't need to bother with things like file names, but rather
just get an open BlockDriverState for the underlying protocol. This patch
introduces this behaviour for bdrv_open implementation. For protocols which
need to access the filename to open their file/device/connection/... a new
callback bdrv_file_open is introduced which doesn't get an underlying file
opened.

For now, also some of the more obscure formats use bdrv_file_open because they
open() the file themselves instead of using the block.c functions. They need to
be fixed in later patches.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Move bdrv_open_common so it's defined before its callers and remove the forward
declaration.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

bdrv_open contains quite some code that is only useful for opening images (as
opposed to opening files by a protocol), for example snapshots.

This patch splits the code so that we have bdrv_open_file() for files (uses
protocols), bdrv_open() for images (uses format drivers) and bdrv_open_common()
for the code common for opening both images and files.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

We're running into various problems because the "raw" file access, which
is used internally by the various image formats is entangled with the
"raw" image format, which maps the VM view 1:1 to a file system.

This patch renames the raw file backends to the file protocol which
is treated like other protocols (e.g. nbd and http) and adds a new
"raw" image format which is just a wrapper around calls to the underlying
protocol.

The patch is surprisingly simple, besides changing the probing logical
in block.c to only look for image formats when using bdrv_open and
renaming of the old raw protocols to file there's almost nothing in there.

For creating images, a new bdrv_create_file is introduced which guesses the
protocol to use. This allows using qemu-img create -f raw (or just using the
default) for both files and host devices. Converting the other format drivers
to use this function to create their images is left for later patches.

The only issues still open are in the handling of the host devices.
Firstly in current qemu we can specifiy the host* format names
on various command line acceping images, but the new code can't
do that without adding some translation.  Second the layering breaks
the no_zero_init flag in the BlockDriver used by qemu-img.  I'm not
happy how this is done per-driver instead of per-state so I'll
prepare a separate patch to clean this up.

There's some more cleanup opportunity after this patch, e.g. using
separate lists and registration functions for image formats vs
protocols and maybe even host drivers, but this can be done at a
later stage.

Also there's a check for protocol in bdrv_open for the BDRV_O_SNAPSHOT
case that I don't quite understand, but which I fear won't work as
expected - possibly even before this patch.

Note that this patch requires various recent block patches from Kevin
and me, which should all be in his block queue.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

1) Qemu is not only a PC emulator.
2) "image image" has already been changed to "disk image" in qemu-doc.texi
Signed-off-by: NThomas Monjalon <thomas@monjalon.net>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

MAX_OPC_PARAM is intended to refer to the maximum number of entries used
in gen_opparam_buf[] for any single helper call.  It is currently defined
as 10, but for 32-bit archs, the correct value (with a maximum for four
helper arguments) is 14, and for 64-bit archs, only 9 entries are needed.

tcg_gen_callN() fills four entries with the function address, flags,
number of args, etc. and on 32-bit archs uses a further two entries per
argument (with a maximum of four helper arguments), plus two more for the
return value.  On 64-bit archs, only half as many entries are used for the
args and the return value.

In reality, TBs tend not to consist purely of helper calls exceeding the
stated 10 gen_opparam_buf[] entries, so this would never actually be a
problem on 32-bit archs, but the definition is still rather confusing.
Signed-off-by: NStuart Brady <sdb@zubnet.me.uk>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

We cannot install different opaque pointer for read and write
of the same i/o address.

- handle zero address in bmdma_writeb_common and install
  the same opaque pointer for both read and write access.
Signed-off-by: NIgor V. Kovalenko <igor.v.kovalenko@gmail.com>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

Instead of doing tricks to get the pci_dev, just pass it in the 1st
place.  Patch is a bit longer that reverting the pci_dev field, but it
states more clearly (IMHO) what we are doing.

It also fixes the bm test, now that you told me that ->unit is not
always valid.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NIgor V. Kovalenko <igor.v.kovalenko@gmail.com>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

Individual ports can now signal to the virtio-serial core to stop
sending data if the ports cannot immediately handle new data.  When a
port later unthrottles, any data queued up in the virtqueue are sent to
the port.

Disable throttling once a port is closed (and we discard all the
unconsumed buffers in the vq).

The guest kernel can reclaim the buffers when it receives the port close
event or when a port is being removed. Ensure we free up the buffers
before we send out any events to the guest.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Before the earlier patch, we relied on incorrect virtio api usage to
signal to the guest that a particular buffer wasn't consumed by the
host.

After fixing that, we now just discard the data the guest sends us while
a host port is disconnected or doesn't have a handler registered for
consuming data.

This commit really doesn't change anything from the current behaviour,
just makes the code slightly better by spinning off data handling to
ports in another function.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

We cannot indicate to the guest how much data was consumed by an app for
out_bufs.  So we just have to assume the apps will consume all the data
that are handed over to them.

Fix the virtio api abuse in control_out() and handle_output().
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Current guests don't send more than one iov but it can change later.
Ensure we handle that case.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
CC: Avi Kivity <avi@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Current control messages are small enough to not be split into multiple
buffers but we could run into such a situation in the future or a
malicious guest could cause such a situation.

So handle the entire iov request for control messages.

Also ensure the size of the control request is >= what we expect
otherwise we risk accessing memory that we don't own.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
CC: Avi Kivity <avi@redhat.com>
Reported-by: NAvi Kivity <avi@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

iov_to_buf() puts the buffer contents in the iov in a linearized buffer.

iov_size() gets the length of the contents in the iov.

The iov_to_buf() function is the memcpy_to_iovec() function that was
used in virtio-ballon.c.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The virtio-net code uses iov_fill() which fills an iov from a linear
buffer. The virtio-serial-bus code does something similar in an
open-coded function.

Create a new iov.c file that has iov_from_buf().

Convert virtio-net and virtio-serial-bus over to use this functionality.
virtio-net used ints to hold sizes, the new function is going to use
size_t types.

Later commits will add the opposite functionality -- going from an iov
to a linear buffer.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Data should be written only when ports are open.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

If adding of ports or devices in the guest fails we can send out a QMP
event so that management software can deal with it.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The check for a 0-sized write request to a guest port is not necessary;
the while loop below won't be executed in this case and all will be
fine.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The virtio-serial code doesn't mix declarations and definitions, so
separate them out on different lines.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Allow the port 'id's to be set by a user on the command line. This is
needed by management apps that will want a stable port numbering scheme
for hot-plug/unplug and migration.

Since the port numbers are shared with the guest (to identify ports in
control messages), we just send a control message to the guest
indicating addition of new ports (hot-plug) or notifying the guest of
the available ports when the guest sends us a DEVICE_READY control
message.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

If the host connection to a port is closed on the destination machine
after migration, whereas the connection was open on the source, the
guest has to be informed of that.

Similar for a host connection open on the destination.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

If some ports that were hot-plugged on the source are not available on
the destination, fail migration instead of trying to deref a NULL
pointer.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Reported-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The number of ports on the source as well as the destination machines
should match. If they don't, it means some ports that got hotplugged on
the source aren't instantiated on the destination. Or that ports that
were hot-unplugged on the source are created on the destination.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Reported-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The target could be started with max_nr_ports for a virtio-serial device
lesser than what was available on the source machine. Fail the migration
in such a case.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Reported-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Using GCC-4.2.4-1ubuntu4, there is a warning:
	microblaze-dis.c:792: warning: unused variable 'fprintf'

Indeed, fprintf() is shadowed by a custom redefinition but is not used because
of FORTIFY_SOURCE option which replace calls to fprintf() by fprintf_chk().
So, fprintf refers to the libc implementation instead of the qemu one.
It's a bug.

It is fixed by renaming the variable to something different of "fprintf".
It prevents from hazardous shadowing.
Signed-off-by: NThomas Monjalon <thomas@monjalon.net>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

Signed-off-by: NRichard Henderson <rth@twiddle.net>
Signed-off-by: NAurelien Jarno <aurelien@aurel32.net>