The 'drive-mirror' tests often issue 'block-job-complete' and wait for
the QMP completion event.  Other types of block jobs also want to wait
for completion but they may not need to issue 'block-job-complete'.

Extract wait_until_completed() from 041 and put it into iotests.py.
Return the QMP event object so the caller can make additional
assertions, if necessary.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The Abort action can be used to test QMP 'transaction' failure.  Add it
as the last action to exercise the .abort() and .cleanup() code paths
for all previous actions.
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch adds a transactional version of the drive-backup QMP command.
It allows atomic snapshots of multiple drives along with automatic
cleanup if there is a failure to start one of the backup jobs.

Note that QMP events are emitted for block job completion/cancellation
and the block job will be listed by query-block-jobs.

@device: the name of the device whose writes should be mirrored.

@target: the target of the new image. If the file exists, or if it
         is a device, the existing file/device will be used as the new
         destination.  If it does not exist, a new file will be created.

@format: #optional the format of the new destination, default is to
         probe if @mode is 'existing', else the format of the source

@mode: #optional whether and how QEMU should create a new image, default is
       'absolute-paths'.

@speed: #optional the maximum speed, in bytes per second

@on-source-error: #optional the action to take on an error on the source,
                  default 'report'.  'stop' and 'enospc' can only be used
                  if the block device supports io-status (see BlockInfo).

@on-target-error: #optional the action to take on an error on the target,
                  default 'report' (no limitations, since this applies to
                  a different block device than @device).
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Some QMP 'transaction' types don't need to do anything on .commit().
Make .commit() optional just like .abort().

The "drive-backup" action will take advantage of this, it only needs to
cancel the block job on .abort().  Other block job actions will probably
follow the same pattern, so allow .commit() to be NULL.
Suggested-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The QMP 'transaction' command keeps a list of in-flight transactions.
The transaction state structure is called BlkTransactionStates even
though it only deals with a single transaction.  The only plural thing
is the linked list of transaction states.

I find it confusing to call the single structure "States".  This patch
renames it to "State", just like BlockDriverState is singular.
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

@drive-backup

Start a point-in-time copy of a block device to a new destination.  The
status of ongoing drive-backup operations can be checked with
query-block-jobs where the BlockJobInfo.type field has the value 'backup'.
The operation can be stopped before it has completed using the
block-job-cancel command.

@device: the name of the device which should be copied.

@target: the target of the new image. If the file exists, or if it
         is a device, the existing file/device will be used as the new
         destination.  If it does not exist, a new file will be created.

@format: #optional the format of the new destination, default is to
         probe if @mode is 'existing', else the format of the source

@mode: #optional whether and how QEMU should create a new image, default is
       'absolute-paths'.

@speed: #optional the maximum speed, in bytes per second

@on-source-error: #optional the action to take on an error on the source,
                  default 'report'.  'stop' and 'enospc' can only be used
                  if the block device supports io-status (see BlockInfo).

@on-target-error: #optional the action to take on an error on the target,
                  default 'report' (no limitations, since this applies to
                  a different block device than @device).

Note that @on-source-error and @on-target-error only affect background I/O.
If an error occurs during a guest write request, the device's rerror/werror
actions will be used.

Returns: nothing on success
         If @device is not a valid block device, DeviceNotFound

Since 1.6
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Use bdrv_getlength() for its byte units and error return instead of
bdrv_get_geometry().
Reported-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

It is not necessary to check that we can find a protocol block driver
since we create or open the image file.  This produces the error that we
need anyway.

Besides, the QERR_INVALID_BLOCK_FORMAT is inappropriate since the
protocol is incorrect rather than the format.

Also drop an empty line between bdrv_open() and checking its return
value.  This may be due to copy-pasting from earlier code that performed
other operations before handling errors.
Reported-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NWenchao Xia <xiawenc@linux.vnet.ibm.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

backup_start() creates a block job that copies a point-in-time snapshot
of a block device to a target block device.

We call backup_do_cow() for each write during backup. That function
reads the original data from the block device before it gets
overwritten.  The data is then written to the target device.

Currently backup cluster size is hardcoded to 65536 bytes.

[I made a number of changes to Dietmar's original patch and folded them
in to make code review easy.  Here is the full list:

 * Drop BackupDumpFunc interface in favor of a target block device
 * Detect zero clusters with buffer_is_zero() and use bdrv_co_write_zeroes()
 * Use 0 delay instead of 1us, like other block jobs
 * Unify creation/start functions into backup_start()
 * Simplify cleanup, free bitmap in backup_run() instead of cb
 * function
 * Use HBitmap to avoid duplicating bitmap code
 * Use bdrv_getlength() instead of accessing ->total_sectors
 * directly
 * Delete the backup.h header file, it is no longer necessary
 * Move ./backup.c to block/backup.c
 * Remove #ifdefed out code
 * Coding style and whitespace cleanups
 * Use bdrv_add_before_write_notifier() instead of blockjob-specific hooks
 * Keep our own in-flight CowRequest list instead of using block.c
   tracked requests.  This means a little code duplication but is much
   simpler than trying to share the tracked requests list and use the
   backup block size.
 * Add on_source_error and on_target_error error handling.
 * Use trace events instead of DPRINTF()

-- stefanha]
Signed-off-by: NDietmar Maurer <dietmar@proxmox.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The bdrv_add_before_write_notifier() function installs a callback that
is invoked before a write request is processed.  This will be used to
implement copy-on-write point-in-time snapshots where we need to copy
out old data before overwriting it.

Note that BdrvTrackedRequest is moved to block_int.h since it is passed
to .notify() functions.
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

notifier_list_notify() has no return value.  This is fine when we just
want to invoke side-effects.

Sometimes it's useful for notifiers to produce a return value.  This
allows notifiers to "veto" an operation and will be used by the block
layer before-write notifier.
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The raw-posix driver has code to provide a /dev/cdrom on OS X even
though it doesn't really exist. However, since commit c66a6157 the real
filename is dismissed after finding it, so opening /dev/cdrom fails.
Put the filename back into the options QDict to make this work again.

Cc: qemu-stable@nongnu.org
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

Fix compilation failures for linux-user targets following recent
migration related commits bd2fa51f and 43487c67.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 1372362818-4740-1-git-send-email-peter.maydell@linaro.org
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

# By Michael R. Hines (9) and others
# Via Juan Quintela
* quintela/migration.next:
  rdma: introduce capability x-rdma-pin-all
  rdma: new QEMUFileOps hooks
  rdma: introduce qemu_ram_foreach_block()
  rdma: export qemu_fflush()
  rdma: introduce qemu_file_mode_is_not_valid()
  rdma: export throughput w/ MigrationStats QMP
  rdma: export yield_until_fd_readable()
  rdma: introduce qemu_update_position()
  rdma: add documentation
  migration: do not overwrite zero pages
  Revert "migration: do not sent zero pages in bulk stage"
  arch_init/ram_load: add error message for block length mismatch

Message-id: 1372329455-5995-1-git-send-email-quintela@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

This capability allows you to disable dynamic chunk registration
for better throughput on high-performance links.

For example, using an 8GB RAM virtual machine with all 8GB of memory in
active use and the VM itself is completely idle using a 40 gbps infiniband link:

1. x-rdma-pin-all disabled total time: approximately 7.5 seconds @ 9.5 Gbps
2. x-rdma-pin-all enabled total time: approximately 4 seconds @ 26 Gbps

These numbers would of course scale up to whatever size virtual machine
you have to migrate using RDMA.

Enabling this feature does *not* have any measurable affect on
migration *downtime*. This is because, without this feature, all of the
memory will have already been registered already in advance during
the bulk round and does not need to be re-registered during the successive
iteration rounds.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NChegu Vinod <chegu_vinod@hp.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Tested-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

These are the prototypes and implementation of new hooks that
RDMA takes advantage of to perform dynamic page registration.

An optional hook is also introduced for a custom function
to be able to override the default save_page function.

Also included are the prototypes and accessor methods used by
arch_init.c which invoke funtions inside savevm.c to call out
to the hooks that may or may not have been overridden
inside of QEMUFileOps.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

This is used during RDMA initialization in order to
transmit a description of all the RAM blocks to the
peer for later dynamic chunk registration purposes.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

RDMA uses this to flush the control channel before sending its
own message to handle page registrations.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

QEMUFileRDMA also has read and write modes. This function is now
shared to reduce code duplication.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

This exposes throughput (in megabits/sec) through QMP.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The RDMA event channel can be made non-blocking just like a TCP
socket. Exporting this function allows us to yield so that the
QEMU monitor remains available.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

RDMA writes happen asynchronously, and thus the performance accounting
also needs to be able to occur asynchronously. This allows anybody
to call into savevm.c to update both f->pos as well as into arch_init.c
to update the acct_info structure with up-to-date values when
the RDMA transfer actually completes.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Tested-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

docs/rdma.txt contains full documentation,
wiki links, github url and contact information.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NChegu Vinod <chegu_vinod@hp.com>
Tested-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

on incoming migration do not memset pages to zero if they already read as zero.
this will allocate a new zero page and consume memory unnecessarily. even
if we madvise a MADV_DONTNEED later this will only deallocate the memory
asynchronously.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Not sending zero pages breaks migration if a page is zero
at the source but not at the destination. This can e.g. happen
if different BIOS versions are used at source and destination.
It has also been reported that migration on pseries is completely
broken with this patch.

This effectively reverts commit f1c72795.

Conflicts:

	arch_init.c
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Makes it easier to debug situations where the source and target have
different ram blocks in a device and migration fails due to that, for
instance a BAR size change on a PCI device.
Signed-off-by: NAlon Levy <alevy@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

# By Peter Crosthwaite (3) and others
# Via Peter Maydell
* pmaydell/arm-devs.for-upstream:
  nand: Don't inherit from Sysbus
  block/nand: Convert Sysbus::init to Device::realize
  block/nand: QOM casting sweep
  i.MX31: Fix PRCS bit test
  arm/boot: Free dtb blob memory after use
  i.MX: Rework functions/types name and use new style initialization
  i.MX: Implement a more complete version of the GPT timer.
  ARM: Allow dumping of device tree

Message-id: 1372184516-32397-1-git-send-email-peter.maydell@linaro.org
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

# By Peter Maydell
# Via Peter Maydell
* pmaydell/target-arm.for-upstream:
  target-arm: Make LPAE feature imply V7MP
  target-arm: Use tuple list to sync cp regs with KVM
  target-arm: Reinitialize all KVM VCPU registers on reset
  target-arm: Initialize cpreg list from KVM when using KVM
  target-arm: Convert TCG to using (index,value) list for cp migration
  target-arm: mark up cpregs for no-migrate or raw access
  target-arm: Add raw_readfn and raw_writefn to ARMCPRegInfo
  target-arm: Allow special cpregs to have flags set

Message-id: 1372181592-32170-1-git-send-email-peter.maydell@linaro.org
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

# By Cornelia Huck (2) and Christian Borntraeger (1)
# Via Cornelia Huck
* cohuck/virtio-ccw-upstr:
  virtio-ccw: Wire up guest and host notifies.
  virtio-ccw: Wire up ioeventfd.
  s390/virtio-ccw: Fix virtio reset

Message-id: 1372177538-9812-1-git-send-email-cornelia.huck@de.ibm.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

# By Paul Durrant (2) and Stefano Stabellini (1)
# Via Stefano Stabellini
* sstabellini/xen-20130625:
  Move hardcoded initialization of xen-platform device.
  Allow use of pc machine type (accel=xen) for Xen HVM domains.
  Revert "xen: start PCI hole at 0xe0000000 (same as pc_init1 and qemu-xen-traditional)"

Message-id: alpine.DEB.2.02.1306251323220.4782@kaball.uk.xensource.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Nand chips are not sysbus devices - they do not have any sense of MMIO,
nor interrupts. Re-parent to TYPE_DEVICE accordingly.

Cc: afaerber@suse.de
Signed-off-by: NPeter Crosthwaite <peter.crosthwaite@xilinx.com>
Reviewed-by: NAndreas Färber <afaerber@suse.de>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

The prescribed transition from Sysbus::init function to a
Device::realize.

Cc: afaerber@suse.de
Signed-off-by: NPeter Crosthwaite <peter.crosthwaite@xilinx.com>
Reviewed-by: NAndreas Färber <afaerber@suse.de>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Define and use standard QOM cast macro. Remove usages of DO_UPCAST and
direct -> style casting.

Cc: afaerber@suse.de
Signed-off-by: NPeter Crosthwaite <peter.crosthwaite@xilinx.com>
Reviewed-by: NAndreas Färber <afaerber@suse.de>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

cppcheck detected a condition which was always false.

According to the MCIMX31 Reference Manual, the PRCS bits have to be 01
to select the Frequency Pre-Multiplier (FPM). PRCS uses bits 1 and 2,
so we have to test for 2.
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NPeter Chubb <peter.chubb@nicta.com.au>
Message-id: 1370810662-32320-1-git-send-email-sw@weilnetz.de
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

The dtb blob returned by load_device_tree() is in memory allocated
with g_malloc(). Free it accordingly once we have copied its
contents into the guest memory. To make this easy, we need also to
clean up the error handling in load_dtb() so that we consistently
handle errors in the same way (by printing a message and then
returning -1, rather than either plowing on or exiting immediately).
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NAndreas Färber <afaerber@suse.de>
Message-id: 1371209256-11408-1-git-send-email-peter.maydell@linaro.org

* use dynamic cast whenever possible
* Change function names to some more meaningful prefix
* Change type names to a more meaningful one
* use new style device initialization
Signed-off-by: NJean-Christophe DUBOIS <jcd@tribudubois.net>
Message-id: 1369898943-1993-3-git-send-email-jcd@tribudubois.net
Reviewed-by: NPeter Chubb <peter.chubb@nicta.com.au>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

* implement compare 1 2 and 3 registers
* simplify Debug printf
Signed-off-by: NJean-Christophe DUBOIS <jcd@tribudubois.net>
Message-id: 1369898943-1993-2-git-send-email-jcd@tribudubois.net
Reviewed-by: NPeter Chubb <peter.chubb@nicta.com.au>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

By calling qemu_devtree_dumpdtb near the end of load_dtb.
Signed-off-by: NJohn Rigby <john.rigby@linaro.org>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

The v7 ARM ARM specifies that the Large Physical Address
Extension requires implementation of the Multiprocessing
Extensions, so make our LPAE feature imply V7MP rather
than specifying both in the A15 CPU initfn.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NAndreas Färber <afaerber@suse.de>
Message-id: 1371127899-10364-1-git-send-email-peter.maydell@linaro.org

Use the tuple list of cp registers for syncing KVM state to QEMU,
rather than only syncing a very minimal set by hand.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>