Change XBZRLE cache size in bytes (the size should be a power of 2, it will be
rounded down to the nearest power of 2).
If XBZRLE cache size is too small there will be many cache miss.

New query-migrate-cache-size QMP command and 'info migrate_cache_size' HMP
command to query cache value.
Signed-off-by: NBenoit Hudzia <benoit.hudzia@sap.com>
Signed-off-by: NPetter Svard <petters@cs.umu.se>
Signed-off-by: NAidan Shribman <aidan.shribman@sap.com>
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

In the outgoing migration check to see if the page is cached and
changed, then send compressed page by using save_xbrle_page function.
In the incoming migration check to see if RAM_SAVE_FLAG_XBZRLE is set
and decompress the page (by using load_xbrle function).
Signed-off-by: NBenoit Hudzia <benoit.hudzia@sap.com>
Signed-off-by: NPetter Svard <petters@cs.umu.se>
Signed-off-by: NAidan Shribman <aidan.shribman@sap.com>
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

For performance we are encoding long word at a time.
For nzrun we use long-word-at-a-time NULL-detection tricks from strcmp():
using ((lword - 0x0101010101010101) & (~lword) & 0x8080808080808080) test
to find out if any byte in the long word is zero.
Signed-off-by: NBenoit Hudzia <benoit.hudzia@sap.com>
Signed-off-by: NPetter Svard <petters@cs.umu.se>
Signed-off-by: NAidan Shribman <aidan.shribman@sap.com>
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Implement Unsigned Little Endian Base 128.
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Add MRU page cache mechanism.
The page are accessed by their address.
Signed-off-by: NBenoit Hudzia <benoit.hudzia@sap.com>
Signed-off-by: NPetter Svard <petters@cs.umu.se>
Signed-off-by: NAidan Shribman <aidan.shribman@sap.com>
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The management can enable/disable a capability for the next migration by using
migrate-set-capabilities QMP command.
The user can use migrate_set_capability HMP command.
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The management can query the current migration capabilities using
query-migrate-capabilities QMP command.
The user can use 'info migrate_capabilities' HMP command.
Currently only XBZRLE capability is available.
Signed-off-by: NOrit Wasserman <owasserm@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

* kraxel/usb.58:
  usb-storage: fix SYNCHRONIZE_CACHE
  usb-storage: improve debug logging

* kwolf/for-anthony:
  qemu-img: use QemuOpts instead of QEMUOptionParameter in resize function
  qemu-iotests: Be more flexible with image creation options
  qemu-iotests: add 039 qcow2 lazy refcounts test
  qemu-io: add "abort" command to simulate program crash
  qcow2: implement lazy refcounts
  qemu-iotests: ignore qemu-img create lazy_refcounts output
  docs: add lazy refcounts bit to qcow2 specification
  qcow2: introduce dirty bit
  docs: add dirty bit to qcow2 specification
  qemu-iotests: add qed.py image manipulation utility
  qapi: generalize documentation of streaming commands
  ide scsi: Mess with geometry only for hard disk devices

Commit 59310659 is incomplete,
we'll arrive in the scsi command complete callback in CSW state
and must handle that case correctly.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

in_addr_t isn't available on mingw32.  Just use an unsigned long instead.  I
considered typedef'ing in_addr_t on mingw32 but this would potentially be
brittle if mingw32 did introduce the type.

Cc: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NDong Xu Wang <wdongxu@linux.vnet.ibm.com>
Reviewed-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

qemu-iotests already filters out image creation options that may be
present or not in order to get the same output in both cases. However,
often it only considers the default value of the option. Cover all valid
values instead so that ./check -o name=value can be used successfull for
all of them.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This tests establishes the basic post-conditions of the qcow2 lazy
refcounts features:

  1. If the image was closed normally, it is marked clean.

  2. If an allocating write was performed and the image was not closed
     normally, then it is marked dirty.

     a. Written data can be read back successfully.
     b. The image file can be repaired and will be marked clean again.
     c. The image file is automatically repaired when opened read/write.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Avoiding data loss and corruption is the top requirement for image file
formats.  The qemu-io "abort" command makes it possible to simulate
program crashes and does not give the image format a chance to cleanly
shut down.  This command is useful for data integrity test cases.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Lazy refcounts is a performance optimization for qcow2 that postpones
refcount metadata updates and instead marks the image dirty.  In the
case of crash or power failure the image will be left in a dirty state
and repaired next time it is opened.

Reducing metadata I/O is important for cache=writethrough and
cache=directsync because these modes guarantee that data is on disk
after each write (hence we cannot take advantage of caching updates in
RAM).  Refcount metadata is not needed for guest->file block address
translation and therefore does not need to be on-disk at the time of
write completion - this is the motivation behind the lazy refcount
optimization.

The lazy refcount optimization must be enabled at image creation time:

  qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on a.qcow2 10G
  qemu-system-x86_64 -drive if=virtio,file=a.qcow2,cache=writethrough

Update qemu-iotests 031 and 036 since the extension header size changes
when we add feature bit table entries.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Hide the default lazy_refcounts=off output from qemu-img like we do with
other image creation options.  This ensures that existing golden outputs
continue to pass despite the new option that has been added.

Note that this patch applies before the one that actually introduces the
lazy_refcounts=on|off option.  This ensures git-bisect(1) continues to
work.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The lazy refcounts bit indicates that this image can take advantage of
the dirty bit and that refcount updates can be postponed.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch adds an incompatible feature bit to mark images that have not
been closed cleanly.  When a dirty image file is opened a consistency
check and repair is performed.

Update qemu-iotests 031 and 036 since the extension header size changes
when we add feature bit table entries.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The dirty bit will make it possible to perform lazy refcount updates,
where the image file is not kept consistent all the time.  Upon opening
a dirty image file, it is necessary to perform a consistency check and
repair any incorrect refcounts.

Therefore the dirty bit must be an incompatible feature bit.  We don't
want old programs accessing a file with stale refcounts.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The qed.py utility can inspect and manipulate QED image files.  It can
be used for testing to see the state of image metadata and also to
inject corruptions into the image file.  It also has a scrubbing feature
to copy just the metadata out of an image file, allowing users to share
broken image files without revealing data in bug reports.

This has lived in my local repo for a long time but could be useful
to others.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Talk about background operations in general, rather than specifically
about streaming.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Legacy -drive cyls=... are now ignored completely when the drive
doesn't back a hard disk device.  Before, they were first checked
against a hard disk's limits, then ignored.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Acked-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Commit b1f416aa breaks vhost_net
because it always registers the virtio_pci_host_notifier_read() handler
function on the ioeventfd, even when vhost_net.ko is using the ioeventfd.
The result is both QEMU and vhost_net.ko polling on the same eventfd
and the virtio_net.ko guest driver seeing inconsistent results:

  # ifconfig eth0 192.168.0.1 netmask 255.255.255.0
  virtio_net virtio0: output:id 0 is not a head!

To fix this, proceed the same as we do for irqfd: add a parameter to
virtio_queue_set_host_notifier_fd_handler and in that case only set
the notifier, not the handler.

Cc: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Tested-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Reviewed-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

* kiszka/queues/slirp:
  slirp: Handle whole 127.0.0.0/8 network as local addresses.

* 'axp-next' of git://repo.or.cz/qemu/rth:
  alpha-linux-user: Fix the getpriority syscall
  alpha-linux-user: Properly handle the non-rt sigprocmask syscall.
  alpha-linux-user: Fix a3 error return with v0 error bypass.
  linux-user: Translate pipe2 flags; add to strace
  linux-user: Allocate the right amount of space for non-fixed file maps
  linux-user: Handle O_SYNC, O_NOATIME, O_CLOEXEC, O_PATH
  linux-user: Sync fcntl.h bits with the kernel
  alpha-linux-user: Handle TARGET_SSI_IEEE_RAISE_EXCEPTION properly
  alpha-linux-user: Work around hosted mmap allocation problems
  alpha-linux-user: Fix signal handling

Alpha uses unbiased priority values in the syscall, with the a3
return value signaling error conditions.  Therefore, properly
interpret the libc getpriority as needed for the guest rather
than passing the host value through unchanged.
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Name the syscall properly for QEMU, kernel source notwithstanding.
Fix syntax errors in the code thus enabled within do_syscall.
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

We were failing to initialize a3 for syscalls that bypass the
negative return value error check.
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

If we let the kernel handle the implementation of mmap_find_vma,
via an anon mmap, we must use the size as indicated by the user
and not the size truncated to the filesize.

This happens often in ld.so, where we initially mmap the file to
the size of the text+data+bss to reserve an area, then mmap+fixed
over the top to properly handle data and bss.
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

For each target, only define the bits that appear in
arch/target/include/asm/fcntl.h.  Mirror the kernel's
asm-generic layout by handling anything undefined afterward.
Signed-off-by: NRichard Henderson <rth@twiddle.net>

We weren't aggregating the exceptions, nor raising signals properly.
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Signed-off-by: NRichard Henderson <rth@twiddle.net>

Proper signal numbers were not defined, and EXCP_INTERRUPT
was unhandled, leading to all sorts of subtle confusion.
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Qualifier 'volatile' is not useful for applications, it's too strict
for single threaded code but does not give the real atomicity guarantees
needed for multithreaded code.

Drop them and now useless casts.
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

The Xen 4.1 probe never uses the return value from xc_interface_open(),
so was provoking a compiler warning on newer gcc. Fix by not bothering
to put the return value anywhere.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>