We now always send a JSON blob describing the migration file format as part
of the migration stream. However, some tools built around QEMU have proven
to stumble over this.

This patch gives the user the chance to disable said self-describing part of
the migration stream. To disable vmdesc submission, just add

  -machine suppress-vmdesc=on

to your QEMU command line.
Signed-off-by: NAlexander Graf <agraf@suse.de>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

One of the really nice things about the VM description format is that it goes
over the wire when live migration is happening. Unfortunately QEMU today closes
any socket once it sees VM_EOF coming, so we never give the VMDESC the chance to
actually land on the wire.

This patch makes QEMU read the description as well. This way we ensure that
anything wire tapping us in between will get the chance to also interpret the
stream.

Along the way we also fix virt tests that assume that number_bytes_sent on the
sender side is equal to number_bytes_read which was true before the VMDESC
patches and is true again with this patch.
Signed-off-by: NAlexander Graf <agraf@suse.de>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Tested-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

It has always been a page header, not a block header.  Once there, the
flag argument was only passed to make a bit or with it, just do the or
on the caller.
Signed-off-by: NJuan Quintela <quintela@redhat.com>

No need to pass it through all the callers.  Once there, update
last_sent_block here.
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add a parameter to pass the number of bytes written, and make it return
the number of pages written instead.
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add a parameter to pass the number of bytes written, and make it return
the number of pages written instead.
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add a parameter to pass the number of bytes written, and make it return
the number of pages written instead.
Signed-off-by: NJuan Quintela <quintela@redhat.com>

It used to be an int, but then we can't pass directly the
bytes_transferred parameter, that would happen later in the series.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>

Add migrate_incoming/migrate-incoming to start an incoming
migration.

Once a qemu has been started with
    -incoming defer

the migration can be started by issuing:
    migrate_incoming uri
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

-incoming defer causes qemu to wait for an incoming migration
to be specified later.  The monitor can be used to set migration
capabilities that may affect the incoming connection process.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Final batch of s390x enhancements/fixes for 2.3:
- handle TOD clock during migration
- CPACF key wrap options
- limit amount of pci device code we build
- ensure big endian accesses for ccws
- various fixes and cleanups

# gpg: Signature made Mon Mar 16 10:01:44 2015 GMT using RSA key ID C6F02FAF
# gpg: Good signature from "Cornelia Huck <huckc@linux.vnet.ibm.com>"
# gpg:                 aka "Cornelia Huck <cornelia.huck@de.ibm.com>"

* remotes/cohuck/tags/s390x-20150316:
  s390x/config: Do not include full pci.mak
  s390x/pci: fix length in sei_nt2 event
  s390x/ipl: remove dead code
  s390x/virtio-bus: Remove unused function s390_virtio_bus_console()
  s390x: CPACF: Handle key wrap machine options
  s390x/kvm: make use of generic vm attribute check
  kvm: encapsulate HAS_DEVICE for vm attrs
  virtio-ccw: assure BE accesses
  s390x/kvm: Guest Migration TOD clock synchronization
  s390x: Replace unchecked qdev_init() by qdev_init_nofail()
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

seabios: update to 1.8.1 stable release

# gpg: Signature made Mon Mar 16 08:09:25 2015 GMT using RSA key ID D3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"

* remotes/kraxel/tags/pull-seabios-1.8.1-20150316-1:
  seabios: update to 1.8.1 stable release
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Pool TCG data, and ALWAYS/NEVER fix

# gpg: Signature made Fri Mar 13 20:09:09 2015 GMT using RSA key ID 4DD0279B
# gpg: Good signature from "Richard Henderson <rth7680@gmail.com>"
# gpg:                 aka "Richard Henderson <rth@redhat.com>"
# gpg:                 aka "Richard Henderson <rth@twiddle.net>"

* remotes/rth/tags/tcg-pull-20150313:
  tcg: Complete handling of ALWAYS and NEVER
  tcg: Use tcg_malloc to allocate TCGLabel
  tcg: Change generator-side labels to a pointer
  tcg: Change translator-side labels to a pointer
  tcg-ia64: Use tcg_malloc to allocate TCGLabelQemuLdst
  tcg: Use tcg_malloc to allocate TCGLabelQemuLdst
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

pci.mak includes a lot of devices - and most of them do not make
sense on s390x, like USB controllers or audio cards. These devices
also show up when running "qemu-system-s390x -device help" and thus
could raise the hope for the users that they could use these kind
of devices with qemu-system-s390x. To avoid this confusion, we
should not include pci.mak and rather include the bare minimum
manually instead.
Signed-off-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Acked-by: NFrank Blaschka <blaschka@linux.vnet.ibm.com>
Message-Id: <1426169954-6062-1-git-send-email-thuth@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

The sei_nt2 event must contain the length of the event.
Signed-off-by: NFrank Blaschka <blaschka@linux.vnet.ibm.com>
Signed-off-by: NJens Freimann <jfrei@linux.vnet.ibm.com>
Message-Id: <1426164834-38648-7-git-send-email-jfrei@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

load_image_targphys already checks the max size and will return
an error code. So the follow-on check will never trigger.
Signed-off-by: NDominik Dingel <dingel@linux.vnet.ibm.com>
Reviewed-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Reviewed-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: NJens Freimann <jfrei@linux.vnet.ibm.com>
Message-Id: <1426164834-38648-6-git-send-email-jfrei@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

The function s390_virtio_bus_console() is completely unused and thus
can be removed safely.
Signed-off-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Reviewed-by: NJens Freimann <jfrei@linux.vnet.ibm.com>
Signed-off-by: NJens Freimann <jfrei@linux.vnet.ibm.com>
Message-Id: <1426164834-38648-5-git-send-email-jfrei@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Check for the aes_key_wrap and dea_key_wrap machine options and set the
appropriate KVM device attribute(s) to tell the kernel to enable or disable
the AES/DEA protected key functions for the guest domain.

This patch introduces two new machine options for indicating the state of
AES/DEA key wrapping functions.  This controls whether the guest will
have access to the AES/DEA crypto functions.

aes_key_wrap="on | off" is changed to aes-key-wrap="on | off"
dea_key_wrap="on | off" is changed to dea-key-wrap="on | off"

Check for the aes-key-wrap and dea-key-wrap machine options and set the
appropriate KVM device attribute(s) to tell the kernel to enable or disable
the AES/DEA protected key functions for the guest domain.
Reviewed-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NTony Krowiak <akrowiak@linux.vnet.ibm.com>
Signed-off-by: NJens Freimann <jfrei@linux.vnet.ibm.com>
Message-Id: <1426164834-38648-4-git-send-email-jfrei@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

By using the new introduced generic interface we
can remove redundancies and clean up.
Reviewed-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Suggested-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: NDominik Dingel <dingel@linux.vnet.ibm.com>
Signed-off-by: NJens Freimann <jfrei@linux.vnet.ibm.com>
Message-Id: <1426164834-38648-3-git-send-email-jfrei@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

More and more virtual machine specifics between kvm and qemu will be
transferred with vm attributes.
So we encapsulate the common logic in a generic function.

Additionally we need only to check during initialization if kvm supports
virtual machine attributes.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Suggested-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Reviewed-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: NDominik Dingel <dingel@linux.vnet.ibm.com>
Signed-off-by: NJens Freimann <jfrei@linux.vnet.ibm.com>
Message-Id: <1426164834-38648-2-git-send-email-jfrei@linux.vnet.ibm.com>
Acked-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

All fields in structures transmitted by ccws are big endian; assure
we handle them as such.
Reviewed-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Reviewed-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Message-Id: <1426067871-17693-2-git-send-email-cornelia.huck@de.ibm.com>

Synchronizes the guest TOD clock across a migration by sending the guest TOD
clock value to the destination system. If the guest TOD clock is not preserved
across a migration then the guest's view of time will snap backwards if the
destination host clock is behind the source host clock. This will cause the
guest to hang immediately upon resuming on the destination system.
Reviewed-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NJason J. Herne <jjherne@linux.vnet.ibm.com>
Signed-off-by: NJens Freimann <jfrei@linux.vnet.ibm.com>

Message-Id: <1425912968-54387-1-git-send-email-jfrei@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

s390_flic_init() is a helper to create and realize either
"s390-flic-kvm" or "s390-flic-qemu".  When qdev_init() fails, it
complains to stderr and succeeds.

Except it can't actually fail, because the "s390-flic-qemu" is a dummy
without a realize method, and "s390-flic-kvm"'s realize can't fail,
even when the kernel device is really unavailable.  Odd.

Replace qdev_init() by qdev_init_nofail() to make "can't fail" locally
obvious, and get rid of the unreachable error reporting.

Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Cornelia Huck <cornelia.huck@de.ibm.com>
Cc: Alexander Graf <agraf@suse.de>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Acked-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Message-Id: <1423128889-18260-4-git-send-email-armbru@redhat.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Carries two bugfixes and support for multiple pci root buses.

git shortlog rel-1.8.0..rel-1.8.1
=================================

Ameya Palande (1):
      x86: add barrier to read{b,w,l} and write{b,w,l} functions

Kevin O'Connor (1):
      smp: Fix smp race introduced in 0673b787

Marcel Apfelbaum (2):
      fw/pci: scan all buses if extraroots romfile is present
      fw/pci: map memory and IO regions for multiple pci root buses
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Missing from movcond, and brcondi_i32 (but not brcondi_i64).
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Pre-allocating 512 of them per TB is a waste.
Reviewed-by: NBastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

This is less about improved type checking than enabling a
subsequent change to the representation of labels.
Acked-by: NClaudio Fontana <claudio.fontana@huawei.com>
Tested-by: NClaudio Fontana <claudio.fontana@huawei.com>
Cc: Andrzej Zaborowski <balrogg@gmail.com>
Cc: Peter Maydell <peter.maydell@linaro.org>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Cc: Blue Swirl <blauwirbel@gmail.com>
Cc: Stefan Weil <sw@weilnetz.de>
Reviewed-by: NBastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

This is improved type checking for the translators -- it's no longer
possible to accidentally swap arguments to the branch functions.

Note that the code generating backends still manipulate labels as int.

With notable exceptions, the scope of the change is just a few lines
for each target, so it's not worth building extra machinery to do this
change in per-target increments.

Cc: Peter Maydell <peter.maydell@linaro.org>
Cc: Edgar E. Iglesias <edgar.iglesias@gmail.com>
Cc: Michael Walle <michael@walle.cc>
Cc: Leon Alrae <leon.alrae@imgtec.com>
Cc: Anthony Green <green@moxielogic.com>
Cc: Jia Liu <proljc@gmail.com>
Cc: Alexander Graf <agraf@suse.de>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Cc: Blue Swirl <blauwirbel@gmail.com>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Reviewed-by: NBastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Reviewed-by: NBastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Pre-allocating 640 of them per TB is a waste.
Reviewed-by: NBastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

A couple of #ifdef changes necessary to use NetBSD's ucontext
structs on sparc64 and arm.
Signed-off-by: NTobias Nygren <tnn@NetBSD.org>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 1425591461-17550-1-git-send-email-tnn@NetBSD.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

misc ui patches, mostly sdl related.

# gpg: Signature made Thu Mar 12 14:51:07 2015 GMT using RSA key ID D3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"

* remotes/kraxel/tags/pull-sdl-20150312-2:
  pixman: add a bunch of PIXMAN_BE_* defines for 32bpp
  Allow the use of X11 from a non standard location.
  configure: opengl overhaul
  sdl: Fix crash when calling sdl_switch() with NULL surface
  sdl: Refresh debug statements
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Update OpenBIOS images

# gpg: Signature made Fri Mar 13 11:04:07 2015 GMT using RSA key ID AE0F321F
# gpg: Good signature from "Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>"

* remotes/mcayland/tags/qemu-openbios-signed:
  Update OpenBIOS images
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Update OpenBIOS images to SVN r1334 built from submodule.
Signed-off-by: NMark Cave-Ayland <mark.cave-ayland@ilande.co.uk>

# gpg: Signature made Thu Mar 12 20:06:50 2015 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/net-pull-request:
  tests: rtl8139: test timers and interrupt
  net: synchronize net_host_device_remove with host_net_remove_completion
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

# gpg: Signature made Thu Mar 12 19:09:26 2015 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request:
  qcow2: fix the macro QCOW_MAX_L1_SIZE's use
  queue: fix QSLIST_INSERT_HEAD_ATOMIC race
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Test behaviour of timers and interrupts related to timeouts.
Signed-off-by: NFrediano Ziglio <freddy77@gmail.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 1420742303-3030-1-git-send-email-freddy77@gmail.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Using net_host_check_device is unnecessary.  qemu_del_net_client asserts
for the non-peer case that it can only process NIC type NetClientStates,
and that assertion is valid for the peered case as well, so move it and
use the same check in net_host_device_remove.  host_net_remove_completion
is already checking the type.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NJason Wang <jasowang@redhat.com>
Message-id: 1419353600-30519-2-git-send-email-pbonzini@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

QCOW_MAX_L1_SIZE's unit is byte, and l1_size's unit
is l1 table entry size(8 bytes).
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>
Message-id: 54FFB0F1.5010307@cn.fujitsu.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

There is a not-so-subtle race in QSLIST_INSERT_HEAD_ATOMIC.

Because atomic_cmpxchg returns the old value instead of a success flag,
QSLIST_INSERT_HEAD_ATOMIC was checking for success by comparing against
the second argument to atomic_cmpxchg.  Unfortunately, this only works
if the second argument is a local or thread-local variable.

If it is in memory, it can be subject to common subexpression elimination
(and then everything's fine) or reloaded after the atomic_cmpxchg,
depending on the compiler's whims.  If the latter happens, the race can
happen.  A thread can sneak in, doing something on elm->field.sle_next
after the atomic_cmpxchg and before the comparison.  This causes a wrong
failure, and then two threads are using "elm" at the same time.  In the
case discovered by Christian, the sequence was likely something like this:

    thread 1                   | thread 2
    QSLIST_INSERT_HEAD_ATOMIC  |
      atomic_cmpxchg succeeds  |
      elm added to list        |
                               | steal release_pool
                               | QSLIST_REMOVE_HEAD
                               | elm removed from list
                               | ...
                               | QSLIST_INSERT_HEAD_ATOMIC
                               |   (overwrites sle_next)
      spurious failure         |
      atomic_cmpxchg succeeds  |
      elm added to list again  |
                               |
    steal release_pool         |
    QSLIST_REMOVE_HEAD         |
    elm removed again          |

The last three steps could be done by a third thread as well.
A reproducer that failed in a matter of seconds is as follows:

- the guest has 32 VCPUs on a 28 core host (hyperthreading was enabled),
  memory was 16G just to err on the safe side (the host has 64G, but hey
  at least you need no s390)

- the guest has 24 null-aio virtio-blk devices using dataplane
  (-object iothread,id=ioN -drive if=none,id=blkN,driver=null-aio,size=500G
  -device virtio-blk-pci,iothread=ioN,drive=blkN)

- the guest also has a single network interface.  It's only doing loopback
  tests so slirp vs. tap and the model doesn't matter.

- the guest is running fio with the following script:

     [global]
     rw=randread
     blocksize=16k
     ioengine=libaio
     runtime=10m
     buffered=0
     fallocate=none
     time_based
     iodepth=32

     [virtio1a]
     filename=/dev/block/252\:16

     [virtio1b]
     filename=/dev/block/252\:16

     ...

     [virtio24a]
     filename=/dev/block/252\:384

     [virtio24b]
     filename=/dev/block/252\:384

     [listen1]
     protocol=tcp
     ioengine=net
     port=12345
     listen
     rw=read
     bs=4k
     size=1000g

     [connect1]
     protocol=tcp
     hostname=localhost
     ioengine=net
     port=12345
     protocol=tcp
     rw=write
     startdelay=1
     size=1000g

     ...

     [listen8]
     protocol=tcp
     ioengine=net
     port=12352
     listen
     rw=read
     bs=4k
     size=1000g

     [connect8]
     protocol=tcp
     hostname=localhost
     ioengine=net
     port=12352
     rw=write
     startdelay=1
     size=1000g

Moral of the story: I should refrain from writing more clever stuff.
At least it looks like it is not too clever to be undebuggable.
Reported-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Tested-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 1426002357-6889-1-git-send-email-pbonzini@redhat.com
Fixes: c740ad92Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>