migration/next for 20170628

# gpg: Signature made Wed 28 Jun 2017 12:16:44 BST
# gpg:                using RSA key 0xF487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>"
# gpg:                 aka "Juan Quintela <quintela@trasno.org>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/migration/20170628:
  exec: fix access to ram_list.dirty_memory when sync dirty bitmap
  migration: add "return-path" capability
  vmstate: error hint for failed equal checks
  migration: add comment for TYPE_MIGRATE
  migration: hmp: dump globals
  migration: merge enforce_config_section somewhat
  migration: move skip_section_footers
  migration: move skip_configuration out
  migration: move only_migratable to MigrationState
  migration: move global_state.optional out
  migration: let MigrationState be a qdev
  vl: clean up global property registration
  accel: introduce AccelClass.global_props
  machine: export register_compat_prop()
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Xen 2017/06/27

# gpg: Signature made Tue 27 Jun 2017 23:02:43 BST
# gpg:                using RSA key 0x894F8F4870E1AE90
# gpg: Good signature from "Stefano Stabellini <stefano.stabellini@eu.citrix.com>"
# gpg:                 aka "Stefano Stabellini <sstabellini@kernel.org>"
# Primary key fingerprint: D04E 33AB A51F 67BA 07D3  0AEA 894F 8F48 70E1 AE90

* remotes/sstabellini/tags/xen-20170627-tag:
  xen-disk: add support for multi-page shared rings
  xen-disk: only advertize feature-persistent if grant copy is not available
  xen/disk: don't leak stack data via response ring
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

The 32-bit PPC auxv is a bit complicated because in the
mists of time it used to be 16-aligned rather than directly
after the environment. Older glibc versions had code to
try to probe for whether it needed alignment or not:
https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/powerpc/dl-sysdep.c;hb=e84eabb3871c9b39e59323bf3f6b98c2ca9d1cd0
and the kernel has code which puts some magic entries at
the bottom to ensure that the alignment probe fails:
http://elixir.free-electrons.com/linux/latest/source/arch/powerpc/include/asm/elf.h#L158

QEMU has similar code too, but it was broken by commit
7c4ee5bc, which changed elfload.c from filling in
the auxv starting at the highest address and working down
to starting at the lowest address and working up. This
means that the ARCH_DLINFO hook must now be invoked first
rather than last, and the entries in it for PPC must
be reversed so that the magic AT_IGNOREPPC entries come
at the lowest address in the auxv as they should.

The effect of this was that if running a guest binary that
used an old glibc with the alignment probing the guest ld.so
code would segfault if the size of the guest environment and
argv happened to put the auxv at an address that triggered
the alignment code in the guest glibc.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NRichard Henderson <rth@twiddle.net>
Tested-by: NRichard Henderson <rth@twiddle.net>
Message-id: 1498582198-6649-1-git-send-email-peter.maydell@linaro.org

In cpu_physical_memory_sync_dirty_bitmap(rb, start, ...), the 2nd
argument 'start' is relative to the start of the ramblock 'rb'. When
it's used to access the dirty memory bitmap of ram_list (i.e.
ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]->blocks[]), an offset to
the start of all RAM (i.e. rb->offset) should be added to it, which has
however been missed since c/s 6b6712ef. For a ramblock of host memory
backend whose offset is not zero, cpu_physical_memory_sync_dirty_bitmap()
synchronizes the incorrect part of the dirty memory bitmap of ram_list
to the per ramblock dirty bitmap. As a result, a guest with host
memory backend may crash after migration.

Fix it by adding the offset of ramblock when accessing the dirty memory
bitmap of ram_list in cpu_physical_memory_sync_dirty_bitmap().
Reported-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NHaozhong Zhang <haozhong.zhang@intel.com>
Message-Id: <20170628083704.24997-1-haozhong.zhang@intel.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Tested-by: NJuan Quintela <quintela@redhat.com>
Tested-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

When this capability is enabled, QEMU will use the return path even for
precopy migration. This is helpful at least in one case when destination
failed to load the image while source quited without confirmation. With
return path, source will wait for the last response from destination,
and if destination fails, it'll fail the migration on source, then the
guest can be run again on the source (rather than assuming to be good,
then the guest will be lost after source quits).

It needs to be enabled explicitly on source, otherwise disabled.
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498472935-14461-1-git-send-email-peterx@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

In some cases a failing VMSTATE_*_EQUAL does not mean we detected a bug,
but it's actually the best we can do. Especially in these cases a verbose
error message is required.

Let's introduce infrastructure for specifying a error hint to be used if
equal check fails. Let's do this by adding a parameter to the _EQUAL
macros called _err_hint. Also change all current users to pass NULL as
last parameter so nothing changes for them.
Signed-off-by: NHalil Pasic <pasic@linux.vnet.ibm.com>

Message-Id: <20170623144823.42936-1-pasic@linux.vnet.ibm.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

It'll be strange that the migration object inherits TYPE_DEVICE. Add
some explanations to it.
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498634144-26508-1-git-send-email-peterx@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Now we have some globals that can be configured for migration. Dump them
in HMP info migration for better debugging.

(we can also use this to monitor whether COMPAT fields are applied
correctly on compatible machines)
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-11-git-send-email-peterx@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

These two parameters:

- MachineState::enforce_config_section
- MigrationState::send_configuration

are playing similar role here. This patch merges the first one into
second, then we'll have a single place to reference whether we need to
send the configuration section.

I didn't remove the MachineState.enforce_config_section field since when
applying that machine property (in machine_set_property()) we haven't
yet initialized global properties and migration object. Then, it's
still not easy to pass that boolean to MigrationState at such an early
time.

A natural benefit for current patch is that now we kept the meaning of
"enforce-config-section" since it'll still have the highest
priority (that's what "enforce" mean I guess).
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-10-git-send-email-peterx@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Move it into MigrationState, revert its meaning and renaming it to
send_section_footer, with a property bound to it. Same trick is played
like previous patches.

Removing savevm_skip_section_footers().
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-9-git-send-email-peterx@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

It was in SaveState but now moved to MigrationState altogether, reverted
its meaning, then renamed to "send_configuration". Again, using
HW_COMPAT_2_3 for old PC/SPAPR machines, and accel_register_prop() for
xen_init().

Removing savevm_skip_configuration().
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-8-git-send-email-peterx@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

One less global variable, and it does only matter with migration.

We keep the old "--only-migratable" option, but also now we support:

  -global migration.only-migratable=true

Currently still keep the old interface.

Hmm, now vl.c has no way to access migrate_get_current(). Export a
function for it to setup only_migratable.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-7-git-send-email-peterx@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Put it into MigrationState then we can use the properties to specify
whether to enable storing global state.

Removing global_state_set_optional() since now we can use HW_COMPAT_2_3
for x86/power, and AccelClass.global_props for Xen.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-6-git-send-email-peterx@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Let the old man "MigrationState" join the object family. Direct benefit
is that we can start to use all the property features derived from
current QDev, like: HW_COMPAT_* bits, command line setup for migration
parameters (so will never need to set them up each time using HMP/QMP,
this is really, really attractive for test writters), etc.

I see no reason to disallow this happen yet. So let's start from this
one, to see whether it would be anything good.

Now we init the MigrationState struct statically in main() to make sure
it's initialized after global properties are applied, since we'll use
them during creation of the object.

No functional change at all.
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-5-git-send-email-peterx@redhat.com>
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

It's not that clear on how the global properties are registered to
global_props (and also its priority relationship). Let's provide a
single function to be called in main() for that, with comment to explain
it a bit.
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-4-git-send-email-peterx@redhat.com>
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Introduce this new field for the accelerator classes so that each
specific accelerator in the future can register its own global
properties to be used further by the system. It works just like how the
old machine compatible properties do, but only tailored for
accelerators.

Introduce register_compat_props_array() for it. Export it so that it may
be used in other codes as well in the future.
Suggested-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-3-git-send-email-peterx@redhat.com>
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

We have HW_COMPAT_*, however that's only bound to machines, not other
things (like accelerators).  Behind it, it was register_compat_prop()
that played the trick.  Let's export the function for further use
outside HW_COMPAT_* magic.

Meanwhile, move it to qdev-properties.c where seems more proper (since
it'll be used not only in machine codes).
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <1498536619-14548-2-git-send-email-peterx@redhat.com>
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The blkif protocol has had provision for negotiation of multi-page shared
rings for some time now and many guest OS have support in their frontend
drivers.

This patch makes the necessary modifications to xen-disk support a shared
ring up to order 4 (i.e. 16 pages).
Signed-off-by: NPaul Durrant <paul.durrant@citrix.com>
Signed-off-by: NStefano Stabellini <sstabellini@kernel.org>
Reviewed-by: NStefano Stabellini <sstabellini@kernel.org>

If grant copy is available then it will always be used in preference to
persistent maps. In this case feature-persistent should not be advertized
to the frontend, otherwise it may needlessly copy data into persistently
granted buffers.
Signed-off-by: NPaul Durrant <paul.durrant@citrix.com>
Signed-off-by: NStefano Stabellini <sstabellini@kernel.org>
Reviewed-by: NStefano Stabellini <sstabellini@kernel.org>

Rather than constructing a local structure instance on the stack, fill
the fields directly on the shared ring, just like other (Linux)
backends do. Build on the fact that all response structure flavors are
actually identical (aside from alignment and padding at the end).

This is XSA-216.

Reported by: Anthony Perard <anthony.perard@citrix.com>
Signed-off-by: NJan Beulich <jbeulich@suse.com>
Signed-off-by: NStefano Stabellini <sstabellini@kernel.org>
Acked-by: NAnthony PERARD <anthony.perard@citrix.com>

edgar/mmio-exec-v2.for-upstream

# gpg: Signature made Tue 27 Jun 2017 16:22:30 BST
# gpg:                using RSA key 0x29C596780F6BCA83
# gpg: Good signature from "Edgar E. Iglesias (Xilinx key) <edgar.iglesias@xilinx.com>"
# gpg:                 aka "Edgar E. Iglesias <edgar.iglesias@gmail.com>"
# Primary key fingerprint: AC44 FEDC 14F7 F1EB EDBF  4151 29C5 9678 0F6B CA83

* remotes/edgar/tags/edgar/mmio-exec-v2.for-upstream:
  xilinx_spips: allow mmio execution
  exec: allow to get a pointer for some mmio memory region
  introduce mmio_interface
  qdev: add MemoryRegion property
  cputlb: fix the way get_page_addr_code fills the tlb
  cputlb: move get_page_addr_code
  cputlb: cleanup get_page_addr_code to use VICTIM_TLB_HIT
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

This allows to execute from the lqspi area.

When the request_ptr is called the device loads 1024bytes from the SPI device.
Then this code can be executed by the guest.
Tested-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Reviewed-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: NKONRAD Frederic <fred.konrad@greensocs.com>
Signed-off-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>

This introduces a special callback which allows to run code from some MMIO
devices.

SysBusDevice with a MemoryRegion which implements the request_ptr callback will
be notified when the guest try to execute code from their offset. Then it will
be able to eg: pre-load some code from an SPI device or ask a pointer from an
external simulator, etc..

When the pointer or the data in it are no longer valid the device has to
invalidate it.
Reviewed-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: NKONRAD Frederic <fred.konrad@greensocs.com>
Signed-off-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>

This introduces mmio_interface object which contains a MemoryRegion
and can be hotplugged/hotunplugged.
Reviewed-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: NKONRAD Frederic <fred.konrad@greensocs.com>
Signed-off-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>

We need to pass a pointer to a MemoryRegion for mmio_interface.
So this just adds that.
Reviewed-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: NKONRAD Frederic <fred.konrad@greensocs.com>
Signed-off-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>

get_page_addr_code(..) does a cpu_ldub_code to fill the tlb:
This can lead to some side effects if a device is mapped at this address.

So this patch replaces the cpu_memory_ld by a tlb_fill.
Reviewed-by: NRichard Henderson <rth@twiddle.net>
Reviewed-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: NKONRAD Frederic <fred.konrad@greensocs.com>
Signed-off-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>

This just moves the code before VICTIM_TLB_HIT macro definition
so we can use it.
Reviewed-by: NRichard Henderson <rth@twiddle.net>
Reviewed-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: NKONRAD Frederic <fred.konrad@greensocs.com>
Signed-off-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>

This replaces env1 and page_index variables by env and index
so we can use VICTIM_TLB_HIT macro later.
Reviewed-by: NRichard Henderson <rth@twiddle.net>
Reviewed-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: NKONRAD Frederic <fred.konrad@greensocs.com>
Signed-off-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>

Block layer patches

# gpg: Signature made Mon 26 Jun 2017 14:07:32 BST
# gpg:                using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (60 commits)
  qemu-img: don't shadow opts variable in img_dd()
  block: Do not strcmp() with NULL uri->scheme
  blkverify: Catch bs->exact_filename overflow
  blkdebug: Catch bs->exact_filename overflow
  fix: avoid an infinite loop or a dangling pointer problem in img_commit
  block: change variable names in BlockDriverState
  block: Remove bdrv_aio_readv/writev/flush()
  qed: Use bdrv_co_* for coroutine_fns
  qed: Add coroutine_fn to I/O path functions
  qed: Use a coroutine for need_check_timer
  qed: Simplify request handling
  qed: Use CoQueue for serialising allocations
  qed: Implement .bdrv_co_readv/writev
  qed: Remove recursion in qed_aio_next_io()
  qed: Remove ret argument from qed_aio_next_io()
  qed: Add return value to qed_aio_read/write_data()
  qed: Add return value to qed_aio_write_inplace/alloc()
  qed: Add return value to qed_aio_write_cow()
  qed: Add return value to qed_aio_write_main()
  qed: Add return value to qed_aio_write_l2_update()
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Block patches for the block queue

# gpg: Signature made Mon Jun 26 14:56:24 2017 CEST
# gpg:                using RSA key 0xF407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* mreitz/tags/pull-block-2017-06-26:
  qemu-img: don't shadow opts variable in img_dd()
  block: Do not strcmp() with NULL uri->scheme
  blkverify: Catch bs->exact_filename overflow
  blkdebug: Catch bs->exact_filename overflow
  fix: avoid an infinite loop or a dangling pointer problem in img_commit
  block: change variable names in BlockDriverState
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

It's confusing when two different variables have the same name in one
function.

Cc: Reda Sallahi <fullmanet@gmail.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Message-id: 20170619150002.3033-1-stefanha@redhat.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

uri_parse(...)->scheme may be NULL. In fact, probably every field may be
NULL, and the callers do test this for all of the other fields but not
for scheme (except for block/gluster.c; block/vxhs.c does not access
that field at all).

We can easily fix this by using g_strcmp0() instead of strcmp().

Cc: qemu-stable@nongnu.org
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20170613205726.13544-1-mreitz@redhat.com
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

The bs->exact_filename field may not be sufficient to store the full
blkverify node filename. In this case, we should not generate a filename
at all instead of an unusable one.

Cc: qemu-stable@nongnu.org
Reported-by: NQu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20170613172006.19685-3-mreitz@redhat.com
Reviewed-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

The bs->exact_filename field may not be sufficient to store the full
blkdebug node filename. In this case, we should not generate a filename
at all instead of an unusable one.

Cc: qemu-stable@nongnu.org
Reported-by: NQu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20170613172006.19685-2-mreitz@redhat.com
Reviewed-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

img_commit could fall into an infinite loop calling run_block_job() if
its blockjob fails on any I/O error, fix this already known problem.
Signed-off-by: Nsochin.jiang <sochin.jiang@huawei.com>
Message-id: 1497509253-28941-1-git-send-email-sochin.jiang@huawei.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Change the 'int count' parameter in *pwrite_zeros, *pdiscard related
functions (and some others) to 'int bytes', as they both refer to bytes.
This helps with code legibility.
Signed-off-by: NManos Pitsidianakis <el13635@mail.ntua.gr>
Message-id: 20170609101808.13506-1-el13635@mail.ntua.gr
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

These functions are unused now.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

All functions that are marked coroutine_fn can directly call the
bdrv_co_* version of functions instead of going through the wrapper.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NManos Pitsidianakis <el13635@mail.ntua.gr>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

Now that we stay in coroutine context for the whole request when doing
reads or writes, we can add coroutine_fn annotations to many functions
that can do I/O or yield directly.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

This fixes the last place where we degraded from AIO to actual blocking
synchronous I/O requests. Putting it into a coroutine means that instead
of blocking, the coroutine simply yields while doing I/O.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>