Commit d4e5ec87 already fixed things to work around Python 3's
lame bug of having LC_ALL=C not be 8-bit clean, when parsing the
main QMP qapi files; but failed to do likewise in the tests
directory.  As a result, running 'LC_ALL=C make check' fails on
escape-too-big and unicode-str when using python 3 with a nasty
stack trace instead of the intended graceful error message that
QAPI doesn't yet support 8-bit data (the two tests contain
Unicode é, when parsed in UTF-8; they represent something
different when parsed in a proper single-byte C locale, but that
doesn't matter to the error message printed out, provided that
brain-dead Python hasn't first choked on the input instead of
being 8-bit clean).

Ideally, we'd teach the qapi generator scripts to automatically
slurp things in using UTF-8 regardless of locale, and to honor
content that is not limited to 7 bit data rather than gracefully
erroring out; but until then, since our graceful error depends
on python parsing 8-bit data (even if nothing we generate uses
8-bit data), our quick fix is to use the right locale when
running these tests.
Reported-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180319205040.1113423-1-eblake@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

This reverts commit 3fd2457d.

Enabling OOB caused several iotests failures; due to the imminent
2.12 release, the safest action is to disable OOB for now.  If
other patches fix the issues that iotests exposed, it may be turned
back on in time for the release, otherwise it will be 2.13 material;
either way, the framework changes not reverted now do not hurt if
they remain as part of the 2.12 release.

Additionally, revert the tests in the patch 02130314 ("qmp: introduce
QMPCapability", 2018-03-19), as both parts must be reverted at once
to keep 'make check' passing.
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <20180323140821.28957-2-peterx@redhat.com>
Tested-by: NChristian Borntraeger <borntraeger@de.ibm.com>
[eblake: reorder/squash commits, enhance commit message]
Signed-off-by: NEric Blake <eblake@redhat.com>

This reverts commit 91ad4506.

Enabling OOB caused several iotests failures; due to the imminent
2.12 release, the safest action is to disable OOB, but first we
have to revert tests that rely on OOB.
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <20180323140821.28957-4-peterx@redhat.com>
Tested-by: NChristian Borntraeger <borntraeger@de.ibm.com>
[eblake: reorder commits, enhance commit message]
Signed-off-by: NEric Blake <eblake@redhat.com>

This reverts commit d003f7a8.

Enabling OOB caused several iotests failures; due to the imminent
2.12 release, the safest action is to disable OOB, but first we
have to revert tests that rely on OOB.
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <20180323140821.28957-3-peterx@redhat.com>
Tested-by: NChristian Borntraeger <borntraeger@de.ibm.com>
[eblake: reorder commits, enhance commit message]
Signed-off-by: NEric Blake <eblake@redhat.com>

s390x: Fixes for 2.12

- Fix for the s390 cpumodel
- Forbid multifunction PCI devices

# gpg: Signature made Fri 23 Mar 2018 09:06:31 GMT
# gpg:                using RSA key 117BBC80B5A61C7C
# gpg: Good signature from "Christian Borntraeger (IBM) <borntraeger@de.ibm.com>"
# Primary key fingerprint: F922 9381 A334 08F9 DBAB  FBCA 117B BC80 B5A6 1C7C

* remotes/borntraeger/tags/s390x-20180323:
  s390x/cpumodel: fix feature groups and breakage of MSA8
  s390x/pci: forbid multifunction pci device
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Since commit 46a99c9f ("s390x/cpumodel: model PTFF subfunctions
for Multiple-epoch facility") -cpu help no longer shows the MSA8
feature group. Turns out that we forgot to add the new MEPOCH_PTFF
group enum.

Fixes: 46a99c9f ("s390x/cpumodel: model PTFF subfunctions for Multiple-epoch facility")
Reviewed-by: NDavid Hildenbrand <david@redhat.com>
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>

Currently we don't support pci multifunction. If a pci with
multifucntion is plugged, the guest will spin forever. This patch fixes
this.
Signed-off-by: NYi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: NPierre Morel <pmorel@linux.vnet.ibm.com>
Reviewed-by: NThomas Huth <thuth@redhat.com>
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>

We have a mirror of the qemu-palcode repository on
git.qemu.org; use that instead of the upstream github,
in line with our general policy of keeping and using
a mirror for submodules.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180319131743.3885-1-peter.maydell@linaro.org

Multiboot patches

# gpg: Signature made Wed 21 Mar 2018 14:38:36 GMT
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  tests/multiboot: Add .gitignore
  tests/multiboot: Add tests for the a.out kludge
  tests/multiboot: Test exit code for every qemu run
  multiboot: Check validity of mh_header_addr
  multiboot: Reject kernels exceeding the address space
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Pull request

# gpg: Signature made Wed 21 Mar 2018 14:37:05 GMT
# gpg:                using RSA key DAE8E10975969CE5
# gpg: Good signature from "Marc-André Lureau <marcandre.lureau@redhat.com>"
# gpg:                 aka "Marc-André Lureau <marcandre.lureau@gmail.com>"
# Primary key fingerprint: 87A9 BD93 3F87 C606 D276  F62D DAE8 E109 7596 9CE5

* remotes/elmarco/tags/dump-pull-request:
  dump-guest-memory: more descriptive lookup_type failure
  dump.c: allow fd_write_vmcore to return errno on failure
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Merge tpm 2018/03/21 v1

# gpg: Signature made Wed 21 Mar 2018 12:02:06 GMT
# gpg:                using RSA key 75AD65802A0B4211
# gpg: Good signature from "Stefan Berger <stefanb@linux.vnet.ibm.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B818 B9CA DF90 89C2 D5CE  C66B 75AD 6580 2A0B 4211

* remotes/stefanberger/tags/pull-tpm-2018-03-21-1:
  tpm: CRB: query backend for TPM established flag
  tpm: CRB: reset locAssigned upon relinquishing locality
  tpm: CRB: set registers to 0 by default
  tpm: CRB: Set tpmRegValidSts flag to '1' in device reset
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

# gpg: Signature made Tue 20 Mar 2018 20:43:37 GMT
# gpg:                using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>"
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>"
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>"
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-2.12-pull-request:
  linux-user: init_guest_space: Try to make ARM space+commpage continuous
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJack Schwartz <jack.schwartz@oracle.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJack Schwartz <jack.schwartz@oracle.com>

Testing the exit code only once after a whole group of tests has
completed is not enough, it catches errors only in the very last qemu
invocation. We need to have the check after each qemu run.

The logging and diff with the reference output is still done once per
group to keep things more managable. This is not a problem because the
log file accumulates the output of all runs.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJack Schwartz <jack.schwartz@oracle.com>

I couldn't find a case where this prevents something bad from happening
that isn't already caught by other checks, but let's err on the safe
side and check that mh_header_addr is as expected.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJack Schwartz <jack.schwartz@oracle.com>

The code path where mh_load_end_addr is non-zero in the Multiboot
header checks that mh_load_end_addr >= mh_load_addr and so
mb_load_size is checked.  However, mb_load_size is not checked when
calculated from the file size, when mh_load_end_addr is 0.

If the kernel binary size is larger than can fit in the address space
after load_addr, we ended up with a kernel_size that is smaller than
load_size, which means that we read the file into a too small buffer.

Add a check to reject kernel files with such Multiboot headers.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJack Schwartz <jack.schwartz@oracle.com>

We've seen a few reports of

 (gdb) source /usr/share/qemu-kvm/dump-guest-memory.py
 Traceback (most recent call last):
   File "/usr/share/qemu-kvm/dump-guest-memory.py", line 19, in <module>
     UINTPTR_T = gdb.lookup_type("uintptr_t")
 gdb.error: No type named uintptr_t.

This occurs when symbols haven't been loaded first, i.e. neither a
QEMU binary was loaded nor a QEMU process was attached first. Let's
better inform the user of how to fix the issue themselves in order
to avoid more reports.
Acked-by: NJanosch Frank <frankja@linux.vnet.ibm.com>
Signed-off-by: NAndrew Jones <drjones@redhat.com>
Message-Id: <20180314153820.18426-1-drjones@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Tested-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NLaszlo Ersek <lersek@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: NMarc-André Lureau <marcandre.lureau@redhat.com>

fd_write_vmcore can fail to execute for a lot of reasons that can be
retrieved by errno, but it only returns -1. This makes difficult for
the caller to know what happened and only a generic error message is
propagated back to the user. This is an example using dump-guest-memory:

(qemu) dump-guest-memory /home/yasmin/mnt/test.dump
dump: failed to save memory

All callers of fd_write_vmcore of dump.c does error handling via
error_setg(), so at first it seems feasible to add the Error pointer as
an argument of fd_write_vmcore. This proved to be more complex than it
first looked. fd_write_vmcore is used by write_elf64_notes and
write_elf32_notes as a WriteCoreDumpFunction prototype. WriteCoreDumpFunction
is declared in include/qom/cpu.h and is used all around the code. This
leaves us with few alternatives:

- change the WriteCoreDumpFunction prototype to include an error pointer.
This would require to change all functions that implements this prototype
to also receive an Error pointer;

- change both write_elf64_notes and write_elf32_notes to no use the
WriteCoreDumpFunction. These functions use not only fd_write_vmcore
but also buf_write_note, so this would require to change buf_write_note
to handle an Error pointer. Considerable easier than the alternative
above, but it's still a lot of code just for the benefit of the callers
of fd_write_vmcore.

This patch presents an easier solution that benefits all fd_write_vmcore
callers:

- instead of returning -1 on error, return -errno. All existing callers
already checks for ret < 0 so there is no need to change the caller's
logic too much. This also allows the retrieval of the errno.

- all callers were updated to use error_setg_errno instead of just
errno_setg. Now that fd_write_vmcore can return an errno, let's update
all callers so they can benefit from a more detailed error message.

This is the same dump-guest-memory example with this patch applied:

(qemu) dump-guest-memory /home/yasmin/mnt/test.dump
dump: failed to save memory: No space left on device
(qemu)

This example illustrates an error of fd_write_vmcore when called
from write_data. All other callers will benefit from better
error messages as well.

Reported-by: yilzhang@redhat.com
Cc: Jose Ricardo Ziviani <joserz@linux.vnet.ibm.com>
Signed-off-by: NYasmin Beatriz <yasmins@linux.vnet.ibm.com>
Signed-off-by: NDaniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
Message-Id: <20180212142506.28445-2-danielhb@linux.vnet.ibm.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: NMarc-André Lureau <marcandre.lureau@redhat.com>

Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>

Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>

Initialize all registers of the CRB device to 0. This clears a few
flags upon a reset.
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>

Fix the initialization of the tpmRegValidSts flag and set it to '1'
during device reset without expecting a write to another register.
This seems to also be the default behavior of real hardware.
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>

Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

HMP fixes for 2.12

# gpg: Signature made Tue 20 Mar 2018 12:39:24 GMT
# gpg:                using RSA key 0516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>"
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-hmp-20180320:
  hmp: free sev info
  HMP: Initialize err before using
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

At a fixed distance after the usable memory that init_guest_space maps, for
32-bit ARM targets we also need to map a commpage.  The normal
init_guest_space logic doesn't keep this in mind when searching for an
address range.

If !host_start, then try to find a big continuous segment where we can put
both the usable memory and the commpage; we then munmap that segment and
set current_start to that address; and let the normal code mmap the usable
memory and the commpage separately.  That is: if we don't have hint of
where to start looking for memory, come up with one that is better than
NULL.  Depending on host_size and guest_start, there may or may not be a
gap between the usable memory and the commpage, so this is slightly more
restrictive than it needs to be; but it's only a hint, so that's OK.

We only do that for !host start, because if host_start, then either:
 - we got an address passed in with -B, in which case we don't want to
   interfere with what the user said;
 - or host_start is based off of the ELF image's loaddr.  The check "if
   (host_start && real_start != current_start)" suggests that we really
   want lowest available address that is >= loaddr.  I don't know why that
   is, but I'm trusting that Paul Brook knew what he was doing when he
   wrote the original version of that check in
   c581deda way back in 2010.
Signed-off-by: NLuke Shumaker <lukeshu@parabola.nu>
Message-Id: <20171228180814.9749-11-lukeshu@lukeshu.com>
Signed-off-by: NLaurent Vivier <laurent@vivier.eu>

virtio,vhost,pci,pc: features, cleanups

SRAT tables for DIMM devices
new virtio net flags for speed/duplex
post-copy migration support in vhost
cleanups in pci
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Tue 20 Mar 2018 14:40:43 GMT
# gpg:                using RSA key 281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>"
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>"
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream: (51 commits)
  postcopy shared docs
  libvhost-user: Claim support for postcopy
  postcopy: Allow shared memory
  vhost: Huge page align and merge
  vhost+postcopy: Wire up POSTCOPY_END notify
  vhost-user: Add VHOST_USER_POSTCOPY_END message
  libvhost-user: mprotect & madvises for postcopy
  vhost+postcopy: Call wakeups
  vhost+postcopy: Add vhost waker
  postcopy: postcopy_notify_shared_wake
  postcopy: helper for waking shared
  vhost+postcopy: Resolve client address
  postcopy-ram: add a stub for postcopy_request_shared_page
  vhost+postcopy: Helper to send requests to source for shared pages
  vhost+postcopy: Stash RAMBlock and offset
  vhost+postcopy: Send address back to qemu
  libvhost-user+postcopy: Register new regions with the ufd
  migration/ram: ramblock_recv_bitmap_test_byte_offset
  postcopy+vhost-user: Split set_mem_table for postcopy
  vhost+postcopy: Transmit 'listen' to slave
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

# Conflicts:
#	scripts/update-linux-headers.sh

Add some notes to the migration documentation for shared memory
postcopy.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Tell QEMU we understand the protocol features needed for postcopy.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Now that we have the mechanisms in here, allow shared memory in a
postcopy.

Note that QEMU can't tell who all the users of shared regions are
and thus can't tell whether all the users of the shared regions
have appropriate support for postcopy.  Those devices that explicitly
support shared memory (e.g. vhost-user) must check, but it doesn't
stop weirder configurations causing problems.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Align RAMBlocks to page size alignment, and adjust the merging code
to deal with partial overlap due to that alignment.

This is needed for postcopy so that we can place/fetch whole hugepages
when under userfault.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Wire up a call to VHOST_USER_POSTCOPY_END message to the vhost clients
right before we ask the listener thread to shutdown.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

This message is sent just before the end of postcopy to get the
client to stop using userfault since we wont respond to any more
requests.  It should close userfaultfd so that any other pages
get mapped to the backing file automatically by the kernel, since
at this point we know we've received everything.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NPeter Xu <peterx@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Clear the area and turn off THP.
PROT_NONE the area until after we've userfault advised it
to catch any unexpected changes.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Cause the vhost-user client to be woken up whenever:
  a) We place a page in postcopy mode
  b) We get a fault and the page has already been received
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Register a waker function in vhost-user code to be notified when
pages arrive or requests to previously mapped pages get requested.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Add a hook to allow a client userfaultfd to be 'woken'
when a page arrives, and a walker that calls that
hook for relevant clients given a RAMBlock and offset.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NPeter Xu <peterx@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Provide a helper to send a 'wake' request on a userfaultfd for
a shared process.
The address in the clients address space is specified together
with the RAMBlock it was resolved to.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Resolve fault addresses read off the clients UFD into RAMBlock
and offset, and call back to the postcopy code to ask for the page.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NPeter Xu <peterx@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

This fixes the build on systems without userfaultfd.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>