Commit cf869d53 "qmp: support out-of-band (oob) execution" added a
general mechanism for command-independent arguments just for an
out-of-band flag:

    The "control" key is introduced to store this extra flag.  "control"
    field is used to store arguments that are shared by all the commands,
    rather than command specific arguments.  Let "run-oob" be the first.

However, it failed to reject unknown members of "control".  For
instance, in QMP command

    {"execute": "query-name", "id": 42, "control": {"crap": true}}

"crap" gets silently ignored.

Instead of fixing this, revert the general "control" mechanism
(because YAGNI), and do it the way I initially proposed, with key
"exec-oob".  Simpler code, simpler interface.

An out-of-band command

    {"execute": "migrate-pause", "id": 42, "control": {"run-oob": true}}

becomes

    {"exec-oob": "migrate-pause", "id": 42}
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-13-armbru@redhat.com>
[Commit message typo fixed]

Commit cf869d53 "qmp: support out-of-band (oob) execution"
accidentally made qemu-ga accept and ignore "control".  Fix that.

Out-of-band execution in a monitor that doesn't support it now fails
with

    {"error": {"class": "GenericError", "desc": "QMP input member 'control' is unexpected"}}

instead of

    {"error": {"class": "GenericError", "desc": "Please enable out-of-band first for the session during capabilities negotiation"}}

The old description is suboptimal when out-of-band cannot not be
enabled, or the command doesn't support out-of-band execution.

The new description is a bit unspecific, but it'll do.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-12-armbru@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-11-armbru@redhat.com>

Commit cf869d53 "qmp: support out-of-band (oob) execution" changed
how we check "id":

    Note that in the patch I exported qmp_dispatch_check_obj() to be
    used to check the request earlier, and at the same time allowed
    "id" field to be there since actually we always allow that.

The part after "and" is ill-advised: it makes qemu-ga accept and
ignore "id".  Revert.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-10-armbru@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-9-armbru@redhat.com>

Commit cf869d53 "qmp: support out-of-band (oob) execution" made
"id" mandatory for all commands when the client accepted capability
"oob".  This is rather onerous when you play with QMP by hand, and
unnecessarily so: only out-of-band commands need an ID for reliable
matching of response to command.

Revert that part of commit cf869d53 for now, but have documentation
advise on the need to use "id" with out-of-band commands.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20180703085358.13941-8-armbru@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-7-armbru@redhat.com>

tests/qmp-test tests an out-of-band command overtaking a slow in-band
command.  To do that, it needs:

1. An in-band command that *reliably* takes long enough to be
   overtaken.

2. An out-of-band command to do the overtaking.

3. To avoid delays, a way to make the in-band command complete quickly
   after it was overtaken.

To satisfy these needs, commit 469638f9 provides the rather
peculiar oob-capable QMP command x-oob-test:

* With "lock": true, it waits for a global semaphore.

* With "lock": false, it signals the global semaphore.

To satisfy 1., the test runs x-oob-test in-band with "lock": true.
To satisfy 2. and 3., it runs x-oob-test out-of-band with "lock": false.

Note that waiting for a semaphore violates the rules for oob-capable
commands.  Running x-oob-test with "lock": true hangs the monitor
until you run x-oob-test with "lock": false on another monitor (which
you might not have set up).

Having an externally visible QMP command that may hang the monitor is
not nice.  Let's apply a little more ingenuity to the problem.  Idea:
have an existing command block on reading a FIFO special file, unblock
it by opening the FIFO for writing.

For 1., use

    {"execute": "blockdev-add",  "id": ID1,
     "arguments": {
        "driver": "blkdebug", "node-name": ID1, "config": FIFO,
        "image": { "driver": "null-co"}}}

where ID1 is an arbitrary string, and FIFO is the name of the FIFO.

For 2., use

    {"execute": "migrate-pause", "id": ID2, "control": {"run-oob": true}}

where ID2 is a different arbitrary string.  Since there's no migration
to pause, the command will fail, but that's fine; instant failure is
still a test of out-of-band responses overtaking in-band commands.

For 3., open FIFO for writing.

Drop QMP command x-oob-test.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-6-armbru@redhat.com>
[Error checking tweaked]

Events are broadcast to all monitors.  If another monitor's client has
a command with the same ID in flight, the event will incorrectly claim
that command was dropped.  This must be fixed before out-of-band
execution can graduate from "experimental".
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-5-armbru@redhat.com>

OOB documentation is spread over qmp-spec.txt sections 2.2.1
Capabilities and 2.3 Issuing Commands.  The amount of detail is a bit
distracting there.  Move the meat of the matter to new section 2.3.1
Out of band execution.

Throw in a few other improvements while there:

* 2.2 Server Greeting: Drop advice to search entire capabilities
  array; should be obvious.

* 3. QMP Examples

  - 3.1 Server Greeting: Update greeting to the one we expect for the
    release.  Now shows capability "oob".  Update qmp-intro.txt
    likewise.

  - 3.2 Capabilities negotiation: Show client accepting capability
    "oob".

  - 3.7 Out-of-band execution: New.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-4-armbru@redhat.com>
[Whitespace tidied up]

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-3-armbru@redhat.com>

Affects documentation and a few error messages.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-2-armbru@redhat.com>

* IEC units series (Philippe)
* Hyper-V PV TLB flush (Vitaly)
* git archive detection (Daniel)
* host serial passthrough fix (David)
* NPT support for SVM emulation (Jan)
* x86 "info mem" and "info tlb" fix (Doug)

# gpg: Signature made Mon 02 Jul 2018 16:18:21 BST
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (50 commits)
  tcg: simplify !CONFIG_TCG handling of tb_invalidate_*
  i386/monitor.c: make addresses canonical for "info mem" and "info tlb"
  target-i386: Add NPT support
  serial: Open non-block
  bsd-user: Use the IEC binary prefix definitions
  linux-user: Use the IEC binary prefix definitions
  tests/crypto: Use the IEC binary prefix definitions
  vl: Use the IEC binary prefix definitions
  monitor: Use the IEC binary prefix definitions
  cutils: Do not include "qemu/units.h" directly
  hw/rdma: Use the IEC binary prefix definitions
  hw/virtio: Use the IEC binary prefix definitions
  hw/vfio: Use the IEC binary prefix definitions
  hw/sd: Use the IEC binary prefix definitions
  hw/usb: Use the IEC binary prefix definitions
  hw/net: Use the IEC binary prefix definitions
  hw/i386: Use the IEC binary prefix definitions
  hw/ppc: Use the IEC binary prefix definitions
  hw/mips: Use the IEC binary prefix definitions
  hw/mips/r4k: Constify params_size
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Assorted tlb and tb caching fixes

# gpg: Signature made Mon 02 Jul 2018 17:03:07 BST
# gpg:                using RSA key 64DF38E8AF7E215F
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>"
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A  05C0 64DF 38E8 AF7E 215F

* remotes/rth/tags/pull-tcg-20180702:
  cpu: Assert asidx_from_attrs return value in range
  accel/tcg: Avoid caching overwritten tlb entries
  accel/tcg: Don't treat invalid TLB entries as needing recheck
  accel/tcg: Correct "is this a TLB miss" check in get_page_addr_code()
  tcg: Define and use new tlb_hit() and tlb_hit_page() functions
  translate-all: fix locking of TBs whose two pages share the same physical page
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/kraxel/tags/seabios-1.11.2-20180702-pull-request' into staging

seabios: update to release 1.11.2, add/update configuration.
vgabios: remove (old unused lgpl'ed vgabios).
ramfb,bochs-display: use new vgabios roms.

# gpg: Signature made Mon 02 Jul 2018 16:42:06 BST
# gpg:                using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/seabios-1.11.2-20180702-pull-request:
  ramfb: enable vgabios
  bochs-display: enable vgabios
  seabios: update bios and vgabios binaries
  vgabios: remove submodule and build rules.
  seabios: enable ide dma
  seabios: add vga configs for bochs-display and ramfb
  seabios: update submodule to release 1.11.2
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Add vgabios binary to fw_cfg vgaroms.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Add vgabios binary to pci rom bar.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Adds two new vgabios binaries, for ramfb and bochs-display.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

It's the old, lgpl vgabios implementation.

Was left in as fallback when we switched to seavgabios, so we could
easily switch back in case we see regressions.  It's unused since years
now, reportedly doesn't even build, and lacks support for recently (and
not so recently) added display devices.

Zap it.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

QNX reportedly requires this to boot.
Should also speed up booting other guests.

Note: Upstream seabios defaults this to 'n' to due to known problems
on physical hardware (qemu not affected), and wouldn't flip the default
to 'y'.  So we adjust our local build config accordingly.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Both bochs-display and ramfb are devices with a simple framebuffer and
no vga emulation or text mode.  seavgabios has support for text mode
emulation (at vgabios call level), we are using that to provide some
vga compatibility support for these devices.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

git shortlog rel-1.11.1..rel-1.11.2
-----------------------------------

Gerd Hoffmann (11):
      optionrom: enable non-vga display devices
      cbvga: factor out cbvga_setup_modes()
      qemu: add bochs-display support
      cbvga_setup_modes: use real mode number instead of 0x140
      cbvga_list_modes: don't list current mode twice
      cbvga_set_mode: disable clearmem in windows x86 emulator.
      bochs_display_setup: return error on failure
      pmm: use tmp zone on oom
      vgasrc: add allocate_pmm()
      qemu: add qemu ramfb support
      cbvga_set_mode: refine clear display logic
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

When installing a TLB entry, remove any cached version of the
same page in the VTLB.  If the existing TLB entry matches, do
not copy into the VTLB, but overwrite it.
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

In get_page_addr_code() when we check whether the TLB entry
is marked as TLB_RECHECK, we should not go down that code
path if the TLB entry is not valid at all (ie the TLB_INVALID
bit is set).
Tested-by: NLaurent Vivier <laurent@vivier.eu>
Reported-by: NLaurent Vivier <laurent@vivier.eu>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Message-Id: <20180629161731.16239-1-peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

In commit 71b9a453 we changed the condition we use
to determine whether we need to refill the TLB in
get_page_addr_code() to
    if (unlikely(env->tlb_table[mmu_idx][index].addr_code !=
                 (addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK)))) {

This isn't the right check (it will falsely fail if the
input addr happens to have the low bit corresponding to
TLB_INVALID_MASK set, for instance). Replace it with a
use of the new tlb_hit() function, which is the correct test.
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Message-Id: <20180629162122.19376-3-peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

The condition to check whether an address has hit against a particular
TLB entry is not completely trivial. We do this in various places, and
in fact in one place (get_page_addr_code()) we have got the condition
wrong. Abstract it out into new tlb_hit() and tlb_hit_page() inline
functions (one for a known-page-aligned address and one for an
arbitrary address), and use them in all the places where we had the
condition correct.

This is a no-behaviour-change patch; we leave fixing the buggy
code in get_page_addr_code() to a subsequent patch.
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Message-Id: <20180629162122.19376-2-peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

Commit 0b5c91f7 ("translate-all: use per-page locking in !user-mode",
2018-06-15) introduced per-page locking. It assumed that the physical
pages corresponding to a TB (at most two pages) are always distinct,
which is wrong. For instance, an xtensa test provided by Max Filippov
is broken by the commit, since the test maps two virtual pages
to the same physical page:

	virt1: 7fff, virt2: 8000
	phys1 6000fff, phys2 6000000

Fix it by removing the assumption from page_lock_pair.
If the two physical page addresses are equal, we only lock
the PageDesc once. Note that the two callers of page_lock_pair,
namely page_unlock_tb and tb_link_page, are also updated so that
we do not try to unlock the same PageDesc twice.

Fixes: 0b5c91f7Reported-by: NMax Filippov <jcmvbkbc@gmail.com>
Tested-by: NMax Filippov <jcmvbkbc@gmail.com>
Tested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Signed-off-by: NEmilio G. Cota <cota@braap.org>
Message-Id: <1529944302-14186-1-git-send-email-cota@braap.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

s390x updates:
- add bpb/ppa15 features to default cpu model for z196 and later
- rework TOD handling and fix cpu hotplug under tcg
- various fixes

# gpg: Signature made Mon 02 Jul 2018 12:09:40 BST
# gpg:                using RSA key DECF6B93C6F02FAF
# gpg: Good signature from "Cornelia Huck <conny@cornelia-huck.de>"
# gpg:                 aka "Cornelia Huck <huckc@linux.vnet.ibm.com>"
# gpg:                 aka "Cornelia Huck <cornelia.huck@de.ibm.com>"
# gpg:                 aka "Cornelia Huck <cohuck@kernel.org>"
# gpg:                 aka "Cornelia Huck <cohuck@redhat.com>"
# Primary key fingerprint: C3D0 D66D C362 4FF6 A8C0  18CE DECF 6B93 C6F0 2FAF

* remotes/cohuck/tags/s390x-20180702:
  s390x/tcg: fix locking problem with tcg_s390_tod_updated
  s390x/kvm: indicate alignment in legacy_s390_alloc()
  s390x/kvm: legacy_s390_alloc() only supports one allocation
  s390x/tcg: fix CPU hotplug with single-threaded TCG
  s390x/tcg: rearm the CKC timer during migration
  s390x/tcg: implement SET CLOCK
  s390x/tcg: SET CLOCK COMPARATOR can clear CKC interrupts
  s390x/tcg: properly implement the TOD
  s390x/tcg: drop tod_basetime
  s390x/tod: factor out TOD into separate device
  s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*()
  s390x/tcg: avoid overflows in time2tod/tod2time
  s390x/cpumodel: default enable bpb and ppa15 for z196 and later
  loader: Check access size when calling rom_ptr() to avoid crashes
  s390/ipl: fix ipl with -no-reboot
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

There is no need for a stub, since tb_invalidate_phys_addr can be excised
altogether when TCG is disabled.  This is a bit cleaner since it avoids
using code that is clearly specific to user-mode emulation (it calls
mmap_lock/unlock) for the !CONFIG_TCG case.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Correct the output of the "info mem" and "info tlb" monitor commands to
correctly show canonical addresses.

In 48-bit addressing mode, the upper 16 bits of linear addresses are
equal to bit 47. In 57-bit addressing mode (LA57), the upper 7 bits of
linear addresses are equal to bit 56.
Signed-off-by: NDoug Gale <doug16k@gmail.com>
Message-Id: <20180617084025.29198-1-doug16k@gmail.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This implements NPT suport for SVM by hooking into
x86_cpu_handle_mmu_fault where it reads the stage-1 page table. Whether
we need to perform this 2nd stage translation, and how, is decided
during vmrun and stored in hflags2, along with nested_cr3 and
nested_pg_mode.

As get_hphys performs a direct cpu_vmexit in case of NPT faults, we need
retaddr in that function. To avoid changing the signature of
cpu_handle_mmu_fault, this passes the value from tlb_fill to get_hphys
via the CPU state.

This was tested successfully via the Jailhouse hypervisor.
Signed-off-by: NJan Kiszka <jan.kiszka@siemens.com>
Message-Id: <567473a0-6005-5843-4c73-951f476085ca@web.de>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

On a real serial device, the open can block if the handshake
lines are in a particular state.  If a QEMU is passing the serial
device to the guest, the QEMU startup is blocked opening the device
(with a symptom seen as a timeout from libvirt).

Open the serial port with O_NONBLOCK.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

It eases code review, unit is explicit.

Patch generated using:

  $ git grep -n '[<>][<>]= ?[1-5]0'

and modified manually.
Suggested-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180625124238.25339-47-f4bug@amsat.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

It eases code review, unit is explicit.

Patch generated using:

  $ git grep -n '[<>][<>]= ?[1-5]0'

and modified manually.
Suggested-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180625124238.25339-46-f4bug@amsat.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

It eases code review, unit is explicit.

Patch generated using:

  $ git grep -n '[<>][<>]= ?[1-5]0'

and modified manually.
Suggested-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180625124238.25339-45-f4bug@amsat.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

It eases code review, unit is explicit.

Patch generated using:

  $ git grep -n '[<>][<>]= ?[1-5]0'

and modified manually.
Suggested-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180625124238.25339-44-f4bug@amsat.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

It eases code review, unit is explicit.

Patch generated using:

  $ git grep -n '[<>][<>]= ?[1-5]0'

and modified manually.
Suggested-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180625124238.25339-43-f4bug@amsat.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

All files using "qemu/units.h" definitions already include it directly,
we can now remove it from "qemu/cutils.h".
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NMarcel Apfelbaum <marcel@redhat.com>
Message-Id: <20180625124238.25339-41-f4bug@amsat.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

It eases code review, unit is explicit.

Patch generated using:

  $ git grep -E '(1024|2048|4096|8192|(<<|>>).?(10|20|30))' hw/ include/hw/

and modified manually.
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NMarcel Apfelbaum <marcel@redhat.com>
Reviewed-by: NYuval Shaia <yuval.shaia@oracle.com>
Message-Id: <20180625124238.25339-40-f4bug@amsat.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>