Introduce some hooks for the shared part of qemu thread between POSIX
and Windows implementations.  Note that in qemu_mutex_unlock_impl() we
moved the call before unlock operation which should make more sense.
And we don't need qemu_mutex_post_unlock() hook.

Put all these shared hooks into the header files.  It should be internal
to qemu-thread but not for qemu-thread users, hence put into util/
directory.
Reviewed-by: NEmilio G. Cota <cota@braap.org>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <20180425025459.5258-3-peterx@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This allows us to use atomic-add-bench as a microbenchmark
for evaluating qemu_mutex_lock's performance.
Signed-off-by: NEmilio G. Cota <cota@braap.org>
[cherry picked from https://github.com/cota/qemu/commit/f04f34df]
Signed-off-by: NPeter Xu <peterx@redhat.com>
Message-Id: <20180425025459.5258-2-peterx@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

According to KVM commit 75d61fbc, it needs to delete the slot before
changing the KVM_MEM_READONLY flag. But QEMU commit 235e8982 only check
whether KVM_MEM_READONLY flag is set instead of changing. It doesn't
need to delete the slot if the KVM_MEM_READONLY flag is not changed.

This fixes a issue that migrating a VM at the OVMF startup stage and
VM is executing the codes in rom. Between the deleting and adding the
slot in kvm_set_user_memory_region, there is a chance that guest access
rom and trap to KVM, then KVM can't find the corresponding memslot.
While KVM (on ARM) injects an abort to guest due to the broken hva, then
guest will get stuck.
Signed-off-by: NShannon Zhao <zhaoshenglong@huawei.com>
Message-Id: <1526462314-19720-1-git-send-email-zhaoshenglong@huawei.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Right now we can crash QEMU using e.g.

qemu-system-x86_64 -m 256M,maxmem=20G,slots=2 \
 -object memory-backend-file,id=mem0,size=12288,mem-path=/dev/zero,align=12288 \
 -device pc-dimm,id=dimm1,memdev=mem0

qemu-system-x86_64: util/mmap-alloc.c:115:
 qemu_ram_mmap: Assertion `is_power_of_2(align)' failed

Fix this by adding a proper check.
Signed-off-by: NDavid Hildenbrand <david@redhat.com>
Message-Id: <20180607154705.6316-3-david@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

The start of the address space indicates which maximum alignment is
supported by our machine (e.g. ppc, x86 1GB). This is helpful to
catch fragmenting guest physical memory in strange fashions.

Right now we can crash QEMU by e.g. (there might be easier examples)

qemu-system-x86_64 -m 256M,maxmem=20G,slots=2 \
 -object memory-backend-file,id=mem0,size=8192M,mem-path=/dev/zero,align=8192M \
 -device pc-dimm,id=dimm1,memdev=mem0
Signed-off-by: NDavid Hildenbrand <david@redhat.com>
Message-Id: <20180607154705.6316-2-david@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Not included by mistake in commit 327fccb2.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

The implementation of these two instructions was swapped.
At the same time, unify the setup of eflags for the insn group.
Reported-by: NRicardo Ribalda Delgado <ricardo.ribalda@gmail.com>
Signed-off-by: NRichard Henderson <rth@twiddle.net>
Message-Id: <20170712192902.15493-1-rth@twiddle.net>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Only retry on serial_xmit if qemu_chr_fe_write returns 0, as this is the
only recoverable error.

Retrying with any other scenario, in addition to being a waste of CPU
cycles, can compromise the Guest stability if by the vCPU issuing the
write and the main loop thread are, by chance or explicit pinning,
running on the same pCPU.

Previous discussion:

https://lists.nongnu.org/archive/html/qemu-devel/2018-05/msg06998.htmlSigned-off-by: NSergio Lopez <slp@redhat.com>
Message-Id: <1528185295-14199-1-git-send-email-slp@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20180602085259.17853-1-stefanha@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>

The stdio chardev finalize method calls term_exit() to restore the
original terminal settings that were saved in the "oldtty" global. If
the qemu_chr_open_stdio() method exited with an error, we might not have
any original terminal settings saved in "oldtty" yet.

eg

  $ qemu-system-x86_64 -monitor stdio -daemonize
  qemu-system-x86_64: -monitor stdio: cannot use stdio with -daemonize

will cause QEMU to splatter the terminal settings with an all-zeros
"struct termios", with predictably unpleasant results. Fortunately the
existing "stdio_in_use" flag is suitable witness for whether "oldtty"
contains settings that need restoring.
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>
Message-Id: <20180604123043.13985-1-berrange@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Place them in exec.c, exec-all.h and ram_addr.h.  This removes
knowledge of translate-all.h (which is an internal header) from
several files outside accel/tcg and removes knowledge of
AddressSpace from translate-all.c (as it only operates on ram_addr_t).
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

When an IOMMUMemoryRegion is in front of a virtio device,
address_space_cache_init does not set cache->ptr as the memory
region is not RAM. However when the device performs an access,
we end up in glue() which performs the translation and then uses
MAP_RAM. This latter uses the unset ptr and returns a wrong value
which leads to a SIGSEV in address_space_lduw_internal_cached_slow,
for instance.

In slow path cache->ptr is NULL and MAP_RAM must redirect to
qemu_map_ram_ptr((mr)->ram_block, ofs).

As MAP_RAM, IS_DIRECT and INVALIDATE are the same in _cached_slow
and non cached mode, let's remove those macros.

This fixes the use cases featuring vIOMMU (Intel and ARM SMMU)
which lead to a SIGSEV.

Fixes: 48564041 (exec: reintroduce MemoryRegion caching)
Signed-off-by: NEric Auger <eric.auger@redhat.com>

Message-Id: <1528895946-28677-1-git-send-email-eric.auger@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

migration/next for 20180627

# gpg: Signature made Wed 27 Jun 2018 13:53:53 BST
# gpg:                using RSA key F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>"
# gpg:                 aka "Juan Quintela <quintela@trasno.org>"
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/migration/20180627:
  migration: fix crash in when incoming client channel setup fails
  postcopy: drop ram_pages parameter from postcopy_ram_incoming_init()
  migration: Stop sending whole pages through main channel
  migration: Remove not needed semaphore and quit
  migration: Wait for blocking IO
  migration: Start sending messages
  migration: Create ram_save_multifd_page
  migration: Create multifd_bytes ram_counter
  migration: Synchronize multifd threads with main thread
  migration: Add block where to send/receive packets
  migration: Multifd channels always wait on the sem
  migration: Add multifd traces for start/end thread
  migration: Abstract the number of bytes sent
  migration: Calculate mbps only during transfer time
  migration: Create multifd packet
  migration: Create multipage support
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

MIPS queue

# gpg: Signature made Wed 27 Jun 2018 19:16:23 BST
# gpg:                using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <amarkovic@wavecomp.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01  DD75 D497 2A89 67F7 5A65

* remotes/amarkovic/tags/mips-queue-2018-06-27:
  target/mips: Fix gdbstub to read/write 64 bit FP registers
  target/mips: Fix data type for offset
  target/mips: Update gen_flt_ldst()
  target/mips: Fix microMIPS on reset
  target/mips: Raise a RI when given fs is n/a from CTC1
  hw/pci-host/xilinx-pcie: don't make "io" region be RAM
  hw/mips/mips_malta: don't make bios region 'nomigrate'
  hw/mips/boston: don't make flash region 'nomigrate'
  MAINTAINERS: update target-mips maintainers
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Pull request

 * Trace TCG atomic memory accesses
 * Document that trace event arguments cannot be floating point

# gpg: Signature made Wed 27 Jun 2018 13:57:40 BST
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/tracing-pull-request:
  trace: forbid floating point types
  trace: enable tracing of TCG atomics
  trace: add trace_mem_build_info_no_se_be/le
  trace: expand mem_info:size_shift to 3 bits
  trace: simplify trace_mem functions
  trace: fix misreporting of TCG access sizes for user-space
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Fix gdbstub to read/write 64 bit FP registers
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Offset can be larger than 16 bit from nanoMIPS,
and immediate field can be larger than 16 bits as well.
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Update gen_flt_ldst() in order to reuse the functions for nanoMIPS
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Fix to activate microMIPS on reset when Config3.ISA == {1, 3}
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Fix to raise a Reserved Instruction exception when given fs is not
available from CTC1.
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Currently we use memory_region_init_rom_nomigrate() to create
the "io" memory region to pass to pci_register_root_bus().
This is a dummy region, because this PCI controller doesn't
support accesses to PCI IO space.

There is no reason for the dummy region to be a RAM region;
it is only used as a place where PCI BARs can be mapped,
and if you could get a PCI card to do a bus master access
to the IO space it should not get acts-like-RAM behaviour.
Use a simple container memory region instead. (We do have
one PCI card model which can do bus master accesses to IO
space -- the LSI53C895A SCSI adaptor.)

This avoids the oddity of having a memory region which is
RAM but where the RAM is not migrated.

Note that the size of the region we use here has no
effect on behaviour.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NAlistair Francis <alistair.francis@wdc.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Currently we use memory_region_init_rom_nomigrate() to create
the "bios.1fc" memory region, and we don't manually register
it with vmstate_register_ram(). This currently means that its
contents are migrated but as a ram block whose name is the empty
string; in future it may mean they are not migrated at all. Use
memory_region_init_ram() instead.

Note that this is a a cross-version migration compatibility break
for the "malta" machine.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NCédric Le Goater <clg@kaod.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NPaul Burton <paul.burton@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Currently we use memory_region_init_rom_nomigrate() to create
the "boston.flash" memory region, and we don't manually register
it with vmstate_register_ram(). This currently means that its
contents are migrated but as a ram block whose name is the empty
string; in future it may mean they are not migrated at all. Use
memory_region_init_ram() instead.

Note that this is a a cross-version migration compatibility break
for the "boston" machine.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NCédric Le Goater <clg@kaod.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NPaul Burton <paul.burton@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Yongbok Kim transfers duties of QEMU for target MIPS maintainer to
myself as he leaves MIPS. Many thanks to Yongbok for his substantial
contributing to QEMU for MIPS over many years and taking care of its
maintainance for almost two years.
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Acked-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

The way we determine if we can start the incoming migration was
changed to use migration_has_all_channels() in:

  commit 428d8908
  Author: Juan Quintela <quintela@redhat.com>
  Date:   Mon Jul 24 13:06:25 2017 +0200

    migration: Create migration_has_all_channels

This method in turn calls multifd_recv_all_channels_created()
which is hardcoded to always return 'true' when multifd is
not in use. This is a latent bug...

...activated in a following commit where that return result
ends up acting as the flag to indicate whether it is possible
to start processing the migration:

  commit 36c2f8be
  Author: Juan Quintela <quintela@redhat.com>
  Date:   Wed Mar 7 08:40:52 2018 +0100

    migration: Delay start of migration main routines

This means that if channel initialization fails with normal
migration, it'll never notice and attempt to start the
incoming migration regardless and crash on a NULL pointer.

This can be seen, for example, if a client connects to a server
requiring TLS, but has an invalid x509 certificate:

qemu-system-x86_64: The certificate hasn't got a known issuer
qemu-system-x86_64: migration/migration.c:386: process_incoming_migration_co: Assertion `mis->from_src_file' failed.

 #0  0x00007fffebd24f2b in raise () at /lib64/libc.so.6
 #1  0x00007fffebd0f561 in abort () at /lib64/libc.so.6
 #2  0x00007fffebd0f431 in _nl_load_domain.cold.0 () at /lib64/libc.so.6
 #3  0x00007fffebd1d692 in  () at /lib64/libc.so.6
 #4  0x0000555555ad027e in process_incoming_migration_co (opaque=<optimized out>) at migration/migration.c:386
 #5  0x0000555555c45e8b in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at util/coroutine-ucontext.c:116
 #6  0x00007fffebd3a6a0 in __start_context () at /lib64/libc.so.6
 #7  0x0000000000000000 in  ()

To handle the non-multifd case, we check whether mis->from_src_file
is non-NULL. With this in place, the migration server drops the
rejected client and stays around waiting for another, hopefully
valid, client to arrive.
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>
Message-Id: <20180619163552.18206-1-berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Not needed. Don't expose last_ram_page().
Signed-off-by: NDavid Hildenbrand <david@redhat.com>
Message-Id: <20180620202736.21399-1-david@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

We have to flush() the QEMUFile because now we sent really few data
through that channel.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

We know quit with shutdwon in the QIO.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
--
Add comment
Use shutdown() instead of unref()

We have three conditions here:
- channel fails -> error
- we have to quit: we close the channel and reads fails
- normal read that success, we are in bussiness

So forget the complications of waiting in a semaphore.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

The function still don't use multifd, but we have simplified
ram_save_page, xbzrle and RDMA stuff is gone.  We have added a new
counter.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

--
Add last_page parameter
Add commets for done and address
Remove multifd field, it is the same than normal pages
Merge next patch, now we send multiple pages at a time
Remove counter for multifd pages, it is identical to normal pages
Use iovec's instead of creating the equivalent.
Clear memory used by pages (dave)
Use g_new0(danp)
define MULTIFD_CONTINUE
now pages member is a pointer
Fix off-by-one in number of pages in one packet
Remove RAM_SAVE_FLAG_MULTIFD_PAGE
s/multifd_pages_t/MultiFDPages_t/
add comment explaining what it means

This will include how many bytes they are sent through multifd.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

We synchronize all threads each RAM_SAVE_FLAG_EOS.  Bitmap
synchronizations don't happen inside a  ram section, so we are safe
about two channels trying to overwrite the same memory.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

--
seq needs to be atomic now, will also be accessed from main thread.
Fix the if (true || ...) leftover
We are back to non-atomics

Once there add tracepoints.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

Either for quit, sync or packet, we first wake them.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

We want to know how many pages/packets each channel has sent.  Add
counters for those.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

--
sort trace-events (dave)

Right now we use the "position" inside the QEMUFile, but things like
RDMA already do weird things to be able to maintain that counter
right, and multifd will have some similar problems.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

We used to include in this calculation the setup time, but that can be
quite big in rdma or multifd.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

We still don't put anything there.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

--
fix magic (dave)
check offset/ramblock  (dave)
s/seq/packet_num/ and make it 64bit

We only create/destry the page list here.  We will use it later.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>