Merge I/O channels base classes

# gpg: Signature made Fri 18 Dec 2015 12:18:38 GMT using RSA key ID 15104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>"
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>"

* remotes/berrange/tags/pull-io-channel-base-2015-12-18-1:
  io: add QIOChannelBuffer class
  io: add QIOChannelCommand class
  io: add QIOChannelWebsock class
  io: add QIOChannelTLS class
  io: add QIOChannelFile class
  io: add QIOChannelSocket class
  io: add QIOTask class for async operations
  io: add helper module for creating watches on FDs
  io: add abstract QIOChannel classes
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Add a QIOChannel subclass that is capable of performing I/O
to/from a memory buffer. This implementation does not attempt
to support concurrent readers & writers. It is designed for
serialized access where by a single thread at a time may write
data, seek and then read data back out.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a QIOChannel subclass that is capable of performing I/O
to/from a separate process, via a pair of pipes. The command
can be used for unidirectional or bi-directional I/O.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a QIOChannel subclass that can run the websocket protocol over
the top of another QIOChannel instance. This initial implementation
is only capable of acting as a websockets server. There is no support
for acting as a websockets client yet.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a QIOChannel subclass that can run the TLS protocol over
the top of another QIOChannel instance. The object provides a
simplified API to perform the handshake when starting the TLS
session. The layering of TLS over the underlying channel does
not have to be setup immediately. It is possible to take an
existing QIOChannel that has done some handshake and then swap
in the QIOChannelTLS layer. This allows for use with protocols
which start TLS right away, and those which start plain text
and then negotiate TLS.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a QIOChannel subclass that is capable of operating on things
that are files, such as plain files, pipes, character/block
devices, but notably not sockets.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Implement a QIOChannel subclass that supports sockets I/O.
The implementation is able to manage a single socket file
descriptor, whether a TCP/UNIX listener, TCP/UNIX connection,
or a UDP datagram. It provides APIs which can listen and
connect either asynchronously or synchronously. Since there
is no asynchronous DNS lookup API available, it uses the
QIOTask helper for spawning a background thread to ensure
non-blocking operation.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

A number of I/O operations need to be performed asynchronously
to avoid blocking the main loop. The caller of such APIs need
to provide a callback to be invoked on completion/error and
need access to the error, if any. The small QIOTask provides
a simple framework for dealing with such probes. The API
docs inline provide an outline of how this is to be used.

Some functions don't have the ability to run asynchronously
(eg getaddrinfo always blocks), so to facilitate their use,
the task class provides a mechanism to run a blocking
function in a thread, while triggering the completion
callback in the main event loop thread. This easily allows
any synchronous function to be made asynchronous, albeit
at the cost of spawning a thread.

In this series, the QIOTask class will be used for things like
the TLS handshake, the websockets handshake and TCP connect()
progress.

The concept of QIOTask is inspired by the GAsyncResult
interface / GTask class in the GIO libraries. The min
version requirements on glib don't allow those to be
used from QEMU, so QIOTask provides a facsimilie which
can be easily switched to GTask in the future if the
min version is increased.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

A number of the channel implementations will require the
ability to create watches on file descriptors / sockets.
To avoid duplicating this code in each channel, provide a
helper API for dealing with file descriptor watches.

There are two watch implementations provided. The first
is useful for bi-directional file descriptors such as
sockets, regular files, character devices, etc. The
second works with a pair of unidirectional file descriptors
such as pipes.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Start the new generic I/O channel framework by defining a
QIOChannel abstract base class. This is designed to feel
similar to GLib's GIOChannel, but with the addition of
support for using iovecs, qemu error reporting, file
descriptor passing, coroutine integration and use of
the QOM framework for easier sub-classing.

The intention is that anywhere in QEMU that almost
anywhere that deals with sockets will use this new I/O
infrastructure, so that it becomes trivial to then layer
in support for TLS encryption. This will at least include
the VNC server, char device backend and migration code.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

* KVM: synic support, split irqchip support
* memory: cleanups, optimizations, ioeventfd emulation
* SCSI: small fixes, vmw_pvscsi compatibility improvements
* qemu_log cleanups
* Coverity model improvements

# gpg: Signature made Thu 17 Dec 2015 16:35:21 GMT using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"

* remotes/bonzini/tags/for-upstream: (45 commits)
  coverity: Model g_memdup()
  coverity: Model g_poll()
  scsi: always call notifier on async cancellation
  scsi: use scsi_req_cancel_async when purging requests
  target-i386: kvm: clear unusable segments' flags in migration
  rcu: optimize rcu_read_lock
  memory: try to inline constant-length reads
  memory: inline a few small accessors
  memory: extract first iteration of address_space_read and address_space_write
  memory: split address_space_read and address_space_write
  memory: avoid unnecessary object_ref/unref
  memory: reorder MemoryRegion fields
  exec: make qemu_ram_ptr_length more similar to qemu_get_ram_ptr
  exec: always call qemu_get_ram_ptr within rcu_read_lock
  linux-user: convert DEBUG_SIGNAL logging to tracepoints
  linux-user: avoid "naked" qemu_log
  user: introduce "-d page"
  xtensa: avoid "naked" qemu_log
  tricore: avoid "naked" qemu_log
  ppc: cleanup logging
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

We model all the non-deprecated memory allocation functions from
https://developer.gnome.org/glib/stable/glib-Memory-Allocation.html
except for g_memdup(), g_clear_pointer(), g_steal_pointer().  We don't
use the latter two.  Model the former.

Coverity now reports an OVERRUN
vl.c:2317: alloc_strlen: Allocating insufficient memory for the terminating null of the string.
Correct, but we omit the terminating null intentionally there.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1448901152-11716-1-git-send-email-armbru@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

In my testing, Coverity reported two more CHECKED_RETURN:

* qemu-char.c:1248: fixed in commit c1f24489: "qemu-char: retry g_poll
  on EINTR".

* migration/qemu-file-unix.c:75: harmless, cleaned up in commit
  4e39f57c "migration: Clean up use of g_poll() in
  socket_writev_buffer()
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1450336833-27710-1-git-send-email-armbru@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This was found by code inspection.  If the request is cancelled twice,
the notifier is never called on the second cancellation request,
and hence for example a TMF might never finish.

All the calls in scsi_req_cancel_async are idempotent, so the change
is safe.
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-Id: <1450290827-30508-2-git-send-email-pbonzini@redhat.com>

This avoids calls to aio_poll without having acquired the context first.
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-Id: <1450290827-30508-1-git-send-email-pbonzini@redhat.com>

This commit fixes migration of a QEMU/KVM guest from kernel >= v3.9 to
kernel <= v3.7 (e.g. from RHEL 7 to RHEL 6). Without this commit a guest
migrated across these kernel versions fails to resume on the target host
as its segment descriptors are invalid.

Two separate kernel commits combined together to result in this bug:

  commit f0495f9b9992f80f82b14306946444b287193390
  Author: Avi Kivity <avi@redhat.com>
  Date:   Thu Jun 7 17:06:10 2012 +0300

      KVM: VMX: Relax check on unusable segment

      Some userspace (e.g. QEMU 1.1) munge the d and g bits of segment
      descriptors, causing us not to recognize them as unusable segments
      with emulate_invalid_guest_state=1.  Relax the check by testing for
      segment not present (a non-present segment cannot be usable).
Signed-off-by: NAvi Kivity <avi@redhat.com>

  commit 25391454e73e3156202264eb3c473825afe4bc94
  Author: Gleb Natapov <gleb@redhat.com>
  Date:   Mon Jan 21 15:36:46 2013 +0200

      KVM: VMX: don't clobber segment AR of unusable segments.

      Usability is returned in unusable field, so not need to clobber entire
      AR. Callers have to know how to deal with unusable segments already
      since if emulate_invalid_guest_state=true AR is not zeroed.
Signed-off-by: NGleb Natapov <gleb@redhat.com>
Signed-off-by: NMarcelo Tosatti <mtosatti@redhat.com>

The first commit changed the KVM_SET_SREGS ioctl so that it did no treat
segment flags == 0 as an unusable segment, instead only looking at the
"present" flag.

The second commit changed KVM_GET_SREGS so that it did not clear the
flags of an unusable segment.

Since QEMU does not itself maintain the "unusable" flag across a
migration, the end result is that unusable segments read from a kernel
with these commits and loaded into a kernel without these commits are
not properly recognised as being unusable.

This commit updates both get_seg and set_seg so that the problem is
avoided even when migrating to or migrating from a QEMU without this
commit. In get_seg, we clear the segment flags if the segment is marked
unusable. In set_seg, we mark the segment unusable if the segment's
"present" flag is not set.
Signed-off-by: NMichael Chapman <mike@very.puzzling.org>
Message-Id: <1449464047-17467-1-git-send-email-mike@very.puzzling.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

rcu_read_lock cannot change rcu_gp_ongoing from true to false
(the previous value of p_rcu_reader->ctr is zero), hence
there is no need to check p_rcu_reader->waiting and wake up
a concurrent synchronize_rcu.

While at it mark the wakeup as unlikely in rcu_read_unlock.
Reviewed-by: NWen Congyang <wency@cn.fujitsu.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-Id: <1450265542-4323-1-git-send-email-pbonzini@redhat.com>

memcpy can take a large amount of time for small reads and writes.
Handle the common case of reading s/g descriptors from memory (there
is no corresponding "write" case that is as common, because writes
often use address_space_st* functions) by inlining the relevant
parts of address_space_read into the caller.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

These are used in the address_space_* fast paths.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

We want to inline the case where there is only one iteration, because
then the compiler can also inline the memcpy.  As a start, extract
everything after the first address_space_translate call.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Rather than dispatching on is_write for every iteration, make
address_space_rw call one of the two functions.  The amount of
duplicate logic is pretty small, and memory_access_is_direct can
be tweaked so that it inlines better in the callers.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

For the common case of DMA into non-hotplugged RAM, it is unnecessary
but expensive to do object_ref/unref.  Add back an owner field to
MemoryRegion, so that these memory regions can skip the reference
counting.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Order fields so that all fields accessed during a RAM read/write fit in
the same cache line.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Notably, use qemu_get_ram_block to enjoy the MRU optimization.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Simplify the code and document the assumption.  The only caller
that is not within rcu_read_lock is memory_region_get_ram_ptr.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

"Unimplemented" messages go to stderr, everything else goes to tracepoints
Reviewed-by: NLaurent Vivier <lvivier@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Ensure that all log writes are protected by qemu_loglevel_mask or,
in serious cases, go to both the log and stderr.
Reviewed-by: NLaurent Vivier <lvivier@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Cc: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Cc: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Avoid "naked" qemu_log, bring documentation for DEBUG #defines
up to date.
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Convert to debug-only qemu_log.

Cc: Alexander Graf <agraf@suse.de>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Reviewed-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Reviewed-by: NEdgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Reviewed-by: NRichard Henderson <rth@twiddle.net>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

In some cases, the same message is printed both on stderr and in the log.
Avoid duplicate output in the default case where stderr _is_ the log,
and standardize this to stderr+log where it used to use stdio+log.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Our QA team wants to preserve serial output of the guest in between QEMU
runs to perform post-analysis.

By default this behavior is off (file is truncated each time QEMU is
started or device is plugged).
Signed-off-by: NOlga Krishtal <okrishtal@virtuozzo.com>
Signed-off-by: NDenis V. Lunev <den@openvz.org>
CC: Eric Blake <eblake@redhat.com>
CC: Markus Armbruster <armbru@redhat.com>
CC: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1449211324-17856-1-git-send-email-den@openvz.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This patch adds support for split IRQ chip mode. When
KVM_CAP_SPLIT_IRQCHIP is enabled:

    1.) The PIC, PIT, and IOAPIC are implemented in userspace while
    the LAPIC is implemented by KVM.

    2.) The software IOAPIC delivers interrupts to the KVM LAPIC via
    kvm_set_irq. Interrupt delivery is configured via the MSI routing
    table, for which routes are reserved in target-i386/kvm.c then
    configured in hw/intc/ioapic.c

    3.) KVM delivers IOAPIC EOIs via a new exit KVM_EXIT_IOAPIC_EOI,
    which is handled in target-i386/kvm.c and relayed to the software
    IOAPIC via ioapic_eoi_broadcast.
Signed-off-by: NMatt Gingell <gingell@google.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This patch adds the initial plumbing for split IRQ chip mode via
KVM_CAP_SPLIT_IRQCHIP. In addition to option processing, a number of
kvm_*_in_kernel macros are defined to help clarify which component is
where.
Signed-off-by: NMatt Gingell <gingell@google.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Hyper-V SynIC timers are host timers that are configurable
by guest through corresponding MSR's (HV_X64_MSR_STIMER*).
Guest setup and use fired by host events(SynIC interrupt
and appropriate timer expiration message) as guest clock
events.

The state of Hyper-V SynIC timers are stored in corresponding
MSR's. This patch seria implements such MSR's support and migration.
Signed-off-by: NAndrey Smetanin <asmetanin@virtuozzo.com>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: Richard Henderson <rth@twiddle.net>
CC: Eduardo Habkost <ehabkost@redhat.com>
CC: "Andreas Färber" <afaerber@suse.de>
CC: Marcelo Tosatti <mtosatti@redhat.com>
CC: Denis V. Lunev <den@openvz.org>
CC: Roman Kagan <rkagan@virtuozzo.com>
CC: kvm@vger.kernel.org

Message-Id: <1448464885-8300-3-git-send-email-asmetanin@virtuozzo.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>