This enables byte granularity requests on quorum nodes.

Note that the QMP events emitted by the driver are an external API that
we were careless enough to define as sector based. The offset and length
of requests reported in events are rounded therefore.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>

Replacing it with bdrv_co_pwritev() prepares us for byte granularity
requests and gets us rid of the last bdrv_aio_*() user in quorum.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

This is a conversion to a more natural coroutine style and improves the
readability of the driver.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Instead of calling quorum_aio_finalize() deeply nested in what used
to be an AIO callback, do it in the same functions that allocated the
AIOCB.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

This converts the quorum block driver from implementing callback-based
interfaces for read/write to coroutine-based ones. This is the first
step that will allow us further simplification of the code.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>

There is no point in passing the value of bs->opaque in order to
overwrite it with itself.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>

In the context of asynchronous work, if we have a worker coroutine that
didn't yield, the parent coroutine cannot be reentered because it hasn't
yielded yet. In this case we don't even have to reenter the parent
because it will see that the work is already done and won't even yield.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>

With aio=native (qemu-img bench -n) one or more requests can be completed
when a new request is submitted.  This in turn can cause bench_cb to
recurse before b->in_flight is updated.  This causes multiple I/Os
to be submitted with the same offset and, furthermore, the blk_aio_*
coroutines are never freed and qemu-img aborts.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

cryptodev patches

- add xts mode support
- add 3DES algorithm support
- other trivial fixes

# gpg: Signature made Sat 24 Dec 2016 05:56:44 GMT
# gpg:                using RSA key 0x2ED7FDE9063C864D
# gpg: Good signature from "Gonglei <arei.gonglei@huawei.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 3EF1 8E53 3459 E6D1 963A  3C05 2ED7 FDE9 063C 864D

* remotes/gonglei/tags/cryptodev-next-20161224:
  cryptodev: add 3des-ede support
  cryptodev: remove single-DES support in cryptodev
  cryptodev: add xts(aes) support
  cryptodev: fix the check of aes algorithm
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

# gpg: Signature made Fri 06 Jan 2017 02:55:49 GMT
# gpg:                using RSA key 0xEF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F  3562 EF04 965B 398D 6211

* remotes/jasowang/tags/net-pull-request:
  fsl_etsec: Fix Tx BD ring wrapping handling
  rtl8139: correctly handle PHY reset
  record/replay: add network support
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Current code that handles Tx buffer desciprtor ring scanning employs the
following algorithm:

	1. Restore current buffer descriptor pointer from TBPTRn

	2. Process current descriptor

	3. If current descriptor has BD_WRAP flag set set current
	   descriptor pointer to start of the descriptor ring

	4. If current descriptor points to start of the ring exit the
	   loop, otherwise increment current descriptor pointer and go
	   to #2

	5. Store current descriptor in TBPTRn

The way the code is implemented results in buffer descriptor ring being
scanned starting at offset/descriptor #0. While covering 99% of the
cases, this algorithm becomes problematic for a number of edge cases.

Consider the following scenario: guest OS driver initializes descriptor
ring to N individual descriptors and starts sending data out. Depending
on the volume of traffic and probably guest OS driver implementation it
is possible that an edge case where a packet, spread across 2
descriptors is placed in descriptors N - 1 and 0 in that order(it is
easy to imagine similar examples involving more than 2 descriptors).

What happens then is aforementioned algorithm starts at descriptor 0,
sees a descriptor marked as BD_LAST, which it happily sends out as a
separate packet(very much malformed at this point) then the iteration
continues and the first part of the original packet is tacked to the
next transmission which ends up being bogus as well.

This behvaiour can be pretty reliably observed when scp'ing data from a
guest OS via TAP interface for files larger than 160K (every time for
700K+).

This patch changes the scanning algorithm to do the following:

	1. Restore "current" buffer descriptor pointer from
	   TBPTRn

	2. If "current" descriptor does not have BD_TX_READY set, goto #6

	3. Process current descriptor

	4. If "current" descriptor has BD_WRAP flag set "current"
	   descriptor pointer to start of the descriptor ring otherwise
	   set increment "current" by the size of one descriptor

	5. Goto #1

	6. Save "current" buffer descriptor in TBPTRn

This way we preserve the information about which descriptor was
processed last and always start where we left off avoiding the original
problem. On top of that, judging by the following excerpt from
MPC8548ERM (p. 14-48):

"... When the end of the TxBD ring is reached, eTSEC initializes TBPTRn
to the value in the corresponding TBASEn. The TBPTR register is
internally written by the eTSEC’s DMA controller during
transmission. The pointer increments by eight (bytes) each time a
descriptor is closed successfully by the eTSEC..."

revised algorithm might also a more correct way of emulating this aspect
of eTSEC peripheral.

Cc: Alexander Graf <agraf@suse.de>
Cc: Scott Wood <scottwood@freescale.com>
Cc: Jason Wang <jasowang@redhat.com>
Cc: qemu-devel@nongnu.org
Signed-off-by: NAndrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: NJason Wang <jasowang@redhat.com>

According to datasheet:
"[Bit 15 of Basic Mode Control Register] sets the status and control registers
of the PHY (register 0062-0074) in a default state. This bit is self-clearing.
1 = software reset; 0 = normal operation."

This fixes the netcard detection failure in Minoca OS.
Signed-off-by: NHervé Poussineau <hpoussin@reactos.org>
Signed-off-by: NJason Wang <jasowang@redhat.com>

This patch adds support of recording and replaying network packets in
irount rr mode.

Record and replay for network interactions is performed with the network filter.
Each backend must have its own instance of the replay filter as follows:
 -netdev user,id=net1 -device rtl8139,netdev=net1
 -object filter-replay,id=replay,netdev=net1

Replay network filter is used to record and replay network packets. While
recording the virtual machine this filter puts all packets coming from
the outer world into the log. In replay mode packets from the log are
injected into the network device. All interactions with network backend
in replay mode are disabled.

v5 changes:
 - using iov_to_buf function instead of loop
Signed-off-by: NPavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
Signed-off-by: NJason Wang <jasowang@redhat.com>

# gpg: Signature made Wed 04 Jan 2017 13:29:09 GMT
# gpg:                using RSA key 0x9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  iothread: add poll-grow and poll-shrink parameters
  aio: self-tune polling time
  virtio: disable virtqueue notifications during polling
  aio: add .io_poll_begin/end() callbacks
  virtio: turn vq->notification into a nested counter
  virtio-scsi: suppress virtqueue kick during processing
  virtio-blk: suppress virtqueue kick during processing
  iothread: add polling parameters
  linux-aio: poll ring for completions
  virtio: poll virtqueues for new buffers
  aio: add polling mode to AioContext
  aio: add AioPollFn and io_poll() interface
  aio: add flag to skip fds to aio_dispatch()
  HACKING: document #include order
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

- transport specific callbacks (for Xen)
- fix crash (2.8 regression)
- 9p functional tests

# gpg: Signature made Tue 03 Jan 2017 17:30:58 GMT
# gpg:                using DSA key 0x02FC3AEB0101DBC2
# gpg: Good signature from "Greg Kurz <groug@kaod.org>"
# gpg:                 aka "Greg Kurz <groug@free.fr>"
# gpg:                 aka "Greg Kurz <gkurz@fr.ibm.com>"
# gpg:                 aka "Greg Kurz <gkurz@linux.vnet.ibm.com>"
# gpg:                 aka "Gregory Kurz (Groug) <groug@free.fr>"
# gpg:                 aka "Gregory Kurz (Cimai Technology) <gkurz@cimai.com>"
# gpg:                 aka "Gregory Kurz (Meiosys Technology) <gkurz@meiosys.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 2BD4 3B44 535E C0A7 9894  DBA2 02FC 3AEB 0101 DBC2

* remotes/gkurz/tags/for-upstream:
  tests: virtio-9p: ".." cannot be used to walk out of the shared directory
  tests: virtio-9p: no slash in path elements during walk
  tests: virtio-9p: add walk operation test
  tests: virtio-9p: add attach operation test
  tests: virtio-9p: add version operation test
  9pfs: fix P9_NOTAG and P9_NOFID macros
  tests: virtio-9p: code refactoring
  tests: virtio-9p: rename PCI configuration test
  9pfs: fix crash when fsdev is missing
  9pfs: introduce init_out/in_iov_from_pdu
  9pfs: call v9fs_init_qiov_from_pdu before v9fs_pack
  9pfs: introduce transport specific callbacks
  9pfs: move pdus to V9fsState
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

virtio-gpu: misc bugfixes.

# gpg: Signature made Tue 03 Jan 2017 14:48:04 GMT
# gpg:                using RSA key 0x4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/pull-vga-20170103-1:
  virtio-gpu: fix memory leak in resource attach backing
  virtio-gpu-3d: fix memory leak in resource attach backing
  virtio-gpu: call cleanup mapping function in resource destroy
  virtio-gpu: track and limit host memory allocations
  display: virtio-gpu-3d: check virgl capabilities max_size
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

These parameters control the poll time self-tuning algorithm.  They are
optional and will default to sane values if omitted.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-14-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This patch is based on the algorithm for the kvm.ko halt_poll_ns
parameter in Linux.  The initial polling time is zero.

If the event loop is woken up within the maximum polling time it means
polling could be effective, so grow polling time.

If the event loop is woken up beyond the maximum polling time it means
polling is not effective, so shrink polling time.

If the event loop makes progress within the current polling time then
the sweet spot has been reached.

This algorithm adjusts the polling time so it can adapt to variations in
workloads.  The goal is to reach the sweet spot while also recognizing
when polling would hurt more than help.

Two new trace events, poll_grow and poll_shrink, are added for observing
polling time adjustment.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-13-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This is a performance optimization to eliminate vmexits during polling.
It also avoids spurious ioeventfd processing after polling ends.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-12-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The begin and end callbacks can be used to prepare for the polling loop
and clean up when polling stops.  Note that they may only be called once
for multiple aio_poll() calls if polling continues to succeed.  Once
polling fails the end callback is invoked before aio_poll() resumes file
descriptor monitoring.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-11-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Polling should disable virtqueue notifications but that requires nested
virtio_queue_set_notification() calls.  Turn vq->notification into a
counter so it is possible to do nesting.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-10-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The guest does not need to kick the virtqueue while we are processing
it.  This reduces the number of vmexits during periods of heavy I/O.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-9-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The guest does not need to kick the virtqueue while we are processing
it.  This reduces the number of vmexits during periods of heavy I/O.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-8-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Poll mode can be configured with -object iothread,poll-max-ns=NUM.
Polling is disabled with a value of 0 nanoseconds.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-7-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The Linux AIO userspace ABI includes a ring that is shared with the
kernel.  This allows userspace programs to process completions without
system calls.

Add an AioContext poll handler to check for completions in the ring.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-6-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Add an AioContext poll handler to detect new virtqueue buffers without
waiting for a guest->host notification.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-5-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The AioContext event loop uses ppoll(2) or epoll_wait(2) to monitor file
descriptors or until a timer expires.  In cases like virtqueues, Linux
AIO, and ThreadPool it is technically possible to wait for events via
polling (i.e. continuously checking for events without blocking).

Polling can be faster than blocking syscalls because file descriptors,
the process scheduler, and system calls are bypassed.

The main disadvantage to polling is that it increases CPU utilization.
In classic polling configuration a full host CPU thread might run at
100% to respond to events as quickly as possible.  This patch implements
a timeout so we fall back to blocking syscalls if polling detects no
activity.  After the timeout no CPU cycles are wasted on polling until
the next event loop iteration.

The run_poll_handlers_begin() and run_poll_handlers_end() trace events
are added to aid performance analysis and troubleshooting.  If you need
to know whether polling mode is being used, trace these events to find
out.

Note that the AioContext is now re-acquired before disabling notify_me
in the non-polling case.  This makes the code cleaner since notify_me
was enabled outside the non-polling AioContext release region.  This
change is correct since it's safe to keep notify_me enabled longer
(disabling is an optimization) but potentially causes unnecessary
event_notifer_set() calls.  I think the chance of performance regression
is small here.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-4-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The new AioPollFn io_poll() argument to aio_set_fd_handler() and
aio_set_event_handler() is used in the next patch.

Keep this code change separate due to the number of files it touches.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-3-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Polling mode will not call ppoll(2)/epoll_wait(2).  Therefore we know
there are no fds ready and should avoid looping over fd handlers in
aio_dispatch().
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161201192652.9509-2-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

It was not obvious to me why "qemu/osdep.h" must be the first #include.
This documents the rationale and the overall #include order.

Cc: Fam Zheng <famz@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Eric Blake <eblake@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Message-id: 1479307161-24658-1-git-send-email-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

According to the 9P spec at http://man.cat-v.org/plan_9/5/intro, the
parent directory of the root directory of a server's tree is itself.
This test hence checks that the qid of the root directory as returned by
attach is the same as the qid of ".." when walking from the root directory.
Signed-off-by: NGreg Kurz <groug@kaod.org>

The walk operation is expected to fail and to return ENOENT.
Signed-off-by: NGreg Kurz <groug@kaod.org>

The walk operation is used to traverse the directory tree and to associate
paths to fids. A single walk can be used to traverse up to P9_MAXWELEM path
elements at the same time.

The test creates a path with P9_MAXWELEM elements on the backend (à la
'mkdir -p') and issues a walk operation. The walk is expected to succeed
without error.

Reference:

http://man.cat-v.org/plan_9/5/walkSigned-off-by: NGreg Kurz <groug@kaod.org>

The attach operation is used to establish a connection between the
client and the server. After this, the client is able to access the
underlying filesystem and do I/O.

This test simply ensures the operation succeeds without error.

Reference:

http://man.cat-v.org/plan_9/5/attachSigned-off-by: NGreg Kurz <groug@kaod.org>

This patch lays the foundations to be able to test 9P operations and
provides a test for the version operation as a first example.

A 9P request is composed of a T-message sent by the client (guest) to the
server (QEMU), and a R-message sent by the server back to the client.

The following general calls are available to implement requests for any
9P operations:

v9fs_req_init(): allocates the request structure and the guest memory for
                 the T-message

v9fs_req_send(): allocates the guest memory for the R-message and sends the
                 T-message to QEMU

v9fs_req_recv(): waits for QEMU to answer and does some sanity checks on the
                 returned R-message header

v9fs_req_free(): releases the guest memory and the request structure

Helpers are provided, to be used by each specific 9P operation to copy data
to/from the guest memory.

The version operation is used to negotiate the 9P protocol version to be
used and the maximum buffer size for exchanged data. It is necessarily
the first message of a 9P session. For simplicity, the maximum buffer size
is hardcoded to 4k, which should be enough for functional tests.

The test simply advertises the "9P2000.L" version to QEMU and expects QEMU
to answer it is supported.

References:

http://man.cat-v.org/plan_9/5/intro
http://man.cat-v.org/plan_9/5/versionSigned-off-by: NGreg Kurz <groug@kaod.org>

The u16 and u32 types don't exist in QEMU common headers. It never broke
build because these two macros aren't use by the current code, but this
is about to change with the future addition of functional tests for 9P.

Also, these should have enclosing parenthesis to be usable in any
syntactical situation.

As suggested by Eric Blake, let's use UINT16_MAX and UINT32_MAX to address
both issues.
Signed-off-by: NGreg Kurz <groug@kaod.org>

This moves the test_share static and the QOSState into the QVirtIO9P
structure, and put PCI related code in functions with a _pci_ name.

This will avoid code duplication in future tests, and allow to add
support for non-PCI platforms.
Signed-off-by: NGreg Kurz <groug@kaod.org>

Signed-off-by: NGreg Kurz <groug@kaod.org>

If the user passes -device virtio-9p without the corresponding -fsdev, QEMU
dereferences a NULL pointer and crashes.

This is a 2.8 regression introduced by commit 702dbcc2.
Signed-off-by: NGreg Kurz <groug@kaod.org>
Reviewed-by: NLi Qiang <liq3ea@gmail.com>

Not all 9pfs transports share memory between request and response. For
those who don't, it is necessary to know how much memory is required in
the response.

Split the existing init_iov_from_pdu function in two:
init_out_iov_from_pdu (for writes) and init_in_iov_from_pdu (for reads).
init_in_iov_from_pdu takes an additional size parameter to specify the
memory required for the response message.
Signed-off-by: NStefano Stabellini <sstabellini@kernel.org>
Reviewed-by: NGreg Kurz <groug@kaod.org>
Signed-off-by: NGreg Kurz <groug@kaod.org>