- vhost-scsi: add bootindex property
- RCU: fix MemoryRegion lifetime issues in PCI; document the rules;
convert of AddressSpaceDispatch and RAMList
- KVM: add kvm_exit reasons for aarch64

# gpg: Signature made Mon Feb 16 16:32:32 2015 GMT using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (21 commits)
  Convert ram_list to RCU
  exec: convert ram_list to QLIST
  cosmetic changes preparing for the following patches
  exec: protect mru_block with RCU
  rcu: add g_free_rcu
  rcu: introduce RCU-enabled QLIST
  exec: RCUify AddressSpaceDispatch
  exec: make iotlb RCU-friendly
  exec: introduce cpu_reload_memory_map
  docs: clarify memory region lifecycle
  pci: split shpc_cleanup and shpc_free
  pcie: remove mmconfig memory leak and wrap mmconfig update with transaction
  memory: keep the owner of the AddressSpace alive until do_address_space_destroy
  rcu: run RCU callbacks under the BQL
  rcu: do not let RCU callbacks pile up indefinitely
  vhost-scsi: set the bootable value of channel/target/lun
  vhost-scsi: add a property for booting
  vhost-scsi: expose the TYPE_FW_PATH_PROVIDER interface
  vhost-scsi: add bootindex property
  qdev: support to get a device firmware path directly
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Pull request

v2:
 * Fix C11 typedef redefinitions in ahci and libqos malloc [Peter]
 * Fix lx -> PRIx64 format specifiers in ahci [Peter]

# gpg: Signature made Mon Feb 16 15:45:53 2015 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request: (65 commits)
  block: Keep bdrv_check*_request()'s return value
  block: Remove "growable" from BDS
  block: Clamp BlockBackend requests
  qemu-io: Use BlockBackend
  qemu-io: Remove "growable" option
  qemu-io: Use blk_new_open() in openfile()
  qemu-nbd: Use blk_new_open() in main()
  qemu-img: Use BlockBackend as far as possible
  qemu-img: Use blk_new_open() in img_rebase()
  qemu-img: Use blk_new_open() in img_open()
  block/xen: Use blk_new_open() in blk_connect()
  blockdev: Use blk_new_open() in blockdev_init()
  iotests: Add test for driver=qcow2, format=qcow2
  block: Add Error parameter to bdrv_find_protocol()
  block: Add blk_new_open()
  block: Lift some BDS functions to the BlockBackend
  iotests: Add test for qemu-img convert to NBD
  qemu-img: Fix qemu-img convert -n
  qemu-iotests: Add 093 for IO throttling
  qemu-iotests: Allow caller to disable underscore convertion for qmp
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

vnc: fixup some QemuOpts conversion fallout.

# gpg: Signature made Mon Feb 16 08:13:32 2015 GMT using RSA key ID D3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"

* remotes/kraxel/tags/pull-vnc-20150216-1:
  vnc: fix coverity warning
  ui/vnc: optimize full scanline updates
  vnc: auto assian an id when calling change vnc qmp interface
  vnc: introduce an wrapper for auto assign vnc id
  vnc: using bool type instead of int for QEMU_OPT_BOOL
  vnc: correct missing property about vnc_display
  vnc: fix qemu crash when not configure vnc option
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

MIPS patches 2015-02-13

Changes:
* bug fixes, cleanups and minor improvements

# gpg: Signature made Sat Feb 14 17:01:37 2015 GMT using RSA key ID 0B29DA6B
# gpg: Can't check signature: public key not found

* remotes/lalrae/tags/mips-20150213-2:
  linux-user: correct stat structure in MIPS N32
  target-mips: pass 0 instead of -1 as rs in microMIPS LUI instruction
  target-mips: fix broken snapshotting
  target-mips: use CP0EnLo_XI instead of magic number
  target-mips: ll and lld cause AdEL exception for unaligned address
  target-mips: fix detection of the end of the page during translation
  target-mips: Make CP0.Status.CU1 read-only for the 5Kc and 5KEc processors
  isa: remove isa_mem_base variable
  gt64xxx: remove isa_mem_base usage
  piix4: use PCI address space instead of system memory
  mips: remove isa_mem_base usage
  jazz: remove usage of isa_mem_base
  jazz: do not explode QEMUMachineInitArgs structure
  isa: add memory space parameter to isa_bus_new
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Allow "unlocked" reads of the ram_list by using an RCU-enabled QLIST.

The ramlist mutex is kept.  call_rcu callbacks are run with the iothread
lock taken, but that may change in the future.  Writers still take the
ramlist mutex, but they no longer need to assume that the iothread lock
is taken.

Readers of the list, instead, no longer require either the iothread
or ramlist mutex, but they need to use rcu_read_lock() and
rcu_read_unlock().

One place in arch_init.c was downgrading from write side to read side
like this:

    qemu_mutex_lock_iothread()
    qemu_mutex_lock_ramlist()
    ...
    qemu_mutex_unlock_iothread()
    ...
    qemu_mutex_unlock_ramlist()

and the equivalent idiom is:

    qemu_mutex_lock_ramlist()
    rcu_read_lock()
    ...
    qemu_mutex_unlock_ramlist()
    ...
    rcu_read_unlock()
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NMike Day <ncmike@ncultra.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

QLIST has RCU-friendly primitives, so switch to it.
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NMike Day <ncmike@ncultra.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NMike Day <ncmike@ncultra.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Hence, freeing a RAMBlock has to be switched to call_rcu.
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This simplifies calling g_free from an RCU callback.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Add RCU-enabled variants on the existing bsd DQ facility. Each
operation has the same interface as the existing (non-RCU)
version. Also, each operation is implemented as macro.

Using the RCU-enabled QLIST, existing QLIST users will be able to
convert to RCU without using a different list interface.
Signed-off-by: NMike Day <ncmike@ncultra.org>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Note that even after this patch, most callers of address_space_*
functions must still be under the big QEMU lock, otherwise the memory
region returned by address_space_translate can disappear as soon as
address_space_translate returns.  This will be fixed in the next part
of this series.
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

After the previous patch, TLBs will be flushed on every change to
the memory mapping.  This patch augments that with synchronization
of the MemoryRegionSections referred to in the iotlb array.

With this change, it is guaranteed that iotlb_to_region will access
the correct memory map, even once the TLB will be accessed outside
the BQL.
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This for now is a simple TLB flush.  This can change later for two
reasons:

1) an AddressSpaceDispatch will be cached in the CPUState object

2) it will not be possible to do tlb_flush once the TCG-generated code
runs outside the BQL.
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Now that objects actually obey the rules, document them.
Reviewed-by: NMatthew Rosato <mjrosato@linux.vnet.ibm.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

object_unparent should not be called until the parent device is going to be
destroyed.  Only remove the capability and do memory_region_del_subregion
at unrealize time.  Freeing the data structures is left in shpc_free, to
be called from the instance_finalize callback.
Acked-by: NMichael S. Tsirkin <mst@redhat.com>
Reviewed-by: NMatthew Rosato <mjrosato@linux.vnet.ibm.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This memory leak was introduced inadvertently by omitting object_unparent.
A better fix is to use the new memory_region_set_size instead of destroying
and recreating the MMIO region on the fly.

Also, ensure that unmapping and remapping the region is done atomically.
Acked-by: NMichael S. Tsirkin <mst@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Do not throw away the value returned by bdrv_check_request() and
bdrv_check_byte_request().

Fix up some coding style issues in the proximity of the affected hunks.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1423162705-32065-17-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Now that request clamping is done in the BlockBackend, the "growable"
field can be removed from the BlockDriverState. All BDSs are now treated
as being "growable" (that is, they are allowed to grow; they are not
necessarily actually able to).
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-16-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

BlockBackend is used as the interface between the block layer and guest
devices. It should therefore assure that all requests are clamped to the
image size.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1423162705-32065-15-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

qemu-io should behave like a guest, therefore it should use BlockBackend
to access the block layer.

There are a couple of places where that is infeasible: First, the
bdrv_debug_* functions could theoretically be mirrored in the
BlockBackend, but since these are functions internal to the block layer,
they should not be visible externally (qemu-io as a test tool is exempt
from this).

Second, bdrv_get_info() and bdrv_get_specific_info() work on a single
BDS alone, therefore they should stay BDS-specific.

Third, bdrv_is_allocated() mainly works on a single BDS as well. Some
data may be passed through from the BDS's file (if sectors which are
apparently allocated in the file are not really allocated there but just
zero).

[Fixed conflicts around block_acct_start() usage from Fam Zheng's
"qemu-io: Account IO by aio_read and aio_write" commit.  Use
BlockBackend and blk_get_stats() instead of BlockDriverState.
--Stefan]
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-14-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Remove "growable" option from the "open" command and from the qemu-io
command line. qemu-io is about to be converted to BlockBackend which
will make sure that no request exceeds the image size, so the only way
to keep "growable" would be to use BlockBackend if it is not given and
to directly access the BDS if it is.

qemu-io is a debugging tool, therefore removing a rarely used option
will have only a very small impact, if any. There was only one
qemu-iotest which used the option; since it is not critical, this patch
just removes it.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-13-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-12-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1423162705-32065-11-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Although qemu-img already creates BlockBackends, it does not do accesses
to the images through them. This patch converts all of the bdrv_* calls
for which this is currently possible to blk_* calls. Most of the
remaining calls will probably stay bdrv_* calls because they really do
operate on the BDS level instead of the BB level.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-10-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-9-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-8-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

As part of the required changes, this fixes a bug where specifying an
invalid driver would result in the block layer probing the image format;
now it will result in an error, unless "<unset>" is specified as the
driver name. Fixing this would require further work on the xen_disk code
which does not seem worth it (at this point and for this patch).
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-7-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Due to different error propagation, this breaks tests 051 and 087; fix
their output.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-6-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

While specifying a different driver and format is obviously invalid,
specifying the same driver once through driver and once through format
is invalid as well. Add a test for it.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-5-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The argument given to bdrv_find_protocol() is just a file name, which
makes it difficult for the caller to reconstruct what protocol
bdrv_find_protocol() was hoping to find. This patch adds an Error
parameter to that function to solve this issue.
Suggested-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-4-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

blk_new_with_bs() creates a BlockBackend with an empty BlockDriverState
attached to it. Empty BDSs are not nice, therefore add an alternative
function which combines blk_new_with_bs() with bdrv_open().

Note: In contrast to bdrv_open() which takes a BlockDriver parameter,
blk_new_open() does not take such a parameter. This is because
bdrv_open() opens a BlockDriverState, therefore it is natural to be able
to set the BlockDriver for that BDS. The fact that bdrv_open() can open
more than a single BDS is merely some form of a byproduct.

blk_new_open() on the other hand is intended to be used to create a
whole tree of BlockDriverStates. Therefore, setting a single BlockDriver
does not make much sense. Instead, the drivers to be used for each of
the nodes must be configured through the "options" QDict; including the
driver of the root BDS.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1423162705-32065-3-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Create the blk_* counterparts for the following bdrv_* functions (which
make sense to call on the BlockBackend level):
- bdrv_co_write_zeroes()
- bdrv_write_compressed()
- bdrv_truncate()
- bdrv_nb_sectors()
- bdrv_discard()
- bdrv_load_vmstate()
- bdrv_save_vmstate()
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423162705-32065-2-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423666727-20777-3-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

If -n is specified, it does not matter whether the output format and
protocol support image creation; building the creation options should
simply be skipped.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1423666727-20777-2-git-send-email-mreitz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This case utilizes qemu-io command "aio_{read,write} -q" to verify the
effectiveness of IO throttling options.

It's implemented by driving the vm timer from qtest protocol, so the
throttling timers are signaled with determinied time duration. Then we
verify the completed IO requests are within 10% error of bps and iops
limits.

"null" protocol is used as the disk backend so that no actual disk IO is
performed on host, this will make the blockstats much more
deterministic. Both "null-aio" and "null-co" are covered, which is also
a simple cross validation test for the driver code.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 1422586186-9925-6-git-send-email-famz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

QMP command "block_set_io_throttle" expects underscores in parameters
instead of dashes: {iops,bps}_{rd,wr,max}.

Add optional argument conv_keys (defaults to True, backward compatible),
it will be used in IO throttling test case.
Reviewed-by: NBenoit Canet <benoit@irqsave.net>
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 1422586186-9925-5-git-send-email-famz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This will allow test cases to run command in qtest protocol. It's
write-only for now.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 1422586186-9925-4-git-send-email-famz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This adds scripts/qtest.py as a python library for qtest protocol.

This is a skeleton with a basic "cmd" method to execute a command,
reading and parsing of qtest output could be added later on demand.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 1422586186-9925-3-git-send-email-famz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This will enable accounting of aio requests issued from qemu-io aio
read/write commands.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 1422586186-9925-2-git-send-email-famz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

qemu_clock_run_timers() only takes care of main_loop_tlg, we shouldn't
forget aio timer list groups.

Currently, the qemu_clock_deadline_ns_all (a few lines above) counts all
the timergroups of this clock type, including aio tlg, but we don't fire
them, so they are never cleared, which makes a dead loop.

For example, this function hangs when trying to drive throttled block
request queue with qtest clock_step.
Signed-off-by: NFam Zheng <famz@redhat.com>
Acked-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 1421661103-29153-1-git-send-email-famz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>