Commit abf754fe updated 026.out, but forgot to also update
026.out.nocache.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>

block_crypto_open_opts_init() and block_crypto_create_opts_init()
contain a virtual visit of QCryptoBlockOptions and
QCryptoBlockCreateOptions less member "format", respectively.

Change their callers to put member "format" in the QDict, so they can
use the generated visitors for these types instead.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

in_flight and tracked requests need to be tracked in every layer during
recursion. For now the only user is qemu-img convert where overlapping
requests and IOThreads don't exist, therefore this change doesn't make
much difference form user point of view, but it is incorrect as part of
the API. Fix it.
Reported-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

In the beginning of the function, we initialize the local variable to 0,
and in the body of the function, we check the assigned values and exit
the loop immediately. So here it can never be non-zero.
Reported-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This moves the code to resize an image file to the thread pool to avoid
blocking.

Creating large images with preallocation with blockdev-create is now
actually a background job instead of blocking the monitor (and most
other things) until the preallocation has completed.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

When growing an image, block drivers (especially protocol drivers) may
initialise the newly added area. I/O requests to the same area need to
wait for this initialisation to be completed so that data writes don't
get overwritten and reads don't read uninitialised data.

To avoid overhead in the fast I/O path by adding new locking in the
protocol drivers and to restrict the impact to requests that actually
touch the new area, reuse the existing tracked request infrastructure in
block/io.c and mark all discard requests as serialising.

With this change, it is safe for protocol drivers to make
.bdrv_co_truncate actually asynchronous.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

This moves the bdrv_truncate() implementation from block.c to block/io.c
so it can have access to the tracked requests infrastructure.

This involves making refresh_total_sectors() public (in block_int.h).
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

All callers are coroutine_fns now, so we can just directly call
preallocate_co().
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

bdrv_truncate() is an operation that can block (even for a quite long
time, depending on the PreallocMode) in I/O paths that shouldn't block.
Convert it to a coroutine_fn so that we have the infrastructure for
drivers to make their .bdrv_co_truncate implementation asynchronous.

This change could potentially introduce new race conditions because
bdrv_truncate() isn't necessarily executed atomically any more. Whether
this is a problem needs to be evaluated for each block driver that
supports truncate:

* file-posix/win32, gluster, iscsi, nfs, rbd, ssh, sheepdog: The
  protocol drivers are trivially safe because they don't actually yield
  yet, so there is no change in behaviour.

* copy-on-read, crypto, raw-format: Essentially just filter drivers that
  pass the request to a child node, no problem.

* qcow2: The implementation modifies metadata, so it needs to hold
  s->lock to be safe with concurrent I/O requests. In order to avoid
  double locking, this requires pulling the locking out into
  preallocate_co() and using qcow2_write_caches() instead of
  bdrv_flush().

* qed: Does a single header update, this is fine without locking.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

If qcow2_alloc_clusters_at() returns an error, we do need to negate it
to get back the positive errno code for error_setg_errno(), but we still
need to return the negative error code.

Fixes: 772d1f97Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

Add an optional paramter num_queues for device, and set it
to 64 by default.
Signed-off-by: NWeiping Zhang <zhangweiping@didichuxing.com>
Acked-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The error handling policy was traditionally set with -drive, but with
-blockdev it is no longer possible to set frontend options. scsi-disk
(and other block devices) have long supported qdev properties to
configure the error handling policy, so let's add these options to
usb-storage as well and just forward them to the internal scsi-disk
instance.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>

Commit 51f63ec7 tried to change all references to 2.13 into 3.0, but
it failed to achieve this because it was not properly rebased on top of
the series introducing qapi/job.json. Change the references now.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Coverity can't see that qobject_input_visitor_new_flat_confused()
returns non-null when it doesn't set @local_err.  Check the return
value instead, like all the other callers do.

Fixes: CID 1393615
Fixes: CID 1393616
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Commit f1b34a24 replaced less-than-obvious test in
qdict_flatten_qdict() by the obvious one.  Sadly, it made something
else non-obvious: the fact that @new_key passed to qdict_put_obj()
can't be null, because that depends on the function's precondition
(target == qdict) == !prefix.

Tweak the function some more to help Coverity and human readers alike.

Fixes: CID 1393620
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Merge qio 2018-06-28 v3

Misc bug fixes for sockets channels

# gpg: Signature made Thu 28 Jun 2018 13:37:35 BST
# gpg:                using RSA key BE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>"
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>"
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E  8E3F BE86 EBB4 1510 4FDF

* remotes/berrange/tags/qio-next-pull-request:
  Delete AF_UNIX socket after close
  socket: don't free msgfds if error equals EAGAIN
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Pull request

 * Gracefully handle Linux AIO init failure

# gpg: Signature made Wed 27 Jun 2018 15:48:28 BST
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  linux-aio: properly bubble up errors from initialization
  compiler: add a sizeof_field() macro
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

migration/next for 20180627

# gpg: Signature made Wed 27 Jun 2018 13:53:53 BST
# gpg:                using RSA key F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>"
# gpg:                 aka "Juan Quintela <quintela@trasno.org>"
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/migration/20180627:
  migration: fix crash in when incoming client channel setup fails
  postcopy: drop ram_pages parameter from postcopy_ram_incoming_init()
  migration: Stop sending whole pages through main channel
  migration: Remove not needed semaphore and quit
  migration: Wait for blocking IO
  migration: Start sending messages
  migration: Create ram_save_multifd_page
  migration: Create multifd_bytes ram_counter
  migration: Synchronize multifd threads with main thread
  migration: Add block where to send/receive packets
  migration: Multifd channels always wait on the sem
  migration: Add multifd traces for start/end thread
  migration: Abstract the number of bytes sent
  migration: Calculate mbps only during transfer time
  migration: Create multifd packet
  migration: Create multipage support
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

MIPS queue

# gpg: Signature made Wed 27 Jun 2018 19:16:23 BST
# gpg:                using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <amarkovic@wavecomp.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01  DD75 D497 2A89 67F7 5A65

* remotes/amarkovic/tags/mips-queue-2018-06-27:
  target/mips: Fix gdbstub to read/write 64 bit FP registers
  target/mips: Fix data type for offset
  target/mips: Update gen_flt_ldst()
  target/mips: Fix microMIPS on reset
  target/mips: Raise a RI when given fs is n/a from CTC1
  hw/pci-host/xilinx-pcie: don't make "io" region be RAM
  hw/mips/mips_malta: don't make bios region 'nomigrate'
  hw/mips/boston: don't make flash region 'nomigrate'
  MAINTAINERS: update target-mips maintainers
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Since version 2.12.0 AF_UNIX socket created for QMP exchange is not
deleted on instance shutdown.

This is due to the fact that function qio_channel_socket_finalize() is
called after qio_channel_socket_close().
Signed-off-by: NPavel Balaev <mail@void.so>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

If we see EAGAIN, no data was sent over the socket, so we still have to
retry sending of msgfds next time.
Signed-off-by: Nlinzhecheng <linzhecheng@huawei.com>
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>

Pull request

 * Trace TCG atomic memory accesses
 * Document that trace event arguments cannot be floating point

# gpg: Signature made Wed 27 Jun 2018 13:57:40 BST
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/tracing-pull-request:
  trace: forbid floating point types
  trace: enable tracing of TCG atomics
  trace: add trace_mem_build_info_no_se_be/le
  trace: expand mem_info:size_shift to 3 bits
  trace: simplify trace_mem functions
  trace: fix misreporting of TCG access sizes for user-space
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Fix gdbstub to read/write 64 bit FP registers
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Offset can be larger than 16 bit from nanoMIPS,
and immediate field can be larger than 16 bits as well.
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Update gen_flt_ldst() in order to reuse the functions for nanoMIPS
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Fix to activate microMIPS on reset when Config3.ISA == {1, 3}
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Fix to raise a Reserved Instruction exception when given fs is not
available from CTC1.
Signed-off-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Currently we use memory_region_init_rom_nomigrate() to create
the "io" memory region to pass to pci_register_root_bus().
This is a dummy region, because this PCI controller doesn't
support accesses to PCI IO space.

There is no reason for the dummy region to be a RAM region;
it is only used as a place where PCI BARs can be mapped,
and if you could get a PCI card to do a bus master access
to the IO space it should not get acts-like-RAM behaviour.
Use a simple container memory region instead. (We do have
one PCI card model which can do bus master accesses to IO
space -- the LSI53C895A SCSI adaptor.)

This avoids the oddity of having a memory region which is
RAM but where the RAM is not migrated.

Note that the size of the region we use here has no
effect on behaviour.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NAlistair Francis <alistair.francis@wdc.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Currently we use memory_region_init_rom_nomigrate() to create
the "bios.1fc" memory region, and we don't manually register
it with vmstate_register_ram(). This currently means that its
contents are migrated but as a ram block whose name is the empty
string; in future it may mean they are not migrated at all. Use
memory_region_init_ram() instead.

Note that this is a a cross-version migration compatibility break
for the "malta" machine.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NCédric Le Goater <clg@kaod.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NPaul Burton <paul.burton@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Currently we use memory_region_init_rom_nomigrate() to create
the "boston.flash" memory region, and we don't manually register
it with vmstate_register_ram(). This currently means that its
contents are migrated but as a ram block whose name is the empty
string; in future it may mean they are not migrated at all. Use
memory_region_init_ram() instead.

Note that this is a a cross-version migration compatibility break
for the "boston" machine.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NCédric Le Goater <clg@kaod.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NPaul Burton <paul.burton@mips.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

Yongbok Kim transfers duties of QEMU for target MIPS maintainer to
myself as he leaves MIPS. Many thanks to Yongbok for his substantial
contributing to QEMU for MIPS over many years and taking care of its
maintainance for almost two years.
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@mips.com>
Acked-by: NYongbok Kim <yongbok.kim@mips.com>
Reviewed-by: NAleksandar Markovic <aleksandar.markovic@mips.com>

laio_init() can fail for a couple of reasons, which will lead to a NULL
pointer dereference in laio_attach_aio_context().

To solve this, add a aio_setup_linux_aio() function which is called
early in raw_open_common. If this fails, propagate the error up. The
signature of aio_get_linux_aio() was not modified, because it seems
preferable to return the actual errno from the possible failing
initialization calls.

Additionally, when the AioContext changes, we need to associate a
LinuxAioState with the new AioContext. Use the bdrv_attach_aio_context
callback and call the new aio_setup_linux_aio(), which will allocate a
new AioContext if needed, and return errors on failures. If it fails for
any reason, fallback to threaded AIO with an error message, as the
device is already in-use by the guest.

Add an assert that aio_get_linux_aio() cannot return NULL.
Signed-off-by: NNishanth Aravamudan <naravamudan@digitalocean.com>
Message-id: 20180622193700.6523-1-naravamudan@digitalocean.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Determining the size of a field is useful when you don't have a struct
variable handy.  Open-coding this is ugly.

This patch adds the sizeof_field() macro, which is similar to
typeof_field().  Existing instances are updated to use the macro.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Message-id: 20180614164431.29305-1-stefanha@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The way we determine if we can start the incoming migration was
changed to use migration_has_all_channels() in:

  commit 428d8908
  Author: Juan Quintela <quintela@redhat.com>
  Date:   Mon Jul 24 13:06:25 2017 +0200

    migration: Create migration_has_all_channels

This method in turn calls multifd_recv_all_channels_created()
which is hardcoded to always return 'true' when multifd is
not in use. This is a latent bug...

...activated in a following commit where that return result
ends up acting as the flag to indicate whether it is possible
to start processing the migration:

  commit 36c2f8be
  Author: Juan Quintela <quintela@redhat.com>
  Date:   Wed Mar 7 08:40:52 2018 +0100

    migration: Delay start of migration main routines

This means that if channel initialization fails with normal
migration, it'll never notice and attempt to start the
incoming migration regardless and crash on a NULL pointer.

This can be seen, for example, if a client connects to a server
requiring TLS, but has an invalid x509 certificate:

qemu-system-x86_64: The certificate hasn't got a known issuer
qemu-system-x86_64: migration/migration.c:386: process_incoming_migration_co: Assertion `mis->from_src_file' failed.

 #0  0x00007fffebd24f2b in raise () at /lib64/libc.so.6
 #1  0x00007fffebd0f561 in abort () at /lib64/libc.so.6
 #2  0x00007fffebd0f431 in _nl_load_domain.cold.0 () at /lib64/libc.so.6
 #3  0x00007fffebd1d692 in  () at /lib64/libc.so.6
 #4  0x0000555555ad027e in process_incoming_migration_co (opaque=<optimized out>) at migration/migration.c:386
 #5  0x0000555555c45e8b in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at util/coroutine-ucontext.c:116
 #6  0x00007fffebd3a6a0 in __start_context () at /lib64/libc.so.6
 #7  0x0000000000000000 in  ()

To handle the non-multifd case, we check whether mis->from_src_file
is non-NULL. With this in place, the migration server drops the
rejected client and stays around waiting for another, hopefully
valid, client to arrive.
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>
Message-Id: <20180619163552.18206-1-berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Not needed. Don't expose last_ram_page().
Signed-off-by: NDavid Hildenbrand <david@redhat.com>
Message-Id: <20180620202736.21399-1-david@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

We have to flush() the QEMUFile because now we sent really few data
through that channel.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

We know quit with shutdwon in the QIO.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
--
Add comment
Use shutdown() instead of unref()

We have three conditions here:
- channel fails -> error
- we have to quit: we close the channel and reads fails
- normal read that success, we are in bussiness

So forget the complications of waiting in a semaphore.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

The function still don't use multifd, but we have simplified
ram_save_page, xbzrle and RDMA stuff is gone.  We have added a new
counter.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

--
Add last_page parameter
Add commets for done and address
Remove multifd field, it is the same than normal pages
Merge next patch, now we send multiple pages at a time
Remove counter for multifd pages, it is identical to normal pages
Use iovec's instead of creating the equivalent.
Clear memory used by pages (dave)
Use g_new0(danp)
define MULTIFD_CONTINUE
now pages member is a pointer
Fix off-by-one in number of pages in one packet
Remove RAM_SAVE_FLAG_MULTIFD_PAGE
s/multifd_pages_t/MultiFDPages_t/
add comment explaining what it means