Remove the assumption that every QEMUFile implementation has
a file descriptor available by introducing a new function
in QEMUFileOps to change the blocking state of a QEMUFile.

If not set, it will fallback to the original code using
the get_fd method.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-7-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

The QEMUFileOps struct contains the I/O subsystem callbacks
and the migration stage hooks. Split the hooks out into a
separate QEMUFileHooks struct to make it easier to refactor
the I/O side of QEMUFile without affecting the hooks.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-6-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

The QEMUFile writev_buffer / put_buffer functions are expected
to write out the full set of requested data, blocking until
complete. The qemu_fflush() caller does not expect to deal with
partial writes. Clarify the function comments and add a sanity
check to the code to catch mistaken implementations.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-5-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 1453832250-766-2-git-send-email-peter.maydell@linaro.org

Postcopy needs a method to send messages from the destination back to
the source, this is the 'return path'.

Wire it up for 'socket' QEMUFile's.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add a wrapper to change the blocking status on a QEMUFile
rather than having to use qemu_set_block(qemu_get_fd(f));
it seems best to avoid exposing the fd since not all QEMUFile's
really have one.  With this wrapper we could move the implementation
down to be different on different transports.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

qemu_get_buffer always copies the data it reads to a users buffer,
however in many cases the file buffer inside qemu_file could be given
back to the caller, avoiding the copy.  This isn't always possible
depending on the size and alignment of the data.

Thus 'qemu_get_buffer_in_place' either copies the data to a supplied
buffer or updates a pointer to the internal buffer if convenient.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The coroutine files are currently referenced by the block-obj-y
variable. The coroutine functionality though is already used by
more than just the block code. eg migration code uses coroutine
yield. In the future the I/O channel code will also use the
coroutine yield functionality. Since the coroutine code is nicely
self-contained it can be easily built as part of the libqemuutil.a
library, making it widely available.

The headers are also moved into include/qemu, instead of the
include/block directory, since they are now part of the util
codebase, and the impl was never in the block/ directory
either.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

g_new(T, n) is neater than g_malloc(sizeof(T) * n).  It's also safer,
for two reasons.  One, it catches multiplication overflowing size_t.
Two, it returns T * rather than void *, which lets the compiler catch
more type errors.

This commit only touches allocations with size arguments of the form
sizeof(T).  Same Coccinelle semantic patch as in commit b45c03f5.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1442231491-23352-1-git-send-email-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: Nzhanghailiang <zhang.zhanghailiang@huawei.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

This time convert the external functions:
  qemu_get_buffer, qemu_peek_buffer
  qemu_put_buffer and qemu_put_buffer_async
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1439463094-5394-6-git-send-email-dgilbert@redhat.com>
Reviewed-by: Nzhanghailiang <zhang.zhanghailiang@huawei.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

We need the names of RAMBlocks as they're loaded for RDMA,
reuse a slightly modified ram_control_load_hook:
  a) Pass a 'data' parameter to use for the name in the block-reg
     case
  b) Only some hook types now require the presence of a hook function.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>

qemu_peek_buffer currently copies the data it reads into a buffer,
however a future patch wants access to the buffer without the copy,
hence rework to remove the copy to the layer above.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

and use it in loadvm_state and ram_load.

Where ever it's used, check the return and error if it failed.

Minor: ram_load was using a 257 byte array for its string, the
       maximum length is 255 bytes + 0 terminator, so fix to 256
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

qemu_put_compression_data() compress the data and put it to QEMUFile.
qemu_put_qemu_file() put the data in the buffer of source QEMUFile to
destination QEMUFile.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

It used to be an int, but then we can't pass directly the
bytes_transferred parameter, that would happen later in the series.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>

For ftell we flush the output buffer to ensure that we don't have anything
lingering in our internal buffers. This is a very safe thing to do.

However, with the dynamic size measurement that the dynamic vmstate
description will bring this would turn out quite slow.

Instead, we can fast path this specific measurement and just take the
internal buffers into account when telling the kernel our position.

I'm sure I overlooked some corner cases where this doesn't work, so
instead of tuning the safe, existing version, this patch adds a fast
variant of ftell that gets used by the dynamic vmstate description code
which isn't critical when it fails.
Signed-off-by: NAlexander Graf <agraf@suse.de>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add QEMUFile interface to allow a socket to be 'shut down' - i.e. any
reads/writes will fail (and any blocking read/write will be woken).
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Add a cast in qemu_get_be32() to avoid shifting left into the sign
bit of a signed integer (which is undefined behaviour in C).
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

The splitting of qemu-file and addition of the buffered file landed
at the same time; so now split the buffered file code out.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Now we've got multiple QEMUFile source files, some of them need
access to things that were defined in qemu-file.c, so create
a -internal header for them.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

The migration code now occupies a fair chunk of the top level .c
files, it seems time to give it it's own directory.

I've not touched:
   arch_init.c - that's mostly RAM migration but has a few random other
                 bits
   savevm.c    - because it's built target specific

This is purely a code move; no code has changed.
   - it fails checkpatch because of old violations, it feels safer
     to keep this as purely a move and fix those at some mythical future
     date.

The xbzrle and vmstate tests are now only run for softmmu builds
since they require files in the migrate/ directory which is only built
for softmmu.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Separate the QEMUFile interface from the stdio-specific implementation,
to reduce dependencies from code using QEMUFile.

The code that is being moved is similar to the one that was on savevm.c before
it was moved in commit 093c455a, except for
some changes done by Markus, Juan, and myself. So, I am using the copyright and
license header from savevm.c, but CCing Juan and Markus so they can review the
copyright/license header.

Cc: Markus Armbruster <armbru@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Separate the QEMUFile interface from the implementation, to reduce
dependencies from code using QEMUFile.

All the code that is being moved to the new file is exactly the same
code that was on savevm.c (moved by commit
093c455a), so I am using the copyright
and license header from savevm.c for the new file.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Use the existing function which checks if writev_buffer() or
put_buffer() are set, instead of duplicating it.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The QEMUFileStdio code will use qemu_file_is_writable() and will be
moved to a separate file.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The person who created qemu-file.c (me, on commit
093c455a) didn't add a copyright/license
header to the file, even though the whole code was copied from savevm.c
(which had a copyright/license header).

To correct this, copy the copyright information and license from
savevm.c, that's where the original code came from.

Luckily, very few changes were made on qemu-file.c after it was created.
All the authors who touched the code are being CCed, so they can confirm
if they are OK with the copyright/license information being added.

Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Alexey Kardashevskiy <aik@ozlabs.ru>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Acked-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

This is based on Stefan and Joel's patch that creates a QEMUFile that goes
to a memory buffer; from:

http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg05036.html

Using the QEMUFile interface, this patch adds support functions for
operating on in-memory sized buffers that can be written to or read from.
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>
Signed-off-by: NJoel Schopp <jschopp@linux.vnet.ibm.com>

For fixes/tweeks I've done:
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Make qemu_peek_buffer repeatedly call fill_buffer until it gets
all the data it requires, or until there is an error.

  At the moment, qemu_peek_buffer will try one qemu_fill_buffer if there
  isn't enough data waiting, however the kernel is entitled to return
  just a few bytes, and still leave qemu_peek_buffer with less bytes
  than it needed.  I've seen this fail in a dev world, and I think it
  could theoretically fail in the peeking of the subsection headers in
  the current world.

Comment qemu_peek_byte to point out it's not guaranteed to work for
  non-continuous peeks
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NChenLiang <chenliang0016@icloud.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

This replaces DPRINTF macro with tracepoints.

This moves some messages from migration.c to savevm.c.

This adds tracepoint to signal about fileds failed to migrate.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Reviewers accepted v2 of the patch, but what got committed was v1,
with the R-bys for v2.  This is the v1->v2 followup fix.

[Amit:
 This fixes commit aded6539
]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

fwrite() returns the number of items written.  But when there is one
error, it can return a short write.

In the particular bug that I was tracking, I did a migration to a
read-only filesystem.  And it was able to finish the migration
correctly.  fwrite() never returned a negative error code, nor zero,
always 4096. (migration writes chunks of about 14000 bytes).  And it
was able to "complete" the migration with success (yes, reading the
file was a bit more difficult).

To add insult to injury, if your amount of memory was big enough (12GB
on my case), it overwrote some important structure, and from them,
malloc failed.  This check makes the problem go away.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>
Reviewed-by: NOrit Wasserman <owasserm@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>