Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Move declarations out of qemu-common.h for functions declared in
utils/ files: e.g. include/qemu/path.h for utils/path.c.
Move inline functions out of qemu-common.h and into new files (e.g.
include/qemu/bcd.h)
Signed-off-by: NVeronia Bahaa <veroniabahaa@gmail.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

We don't join the listen thread, it does its own cleanup.
Mark as detached not joinable.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reported-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <1457690016-9070-2-git-send-email-dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

MigrationState is destroyed before we can come into bottom half.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Amit Shah <amit.shah@redhat.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1457537708-8622-1-git-send-email-den@openvz.org>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Migration of pseries-2.3 doesn't have configuration section. Unfortunately,
QEMU 2.4/2.4.1/2.5 are buggy and always stream and expect the configuration
section, and break migration both ways.

This patch introduces a property which allows to enforce a configuration
section for machines who don't have one.

It can be set at startup:

-machine enforce-config-section=on

or later from the QEMU monitor:

qom-set /machine enforce-config-section on

It is up to the tooling to set or unset this property according to the
version of the QEMU at the other end of the pipe.
Signed-off-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Reviewed-by: NLaurent Vivier <lvivier@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>

There is a possibility to hit an assert in qcow2_get_specific_info that
s->qcow_version is undefined. This happens when VM in starting from
suspended state, i.e. it processes incoming migration, and in the same
time 'info block' is called.

The problem is that qcow2_invalidate_cache() closes the image and
memset()s BDRVQcowState in the middle.

The patch moves processing of bdrv_invalidate_cache_all out of
coroutine context for postcopy migration to avoid that. This function
is called with the following stack:
  process_incoming_migration_co
  qemu_loadvm_state
  qemu_loadvm_state_main
  loadvm_process_command
  loadvm_postcopy_handle_run
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Tested-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Amit Shah <amit.shah@redhat.com>
Message-Id: <1456304019-10507-3-git-send-email-den@openvz.org>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

In qemu_savevm_state_complete_precopy(), it iterates on each device to add
a json object and transfer related status to destination, while the order
of the last two steps could be refined.

Current order:

    json_start_object()
    	save_section_header()
    	vmstate_save()
    json_end_object()
    	save_section_footer()

After the change:

    json_start_object()
    	save_section_header()
    	vmstate_save()
    	save_section_footer()
    json_end_object()

This patch reorder the code to to make it symmetric. No functional change.
Signed-off-by: NWei Yang <richard.weiyang@gmail.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Message-Id: <1454626230-16334-1-git-send-email-richard.weiyang@gmail.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

state->name does not contain a terminating '\0' and you may get:

Machine type received is 'pseries-2.3y�?' and local is 'pseries-2.4'
load of migration failed: Invalid argument

Let's add a precision modifier to fix this.
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Message-Id: <20160205083201.2201.76109.stgit@bahia.huguette.org>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

qemu_loadvm_state is too long, and we can simplify it by splitting up
with three helper functions.
Signed-off-by: Nzhanghailiang <zhang.zhanghailiang@huawei.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Message-Id: <1452829066-9764-4-git-send-email-zhang.zhanghailiang@huawei.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Rename the 'file' member of MigrationState to 'to_dst_file' to
be consistent with to_src_file, from_src_file and from_dst_file.
Signed-off-by: Nzhanghailiang <zhang.zhanghailiang@huawei.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Message-Id: <1452829066-9764-3-git-send-email-zhang.zhanghailiang@huawei.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 1453832250-766-2-git-send-email-peter.maydell@linaro.org

Commit 6daf194d, be62a2eb and 312fd5f2 got rid of a bunch, but they
keep coming back.  Tracked down with the Coccinelle semantic patch
from commit 312fd5f2.

Cc: Fam Zheng <famz@redhat.com>
Cc: Peter Crosthwaite <crosthwaitepeter@gmail.com>
Cc: Bharata B Rao <bharata@linux.vnet.ibm.com>
Cc: Dominik Dingel <dingel@linux.vnet.ibm.com>
Cc: David Hildenbrand <dahi@linux.vnet.ibm.com>
Cc: Jason J. Herne <jjherne@linux.vnet.ibm.com>
Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Changchun Ouyang <changchun.ouyang@intel.com>
Cc: zhanghailiang <zhang.zhanghailiang@huawei.com>
Cc: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: NMarkus Armbruster <armbru@pond.sub.org>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Acked-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Acked-by: NBharata B Rao <bharata@linux.vnet.ibm.com>
Acked-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <1450452927-8346-17-git-send-email-armbru@redhat.com>

Both error_reportf_err() and monitor_printf() print to the same
destination when monitor_printf() is used correctly, i.e. within an
HMP monitor.  Elsewhere, monitor_printf() does nothing, while
error_reportf_err() reports to stderr.

Both changed functions are HMP command handlers.  These should only
run within an HMP monitor.

Unlike monitor_printf(), error_reportf_err() uses the error whole
instead of just its message obtained with error_get_pretty().  This
avoids suppressing its hint (see commit 50b7b000), but I don't think
the errors touched in this commit can come with hints.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <1450452927-8346-15-git-send-email-armbru@redhat.com>

Both error_report_err() and monitor_printf() print to the same
destination when monitor_printf() is used correctly, i.e. within an
HMP monitor.  Elsewhere, monitor_printf() does nothing, while
error_report_err() reports to stderr.

Most changed functions are HMP command handlers.  These should only
run within an HMP monitor.  The one exception is bdrv_password_cb(),
which should also only run within an HMP monitor.

Four command handlers prefix the error message with the command name:
balloon, migrate_set_capability, migrate_set_parameter, migrate.
Pointless, drop.

Unlike monitor_printf(), error_report_err() uses the error whole
instead of just its message obtained with error_get_pretty().  This
avoids suppressing its hint (see commit 50b7b000).  Example:

    (qemu) device_add ivshmem,id=666
    Parameter 'id' expects an identifier
    Identifiers consist of letters, digits, '-', '.', '_', starting with a letter.
    Try "help device_add" for more information

The "Identifiers consist of..." line is new with this patch.

Coccinelle semantic patch:

    @@
    expression M, E;
    @@
    -    monitor_printf(M, "%s\n", error_get_pretty(E));
    -    error_free(E);
    +    error_report_err(E);
    @r1@
    expression M, E;
    format F;
    position p;
    @@
    -    monitor_printf(M, "...%@F@\n", error_get_pretty(E));@p
    -    error_free(E);
    +    error_report_err(E);
    @script:python@
	p << r1.p;
    @@
    print "%s:%s:%s: prefix dropped" % (p[0].file, p[0].line, p[0].column)
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <1450452927-8346-4-git-send-email-armbru@redhat.com>

I missed the calls to send migration events on the destination side
as we enter postcopy.
Take care when adding them not to do it after state has been freed.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Nzhanghailiang <zhang.zhanghailiang@huawei.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Message-Id: <1450266458-3178-4-git-send-email-dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

basically all bdrv_* operations must be called under aio_context_acquire
except ones with bdrv_all prefix.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The patch also ensures proper locking for the operation.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

State deletion can be performed on running VM which reduces VM downtime
This approach looks a bit more natural.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

to create snapshot for all loaded block drivers.

The patch also ensures proper locking.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

There is no much sense to do the check and write warning.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

to check that snapshot is available for all loaded block drivers.
The check bs != bs1 in hmp_info_snapshots is an optimization. The check
for availability of this snapshot will return always true as the list
of snapshots was collected from that image.

The patch also ensures proper locking.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

We should check that all inserted and not read-only images support
snapshotting. This could be made using already invented helper
bdrv_all_can_snapshot().
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

to switch to snapshot on all loaded block drivers.

The patch also ensures proper locking.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

to delete snapshots from all loaded block drivers.

The patch also ensures proper locking.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The patch enforces proper locking for this operation.
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
Tested-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Where we have iterable, but non-postcopiable devices (e.g. htab
or block migration), complete them before forming the 'package'
but with the CPUs stopped.  This stops them filling up the package.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

since commit
    commit 94f5a437
    Author: Liang Li <liang.z.li@intel.com>
    Date:   Mon Nov 2 15:37:00 2015 +0800

    migration: defer migration_end & blk_mig_cleanup

when actual .cleanup callbacks calling was removed from complete operations.

The patch fixes regression introduced by the commit above results in
100% reliable assert for virtio-scsi VM with iothreads enabled during
'virsh create-snapshot' operation:
    assert(i != mr->ioeventfd_nb);
    memory_region_del_eventfd
    virtio_pci_set_host_notifier_internal
    virtio_pci_set_host_notifier
    virtio_scsi_dataplane_start
    virtio_scsi_handle_cmd
    virtio_queue_notify_vq
    virtio_queue_host_notifier_read
    aio_dispatch
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NLiang Li <liang.z.li@intel.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Amit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Prior to servicing userfault requests we must ensure we've not got
huge pages in the area that might include non-transferred memory,
since a hugepage could incorrectly mark the whole huge page as present.

We mark the area as non-huge page (nhp) just before we perform
discards; the discard code now tells us to discard any areas
that haven't been sent (as well as any that are redirtied);
any already formed transparent-huge-pages get fragmented
by this discard process if they cotnain any discards.

Transparent huge pages that have been entirely transferred
and don't contain any discards are not broken by this mechanism;
they stay as huge pages.

By starting postcopy after a full precopy pass, many of the pages
then stay as huge pages; this is important for maintaining performance
after the end of the migration.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Wire up more of the handlers for the commands on the destination side,
in particular loadvm_postcopy_handle_run now has enough to start the
guest running.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The loading of a device state (during postcopy) may access guest
memory that's still on the source machine and thus might need
a page fill; split off a separate thread that handles the incoming
page data so that the original incoming migration code can finish
off the device data.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

During the postcopy phase we must not call the iterate method on
precopy-only devices, since they may have done some cleanup during
the _complete call at the end of the precopy phase.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Mark the area of RAM as 'userfault'
Start up a fault-thread to handle any userfaults we might receive
from it (to be filled in later)
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Soon we'll be in either ACTIVE or POSTCOPY_ACTIVE when we
complete migration, and we need to know which we expect to be
in to change state safely.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add qemu_savevm_state_complete_postcopy to complement
qemu_savevm_state_complete_precopy together with a new
save_live_complete_postcopy method on devices.

The save_live_complete_precopy method is called on
all devices during a precopy migration, and all non-postcopy
devices during a postcopy migration at the transition.

The save_live_complete_postcopy method is called at
the end of postcopy for all postcopiable devices.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

VMDescription is normally sent at the end, after all
of the devices; however that's not the end for postcopy,
so just don't send it when in postcopy.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Provide a check to see if the OS we're running on has all the bits
needed for postcopy.

Creates postcopy-ram.c which will get most of the other helpers we need.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Modify save_live_pending to return separate postcopiable and
non-postcopiable counts.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

MIG_CMD_PACKAGED is a migration command that wraps a chunk of migration
stream inside a package whose length can be determined purely by reading
its header.  The destination guarantees that the whole MIG_CMD_PACKAGED
is read off the stream prior to parsing the contents.

This is used by postcopy to load device state (from the package)
while leaving the main stream free to receive memory pages.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The state of the postcopy process is managed via a series of messages;
   * Add wrappers and handlers for sending/receiving these messages
   * Add state variable that track the current state of postcopy
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>