Use the order of incoming RAMBlocks from the source to record
an index number; that then allows us to sort the destination
local RAMBlock list to match the source.

Now that the RAMBlocks are known to be in the same order, this
simplifies the RDMA Registration step which previously tried to
match RAMBlocks based on offset (which isn't guaranteed to match).

Looking at the existing compress code, I think it was erroneously
relying on an assumption of matching ordering, which this fixes.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

RDMA uses a hash from block offset->RAM Block; this isn't needed
on the destination, and it becomes harder to maintain after the next
patch in the series that sorts the block list.

Split the hash so that it's only generated on the source.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

In the next patch we remove the hash on the destination,
rdma_delete_block does two things with the hash which can be avoided:
  a) The caller passes the offset and rdma_delete_block looks it up
     in the hash; fixed by getting the caller to pass the block
  b) The hash gets recreated after deletion; fixed by making that
     conditional on the hash being initialised.

While this function is currently only used during cleanup, Michael
asked that we keep it general for future dynamic block registration
work.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

We need the names of RAMBlocks as they're loaded for RDMA,
reuse a slightly modified ram_control_load_hook:
  a) Pass a 'data' parameter to use for the name in the block-reg
     case
  b) Only some hook types now require the presence of a hook function.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The 'offset' field in RDMACompress and 'current_addr' field
in RDMARegister are commented as being offsets within a particular
RAMBlock, however they appear to actually be offsets within the
ram_addr_t space.

The code currently assumes that the offsets on the source/destination
match, this change removes the need for the assumption for these
structures by translating the addresses into the ram_addr_t space of
the destination host.

Note: An alternative would be to change the fields to actually
take the data they're commented for; this would potentially be
simpler but would break stream compatibility for those cases
that currently work.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

In a later patch the block name will be used to match up two views
of the block list.  Keep a copy of the block name with the local block
list.

(At some point it could be argued that it would be best just to let
migration see the innards of RAMBlock and avoid the need to use
foreach).
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

A couple of typo fixes.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Variable "r" going out of scope leaks the storage
it points to in line 3268.
Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NLuiz Capitulino <lcapitulino@redhat.com>

Done with following Coccinelle semantic patch, plus manual cosmetic changes in
net/*.c.

    @@
    expression E1, E2, E3, E4;
    @@
    -   qemu_set_fd_handler2(E1, NULL, E2, E3, E4);
    +   qemu_set_fd_handler(E1, E2, E3, E4);
Signed-off-by: NFam Zheng <famz@redhat.com>
Message-id: 1433400324-7358-8-git-send-email-famz@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

RDMA has two data types that are named confusingly;
   RDMALocalBlock (pointed to indirectly by local_ram_blocks)
   RDMARemoteBlock (pointed to by block in RDMAContext)

RDMALocalBlocks, as the name suggests is a data strucuture that
represents the RDMAable RAM Blocks on the current side of the migration
whichever that is.

RDMARemoteBlocks is always the shape of the RAMBlocks on the
destination, even on the destination.

Rename:
     RDMARemoteBlock -> RDMADestBlock
     context->'block' -> context->dest_blocks
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

check the return value of the function it calls and error if it's non-0
Fixup qemu_rdma_init_one_block that is the only current caller,
  and rdma_add_block the only function it calls using it.

Pass the name of the ramblock to the function; helps in debugging.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Reviewed-by: NMichael R. Hines <mrhines@us.ibm.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Qemu crashes when IPv6 address is specified for migration and access
to any RDMA uverbs device available on the system is blocked using cgroups.
Fix the crash by checking the return value of ibv_open_device routine.
Signed-off-by: NMeghana Cheripady <meghana.cheripady@avagotech.com>
Signed-off-by: NPadmanabh Ratnakar <padmanabh.ratnakar@avagotech.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

As part of commit e325b49a,
order in which resources are destroyed was changed for fixing
a seg fault. Due to this change, CQ will never get destroyed as
CQ should be destroyed after QP destruction. Seg fault is caused
improper cleanup when connection fails. Fixing cleanup after
connection failure and order in which resources are destroyed
in qemu_rdma_cleanup() routine.
Signed-off-by: NMeghana Cheripady <meghana.cheripady@emulex.com>
Signed-off-by: NPadmanabh Ratnakar <padmanabh.ratnakar@emulex.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Fix type casts between pointers and 64 bit integers.
Now 32 bit builds are possible again.
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The current code won't compile on 32 bit hosts because there are lots
of type casts between pointers and 64 bit integers.

Fix some of them.
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Do not check for rdma->host being empty twice.  This removes a large
"if" block, so code indentation is changed.  While at it, remove an
ugly goto from the loop, replacing it with a cleaner if logic.  And
finally, there's no need to initialize `ret' variable since is always
has a value.
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Tested-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

--

fixed space detected by Dave
Signed-off-by: NJuan Quintela <quintela@redhat.com>

It used to be an int, but then we can't pass directly the
bytes_transferred parameter, that would happen later in the series.
Signed-off-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>

* Remove trailing whitespace (fixes 9 errors from checkpatch.pl).
  One comment line was longer than 80 characters, so wrap it
  and fix a typo, too.
* Replace tabs by blanks (fixes 1 error).
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

It looks like the dtrace trace code gets upset if you have trace names
with __ in, which the migration/rdma.c code does.

Rename the functions and the associated traces.

Fixes: 733252deSigned-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reported-by: NAndreas Färber <afaerber@suse.de>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Tested-by: NAndreas Färber <afaerber@suse.de>
Message-id: 1424105885-12149-1-git-send-email-dgilbert@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Warnings from the Sparse static analysis tool:

migration-rdma.c:151:12: warning:
 symbol 'wrid_desc' was not declared. Should it be static?
migration-rdma.c:190:12: warning:
 symbol 'control_desc' was not declared. Should it be static?
migration-rdma.c:3301:19: warning:
 symbol 'rdma_read_ops' was not declared. Should it be static?
migration-rdma.c:3308:19: warning:
 symbol 'rdma_write_ops' was not declared. Should it be static?

Cc: Juan Quintela <quintela@redhat.com>
Cc: Amit Shah <amit.shah@redhat.com>
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NGonglei <arei.gonglei@huawei.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Turn all the D/DD/DDDPRINTFs into trace events
Turn most of the fprintf(stderr, into error_report
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The general feeling is that having migration/migration-blah
is overkill.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

The migration code now occupies a fair chunk of the top level .c
files, it seems time to give it it's own directory.

I've not touched:
   arch_init.c - that's mostly RAM migration but has a few random other
                 bits
   savevm.c    - because it's built target specific

This is purely a code move; no code has changed.
   - it fails checkpatch because of old violations, it feels safer
     to keep this as purely a move and fix those at some mythical future
     date.

The xbzrle and vmstate tests are now only run for softmmu builds
since they require files in the migrate/ directory which is only built
for softmmu.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Since we have supported memory hotplug, VM's ram include pc.ram
and hotplug-memory.

Fix the confused description for rdma migration: pc.ram -> VM's ram
Signed-off-by: Nzhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

1. Fix small memory leak in parsing inet address from command line in data_init()
2. Fix ibv_post_send() return value check and pass error code back up correctly.
3. Fix rdma_destroy_qp() segfault after failure to connect to destination.

Reported-by: frank.yangjie@gmail.com
Reported-by: dgilbert@redhat.com
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

If the networking break or there's something wrong with rdma
device(ib0 with no IP) during rdma migration, the main_loop of
qemu will be blocked in rdma_destroy_id. I add rdma_ack_cm_event
to fix this bug.
Signed-off-by: NMo Yuxiang <Moyuxiang@huawei.com>
Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Reviewed-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

As far as we can tell, all known bugs have been fixed:

1. Parallel migrations are working
2. IPv6 migration is working
3. virt-test is working

I'm not comfortable sending the revised libvirt patch
until this is accepted or review suggestions are addressed,
(including pin-all support. It does not make sense to
remove experimental for one thing and not the other. That's
too many trips through the libvirt community).
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Signed-off-by: NIsaku Yamahata <yamahata@private.email.ne.jp>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

- It can't be determined by RDMAContext::cm_id != NULL if the connection
  is established or not.
- RDMAContext::cm_id is leaked and not destroyed because it is set to NULL
  too early.
- RDMAContext::qp is created by rdma_create_qp() so that it should be destroyed
  by rdma_destroy_qp(). not ibv_destroy_qp()

Cc: Michael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NIsaku Yamahata <yamahata@private.email.ne.jp>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

My bad - but it's very important for us to warn the user that
IPv6 is broken on RoCE in linux right now, until linux releases
a fixed version.
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Most typos were found using a modified version of codespell:

accross -> across
issueing -> issuing
TICNT_THRESHHOLD -> TICNT_THRESHOLD
bandwith -> bandwidth
VCARD_7816_PROPIETARY -> VCARD_7816_PROPRIETARY
occured -> occurred
gaurantee -> guarantee
sofware -> software
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

We've gotten reports from multiple testers (including Frank Yangjie
and myself) that RDMA IPv6 support over RocE (Ethernet) is broken
in linux.

A patch to Linux is still in review:

http://comments.gmane.org/gmane.linux.drivers.rdma/16448

If the user is listening on '[::]', then we will not have a opened a device
yet and have no way of verifying if the device is RoCE or not.

In this case, the source VM will throw an error for ALL types of
connections (both IPv4 and IPv6) if the destination machine does not have
a regular infiniband network available for use.

The only way to gaurantee that an error is thrown for broken kernels is
for the management software to choose a *specific* interface at bind time
and validate what time of hardware it is.

Unfortunately, this puts the user in a fix:

 If the source VM connects with an IPv4 address without knowing that the
 destination has bound to '[::]' the migration will unconditionally fail
 unless the management software is not explicitly listening on the the IPv4
 address while using a RoCE-based device.

 If the source VM connects with an IPv6 address, then we're OK because we can
 throw an error on the source (and similarly on the destination).

 But in mixed environments, this will be broken for a while until it is fixed
 inside linux.

We do provide a *tiny* bit of help in mixed environments, though in this patch:

We can list all of the devices in the system and check to see if all the
devices are RoCE or Infiniband.

If we detect that we have a *pure* RoCE environment, then we can safely
thrown an error even if the management sofware has specified '[::]' as the
bind address.

However, if there is are multiple hetergeneous devices, then we cannot make
this assumption and the user just has to be sure they know what they are doing.
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Message-id: 1376078746-24948-6-git-send-email-mrhines@linux.vnet.ibm.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

getaddrinfo() already knows what it's doing,
but it can potentially return multiple addresses.
We need to handle that...
Reviewed-by: NOrit Wasserman <owasserm@redhat.com>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Message-id: 1376078746-24948-5-git-send-email-mrhines@linux.vnet.ibm.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

RDMAControlHeader::len is provided from remote, so check if the value
match the actual transferred byte_len.
Reviewed-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NIsaku Yamahata <yamahata@private.email.ne.jp>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Message-id: 1376078746-24948-4-git-send-email-mrhines@linux.vnet.ibm.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

RMDAControlHeader::len is provided from remote, so validate it.
Reviewed-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NIsaku Yamahata <yamahata@private.email.ne.jp>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Message-id: 1376078746-24948-3-git-send-email-mrhines@linux.vnet.ibm.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

resp.len is given from remote host. So should be validated before use.
Otherwise memcpy can access beyond the buffer.

Cc: Michael R. Hines <mrhines@us.ibm.com>
Reviewed-by: NOrit Wasserman <owasserm@redhat.com>
Reviewed-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NIsaku Yamahata <yamahata@private.email.ne.jp>
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Message-id: 1376078746-24948-2-git-send-email-mrhines@linux.vnet.ibm.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>