Fix bug that creates a gap in the syncing blocks

When transitioning from the LIB sync mode to the mode where net_plugin catches up to the head of a remote peers best known branch, we calculate the branch to send based on the peers local LIB which could have shifted since the time the request was made due to latency.  If this it the first time we are synchronizing with that branch this means that there could be a gap in delivered blocks resulting in the blocks sent being unlinkable.   The more latency on a connection the more likely it is that the LIB of your block provider moved forward between the packets.

As a fix, we now send, at most, the branch from the LIB we last sent to that peer instead of the current LIB.  In addition, if that peer has reported its head id and we are building on top of that branch, we also squelch the delivery of blocks they already know about.

plugins/net_plugin/net_plugin.cpp

@@ -852,10 +852,25 @@ namespace eosio {
       }
       block_id_type head_id;
       block_id_type lib_id;
-      uint32_t lib_num;
+      block_id_type remote_head_id;
+      uint32_t lib_num = 0;
+      uint32_t remote_head_num = 0;
       try {
+         if (last_handshake_recv.generation >= 1) {
+            remote_head_id = last_handshake_recv.head_id;
+            remote_head_num = block_header::num_from_id(remote_head_id);
+            fc_dlog(logger, "maybe truncating branch at  = ${h}:${id}",("h",remote_head_num)("id",remote_head_id));
+         }
+
+         // base our branch off of the last handshake we sent the peer instead of our current
+         // LIB which could have moved forward in time as packets were in flight.
+         if (last_handshake_sent.generation >= 1) {
+            lib_num = last_handshake_sent.last_irreversible_block_num;
+            lib_id = last_handshake_sent.last_irreversible_block_id;
+         } else {
             lib_num = cc.last_irreversible_block_num();
             lib_id = cc.last_irreversible_block_id();
+         }
          head_id = cc.fork_db_head_block_id();
       }
       catch (const assert_exception &ex) {
@@ -872,6 +887,13 @@ namespace eosio {
       block_id_type null_id;
       for (auto bid = head_id; bid != null_id && bid != lib_id; ) {
          try {
+
+            // if the last handshake received indicates that we are catching up on a fork
+            // that the peer is already partially aware of, no need to resend blocks
+            if (block_header::num_from_id(bid) == remote_head_num && remote_head_id == bid) {
+               break;
+            }
+
             signed_block_ptr b = cc.fetch_block_by_id(bid);
             if ( b ) {
                bid = b->previous;
@@ -886,7 +908,7 @@ namespace eosio {
       }
       size_t count = 0;
       if (!bstack.empty()) {
-         if (bstack.back()->previous == lib_id) {
+         if (bstack.back()->previous == lib_id || bstack.back()->previous == remote_head_id) {
             count = bstack.size();
             while (bstack.size()) {
                enqueue(*bstack.back());