1. 06 7月, 2012 19 次提交
    • S
      libceph: set peer name on con_open, not init · b7a9e5dd
      Sage Weil 提交于
      The peer name may change on each open attempt, even when the connection is
      reused.
      Signed-off-by: NSage Weil <sage@inktank.com>
      b7a9e5dd
    • A
      libceph: drop declaration of ceph_con_get() · 26103021
      Alex Elder 提交于
      For some reason the declaration of ceph_con_get() and
      ceph_con_put() did not get deleted in this commit:
          d59315ca libceph: drop ceph_con_get/put helpers and nref member
      
      Clean that up.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      26103021
    • A
      libceph: add some fine ASCII art · bc18f4b1
      Alex Elder 提交于
      Sage liked the state diagram I put in my commit description so
      I'm putting it in with the code.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      bc18f4b1
    • A
      libceph: small changes to messenger.c · 5821bd8c
      Alex Elder 提交于
      This patch gathers a few small changes in "net/ceph/messenger.c":
        out_msg_pos_next()
          - small logic change that mostly affects indentation
        write_partial_msg_pages().
          - use a local variable trail_off to represent the offset into
            a message of the trail portion of the data (if present)
          - once we are in the trail portion we will always be there, so we
            don't always need to check against our data position
          - avoid computing len twice after we've reached the trail
          - get rid of the variable tmpcrc, which is not needed
          - trail_off and trail_len never change so mark them const
          - update some comments
        read_partial_message_bio()
          - bio_iovec_idx() will never return an error, so don't bother
            checking for it
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      5821bd8c
    • A
      libceph: distinguish two phases of connect sequence · 7593af92
      Alex Elder 提交于
      Currently a ceph connection enters a "CONNECTING" state when it
      begins the process of (re-)connecting with its peer.  Once the two
      ends have successfully exchanged their banner and addresses, an
      additional NEGOTIATING bit is set in the ceph connection's state to
      indicate the connection information exhange has begun.  The
      CONNECTING bit/state continues to be set during this phase.
      
      Rather than have the CONNECTING state continue while the NEGOTIATING
      bit is set, interpret these two phases as distinct states.  In other
      words, when NEGOTIATING is set, clear CONNECTING.  That way only
      one of them will be active at a time.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      7593af92
    • A
      libceph: separate banner and connect writes · ab166d5a
      Alex Elder 提交于
      There are two phases in the process of linking together the two ends
      of a ceph connection.  The first involves exchanging a banner and
      IP addresses, and if that is successful a second phase exchanges
      some detail about each side's connection capabilities.
      
      When initiating a connection, the client side now queues to send
      its information for both phases of this process at the same time.
      This is probably a bit more efficient, but it is slightly messier
      from a layering perspective in the code.
      
      So rearrange things so that the client doesn't send the connection
      information until it has received and processed the response in the
      initial banner phase (in process_banner()).
      
      Move the code (in the (con->sock == NULL) case in try_write()) that
      prepares for writing the connection information, delaying doing that
      until the banner exchange has completed.  Move the code that begins
      the transition to this second "NEGOTIATING" phase out of
      process_banner() and into its caller, so preparing to write the
      connection information and preparing to read the response are
      adjacent to each other.
      
      Finally, preparing to write the connection information now requires
      the output kvec to be reset in all cases, so move that into the
      prepare_write_connect() and delete it from all callers.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      ab166d5a
    • A
      libceph: define and use an explicit CONNECTED state · e27947c7
      Alex Elder 提交于
      There is no state explicitly defined when a ceph connection is fully
      operational.  So define one.
      
      It's set when the connection sequence completes successfully, and is
      cleared when the connection gets closed.
      
      Be a little more careful when examining the old state when a socket
      disconnect event is reported.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      e27947c7
    • A
      libceph: clear NEGOTIATING when done · 3ec50d18
      Alex Elder 提交于
      A connection state's NEGOTIATING bit gets set while in CONNECTING
      state after we have successfully exchanged a ceph banner and IP
      addresses with the connection's peer (the server).  But that bit
      is not cleared again--at least not until another connection attempt
      is initiated.
      
      Instead, clear it as soon as the connection is fully established.
      Also, clear it when a socket connection gets prematurely closed
      in the midst of establishing a ceph connection (in case we had
      reached the point where it was set).
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      3ec50d18
    • A
      libceph: clear CONNECTING in ceph_con_close() · bb9e6bba
      Alex Elder 提交于
      A connection that is closed will no longer be connecting.  So
      clear the CONNECTING state bit in ceph_con_close().  Similarly,
      if the socket has been closed we no longer are in connecting
      state (a new connect sequence will need to be initiated).
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      bb9e6bba
    • A
      libceph: don't touch con state in con_close_socket() · 456ea468
      Alex Elder 提交于
      In con_close_socket(), a connection's SOCK_CLOSED flag gets set and
      then cleared while its shutdown method is called and its reference
      gets dropped.
      
      Previously, that flag got set only if it had not already been set,
      so setting it in con_close_socket() might have prevented additional
      processing being done on a socket being shut down.  We no longer set
      SOCK_CLOSED in the socket event routine conditionally, so setting
      that bit here no longer provides whatever benefit it might have
      provided before.
      
      A race condition could still leave the SOCK_CLOSED bit set even
      after we've issued the call to con_close_socket(), so we still clear
      that bit after shutting the socket down.  Add a comment explaining
      the reason for this.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      456ea468
    • A
      libceph: just set SOCK_CLOSED when state changes · d65c9e0b
      Alex Elder 提交于
      When a TCP_CLOSE or TCP_CLOSE_WAIT event occurs, the SOCK_CLOSED
      connection flag bit is set, and if it had not been previously set
      queue_con() is called to ensure con_work() will get a chance to
      handle the changed state.
      
      con_work() atomically checks--and if set, clears--the SOCK_CLOSED
      bit if it was set.  This means that even if the bit were set
      repeatedly, the related processing in con_work() only gets called
      once per transition of the bit from 0 to 1.
      
      What's important then is that we ensure con_work() gets called *at
      least* once when a socket close event occurs, not that it gets
      called *exactly* once.
      
      The work queue mechanism already takes care of queueing work
      only if it is not already queued, so there's no need for us
      to call queue_con() conditionally.
      
      So this patch just makes it so the SOCK_CLOSED flag gets set
      unconditionally in ceph_sock_state_change().
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      d65c9e0b
    • A
      libceph: don't change socket state on sock event · 188048bc
      Alex Elder 提交于
      Currently the socket state change event handler records an error
      message on a connection to distinguish a close while connecting from
      a close while a connection was already established.
      
      Changing connection information during handling of a socket event is
      not very clean, so instead move this assignment inside con_work(),
      where it can be done during normal connection-level processing (and
      under protection of the connection mutex as well).
      
      Move the handling of a socket closed event up to the top of the
      processing loop in con_work(); there's no point in handling backoff
      etc. if we have a newly-closed socket to take care of.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      188048bc
    • A
      libceph: SOCK_CLOSED is a flag, not a state · a8d00e3c
      Alex Elder 提交于
      The following commit changed it so SOCK_CLOSED bit was stored in
      a connection's new "flags" field rather than its "state" field.
      
          libceph: start separating connection flags from state
          commit 928443cd
      
      That bit is used in con_close_socket() to protect against setting an
      error message more than once in the socket event handler function.
      
      Unfortunately, the field being operated on in that function was not
      updated to be "flags" as it should have been.  This fixes that
      error.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      a8d00e3c
    • A
      libceph: don't use bio_iter as a flag · abdaa6a8
      Alex Elder 提交于
      Recently a bug was fixed in which the bio_iter field in a ceph
      message was not being properly re-initialized when a message got
      re-transmitted:
          commit 43643528
          Author: Yan, Zheng <zheng.z.yan@intel.com>
          rbd: Clear ceph_msg->bio_iter for retransmitted message
      
      We are now only initializing the bio_iter field when we are about to
      start to write message data (in prepare_write_message_data()),
      rather than every time we are attempting to write any portion of the
      message data (in write_partial_msg_pages()).  This means we no
      longer need to use the msg->bio_iter field as a flag.
      
      So just don't do that any more.  Trust prepare_write_message_data()
      to ensure msg->bio_iter is properly initialized, every time we are
      about to begin writing (or re-writing) a message's bio data.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      abdaa6a8
    • A
      libceph: move init of bio_iter · 572c588e
      Alex Elder 提交于
      If a message has a non-null bio pointer, its bio_iter field is
      initialized in write_partial_msg_pages() if this has not been done
      already.  This is really a one-time setup operation for sending a
      message's (bio) data, so move that initialization code into
      prepare_write_message_data() which serves that purpose.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      572c588e
    • A
      libceph: move init_bio_*() functions up · df6ad1f9
      Alex Elder 提交于
      Move init_bio_iter() and iter_bio_next() up in their source file so
      the'll be defined before they're needed.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      df6ad1f9
    • A
      libceph: don't mark footer complete before it is · fd154f3c
      Alex Elder 提交于
      This is a nit, but prepare_write_message() sets the FOOTER_COMPLETE
      flag before the CRC for the data portion (recorded in the footer)
      has been completely computed.  Hold off setting the complete flag
      until we've decided it's ready to send.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      fd154f3c
    • A
      libceph: encapsulate advancing msg page · 84ca8fc8
      Alex Elder 提交于
      In write_partial_msg_pages(), once all the data from a page has been
      sent we advance to the next one.  Put the code that takes care of
      this into its own function.
      
      While modifying write_partial_msg_pages(), make its local variable
      "in_trail" be Boolean, and use the local variable "msg" (which is
      just the connection's current out_msg pointer) consistently.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      84ca8fc8
    • A
      libceph: encapsulate out message data setup · 739c905b
      Alex Elder 提交于
      Move the code that prepares to write the data portion of a message
      into its own function.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      739c905b
  2. 22 6月, 2012 2 次提交
  3. 19 6月, 2012 1 次提交
  4. 16 6月, 2012 3 次提交
  5. 07 6月, 2012 4 次提交
  6. 06 6月, 2012 11 次提交
    • A
      libceph: make ceph_con_revoke_message() a msg op · 8921d114
      Alex Elder 提交于
      ceph_con_revoke_message() is passed both a message and a ceph
      connection.  A ceph_msg allocated for incoming messages on a
      connection always has a pointer to that connection, so there's no
      need to provide the connection when revoking such a message.
      
      Note that the existing logic does not preclude the message supplied
      being a null/bogus message pointer.  The only user of this interface
      is the OSD client, and the only value an osd client passes is a
      request's r_reply field.  That is always non-null (except briefly in
      an error path in ceph_osdc_alloc_request(), and that drops the
      only reference so the request won't ever have a reply to revoke).
      So we can safely assume the passed-in message is non-null, but add a
      BUG_ON() to make it very obvious we are imposing this restriction.
      
      Rename the function ceph_msg_revoke_incoming() to reflect that it is
      really an operation on an incoming message.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      8921d114
    • A
      libceph: make ceph_con_revoke() a msg operation · 6740a845
      Alex Elder 提交于
      ceph_con_revoke() is passed both a message and a ceph connection.
      Now that any message associated with a connection holds a pointer
      to that connection, there's no need to provide the connection when
      revoking a message.
      
      This has the added benefit of precluding the possibility of the
      providing the wrong connection pointer.  If the message's connection
      pointer is null, it is not being tracked by any connection, so
      revoking it is a no-op.  This is supported as a convenience for
      upper layers, so they can revoke a message that is not actually
      "in flight."
      
      Rename the function ceph_msg_revoke() to reflect that it is really
      an operation on a message, not a connection.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      6740a845
    • A
      libceph: have messages take a connection reference · 92ce034b
      Alex Elder 提交于
      There are essentially two types of ceph messages: incoming and
      outgoing.  Outgoing messages are always allocated via ceph_msg_new(),
      and at the time of their allocation they are not associated with any
      particular connection.  Incoming messages are always allocated via
      ceph_con_in_msg_alloc(), and they are initially associated with the
      connection from which incoming data will be placed into the message.
      
      When an outgoing message gets sent, it becomes associated with a
      connection and remains that way until the message is successfully
      sent.  The association of an incoming message goes away at the point
      it is sent to an upper layer via a con->ops->dispatch method.
      
      This patch implements reference counting for all ceph messages, such
      that every message holds a reference (and a pointer) to a connection
      if and only if it is associated with that connection (as described
      above).
      
      
      For background, here is an explanation of the ceph message
      lifecycle, emphasizing when an association exists between a message
      and a connection.
      
      Outgoing Messages
      An outgoing message is "owned" by its allocator, from the time it is
      allocated in ceph_msg_new() up to the point it gets queued for
      sending in ceph_con_send().  Prior to that point the message's
      msg->con pointer is null; at the point it is queued for sending its
      message pointer is assigned to refer to the connection.  At that
      time the message is inserted into a connection's out_queue list.
      
      When a message on the out_queue list has been sent to the socket
      layer to be put on the wire, it is transferred out of that list and
      into the connection's out_sent list.  At that point it is still owned
      by the connection, and will remain so until an acknowledgement is
      received from the recipient that indicates the message was
      successfully transferred.  When such an acknowledgement is received
      (in process_ack()), the message is removed from its list (in
      ceph_msg_remove()), at which point it is no longer associated with
      the connection.
      
      So basically, any time a message is on one of a connection's lists,
      it is associated with that connection.  Reference counting outgoing
      messages can thus be done at the points a message is added to the
      out_queue (in ceph_con_send()) and the point it is removed from
      either its two lists (in ceph_msg_remove())--at which point its
      connection pointer becomes null.
      
      Incoming Messages
      When an incoming message on a connection is getting read (in
      read_partial_message()) and there is no message in con->in_msg,
      a new one is allocated using ceph_con_in_msg_alloc().  At that
      point the message is associated with the connection.  Once that
      message has been completely and successfully read, it is passed to
      upper layer code using the connection's con->ops->dispatch method.
      At that point the association between the message and the connection
      no longer exists.
      
      Reference counting of connections for incoming messages can be done
      by taking a reference to the connection when the message gets
      allocated, and releasing that reference when it gets handed off
      using the dispatch method.
      
      We should never fail to get a connection reference for a
      message--the since the caller should already hold one.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      92ce034b
    • A
      libceph: have messages point to their connection · 38941f80
      Alex Elder 提交于
      When a ceph message is queued for sending it is placed on a list of
      pending messages (ceph_connection->out_queue).  When they are
      actually sent over the wire, they are moved from that list to
      another (ceph_connection->out_sent).  When acknowledgement for the
      message is received, it is removed from the sent messages list.
      
      During that entire time the message is "in the possession" of a
      single ceph connection.  Keep track of that connection in the
      message.  This will be used in the next patch (and is a helpful
      bit of information for debugging anyway).
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      38941f80
    • A
      libceph: tweak ceph_alloc_msg() · 1c20f2d2
      Alex Elder 提交于
      The function ceph_alloc_msg() is only used to allocate a message
      that will be assigned to a connection's in_msg pointer.  Rename the
      function so this implied usage is more clear.
      
      In addition, make that assignment inside the function (again, since
      that's precisely what it's intended to be used for).  This allows us
      to return what is now provided via the passed-in address of a "skip"
      variable.  The return type is now Boolean to be explicit that there
      are only two possible outcomes.
      
      Make sure the result of an ->alloc_msg method call always sets the
      value of *skip properly.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      1c20f2d2
    • A
      libceph: fully initialize connection in con_init() · 1bfd89f4
      Alex Elder 提交于
      Move the initialization of a ceph connection's private pointer,
      operations vector pointer, and peer name information into
      ceph_con_init().  Rearrange the arguments so the connection pointer
      is first.  Hide the byte-swapping of the peer entity number inside
      ceph_con_init()
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      1bfd89f4
    • A
      libceph: init monitor connection when opening · 20581c1f
      Alex Elder 提交于
      Hold off initializing a monitor client's connection until just
      before it gets opened for use.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      20581c1f
    • S
      libceph: drop connection refcounting for mon_client · ec87ef43
      Sage Weil 提交于
      All references to the embedded ceph_connection come from the msgr
      workqueue, which is drained prior to mon_client destruction.  That
      means we can ignore con refcounting entirely.
      Signed-off-by: NSage Weil <sage@newdream.net>
      Reviewed-by: NAlex Elder <elder@inktank.com>
      ec87ef43
    • A
      libceph: embed ceph connection structure in mon_client · 67130934
      Alex Elder 提交于
      A monitor client has a pointer to a ceph connection structure in it.
      This is the only one of the three ceph client types that do it this
      way; the OSD and MDS clients embed the connection into their main
      structures.  There is always exactly one ceph connection for a
      monitor client, so there is no need to allocate it separate from the
      monitor client structure.
      
      So switch the ceph_mon_client structure to embed its
      ceph_connection structure.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      67130934
    • S
      libceph: use con get/put ops from osd_client · 0d47766f
      Sage Weil 提交于
      There were a few direct calls to ceph_con_{get,put}() instead of the con
      ops from osd_client.c.  This is a bug since those ops aren't defined to
      be ceph_con_get/put.
      
      This breaks refcounting on the ceph_osd structs that contain the
      ceph_connections, and could lead to all manner of strangeness.
      
      The purpose of the ->get and ->put methods in a ceph connection are
      to allow the connection to indicate it has a reference to something
      external to the messaging system, *not* to indicate something
      external has a reference to the connection.
      
      [elder@inktank.com: added that last sentence]
      Signed-off-by: NSage Weil <sage@newdream.net>
      Reviewed-by: NAlex Elder <elder@inktank.com>
      0d47766f
    • A
      libceph: osd_client: don't drop reply reference too early · ab8cb34a
      Alex Elder 提交于
      In ceph_osdc_release_request(), a reference to the r_reply message
      is dropped.  But just after that, that same message is revoked if it
      was in use to receive an incoming reply.  Reorder these so we are
      sure we hold a reference until we're actually done with the message.
      Signed-off-by: NAlex Elder <elder@inktank.com>
      Reviewed-by: NSage Weil <sage@inktank.com>
      ab8cb34a