Put all connection IB related resources release in this routine.  One
exception is the cm_id which cannot be destroyed as the routine is
protected by the state mutex.  Also move its position to avoid forward
declaration.  While at it fix qp NULL assignment.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Signed-off-by: NRoi Dayan <roid@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Structure that describes the RDMA relates connection objects.  Static
member of iser_conn.

This patch does not change any functionality
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Two reasons why we choose to do this:

1. No point today calling struct iser_conn by another name ib_conn
2. In the next patches we will restructure iser control plane representation
   - struct iser_conn: connection logical representation
   - struct ib_conn: connection RDMA layout representation

This patch does not change any functionality.
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Signed-off-by: NRoi Dayan <roid@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

We need to fail the bind operation if the iser connection state != UP
(started teardown) and this should be done under the state lock.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

When failing to allocate TX CQ we already allocated RX CQ, so we need to make
sure we release it. Also, when failing to register notification to the RX CQ
we currently leak both RX and TX CQs of the current index, fix that too.
Signed-off-by: NRoi Dayan <roid@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

This is to prevent someone from thinking that this code section is
redundant.
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NRoi Dayan <roid@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Instead of waiting for events and condition changes of the iser
connection state, we wait for explicit completion of connection
establishment and teardown.

Separate connection establishment wait object from the teardown object
to avoid a situation where racing connection establishment and
teardown may concurrently wakeup each other.

ep_poll will wait for up_completion invoked by
iser_connected_handler() and iser release worker will wait for
flush_completion before releasing the connection.

Bound the completion wait with a 30 seconds timeout for cases where
iscsid (the user space iscsi daemon) is too slow or gone.
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

The iser connection state lookups and transitions are not fully protected.

Some transitions are protected with a spinlock, and in some cases the
state is accessed unprotected due to specific assumptions of the flow.

Introduce a new mutex to protect the connection state access. We use a
mutex since we need to also include a scheduling operations executed
under the state lock.

Each state transition/condition and its corresponding action will be
protected with the state mutex.

The rdma_cm events handler acquires the mutex when handling connection
events. Since iser connection state can transition to DOWN
concurrently during connection establishment, we bailout from
addr/route resolution events when the state is not PENDING.

This addresses a scenario where ep_poll retries expire during CMA
connection establishment. In this case ep_disconnect is invoked while
CMA events keep coming (address/route resolution, connected, etc...).
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Make it void.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

iser connection needs asynchronous cleanup completions which are
triggered in ep_disconnect.  As a result we are keeping the
corresponding iscsi_endpoint structure hanging for no good reason. In
order to avoid that, we seperate iser_conn from iscsi_endpoint storage
space to have their destruction being independent.

iscsi_endpoint will be destroyed at ep_disconnect stage, while the
iser connection will wait for asynchronous completions to be released
in an orderly fashion.
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NRoi Dayan <roid@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

The iser initiator is the RDMA responder so it should publish to the
target the max inflight rdma read requests its local HCA can handle in
responder_resources (max_qp_rd_atom).

The iser target should take the min of that and its local HCA max
inflight oustanding rdma read requests (max_qp_init_rd_atom).

We keep initiator_depth set to 1 in order to compat with old targets.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

In case the DISCONNECTED event is not delivered after rdma_disconnect
is called, the CM waits TIMEWAIT seconds and delivers the
TIMEWAIT_EXIT local event. We use this as the notification needed to
continue in the teardown and release sequence.
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NRoi Dayan <roid@mellanox.com>
Reviewed-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Replace struct sockaddr_in with struct sockaddr which supports both
IPv4 and IPv6, and print using the %pIS format directive.
Signed-off-by: NRoi Dayan <roid@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

In case protection information exists over the wire
iscsi header data length is required to include it.
Use protection information aware scsi helpers to set
the correct transfer length.

In order to avoid breakage, remove iser transfer length
checks for each task as they are not always true and
somewhat redundant anyway.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Reviewed-by: NMike Christie <michaelc@cs.wisc.edu>
Acked-by: NMike Christie <michaelc@cs.wisc.edu>
Cc: stable@vger.kernel.org # 3.15+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Logging messages need terminating newlines to avoid possible message
interleaving.  Add them.
Signed-off-by: NRoi Dayan <roid@mellanox.com>
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

In some circumstances (multiple targets), RDMA_CM ESTABLISHED event
and ep_disconnect may race. In this case, the iser connection state
may transition to UP (after ep_disconnect transitioned it to
TERMINATING), while the connection is being torn down.

Upon RDMA_CM event ESTABLISHED we allow iser connection state to
transition to UP only from PENDING. We also make sure to protect this
state change (done under the connection lock).
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Reviewed-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

iSER relies on refcounting to manage iser connections establishment
and teardown.

Following commit 39ff05db ("IB/iser: Enhance disconnection logic
for multi-pathing"), iser connection maintain 3 references:

 - iscsi_endpoint (at creation stage)
 - cma_id (at connection request stage)
 - iscsi_conn (at bind stage)

We can avoid taking explicit refcounts by correctly serializing iser
teardown flows (graceful and non-graceful).

Our approach is to trigger a scheduled work to handle ordered teardown
by gracefully waiting for 2 cleanup stages to complete:

 1. Cleanup of live pending tasks indicated by iscsi_conn_stop completion
 2. Flush errors processing

Each completed stage will notify a waiting worker thread when it is
done to allow teardwon continuation.

Since iSCSI connection establishment may trigger endpoint disconnect
without a successful endpoint connect, we rely on the iscsi <-> iser
binding (.conn_bind) to learn about the teardown policy we should take
wrt cleanup stages.

Since all cleanup worker threads are scheduled (release_wq) in
.ep_disconnect it is safe to assume that when module_exit is called,
all cleanup workers are already scheduled. Thus proper module unload
shall flush all scheduled works before allowing safe exit, to
guarantee no resources got left behind.
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Reviewed-by: NRoi Dayan <roid@mellanox.com>
Reviewed-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Update Mellanox copyrights for 2014 on the iser initiator driver.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Add an iser info print with the local/remote QP information carried
out when the connection is established.  While here, fix a little
leftover from the T10 work and set a debug print to be carried in
debug and not info level.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

The iscsi stack has existing mechanisms to link back and forth between
the iscsi connection and the iscsi transport (e.g iser/tcp) connection.

This is done through a dd_data pointer field in struct iscsi_conn
which can be set to point to the transport connection, etc.

The iscsi_iser_conn structure was used to get this linking done in
another way, which is uneeded and adds extra complication to the iser
code, so we just remove it.
Signed-off-by: NAriel Nahum <arieln@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

The iser disconnection flow isn't done before all the inflight
recv/send buffers posted to the QP are either flushed or normally
completed to the CQ that serves this connection.  The condition check
is done in iser_handle_comp_error().

Currently, it's possible for the send buffer completion that makes the
posted send buffers counter reach zero to be polled in the drain tx
call, which is after the rx cq is fully drained.  Since this
completion might be not an error one (for example, it might be a
completion of the logout request iSCSI PDU) we will skip
iser_handle_comp_error().  So the connection will never terminate from
the iscsi stack point of view, and we hang.

To resolve this race, do the draining of the tx cq before the loop on
the rx cq.
Signed-off-by: NRoi Dayan <roid@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Fix pr_err (printk) format warning:

    drivers/infiniband/ulp/iser/iser_verbs.c:1181:4: warning: format '%lx' expects argument of type 'long unsigned int', but argument 3 has type 'sector_t' [-Wformat]
Signed-off-by: NRandy Dunlap <rdunlap@infradead.org>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

After allocating a scsi_host we set protection types and guard type
supported.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Once the iSCSI transaction is completed we must implement
check_protection in order to notify on DIF errors that may have
occured.

The routine boils down to calling ib_check_mr_status to get the
signature status of the transaction.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Add logic to initialize protection information entities.  Upon each
iSCSI task, we keep the scsi_cmnd in order to query the scsi
protection operations and reference to protection buffers.

Modify iser_fast_reg_mr to receive indication whether it is
registering the data or protection buffers.

In addition introduce iser_reg_sig_mr which performs fast registration
work-request for a signature enabled memory region
(IB_WR_REG_SIG_MR).  In this routine we set all the protection
relevants for the device to offload protection data-transfer and
verification.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

During connection establishment we also initialize T10-PI resources
(QP, PI contexts) in order to support SCSI's protection operations.
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Use modparams to activate protection information support.

pi_enable bool: Based on this parameter iSER will know if it should
support T10-PI.  We don't want to do this by default as it requires to
allocate and initialize extra resources.  In case pi_enable=N, iSER
won't publish to SCSI midlayer any DIF capabilities.

pi_guard int: Based on this parameter iSER will publish DIX guard type
support to SCSI midlayer.  0 means CRC is allowed to be passed in DIX
buffers, 1 (or non-zero) means IP-CSUM is allowed to be passed in DIX
buffers.  Note that over the wire, only CRC is allowed.

In the next phase, it is worth considering passing these parameters
from iscsid via nlmsg.  This will allow these parameters to be
connection based rather than global.
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Unaligned SG-lists may also happen for protection information.
Generalize bounce buffer routine to handle any iser_data_buf which may
be data and/or protection.

This patch does not change any functionality.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

This routines operates on data buffers and may also work with
protection infomation buffers.  So we generalize them to handle an
iser_data_buf which can be the command data or command protection
information.

This patch does not change any functionality.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

In T10-PI support we will have memory keys for protection buffers and
signature transactions.  We prefer to compact indicators rather than
keeping multiple bools.

This commit does not change any functionality.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

For T10-PI offload support, we will need to know the device signature
offload capability upon every connection establishment.

This patch does not change any functionality.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

fastreg descriptor will include protection information context.  In
order to place the logic in one place we introduce iser_create_fr_desc
function.

This patch does not change any functionality.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

This is a preparation step for T10-PI offload support.  We prefer to
push the desicion of which mkey to use (global or fastreg) to
iser_fast_reg_mr.  We choose to do this since in T10-PI we may need to
register for protection buffers and in this case we wish to simplify
iser_fast_reg_mr instead of repeating the logic of which key to use.

This patch does not change any functionality.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

FRWR stands for "fast registration work request". We want to avoid
calling the fastreg pool with that name, instead we name it fastreg
which stands for "fast registration".

This pool will include more elements in the future, so it is a good
idea to generalize the name.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NAlex Tabachnik <alext@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

In case iSER uses fast registration method, it should not request for
successful completions on fast registration nor local invalidate
requests.  We color wr_id with ISER_FRWR_LI_WRID in order to correctly
consume error completions.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

We use "tx_desc" again after we free it.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Acked-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>