As a pre-step to a single CQ, we unite the error completion
handlers to a single handler.

This patch does not change any functionality.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

It is disabled at the moment, we will get that back
in once the target is more stable.

This reverts commit 95b60f07
"Add support for completion interrupt coalescing"
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Currently we have no way to tell that the target stack is in shutdown
sequence. In case we have open connections, the initiator immediately
attempts to reconnect in a DDOS attack style, so we may end up
terminating the iser enabled network portal while it's np_accept_list
still have pending connections.

The workaround is simply release all the connections in the list.
A proper fix will be to start shutdown sequence by shutting the
network portal to avoid initiator immediate reconnect attempts.

But the temporary work around seems to work at this point, so I think
we can do this for now...
Reported-by: NSlava Shwartsman <valyushash@gmail.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

iSER will report supported protection operations based on
the tpg attribute t10_pi settings and HCA PI offload capabilities.
If the HCA does not support PI offload or tpg attribute t10_pi is
not set, we fall to SW PI mode.

In order to do that, we move iscsit_get_sup_prot_ops after connection
tpg assignment.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.14+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Fallback to software mode DIF if HCA does not support
PI (without crashing obviously). It is still possible to
run with backend protection and an unprotected frontend,
so looking at the command prot_op is not enough. Check
device PI capability on a per-IO basis (isert_prot_cmd
inline static) to determine if we need to handle protection
information.

Trace:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000010
IP: [<ffffffffa037f8b1>] isert_reg_sig_mr+0x351/0x3b0 [ib_isert]
Call Trace:
 [<ffffffff812b003a>] ? swiotlb_map_sg_attrs+0x7a/0x130
 [<ffffffffa038184d>] isert_reg_rdma+0x2fd/0x370 [ib_isert]
 [<ffffffff8108f2ec>] ? idle_balance+0x6c/0x2c0
 [<ffffffffa0382b68>] isert_put_datain+0x68/0x210 [ib_isert]
 [<ffffffffa02acf5b>] lio_queue_data_in+0x2b/0x30 [iscsi_target_mod]
 [<ffffffffa02306eb>] target_complete_ok_work+0x21b/0x310 [target_core_mod]
 [<ffffffff8106ece2>] process_one_work+0x182/0x3b0
 [<ffffffff8106fda0>] worker_thread+0x120/0x3c0
 [<ffffffff8106fc80>] ? maybe_create_worker+0x190/0x190
 [<ffffffff8107594e>] kthread+0xce/0xf0
 [<ffffffff81075880>] ? kthread_freezable_should_stop+0x70/0x70
 [<ffffffff8159a22c>] ret_from_fork+0x7c/0xb0
 [<ffffffff81075880>] ? kthread_freezable_should_stop+0x70/0x70
Reported-by: NSlava Shwartsman <valyushash@gmail.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.14+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts to allocate PI contexts dynamically in order
avoid a potentially bogus np->tpg_np and associated NULL pointer
dereference in isert_connect_request() during iser-target endpoint
shutdown with multiple network portals.

Also, there is really no need to allocate these at connection
establishment since it is not guaranteed that all the IOs on
that connection will be to a PI formatted device.

We can do it in a lazy fashion so the initial burst will have a
transient slow down, but very fast all IOs will allocate a PI
context.

Squashed:

iser-target: Centralize PI context handling code
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.14+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

In situations such as bond failover, The new session establishment
implicitly invokes the termination of the old connection.

So, we don't want to wait for the old connection wait_conn to completely
terminate before we accept the new connection and post a login response.

The solution is to deffer the comp_wait completion and the conn_put to
a work so wait_conn will effectively be non-blocking (flush errors are
assumed to come very fast).

We allocate isert_release_wq with WQ_UNBOUND and WQ_UNBOUND_MAX_ACTIVE
to spread the concurrency of release works.
Reported-by: NSlava Shwartsman <valyushash@gmail.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The np listener cm_id will also get ADDR_CHANGE event
upcall (in case it is bound to a specific IP). Handle
it correctly by creating a new cm_id and implicitly
destroy the old one.

Since this is the second event a listener np cm_id may
encounter, we move the np cm_id event handling to a
routine.

Squashed:

iser-target: Move cma_id setup to a function
Reported-by: NSlava Shwartsman <valyushash@gmail.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Take isert_conn pointer from cm_id->qp->qp_context. This
will allow us to know that the cm_id context is always
the network portal. This will make the cm_id event check
(connection or network portal) more reliable.

In order to avoid a NULL dereference in cma_id->qp->qp_context
we destroy the qp after we destroy the cm_id (and make the
dereference safe). session stablishment/teardown sequences
can happen in parallel, we should take into account that
connected_handler might race with connection teardown flow.

Also, protect isert_conn->conn_device->active_qps decrement
within the error patch during QP creation failure and the
normal teardown path in isert_connect_release().

Squashed:

iser-target: Decrement completion context active_qps in error flow
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

There is no point in accepting a new CM request only
when we are completely done with the last iscsi login.
Instead we accept immediately, this will also cause the
CM connection to reach connected state and the initiator
is allowed to send the first login. We mark that we got
the initial login and let iscsi layer pick it up when it
gets there.

This reduces the parallel login sequence by a factor of
more then 4 (and more for multi-login) and also prevents
the initiator (who does all logins in parallel) from
giving up on login timeout expiration.

In order to support multiple login requests sequence (CHAP)
we call isert_rx_login_req from isert_rx_completion insead
of letting isert_get_login_rx call it.

Squashed:

iser-target: Use kref_get_unless_zero in connected_handler
iser-target: Acquire conn_mutex when changing connection state
iser-target: Reject connect request in failure path
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

ISER_CONN_UP state is not sufficient to know if
we should wait for completion of flush errors and
disconnected_handler event.

Instead, split it to 2 states:
- ISER_CONN_UP: Got to CM connected phase, This state
indicates that we need to wait for a CM disconnect
event before going to teardown.

- ISER_CONN_FULL_FEATURE: Got to full feature phase
after we posted login response, This state indicates
that we posted recv buffers and we need to wait for
flush completions before going to teardown.

Also avoid deffering disconnected handler to a work,
and handle it within disconnected handler.
More work here is needed to handle DEVICE_REMOVAL event
correctly (cleanup all resources).

Squashed:

iser-target: Don't deffer disconnected handler to a work
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Since commit 0fc4ea70 ("Target/iser: Don't put isert_conn inside
disconnected handler") we put the conn kref in isert_wait_conn, so we
need .wait_conn to be invoked also in the error path.

Introduce call to isert_conn_terminate (called under lock)
which transitions the connection state to TERMINATING and calls
rdma_disconnect. If the state is already teminating, just bail
out back (temination started).

Also, make sure to destroy the connection when getting a connect
error event if didn't get to connected (state UP). Same for the
handling of REJECTED and UNREACHABLE cma events.

Squashed:

iscsi-target: Add call to wait_conn in establishment error flow
Reported-by: NSlava Shwartsman <valyushash@gmail.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

isert has an issue of trying to create a CQ with more CQEs than are
supported by the hardware, that currently results in failures during
isert_device creation during first session login.

This is the isert version of the patch that Minh Tran submitted for
iser, and is simple a workaround required to function with existing
ocrdma hardware.
Signed-off-by: NChris Moore <chris.moore@emulex.com>
Reviewied-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # 3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

In this case the cm_id->context is the isert_np, and the cm_id->qp
is NULL, so use that to distinct the cases.

Since we don't expect any other events on this cm_id we can
just return -1 for explicit termination of the cm_id by the
cma layer.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # 3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds a max_send_sge=2 minimum in isert_conn_setup_qp()
to ensure outgoing control PDU responses with tx_desc->num_sge=2
are able to function correctly.

This addresses a bug with RDMA hardware using dev_attr.max_sge=3,
that in the original code with the ConnectX-2 work-around would
result in isert_conn->max_sge=1 being negotiated.

Originally reported by Chris with ocrdma driver.
Reported-by: NChris Moore <Chris.Moore@emulex.com>
Tested-by: NChris Moore <Chris.Moore@emulex.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Cc: <stable@vger.kernel.org> # 3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

It is not guaranteed to that srp_sq_size is supported
by the HCA. So if we failed to create the QP with ENOMEM,
try with a smaller srp_sq_size. Keep it up until we hit
MIN_SRPT_SQ_SIZE, then fail the connection.
Reported-by: NMark Lehrer <lehrer@gmail.com>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # 3.4+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This fixes ntuple calculation for IPv6 active open request for T5
adapter.  And also removes an duplicate line which got added in commit
92e7ae71 ("iw_cxgb4: Choose appropriate hw mtu index and ISS for
iWARP connections")
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

best_mtu and set_emss were not considering ipv6 header for ipv6 case.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

This fixes a sparse warning.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

In ib_uverbs_open_qp, the sharable xrc target qp is created as a
"pseudo" qp and added to a list of qp's sharing the same physical
QP.  This is done before the "pseudo" qp is assigned a uobject.

There is a race condition here if an async event arrives at the
physical qp.  If the event is handled after the pseudo qp is added to
the list, but before it is assigned a uobject, the kernel crashes in
ib_uverbs_qp_event_handler, due to trying to dereference a NULL
uobject pointer.

Note that simply checking for non-NULL is not enough, due to error
flows in ib_uverbs_open_qp.  If the failure is after assigning the
uobject, but before the qp has fully been created, we still have a
problem.

Thus, in ib_uverbs_qp_event_handler, we test that the uobject is
present, and also that it is live.
Reported-by: NMatthew Finlay <matt@mellanox.com>
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

During create-ah from userspace, uverbs is sending garbage data in
attr.dmac and attr.vlan_id.  This patch sets attr.dmac and
attr.vlan_id to zero.

Fixes: dd5f03be ("IB/core: Ethernet L2 attributes in verbs/cm structures")
Signed-off-by: NDevesh Sharma <devesh.sharma@emulex.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

The kernel used to contain two functions for length-delimited,
case-insensitive string comparison, strnicmp with correct semantics and
a slightly buggy strncasecmp.  The latter is the POSIX name, so strnicmp
was renamed to strncasecmp, and strnicmp made into a wrapper for the new
strncasecmp to avoid breaking existing users.

To allow the compat wrapper strnicmp to be removed at some point in the
future, and to avoid the extra indirection cost, do
s/strnicmp/strncasecmp/g.
Signed-off-by: NRasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Roland Dreier <roland@kernel.org>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Parenthesis around constants serves no purpose, save the bits!
Signed-off-by: NJes Sorensen <Jes.Sorensen@redhat.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

No need to re-invent the wheel here
Signed-off-by: NJes Sorensen <Jes.Sorensen@redhat.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Get rid of obfuscating ocrdma_alloc_mqe() kzalloc() wrapper as all it
did was to make it less visible that the structure was already cleared
on allocation.
Signed-off-by: NJes Sorensen <Jes.Sorensen@redhat.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

If IPV6 is disabled, we get the warning:

    drivers/infiniband/hw/ocrdma/ocrdma_main.c:650:1: warning: label ‘err_notifier6’ defined but not used [-Wunused-label]
Signed-off-by: NLi RongQing <roy.qing.li@gmail.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

In some platforms, when iommu is enabled, the bus address returned by
dma_alloc_coherent is different than the physical address.  ocrdma
should use physical address for mmap-ing the queue memory for the
applications.

This patch adds the use of virt_to_phys() at all such places where
kernel buffer is mapped to user process context.
Signed-off-by: NDevesh Sharma <devesh.sharma@emulex.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

After IP-based GID changes, VLAN id can be obtained from
qp_attr->vlan_id.
Signed-off-by: NDevesh Sharma <devesh.sharma@emulex.com>
Signed-off-by: NSelvin Xavier <selvin.xavier@emulex.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Signed-off-by: NDevesh Sharma <devesh.sharma@emulex.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Expose more signature setting parameters. We modify the signature API
to allow usage of some new execution parameters relevant to data
integrity feature.

This patch modifies ib_sig_domain structure by:

- Deprecate DIF type in signature API (operation will
  be determined by the parameters alone, no DIF type awareness)
- Add APPTAG check bitmask (for input domain)
- Add REFTAG remap (increment) flag for each domain
- Add APPTAG/REFTAG escape options for each domain

The mlx5 driver is modified to follow the new parameters in HW
signature setup.

At the moment the callers (iser/isert) hard-code new parameters (by
DIF type). In the future, callers will retrieve them from the scsi
command structure.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Later there will be more parameters to set, so we want to do it in a
centralized place.

This patch does not change any functionality.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Later there will be more parameters to set, so we want to do it in a
centralized place.

This patch does not change any functionality.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Rather than using the basic BSF layout which utilizes a pre-configured
signature settings (sufficient for current DIF implementation), we use
the extended BSF layout to expose advanced signature settings. These
settings will also be exposed to the user later.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

In the future this will be a per-command parameter so we can lose it,
but in the mean time IP_CSUM is a lot lighter for SW layers to
compute, set it as default.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

We clear the struct before - no need to do 0 assignment.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

In case input and output space parameters match, we can use a copy
mask from input and output space.  Use enums for those.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

When dealing with umem objects, the driver assumed host page sizes
defined by PAGE_SHIFT.  Modify the code to use arbitrary page shift
provided by umem->page_shift to support different page sizes.
Signed-off-by: NYishai Hadas <yishaih@mellanox.com>
Signed-off-by: NEli Cohen <eli@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Some of the fields were set twice. Re-organize to avoid that.
Signed-off-by: NEli Cohen <eli@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

The check to verify that userspace does not provide an invalid index to the
micro UAR was placed too late. Fix this by moving the check before using the
index.

Reported by: Shachar Raindel <raindel@mellanox.com>
Signed-off-by: NEli Cohen <eli@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>