Using kfifos for ID management was limiting the number of QPs and
preventing NP384 MPI jobs.  So replace it with a simple bitmap
allocator.

Remove IDs from the IDR tables before deallocating them.  This bug was
causing the BUG_ON() in insert_handle() to fire because the ID was
getting reused before being removed from the IDR table.
Signed-off-by: NVipul Pandya <vipul@chelsio.com>
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

This allows dumping thousands of QPs.  Log active open failures of
interest.
Signed-off-by: NVipul Pandya <vipul@chelsio.com>
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Add module option db_fc_threshold which is the count of active QPs
that trigger automatic db flow control mode.  Automatically transition
to/from flow control mode when the active qp count crosses
db_fc_theshold.

Add more db debugfs stats

On DB DROP event from the LLD, recover all the iwarp queues.
Signed-off-by: NVipul Pandya <vipul@chelsio.com>
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Use GFP_ATOMIC in _insert_handle() if ints are disabled.

Don't panic if we get an abort with no endpoint found.  Just log a
warning.
Signed-off-by: NVipul Pandya <vipul@chelsio.com>
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Get FULL/EMPTY/DROP events from LLD.  On FULL event, disable normal
user mode DB rings.

Add modify_qp semantics to allow user processes to call into the
kernel to ring doobells without overflowing.

Add DB Full/Empty/Drop stats.

Mark queues when created indicating the doorbell state.

If we're in the middle of db overflow avoidance, then newly created
queues should start out in this mode.

Bump the C4IW_UVERBS_ABI_VERSION to 2 so the user mode library can
know if the driver supports the kernel mode db ringing.
Signed-off-by: NVipul Pandya <vipul@chelsio.com>
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Signed-off-by: NVipul Pandya <vipul@chelsio.com>
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Log a warning and drop the abort message.  Otherwise we will do a
bogus wake_up() and crash.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

This fixes a race where an ingress abort fails to wake up the thread
blocked in rdma_init() causing the app to hang.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Function import_ep() is incorrectly using ep->dst instead of the dst
ptr passed in.  This causes a crash when accepting new rdma connections
becase ep->dst is not initialized yet.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

If the call to mlx4_MAD_IFC() fails in ib_link_query_port() we will
currently do 'return err;' which will leak 'in_mad' and 'out_mad'.  We
should instead do 'goto out;' where we'll properly free the memory we
previously allocated.
Signed-off-by: NJesper Juhl <jj@chaosbits.net>
Acked-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Commit 0b307043 ("IB/mad: Return error response for unsupported
MADs") does not failed MADs (eg those that return
IB_MAD_RESULT_FAILURE) properly -- these MADs should be silently
discarded. (We should not force the lower-layer drivers to return
SUCCESS | CONSUMED in this case, since the MAD is NOT successful).
Unsupported MADs are not failures -- they return SUCCESS, but with an
"unsupported error" status value inside the response MAD.
Reviewed-by: NHal Rosenstock <hal@mellanox.com>
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Commit 0b307043 ("IB/mad: Return error response for unsupported
MADs") does not handle directed-route MADs properly -- it fails to set
the 'D' bit in the response MAD status field.  This is a problem for
SmInfo MADs when the receiver does not have an SM running.
Reviewed-by: NHal Rosenstock <hal@mellanox.com>
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Since commit 96104eda ("RDMA/core: Add SRQ type field"), kernel
users of SRQs need to specify srq_type = IB_SRQT_BASIC in struct
ib_srq_init_attr, or else most low-level drivers will fail in
when srpt_add_one() calls ib_create_srq() and gets -ENOSYS.

(mlx4_ib works OK nearly all of the time, because it just needs
srq_type != IB_SRQT_XRC.  And apparently nearly everyone using
ib_srpt is using mlx4 hardware)
Reported-by: NAlexey Shvetsov <alexxy@gentoo.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Commit e9319b0c ("IB/core: Fix SDR rates in sysfs") changed our
sysfs rate attribute to return EINVAL to userspace if the underlying
device driver returns an invalid rate.  Apparently some drivers do this
when the link is down and some userspace pukes if it gets an error when
reading this attribute, so avoid a regression by not return an error to
match the old code.
Signed-off-by: NRoland Dreier <roland@purestorage.com>

When the IB port is down, the active_speed value returned by the
MAD_IFC command is seven (7) which isn't among the defined IB speeds
in enum ib_port_speed, and this invalid speed value is passed up to
higher layers or applications who do port query.

Fix that by setting the speed to be SDR -- the lowest possible -- when
the port is down.
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Remove all #inclusions of asm/system.h preparatory to splitting and killing
it.  Performed with the following command:

perl -p -i -e 's!^#\s*include\s*<asm/system[.]h>.*\n!!' `grep -Irl '^#\s*include\s*<asm/system[.]h>' *`
Signed-off-by: NDavid Howells <dhowells@redhat.com>

Acked-by: NRoland Dreier <roland@purestorage.com>
Signed-off-by: NCong Wang <amwang@redhat.com>

This patch addresses a bug in srpt_handle_cmd() failure handling where
send_ioctx->kref is being leaked with the local extra reference after init,
causing the expected kref_put() in srpt_handle_send_comp() to not be the final
call to invoke srpt_put_send_ioctx_kref() -> transport_generic_free_cmd() and
perform se_cmd descriptor memory release.

It also fixes a SCF_SCSI_RESERVATION_CONFLICT handling bug where this code
is incorrectly falling through to transport_handle_cdb_direct() after
invoking srpt_queue_status() to send SAM_STAT_RESERVATION_CONFLICT status.

Note this patch is for >= v3.3 mainline code, and current lio-core.git
code has already been converted to target_submit_cmd() + se_cmd->cmd_kref usage,
and internal ioctx->kref usage has been removed.  I'm including this patch
now into target-pending/for-next with a CC' for v3.3 stable.

Cc: Bart Van Assche <bvanassche@acm.org>
Cc: Roland Dreier <roland@purestorage.com>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

To issue a port query, use the QUERY_(Ethernet)_PORT command instead
of the MAD_IFC command, since MAD_IFC attempts to query the firmware
IB SMA, which is irrelevant for IBoE ports.

This allows us to handle both 10Gb/s and 40Gb/s rates (e.g in sysfs),
using QDR speed (10Gb/s) and width of 1X or 4X.
Signed-off-by: NDotan Barak <dotanb@mellanox.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

If an erroneous CQE is polled in the first iteration (i.e. npolled ==
0), we don't update the consumer index and hence the hardware could
get a wrong notion of how many CQEs software polled.  Fix this by
unconditionally updating the doorbell record.  We could change the
check to be something like

	if (npolled || err != -EAGAIN)
		...

but it does not seem worth the effort since a posted write to memory
should not cost too much.
Signed-off-by: NEli Cohen <eli@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

This patch drops the following unused legacy API callers from target_core_fabric.h:

*) TFO->fall_back_to_erl0()
*) TFO->stop_session()
*) TFO->sess_logged_in()
*) TFO->is_state_remove()

This patch also removes the stub usage in loopback, tcm_fc, iscsi_target,
and ib_srpt fabric modules.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Use a bit in wc_flags rather then a whole integer to hold the
"checksum OK" flag.  By itself, this change doesn't reduce the size of
struct ib_wc on 64bit machines -- it stays on 56 bytes because of
padding.  However, it will allow to add more fields in the future
without enlarging the struct.  Also, it will let us have a unified
approach with future libibverbs checksum offload reporting, because a
bit flag doesn't break the library ABI.

This patch was suggested during conversation with Liran Liss
<liranl@mellanox.com>.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Reviewed-by: NSean Hefty <sean.hefty@intel.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

When destroying a listening cmid, the iwcm first marks the state of
the cmid as DESTROYING, then releases the lock and calls into the
iWARP provider to destroy the endpoint.  Since the cmid is not locked,
its possible for the iWARP provider to pass a connection request event
to the iwcm, which will be silently dropped by the iwcm.  This causes
the iWARP provider to never free up the resources from this connection
because the assumption is the iwcm will accept or reject this connection.

The solution is to reject these connection requests.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Since flush_qp() is always called with irqs disabled, all the locking
inside flush_qp() and __flush_qp() doesn't need irq save/restore.

Further, passing the flag variable from iwch_modify_qp() is just wrong
and causes a WARN_ON() in local_bh_enable().
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

While doing the work for commit a6f7feae ("IB/mlx4: pass SMP
vendor-specific attribute MADs to firmware") we realized that the
firmware would respond on all sorts of vendor-specific MADs.
Therefore commit 97285b78 ("mlx4_core: Add extended port
capabilities support") adds redundant code into the driver, since
there's no real reaon to maintain the extended capabilities of the
port, as they can be queried on demand (e.g the FDR10 capability).

This patch reverts commit 97285b78 and removes the check for
extended caps from the mlx4_ib driver port query flow.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

When we destroy a cm_id, we must purge associated events from the
event queue.  If the cm_id is for a listen request, we also purge
corresponding pending connect requests.  This requires destroying
the cm_id's associated with the connect requests by calling
rdma_destroy_id().  rdma_destroy_id() blocks until all outstanding
callbacks have completed.

The issue is that we hold file->mut while purging events from the
event queue.  We also acquire file->mut in our event handler.  Calling
rdma_destroy_id() while holding file->mut can lead to a deadlock,
since the event handler callback cannot acquire file->mut, which
prevents rdma_destroy_id() from completing.

Fix this by moving events to purge from the event queue to a temporary
list.  We can then release file->mut and call rdma_destroy_id()
outside of holding any locks.

Bug report by Or Gerlitz <ogerlitz@mellanox.com>:

    [ INFO: possible circular locking dependency detected ]
    3.3.0-rc5-00008-g79f1e43-dirty #34 Tainted: G          I

    tgtd/9018 is trying to acquire lock:
     (&id_priv->handler_mutex){+.+.+.}, at: [<ffffffffa0359a41>] rdma_destroy_id+0x33/0x1f0 [rdma_cm]

    but task is already holding lock:
     (&file->mut){+.+.+.}, at: [<ffffffffa02470fe>] ucma_free_ctx+0xb6/0x196 [rdma_ucm]

    which lock already depends on the new lock.


    the existing dependency chain (in reverse order) is:

    -> #1 (&file->mut){+.+.+.}:
           [<ffffffff810682f3>] lock_acquire+0xf0/0x116
           [<ffffffff8135f179>] mutex_lock_nested+0x64/0x2e6
           [<ffffffffa0247636>] ucma_event_handler+0x148/0x1dc [rdma_ucm]
           [<ffffffffa035a79a>] cma_ib_handler+0x1a7/0x1f7 [rdma_cm]
           [<ffffffffa0333e88>] cm_process_work+0x32/0x119 [ib_cm]
           [<ffffffffa03362ab>] cm_work_handler+0xfb8/0xfe5 [ib_cm]
           [<ffffffff810423e2>] process_one_work+0x2bd/0x4a6
           [<ffffffff810429e2>] worker_thread+0x1d6/0x350
           [<ffffffff810462a6>] kthread+0x84/0x8c
           [<ffffffff81369624>] kernel_thread_helper+0x4/0x10

    -> #0 (&id_priv->handler_mutex){+.+.+.}:
           [<ffffffff81067b86>] __lock_acquire+0x10d5/0x1752
           [<ffffffff810682f3>] lock_acquire+0xf0/0x116
           [<ffffffff8135f179>] mutex_lock_nested+0x64/0x2e6
           [<ffffffffa0359a41>] rdma_destroy_id+0x33/0x1f0 [rdma_cm]
           [<ffffffffa024715f>] ucma_free_ctx+0x117/0x196 [rdma_ucm]
           [<ffffffffa0247255>] ucma_close+0x77/0xb4 [rdma_ucm]
           [<ffffffff810df6ef>] fput+0x117/0x1cf
           [<ffffffff810dc76e>] filp_close+0x6d/0x78
           [<ffffffff8102b667>] put_files_struct+0xbd/0x17d
           [<ffffffff8102b76d>] exit_files+0x46/0x4e
           [<ffffffff8102d057>] do_exit+0x299/0x75d
           [<ffffffff8102d599>] do_group_exit+0x7e/0xa9
           [<ffffffff8103ae4b>] get_signal_to_deliver+0x536/0x555
           [<ffffffff81001717>] do_signal+0x39/0x634
           [<ffffffff81001d39>] do_notify_resume+0x27/0x69
           [<ffffffff81361c03>] retint_signal+0x46/0x83

    other info that might help us debug this:

     Possible unsafe locking scenario:

           CPU0                    CPU1
           ----                    ----
      lock(&file->mut);
                                   lock(&id_priv->handler_mutex);
                                   lock(&file->mut);
      lock(&id_priv->handler_mutex);

     *** DEADLOCK ***

    1 lock held by tgtd/9018:
     #0:  (&file->mut){+.+.+.}, at: [<ffffffffa02470fe>] ucma_free_ctx+0xb6/0x196 [rdma_ucm]

    stack backtrace:
    Pid: 9018, comm: tgtd Tainted: G          I  3.3.0-rc5-00008-g79f1e43-dirty #34
    Call Trace:
     [<ffffffff81029e9c>] ? console_unlock+0x18e/0x207
     [<ffffffff81066433>] print_circular_bug+0x28e/0x29f
     [<ffffffff81067b86>] __lock_acquire+0x10d5/0x1752
     [<ffffffff810682f3>] lock_acquire+0xf0/0x116
     [<ffffffffa0359a41>] ? rdma_destroy_id+0x33/0x1f0 [rdma_cm]
     [<ffffffff8135f179>] mutex_lock_nested+0x64/0x2e6
     [<ffffffffa0359a41>] ? rdma_destroy_id+0x33/0x1f0 [rdma_cm]
     [<ffffffff8106546d>] ? trace_hardirqs_on_caller+0x11e/0x155
     [<ffffffff810654b1>] ? trace_hardirqs_on+0xd/0xf
     [<ffffffffa0359a41>] rdma_destroy_id+0x33/0x1f0 [rdma_cm]
     [<ffffffffa024715f>] ucma_free_ctx+0x117/0x196 [rdma_ucm]
     [<ffffffffa0247255>] ucma_close+0x77/0xb4 [rdma_ucm]
     [<ffffffff810df6ef>] fput+0x117/0x1cf
     [<ffffffff810dc76e>] filp_close+0x6d/0x78
     [<ffffffff8102b667>] put_files_struct+0xbd/0x17d
     [<ffffffff8102b5cc>] ? put_files_struct+0x22/0x17d
     [<ffffffff8102b76d>] exit_files+0x46/0x4e
     [<ffffffff8102d057>] do_exit+0x299/0x75d
     [<ffffffff8102d599>] do_group_exit+0x7e/0xa9
     [<ffffffff8103ae4b>] get_signal_to_deliver+0x536/0x555
     [<ffffffff810654b1>] ? trace_hardirqs_on+0xd/0xf
     [<ffffffff81001717>] do_signal+0x39/0x634
     [<ffffffff8135e037>] ? printk+0x3c/0x45
     [<ffffffff8106546d>] ? trace_hardirqs_on_caller+0x11e/0x155
     [<ffffffff810654b1>] ? trace_hardirqs_on+0xd/0xf
     [<ffffffff81361803>] ? _raw_spin_unlock_irq+0x2b/0x40
     [<ffffffff81039011>] ? set_current_blocked+0x44/0x49
     [<ffffffff81361bce>] ? retint_signal+0x11/0x83
     [<ffffffff81001d39>] do_notify_resume+0x27/0x69
     [<ffffffff8118a1fe>] ? trace_hardirqs_on_thunk+0x3a/0x3f
     [<ffffffff81361c03>] retint_signal+0x46/0x83
Signed-off-by: NSean Hefty <sean.hefty@intel.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

I'm getting compile failures building this driver, which I narrowed
down to the ilog2 call in ehca_get_max_hwpage_size...

    ERROR: ".____ilog2_NaN" [drivers/infiniband/hw/ehca/ib_ehca.ko]
    undefined!
    make[1]: *** [__modpost] Error 1
    make: *** [modules] Error 2

The use of shca->hca_cap_mr_pgsize is confusing the compiler, and
resulting in the __builtin_constant_p in ilog2 going insane.

I tried making it take the u32 pgsize as an argument and the expansion
of shca->_pgsize in the caller, but that failed as well.

With this patch in place, the driver compiles on my GCC 4.6.2 here.
Suggested-by: NRoland Dreier <roland@purestorage.com>
Signed-off-by: NKyle McMartin <kmcmarti@redhat.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

The kernel IB stack uses one enumeration for IB speed, which wasn't
explicitly specified in the verbs header file.  Add that enum, and use
it all over the code.

The IB speed/width notation is also used by iWARP and IBoE HW drivers,
which use the convention of rate = speed * width to advertise their
port link rate.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

An iser target may send iscsi NO-OP PDUs as soon as it marks the iSER
iSCSI session as fully operative.  This means that there is window
where there are no posted receive buffers on the initiator side, so
it's possible for the iSER RC connection to break because of RNR NAK /
retry errors.  To fix this, rely on the flags bits in the login
request to have FFP (0x3) in the lower nibble as a marker for the
final login request, and post an initial chunk of receive buffers
before sending that login request instead of after getting the login
response.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

We allocate the login dma buffers in iser_verbs.c as part of
alloc_ib_conn_resources(), however we are freeing them in
iser_initiator.c as part of iser_free_rx_descriptors().  This is
needlessly confusing.  We have an alloc_rx_descriptors() and it
doesn't alloc something that the free_rx_descriptors() frees, and we
have an alloc_ib_conn_resources() that allocs something not freed by
free_ib_conn_resources().  Clean that up.
Signed-off-by: NDoug Ledford <dledford@redhat.com>

[ Fix build error in iser_free_ib_conn_res().  - Or ]
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Remove sysfs attributes before removing a target instead of testing
the target state in every sysfs attribute callback method. Note: it is
safe to invoke a sysfs attribute removal method like
device_remove_file() twice on the same attribute.
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NDavid Dillow <dillowda@ornl.gov>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Use pr_fmt() and pr_xxx() instead of more verbose printk() equivalents.
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NDavid Dillow <dillowda@ornl.gov>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Commit 71eeba16 ("IB: Add new InfiniBand link speeds") introduced a bug 
where eg the rate for IB 4X SDR links iss displayed as "8.5 Gb/sec" 
instead of "10 Gb/sec" as it used to be.  Fix that.
Reported-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Davem considers that the argument list of this interface is getting
out of control. This patch tries to address this issue following
his proposal:

struct netlink_dump_control c = { .dump = dump, .done = done, ... };

netlink_dump_start(..., &c);

Suggested by David S. Miller.
Signed-off-by: NPablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

ConnectX devices have a limit on the number of mappings that can be
done on an FMR before having to call sync_tpt.  The current
mlx4_ib driver reports the limit correctly in max_map_per_fmr in
.query_device(), but mlx4_core doesn't check it when actually
allocating FMRs.

Add a max_fmr_maps field to struct mlx4_caps and enforce this maximum
value on FMR allocations.
Signed-off-by: NEli Cohen <eli@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

If the opcode of a work request exceeds the range of valid opcodes,
return the pointer to the offending work request.
Signed-off-by: NEli Cohen <eli@mellanox.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Signed-off-by: NMasanari Iida <standby24x7@gmail.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Set up a response with appropriate error status and send it for MADs
that are not supported by a specific class/version.
Reviewed-by: NHal Rosenstock <hal@mellanox.com>
Signed-off-by: NSwapna Thete <swapna.thete@qlogic.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

Since create_comp_task() creates percpu kthread, it makes sense to use
kthread_create_on_node() to get proper NUMA affinity for kthread
stack.
Signed-off-by: NEric Dumazet <eric.dumazet@gmail.com>
Acked-by: NHoang-Nam Nguyen <hnguyen@de.ibm.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>

The current get portinfo handling filters the LID being sent,
changing zero to 0xffff.

This causes OpenSM to log excessive warning messages.
Reviewed-by: NEdward Mascarenhas <edward.mascarenhas@qlogic.com>
Signed-off-by: NMike Marciniszyn <mike.marciniszyn@qlogic.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>