Make the rxe and rdmavt drivers use dma_virt_ops. Update the
comments that refer to the source files removed by this patch.
Remove struct ib_dma_mapping_ops. Remove ib_device.dma_ops.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Cc: Andrew Boyer <andrew.boyer@dell.com>
Cc: Dennis Dalessandro <dennis.dalessandro@intel.com>
Cc: Jonathan Toppins <jtoppins@redhat.com>
Cc: Alex Estrin <alex.estrin@intel.com>
Cc: Leon Romanovsky <leonro@mellanox.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Prepare for removal of ib_device.dma_device.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Additionally, make it easier to detect skb leaks by issuing a warning
if a leak occurs.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Cc: Andrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Neither rxe->ifc_ops nor any of the function pointers in struct
struct rxe_ifc_ops ever change. Hence remove the rxe->ifc_ops
indirection mechanism.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Avoid that calling check_rkey() for mem->state == RXE_MEM_STATE_FREE
triggers an MR reference leak.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Change do_complete() such that an error completion is not only
generated if a QP is in the error state but also if a work request
failed.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

This change makes the code easier to read and avoids that code is
duplicated.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Since index values equal to or above 'range' can trigger memory
corruption, complain if index >= range.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

It is strongly recommended to report kernel warnings once instead
of every time a condition is hit. Hence change WARN_ON() into
WARN_ON_ONCE() / BUILD_BUG_ON() as appropriate.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Change the argument type of these functions from void * into
struct rxe_pool_entry *.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Let the compiler check the type of the arguments passed to SKB_TO_PKT()
and PKT_TO_SKB().
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Casting a pointer to 'void *' explicitly is not necessary in C code.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The variable 'av' is not used so remove it. Since that change
removes the last user of the 'wqe' argument, remove that argument
too.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Cc: Andrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Avoid that sparse complains about using 0 as a pointer, about
missing function declarations and also avoid that sparse complains
about endianness.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NAndrew Boyer <andrew.boyer@dell.com>
Cc: Moni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

pkt->qp was already dereferenced earlier in the function.

Fixes Smatch complaint:
drivers/infiniband/sw/rxe/rxe_net.c:458 send()
	 warn: variable dereferenced before check 'pkt->qp' (see line 441)
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

If the completer is in the middle of a large read operation, one
lost packet can cause havoc. Going to COMPST_ERROR_RETRY will
cause the requester to resend the request. After that, any packet
from the first attempt still in the receive queue will be
interpreted as an error, restarting the error/retry sequence.
The transfer will quickly exhaust its retries.

This behavior is very noticeable when doing 512KB reads on a
QEMU system configured with 1500B MTU.

Also, a resent request here will prompt the responder on the
other side to immediately start resending, but the resent
packets will get stuck in the already-loaded receive queue and
will never be processed.

Rather than erroring out every time an unexpected future packet
arrives, just drop it. Eventually the retry timer will send a
duplicate request; the completer will be able to make progress since
the queue will start relatively empty.
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

A socket is associated with every QP by the rxe driver but sock_release()
is never called. Add a call to sock_release() in rxe_qp_cleanup().

Fixes: commit 8700e3e7c48A5 ("Add Soft RoCE driver")
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Cc: Moni Shoua <monis@mellanox.com>
Cc: Kamal Heib <kamalh@mellanox.com>
Cc: Amir Vadai <amirv@mellanox.com>
Cc: Haggai Eran <haggaie@mellanox.com>
Cc: <stable@vger.kernel.org>
Reviewed-by: NMoni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Get rid of this warning:

  drivers/infiniband/sw/rdmavt/cq.c: In function ‘rvt_cq_exit’:
  drivers/infiniband/sw/rdmavt/cq.c:542:2: warning: ‘worker’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    kthread_destroy_worker(worker);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

by fixing the function to actually work.

Fixes: 6efaf10f ("IB/rdmavt: Avoid queuing work into a destroyed cq kthread worker")
Cc: Petr Mladek <pmladek@suse.com>
Cc: Doug Ledford <dledford@redhat.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

rvt_create_qp() creates qp->ip only when a qp creation request comes from
userspace (udata is not NULL).  If we exceed the number of available
queue pairs however, the error path always attempts to put a kref to this
structure.  If the requestor is inside the kernel, this leads to a crash.

We fix this by checking that qp->ip is not NULL before caling kref_put().
Signed-off-by: NJim Foraker <foraker1@llnl.gov>
Acked-by: NDennis Dalessandro <dennis.dalessandro@intel.com>
Acked-by: NJonathan Toppins <jtoppins@redhat.com>
Acked-by: NAlex Estrin <alex.estrin@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Use the new API to create and destroy the cq kthread worker.
The API hides some implementation details.

In particular, kthread_create_worker() allocates and initializes
struct kthread_worker. It runs the kthread the right way and stores
task_struct into the worker structure. In addition, the *on_cpu()
variant binds the kthread to the given cpu and the related memory
node.

kthread_destroy_worker() flushes all pending works, stops
the kthread and frees the structure.

This patch does not change the existing behavior. Note that we must
use the on_cpu() variant because the function starts the kthread
and it must bind it to the right CPU before waking. The numa node
is associated for given CPU as well.
Signed-off-by: NPetr Mladek <pmladek@suse.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The memory barrier is not enough to protect queuing works into
a destroyed cq kthread. Just imagine the following situation:

CPU1				CPU2

rvt_cq_enter()
  worker =  cq->rdi->worker;

				rvt_cq_exit()
				  rdi->worker = NULL;
				  smp_wmb();
				  kthread_flush_worker(worker);
				  kthread_stop(worker->task);
				  kfree(worker);

				  // nothing queued yet =>
				  // nothing flushed and
				  // happily stopped and freed

  if (likely(worker)) {
     // true => read before CPU2 acted
     cq->notify = RVT_CQ_NONE;
     cq->triggered++;
     kthread_queue_work(worker, &cq->comptask);

  BANG: worker has been flushed/stopped/freed in the meantime.

This patch solves this by protecting the critical sections by
rdi->n_cqs_lock. It seems that this lock is not much contended
and looks reasonable for this purpose.

One catch is that rvt_cq_enter() might be called from IRQ context.
Therefore we must always take the lock with IRQs disabled to avoid
a possible deadlock.
Signed-off-by: NPetr Mladek <pmladek@suse.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Add struct ib_udata to the signature of create_ah callback that is
implemented by IB device drivers. This allows HW drivers to return extra
data to the userspace library.
This patch prepares the ground for mlx5 driver to resolve destination
mac address for a given GID and return it to userspace.
This patch was previously submitted by Knut Omang as a part of the
patch set to support Oracle's Infiniband HCA (SIF).
Signed-off-by: NKnut Omang <knut.omang@oracle.com>
Signed-off-by: NMoni Shoua <monis@mellanox.com>
Reviewed-by: NYishai Hadas <yishaih@mellanox.com>
Signed-off-by: NLeon Romanovsky <leon@kernel.org>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Increase limit of max CQE from 8K to 32K to allow demanding
applications to work over SoftRoCE with same configuration
as most RoCEv2 HW vendors have.

Fixes: 8700e3e7 ("Soft RoCE driver")
Signed-off-by: NYonatan Cohen <yonatanc@mellanox.com>
Reviewed-by: NMoni Shoua <monis@mellanox.com>
Signed-off-by: NLeon Romanovsky <leon@kernel.org>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

It might be possible for all of a QP's references to be dropped
while one of that QP's tasklets is running.

For example, the completer might run during QP destroy.
If qp->valid is false, it will drop all of the packets on
the resp_pkts list, potentially removing the last reference.
Then it tries to advance the SQ consumer pointer. If the
SQ's buffer has already been destroyed, the system will
panic.

To be safe, hold a reference on the QP for the duration
of each tasklet.
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The system may crash when a malformed request is received and
the error is detected by the responder.

NodeA: $ ibv_rc_pingpong -g 0 -d rxe0 -i 1 -n 1 -s 50000
NodeB: $ ibv_rc_pingpong -g 0 -d rxe0 -i 1 -n 1 -s 1024 <NodeA_ip>

The responder generates a receive error on node B since the incoming
SEND is oversized. If the client tears down the QP before the responder
or the completer finish running, a page fault may occur.

The fix makes the destroy operation spin until the tasks complete, which
appears to be original intent of the design.
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Reviewed-by: NYuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

A ref was added after the call to skb_clone().
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The udata->inlen error path needs to clean up the ref
added by rxe_alloc().
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Peek at the CQ after arming it so that we can return a hint.
This avoids missed completions due to a race between posting
CQEs and arming the CQ.

For example, CM teardown waits on MAD requests to complete with
ib_cq_poll_work(). Without this fix, the last completion might be
left on the CQ, hanging the kthread doing the teardown.

The console backtraces look like this:

[ 4199.911284] Call Trace:
[ 4199.911401]  [<ffffffff9657fe95>] schedule+0x35/0x80
[ 4199.911556]  [<ffffffff965830df>] schedule_timeout+0x22f/0x2c0
[ 4199.911727]  [<ffffffff9657f7a8>] ? __schedule+0x368/0xa20
[ 4199.911891]  [<ffffffff96580903>] wait_for_completion+0xb3/0x130
[ 4199.912067]  [<ffffffff960a17e0>] ? wake_up_q+0x70/0x70
[ 4199.912243]  [<ffffffffc074a06d>] cm_destroy_id+0x13d/0x450 [ib_cm]
[ 4199.912422]  [<ffffffff961615d5>] ? printk+0x57/0x73
[ 4199.912578]  [<ffffffffc074a390>] ib_destroy_cm_id+0x10/0x20 [ib_cm]
[ 4199.912759]  [<ffffffffc076098c>] rdma_destroy_id+0xac/0x340 [rdma_cm]
[ 4199.912941]  [<ffffffffc076f2cc>] 0xffffffffc076f2cc
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The last_psn algorithm fails in the zero-byte case: it calculates
first_psn = N, last_psn = N-1. This makes the operation unretryable since
the res structure will fail the (first_psn <= psn <= last_psn) test in
find_resource().

While here, use BTH_PSN_MASK to mask the calculated last_psn.
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Reviewed-by: NMoni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

skb_out is decremented in rxe_skb_tx_dtor(), which is not called in the
loopback() path. Move the increment to the send() path rather than
rxe_xmit_packet().
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Acked-by: NMoni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

A client might post a read followed by a send. The partner receives
and acknowledges both transactions, posting an RCQ entry for the
send, but something goes wrong with the read ACK. When the client
retries the read, the partner's responder processes the duplicate
read but incorrectly resets the PSN to the value preceding the
original send. When the duplicate send arrives, the responder cannot
tell that it is a duplicate, so the responder generates a duplicate
RCQ entry, confusing the client.
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Reviewed-by: NYonatan Cohen <yonatanc@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

A simple userspace application might poll the CQ, find a completion,
and then attempt to post a new WQE to the SQ. A spurious error can
occur if the userspace application detects a full SQ in the instant
before the kernel is able to advance the SQ consumer pointer.

This is noticeable when using single-entry SQs with ibv_rc_pingpong
if lots of kernel and userspace library debugging is enabled.
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Reviewed-by: NYonatan Cohen <yonatanc@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Avoid smashing the stack when an ICRC error occurs on an IPv6 network.
Signed-off-by: NAndrew Boyer <andrew.boyer@dell.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

It makes me nervous when we cast pointer parameters.  I would estimate
that around 50% of the time, it indicates a bug.  Here the cast is not
needed becaue u32 and and unsigned int are the same thing.  Removing the
cast makes the code more robust and future proof in case any of the
types change.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NYuval Shaia <yuval.shaia@oracle.com>
Acked-by: NMoni Shoua <monis@mellanox.com>
Reviewed-by: NYuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

spinlock can be initialized automatically with DEFINE_SPINLOCK()
rather than explicitly calling spin_lock_init().
Signed-off-by: NWei Yongjun <weiyongjun1@huawei.com>
Reviewed-by: NLeon Romanosky <leonro@mellanox.com>
Reviewed-by: NMoni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

A race condition fix added an rxe_qp structure to the stack in order
to be able to perform rollback in rxe_requester(), but the structure
is large enough to trigger the warning for possible stack overflow:

drivers/infiniband/sw/rxe/rxe_req.c: In function 'rxe_requester':
drivers/infiniband/sw/rxe/rxe_req.c:757:1: error: the frame size of 2064 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]

This changes the rollback function to only save the psn inside
the qp, which is the only field we access in the rollback_qp
anyway.

Fixes: 3050b998 ("IB/rxe: Fix race condition between requester and completer")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>