ib_alloc_fast_reg_mr() and ib_alloc_fast_reg_page_list() returns
ERR_PTR() and not NULL. Compile tested only.
Signed-off-by: NWei Yongjun <yjwei@cn.fujitsu.com>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>

In case the check on ch_count fails the cleanup path is skipped and the
previously allocated memory 'rpl_map', 'chl_map' is not freed.

Reported by Coverity.
Signed-off-by: NChristian Engelmayer <christian.engelmayer@frequentis.com>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>

The svcrdma module was incorrectly unmapping the RPCRDMA header page.
On IBM pserver systems this causes a resource leak that results in
running out of bus address space (10 cthon iterations will reproduce it).
The code was mapping the full page but only unmapping the actual header
length.  The fix is to only map the header length.

I also cleaned up the use of ib_dma_map_page() calls since the unmap
logic always uses ib_dma_unmap_single().  I made these symmetrical.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>

mlx4/connectX FRMR requires local write enable together with remote
rdma write enable. This fixes NFS/RDMA operation over the ConnectX
Infiniband HCA in the default memreg mode.
Signed-off-by: NVu Pham <vu@mellanox.com>
Signed-off-by: NTom Talpey <tmtalpey@gmail.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

These fixes resolved crashes due to resource leak BUG_ON checks. The
resource leaks were detected by introducing asynchronous transport errors.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>

The nfs server rdma transport was mapping rdma read target pages for
TO_DEVICE instead of FROM_DEVICE.  This causes data corruption on non
cache-coherent systems if frmrs are used.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>

printk formats in prior commit were reversed/incorrect.
Compiled without warning on x86 and x86_64, but detected on ppc.
Signed-off-by: NTom Talpey <tmtalpey@gmail.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Certain client rpc's which contain both lengthy page-contained
metadata and a non-empty xdr_tail buffer require careful handling
to avoid overlapped memory copying. Rearranging of existing rpcrdma
marshaling code avoids it; this fixes an NFSv4 symlink creation error
detected with connectathon basic/test8 to multiple servers.
Signed-off-by: NTom Talpey <tmtalpey@gmail.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Certain client-provided RPCRDMA chunk alignments result in an
additional scatter/gather entry, which triggered nfs/rdma server
assertions incorrectly. OpenSolaris nfs/rdma client connectathon
testing was blocked by these in the special/locking section.
Signed-off-by: NTom Talpey <tmtalpey@gmail.com>
Cc: Tom Tucker <tom@opengridcomputing.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Signed-off-by: NIlpo Järvinen <ilpo.jarvinen@helsinki.fi>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Sparse asked whether these could be static.
Signed-off-by: NRoel Kluin <roel.kluin@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

fix this warning:

  net/sunrpc/xprtrdma/verbs.c: In function ‘rpcrdma_conn_upcall’:
  net/sunrpc/xprtrdma/verbs.c:279: warning: unused variable ‘addr’
Signed-off-by: NIngo Molnar <mingo@elte.hu>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

this warning:

  net/sunrpc/xprtrdma/svc_rdma_transport.c: In function ‘svc_rdma_accept’:
  net/sunrpc/xprtrdma/svc_rdma_transport.c:830: warning: ‘dma_mr_acc’ may be used uninitialized in this function

triggers because GCC does not recognize the (correct) flow connection
between need_dma_mr and dma_mr_acc.

Annotate it.
Signed-off-by: NIngo Molnar <mingo@elte.hu>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Using NIPQUAD() with NIPQUAD_FMT, %d.%d.%d.%d or %u.%u.%u.%u
can be replaced with %pI4
Signed-off-by: NHarvey Harrison <harvey.harrison@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The RPC/RDMA connection logic could return early from reconnection
attempts, leading to additional spurious retries.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

The RPC/RDMA code had a constant 5-second reconnect backoff, and
always performed it, even when re-establishing a connection to a
server after the RPC layer closed it due to being idle. Make it
an geometric backoff (up to 30 seconds), and don't delay idle
reconnect.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

The send marshaling code split a particular dprintk across two
lines, which makes it hard to extract from logfiles.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Add defensive timeouts to wait_for_completion() calls in RDMA
address resolution, and make them interruptible. Fix the timeout
units to milliseconds (formerly jiffies) and move to private header.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

The RPC/RDMA code can leak RDMA connection manager endpoints in
certain error cases on connect. Don't signal unwanted events,
and be certain to destroy any allocated qp.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

The xprt_connect call path does not expect such errors as ECONNREFUSED
to be returned from failed transport connection attempts, otherwise it
translates them to EIO and signals fatal errors. For example, mount.nfs
prints simply "internal error". Translate all such errors to ENOTCONN
from RPC/RDMA to match sockets behavior.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

The RPC/RDMA protocol allows clients and servers to avoid RDMA
operations for data which is purely the result of XDR padding.
On the client, automatically insert the necessary padding for
such server replies, and optionally don't marshal such chunks.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

RDMA disconnects yield an upcall from the RDMA connection manager,
which can race with rpc transport close, e.g. on ^C of a mount.
Ensure any rdma cm_id and qp are fully destroyed before continuing.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

An RPC/RDMA client cannot retransmit on an unbroken connection,
doing so violates its flow control with the server.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

This logic sets the connection parameter that configures the local device
and informs the remote peer how many concurrent incoming RDMA_READ
requests are supported. The original logic didn't really do what was
intended for two reasons:

- The max number supported by the device is typically smaller than
any one factor in the calculation used, and

- The field in the connection parameter structure where the value is
stored is a u8 and always overflows for the default settings.

So what really happens is the value requested for responder resources
is the left over 8 bits from the "desired value". If the desired value
happened to be a multiple of 256, the result was zero and it wouldn't
connect at all.

Given the above and the fact that max_requests is almost always larger
than the max responder resources supported by the adapter, this patch
simplifies this logic and simply requests the max supported by the device,
subject to a reasonable limit.

This bug was found by Jim Schutt at Sandia.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>
Acked-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Configure, detect and use "fastreg" support from IB/iWARP verbs
layer to perform RPC/RDMA memory registration.

Make FRMR the default memreg mode (will fall back if not supported
by the selected RDMA adapter).

This allows full and optimal operation over the cxgb3 adapter, and others.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Acked-by: NTom Tucker <tom@opengridcomputing.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

At transport creation, check for, and use, any local dma lkey.
Then, check that the selected memory registration mode is in fact
supported by the RDMA adapter selected for the mount. Fall back
to best alternative if not.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Acked-by: NTom Tucker <tom@opengridcomputing.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Internal RPC/RDMA structure updates in preparation for FRMR support.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Acked-by: NTom Tucker <tom@opengridcomputing.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Refactor the memory registration and deregistration routines.
This saves stack space, makes the code more readable and prepares
to add the new FRMR registration methods.
Signed-off-by: NTom Talpey <talpey@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

The inititator/responder resources in the event have been swapped. They
no represent what the local peer would set their values to in order to
match the peer. Note that iWARP does not exchange these on the wire and
the provider is simply putting in the local device max.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>

Update the svc_rdma_send_error code to use the DMA LKEY which is valid
regardless of the memory registration strategy in use.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>

Use FRMR to map local RPC reply data. This allows RDMA_WRITE to send reply
data using a single WR. The FRMR is invalidated by linking the LOCAL_INV WR
to the RDMA_SEND message used to complete the reply.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>

RPCRDMA requests that specify a read-list are fetched with RDMA_READ. Using
an FRMR to map the data sink improves NFSRDMA security on transports that
place the RDMA_READ data sink LKEY on the wire because the valid lifetime
of the MR is only the duration of the RDMA_READ. The LKEY is invalidated
when the last RDMA_READ WR completes.

Mapping the data sink also allows for very large amounts to data to be
fetched with a single WR, so if the client is also using FRMR, the entire
RPC read-list can be fetched with a single WR.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>

WR can be submitted as linked lists of WR. Update the svc_rdma_send
routine to handle WR chains. This will be used to submit a WR that
uses an FRMR with another WR that invalidates the FRMR.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>

Update the svc_rdma_post_recv routine to use the adapter's global LKEY
instead of sc_phys_mr which is only valid when using a DMA MR.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>

Fast Reg MR introduces a new WR type. Add a service to register the
region with the adapter and update the completion handling to support
completions with a NULL WR context.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>

Query the device capabilities in the svc_rdma_accept function to determine
what advanced memory management capabilities are supported by the device.
Based on the query, select the most secure model available given the
requirements of the transport and capabilities of the adapter.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>

Add services for the allocating, freeing, and unmapping Fast Reg MR. These
services will be used by the transport connection setup, send and receive
routines.
Signed-off-by: NTom Tucker <tom@opengridcomputing.com>

Signed-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>