Fix create WQ to use the given user handle, in addition dropped some
duplicated code from this flow.

Fixes: fd3c7904 ("IB/core: Change idr objects to use the new schema")
Fixes: f213c052 ("IB/uverbs: Add WQ support")
Link: https://lore.kernel.org/r/20200506082444.14502-9-leon@kernel.orgSigned-off-by: NYishai Hadas <yishaih@mellanox.com>
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Both wq_context and srq_context are some leftover from the past in uverbs
layer, they are not really in use, drop them.

Link: https://lore.kernel.org/r/20200506082444.14502-5-leon@kernel.orgSigned-off-by: NYishai Hadas <yishaih@mellanox.com>
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Danil and I will maintain RNBD/RTRS modules.

Link: https://lore.kernel.org/r/20200511135131.27580-26-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

README with description of major sysfs entries, sysfs documentation
are moved to ABI dir as Bart suggested.

Link: https://lore.kernel.org/r/20200511135131.27580-25-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Add rnbd Makefile, Kconfig and also corresponding lines into upper block
layer files.

Link: https://lore.kernel.org/r/20200511135131.27580-24-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This is the sysfs interface to rnbd mapped devices on server side:

  /sys/class/rnbd-server/ctl/devices/<device_name>/
    |- block_dev
    |  *** link pointing to the corresponding block device sysfs entry
    |
    |- sessions/<session-name>/
    |  *** sessions directory
       |
       |- read_only
       |  *** is devices mapped as read only
       |
       |- mapping_path
          *** relative device path provided by the client during mapping

Link: https://lore.kernel.org/r/20200511135131.27580-23-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This provides helper functions for IO submitting to block dev.

Link: https://lore.kernel.org/r/20200511135131.27580-22-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This is main functionality of rnbd-server module, which handles RTRS
events and rnbd protocol requests, like map (open) or unmap (close)
device.  Also server side is responsible for processing incoming IBTRS IO
requests and forward them to local mapped devices.

Link: https://lore.kernel.org/r/20200511135131.27580-21-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This header describes main structs and functions used by rnbd-server
module, namely structs for managing sessions from different clients and
mapped (opened) devices.

Link: https://lore.kernel.org/r/20200511135131.27580-20-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This is the sysfs interface to rnbd block devices on client side:

  /sys/class/rnbd-client/ctl/
    |- map_device
    |  *** maps remote device
    |
    |- devices/
       *** all mapped devices

  /sys/block/rnbd<N>/rnbd/
    |- unmap_device
    |  *** unmaps device
    |
    |- state
    |  *** device state
    |
    |- session
    |  *** session name
    |
    |- mapping_path
       *** path of the dev that was mapped on server

Link: https://lore.kernel.org/r/20200511135131.27580-19-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This is main functionality of rnbd-client module, which provides interface
to map remote device as local block device /dev/rnbd<N> and feeds RTRS
with IO requests.

Link: https://lore.kernel.org/r/20200511135131.27580-18-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This header describes main structs and functions used by rnbd-client
module, mainly for managing RNBD sessions and mapped block devices,
creating and destroying sysfs entries.

Link: https://lore.kernel.org/r/20200511135131.27580-17-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

These are common private headers with rnbd protocol structures, logging,
sysfs and other helper functions, which are used on both client and server
sides.

Link: https://lore.kernel.org/r/20200511135131.27580-16-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Acked-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

README with description of major sysfs entries, sysfs documentation has
been moved to ABI dir as suggested by Bart.

Link: https://lore.kernel.org/r/20200511135131.27580-15-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Add rtrs Makefile, Kconfig and also corresponding lines into upper layer
infiniband/ulp files.

Link: https://lore.kernel.org/r/20200511135131.27580-14-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This is the sysfs interface to rtrs sessions on server side:

  /sys/class/rtrs-server/<SESS-NAME>/
    *** rtrs session accepted from a client peer
    |
    |- paths/<SRC@DST>/
       *** established paths from a client in a session
       |
       |- disconnect
       |  *** disconnect path
       |
       |- hca_name
       |  *** HCA name
       |
       |- hca_port
       |  *** HCA port
       |
       |- stats/
          *** current path statistics
          |
	  |- rdma

Link: https://lore.kernel.org/r/20200511135131.27580-13-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This introduces set of functions used on server side to account statistics
of RDMA data sent/received.

Link: https://lore.kernel.org/r/20200511135131.27580-12-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This is main functionality of rtrs-server module, which accepts set of
RDMA connections (so called rtrs session), creates/destroys sysfs entries
associated with rtrs session and notifies upper layer
(user of RTRS API) about RDMA requests or link events.

Link: https://lore.kernel.org/r/20200511135131.27580-11-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This header describes main structs and functions used by rtrs-server
module, mainly for accepting rtrs sessions, creating/destroying sysfs
entries, accounting statistics on server side.

Link: https://lore.kernel.org/r/20200511135131.27580-10-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This is the sysfs interface to rtrs sessions on client side:

  /sys/class/rtrs-client/<SESS-NAME>/
    *** rtrs session created by rtrs_clt_open() API call
    |
    |- max_reconnect_attempts
    |  *** number of reconnect attempts for session
    |
    |- add_path
    |  *** adds another connection path into rtrs session
    |
    |- paths/<SRC@DST>/
       *** established paths to server in a session
       |
       |- disconnect
       |  *** disconnect path
       |
       |- reconnect
       |  *** reconnect path
       |
       |- remove_path
       |  *** remove current path
       |
       |- state
       |  *** retrieve current path state
       |
       |- hca_port
       |  *** HCA port number
       |
       |- hca_name
       |  *** HCA name
       |
       |- stats/
          *** current path statistics
          |
	  |- cpu_migration
	  |- rdma
	  |- reconnects
	  |- reset_all

Link: https://lore.kernel.org/r/20200511135131.27580-9-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This introduces set of functions used on client side to account statistics
of RDMA data sent/received, amount of IOs inflight, latency, cpu
migrations, etc.  Almost all statistics are collected using percpu
variables.

Link: https://lore.kernel.org/r/20200511135131.27580-8-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This is main functionality of rtrs-client module, which manages set of
RDMA connections for each rtrs session, does multipathing, load balancing
and failover of RDMA requests.

Link: https://lore.kernel.org/r/20200511135131.27580-7-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This header describes main structs and functions used by rtrs-client
module, mainly for managing rtrs sessions, creating/destroying sysfs
entries, accounting statistics on client side.

Link: https://lore.kernel.org/r/20200511135131.27580-6-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

This is a set of library functions existing as a rtrs-core module, used by
client and server modules.

Mainly these functions wrap IB and RDMA calls and provide a bit higher
abstraction for implementing of RTRS protocol on client or server sides.

Link: https://lore.kernel.org/r/20200511135131.27580-5-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

These are common private headers with rtrs protocol structures, logging,
sysfs and other helper functions, which are used on both client and server
sides.

Link: https://lore.kernel.org/r/20200511135131.27580-4-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Introduce public header which provides set of API functions to establish
RDMA connections from client to server machine using RTRS protocol, which
manages RDMA connections for each session, does multipathing and load
balancing.

Main functions for client (active) side:

 rtrs_clt_open() - Creates set of RDMA connections incapsulated
                    in IBTRS session and returns pointer on RTRS
		    session object.
 rtrs_clt_close() - Closes RDMA connections associated with RTRS
                     session.
 rtrs_clt_request() - Requests zero-copy RDMA transfer to/from
                       server.

Main functions for server (passive) side:

 rtrs_srv_open() - Starts listening for RTRS clients on specified
                    port and invokes RTRS callbacks for incoming
		    RDMA requests or link events.
 rtrs_srv_close() - Closes RTRS server context.

Link: https://lore.kernel.org/r/20200511135131.27580-3-danil.kipnis@cloud.ionos.comSigned-off-by: NDanil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Function is going to be used in transport over RDMA module in subsequent
patches, so export it to GPL modules.

Link: https://lore.kernel.org/r/20200511135131.27580-2-danil.kipnis@cloud.ionos.comSigned-off-by: NRoman Pen <roman.penyaev@profitbricks.com>
Acked-by: NTejun Heo <tj@kernel.org>
Cc: linux-kernel@vger.kernel.org
[jwang: extend the commit message]
Signed-off-by: NJack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

The output buffer used in mlx5_cmd_exec_inout() was wrongly changed from
pre-allocated srq_out pointer to an input "out" point. That leads to
unpredictable results in the get_srqc() call later.

Fixes: 31578def ("RDMA/mlx5: Update mlx5_ib to use new cmd interface")
Link: https://lore.kernel.org/r/20200513100809.246315-1-leon@kernel.orgReported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

When drop action is used the matching packet will stop processing in
steering and will be dropped. This functionality will allow users to drop
matching packets.

Link: https://lore.kernel.org/r/20200504054227.271486-1-leon@kernel.orgSigned-off-by: NDaria Velikovsky <daria@mellanox.com>
Reviewed-by: NMaor Gottlieb <maorg@mellanox.com>
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

User can configure default miss rule in order to skip matching in the user
domain and forward the packet to the kernel steering domain.  When user
requests a default miss rule, we add steering rule to forward the traffic
to the next namespace.

Link: https://lore.kernel.org/r/20200504053012.270689-5-leon@kernel.orgSigned-off-by: NMaor Gottlieb <maorg@mellanox.com>
Reviewed-by: NMark Zhang <markz@mellanox.com>
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Move part of the code that get the destinations into function so the code
will be more readable.  In addition change the variables definition to be
in reversed christmas tree.

Link: https://lore.kernel.org/r/20200504053012.270689-4-leon@kernel.orgSigned-off-by: NMaor Gottlieb <maorg@mellanox.com>
Reviewed-by: NMark Zhang <markz@mellanox.com>
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

From the mlx5-next branch at
  git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux

Required for dependencies in following patches

* branch 'mellanox/mlx5-next':
  net/mlx5: Add support in forward to namespace
  {IB/net}/mlx5: Simplify don't trap code
  net/mlx5: Replace zero-length array with flexible-array
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Currently, fs_core supports rule of forward the traffic
to continue matching in the next priority, now we add support
to forward the traffic matching in the next namespace.
Signed-off-by: NMaor Gottlieb <maorg@mellanox.com>
Reviewed-by: NMark Bloch <markb@mellanox.com>
Reviewed-by: NMark Zhang <markz@mellanox.com>
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>

The fs_core already supports creation of rules with multiple
actions/destinations. Refactor fs_core to handle the case
when don't trap rule is created with destination. Adapt the
calling code in the driver.
Signed-off-by: NMaor Gottlieb <maorg@mellanox.com>
Reviewed-by: NMark Zhang <markz@mellanox.com>
Reviewed-by: NMark Bloch <markb@mellanox.com>
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>

The current codebase makes use of the zero-length array language
extension to the C90 standard, but the preferred mechanism to declare
variable-length types such as these ones is a flexible array member[1][2],
introduced in C99:

struct foo {
        int stuff;
        struct boo array[];
};

By making use of the mechanism above, we will get a compiler warning
in case the flexible array does not occur last in the structure, which
will help us prevent some kind of undefined behavior bugs from being
inadvertently introduced[3] to the codebase from now on.

Also, notice that, dynamic memory allocations won't be affected by
this change:

"Flexible array members have incomplete type, and so the sizeof operator
may not be applied. As a quirk of the original implementation of
zero-length arrays, sizeof evaluates to zero."[1]

sizeof(flexible-array-member) triggers a warning because flexible array
members have incomplete type[1]. There are some instances of code in
which the sizeof operator is being incorrectly/erroneously applied to
zero-length arrays and the result is zero. Such instances may be hiding
some bugs. So, this work (flexible-array member conversions) will also
help to get completely rid of those sorts of issues.

This issue was found with the help of Coccinelle.

[1] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
[2] https://github.com/KSPP/linux/issues/21
[3] commit 76497732 ("cxgb3/l2t: Fix undefined behaviour")

Link: https://lore.kernel.org/r/20200507185342.GA14476@embeddedorSigned-off-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

All callers need the 'get', so do it in a central place before returning
the pointer.

Link: https://lore.kernel.org/r/20200506074701.9775-11-leon@kernel.orgSigned-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Just put the expression in the only reader

Link: https://lore.kernel.org/r/20200506074701.9775-10-leon@kernel.orgSigned-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Just call xa_erase directly during cm_destroy_id()

Link: https://lore.kernel.org/r/20200506074701.9775-9-leon@kernel.orgSigned-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

The only caller doesn't care about the timewait, so acquire and return the
cm_id_private from the function.

Link: https://lore.kernel.org/r/20200506074701.9775-8-leon@kernel.orgSigned-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

The way the cm_timewait_info is converted into a work and then freed
is very subtle and surprising, add a note clarifying the lifetime
here.

Link: https://lore.kernel.org/r/20200506074701.9775-7-leon@kernel.orgSigned-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>