Set the address handle and QP address path fields according to the
link layer type (IB/Eth).
Signed-off-by: NAchiad Shochat <achiad@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

These callbacks write into the mlx5 RoCE address table.
Upon del_gid we write a zero'd GID.
Signed-off-by: NAchiad Shochat <achiad@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

When handling a responder completion, if the link layer is Ethernet,
set the work completion network_hdr_type field according to CQE's
info and the IB_WC_WITH_NETWORK_HDR_TYPE flag.
Signed-off-by: NAchiad Shochat <achiad@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Using the vport access functions to retrieve the Ethernet
specific information and return this information in
ib_query_device and ib_query_port.
Signed-off-by: NAchiad Shochat <achiad@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

For Eth ports only:
Maintain a net device pointer in mlx5_ib_device and update it
upon NETDEV_REGISTER and NETDEV_UNREGISTER events if the
net-device and IB device have the same PCI parent device.
Implement the get_netdev callback to return this net device.
Signed-off-by: NAchiad Shochat <achiad@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Make the existing mlx5_ib_port_link_layer() signature match
the ib device callback signature (add port_num parameter).
Refactor it to use a sub function so that the link layer could
be queried also before the ibdev is created.
Signed-off-by: NAchiad Shochat <achiad@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

kzalloc doesn't return ERR_PTR, so there is no need to test for it.

The semantic match that finds this problem is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@@
expression x,e;
@@

* x = kzalloc(...)
... when != x = e
* IS_ERR_OR_NULL(x)
// </smpl>
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Reviewed-by: NDave Goodell <dgoodell@cisco.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NDave Goodell <dgoodell@cisco.com>
Reviewed-by: NReese Faucette <rfaucett@cisco.com>
Reviewed-by: NXuyang Wang <xuywang@cisco.com>
Signed-off-by: NNelson Escobar <neescoba@cisco.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NDave Goodell <dgoodell@cisco.com>
Reviewed-by: NReese Faucette <rfaucett@cisco.com>
Reviewed-by: NXuyang Wang <xuywang@cisco.com>
Signed-off-by: NNelson Escobar <neescoba@cisco.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

They were already implemented at a lower layer, but the upper level
routine placed arbitrary restrictions on which transitions were
permitted.  Simplify the state machine logic to live wholly in
usnic_ib_qp_grp_modify.
Signed-off-by: NDave Goodell <dgoodell@cisco.com>
Reviewed-by: NReese Faucette <rfaucett@cisco.com>
Reviewed-by: NXuyang Wang <xuywang@cisco.com>
Signed-off-by: NNelson Escobar <neescoba@cisco.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NDave Goodell <dgoodell@cisco.com>
Reviewed-by: NReese Faucette <rfaucett@cisco.com>
Reviewed-by: NXuyang Wang <xuywang@cisco.com>
Signed-off-by: NNelson Escobar <neescoba@cisco.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NChristian Benvenuti <benve@cisco.com>
Signed-off-by: NNelson Escobar <neescoba@cisco.com>
Reviewed-by: NDave Goodell <dgoodell@cisco.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NNelson Escobar <neescoba@cisco.com>
Reviewed-by: NDave Goodell <dgoodell@cisco.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

query_protocol() was added in commit 6b90a6d6 ("IB/Verbs:
Implement new callback query_protocol()") and then removed in
commit f9b22e35 ("IB/core: Convert core to use bitfield
for caps").

This left behind an unused prototype.
Signed-off-by: NNelson Escobar <neescoba@cisco.com>
Reviewed-by: NDave Goodell <dgoodell@cisco.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Since RoCEv2 is a protocol over IP header it is required to send IGMP
join and leave requests to the network when joining and leaving
multicast groups.
Signed-off-by: NMoni Shoua <monis@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

ib_ud_header_init() is used to format InfiniBand headers
in a buffer up to (but not with) BTH. For RoCE UDP ENCAP it is
required that this function would be able to build also IP and UDP
headers.
Signed-off-by: NMoni Shoua <monis@mellanox.com>
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Users would like to control the behaviour of rdma_cm.
For example, old applications which don't set the
required RoCE gid type could be executed on RoCE V2
network types. In order to support this configuration,
we implement a configfs for rdma_cm.

In order to use the configfs, one needs to mount it and
mkdir <IB device name> inside rdma_cm directory.

The patch adds support for a single configuration file,
default_roce_mode. The mode can either be "IB/RoCE v1" or
"RoCE v2".
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Currently, cma users can't increase or decrease the cma reference
count. This is necassary when setting cma attributes (like the
default GID type) in order to avoid use-after-free errors.
Adding cma_ref_dev and cma_deref_dev APIs.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

In order to make sure API users don't try to use SGIDs which don't
conform to the routing table, validate the route before searching
the RoCE GID table.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

In order to validate the route, we need an easy way to check if a
net-device belongs to our RDMA device. Move this helper function
to a header file in order to make this check easier.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Reviewed-by: NHaggai Eran <haggaie@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Providers should tell IB core the wc's network type.
This is used in order to search for the proper GID in the
GID table. When using HCAs that can't provide this info,
IB core tries to deep examine the packet and extract
the GID type by itself.

We choose sgid_index and type from all the matching entries in
RDMA-CM based on hint from the IP stack and we set hop_limit for
the IP packet based on above hint from IP stack.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NSomnath Kotur <Somnath.Kotur@Avagotech.Com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Adding RoCE v2 GID type and port type. Vendors
which support this type will get their GID table
populated with RoCE v2 GIDs automatically.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

This patch set adds attributes of net device and gid type to each GID
in the GID table. Users that use verbs directly need to specify
the GID index. Since the same GID could have different types or
associated net devices, users should have the ability to query the
associated GID attributes. Adding these attributes to sysfs.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Previosuly, cm and cma modules supported only IB and RoCE v1 GID type.
In order to support multiple GID types, the gid_type is passed to
cm_init_av_by_path and stored in the path record.

The rdma cm client would use a default GID type that will be saved in
rdma_id_private.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

In order to support multiple GID types, we need to store the gid_type
with each GID. This is also aligned with the RoCE v2 annex "RoCEv2 PORT
GID table entries shall have a "GID type" attribute that denotes the L3
Address type". The currently supported GID is IB_GID_TYPE_IB which is
also RoCE v1 GID type.

This implies that gid_type should be added to roce_gid_table meta-data.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Previously, we've searched the GID table twice: first when we searched
the table for a GID matching the proposed new one, and second when we
didn't find a match, we searched again for an empty GID slot in the
table.  Instead, search the table once noting the first empty slot as
we search for our target GID.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Reviewed-by: NHaggai Eran <haggaie@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Previously, IB GID cached used a lock per entry. This could result
in spending a lot of CPU cycles for locking and unlocking just
in order to find a GID. Changing this in favor of one lock per
a GID table.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Reviewed-by: NHaggai Eran <haggaie@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Refactor ib_dispatch_event into a new function in order to avoid
duplicating code in the next patch.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Reviewed-by: NHaggai Eran <haggaie@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The copy of the attributes present on the device is now used by all consumers
except for uverbs in case of serving user-space query, where dev->query_device
is called.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Instead, use the cached copy of the attributes present on the device.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Use the cached copy of the attributes present on the device, except for
the case of a query originating from user-space, where we have to invoke
the driver query_device entry, so they can fill in their udata.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

This way both the IB core and upper level drivers can access these cached
device attributes rather than querying or caching them on their own.
Signed-off-by: NIra Weiny <ira.weiny@intel.com>
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Use the new CQ abstraction to simplify completions in the iSER
initiator.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Nicer this way.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

We'll need it later with the new CQ abstraction. also switch
login bufs to void pointers.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

This also moves recv completion handling from hardirq context into
softirq context.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Remove struct rdma_iu and instead allocate the struct ib_rdma_wr array
early and fill out directly.  This allows us to chain the WRs, and thus
archives both less lock contention on the HCA workqueue as well as much
simpler error handling.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NBart Van Assche <bart.vanassche@sandisk.com>

This adds an abstraction that allows ULPs to simply pass a completion
object and completion callback with each submitted WR and let the RDMA
core handle the nitty gritty details of how to handle completion
interrupts and poll the CQ.

In detail there is a new ib_cqe structure which just contains the
completion callback, and which can be used to get at the containing
object using container_of.  It is pointed to by the WR and WC as an
alternative to the wr_id field, similar to how many ULPs already use
the field to store a pointer using casts.

A driver using the new completion callbacks allocates it's CQs using
the new ib_create_cq API, which in addition to the number of CQEs and
the completion vectors also takes a mode on how we poll for CQEs.
Three modes are available: direct for drivers that never take CQ
interrupts and just poll for them, softirq to poll from softirq context
using the to be renamed blk-iopoll infrastructure which takes care of
rearming and budgeting, or a workqueue for consumer who want to be
called from user context.

Thanks a lot to Sagi Grimberg who helped reviewing the API, wrote
the current version of the workqueue code because my two previous
attempts sucked too much and converted the iSER initiator to the new
API.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

The remove_keys() logic is performed as garbage collection task. Such
task is intended to be run when no other active processes are running.

The need_resched() will return TRUE if there are user tasks to be
activated in near future.

In such case, we don't execute remove_keys() and postpone
the garbage collection work to try to run in next cycle,
in order to free CPU resources to other tasks.

The possible pseudo-code to trigger such scenario:
1. Allocate a lot of MR to fill the cache above the limit.
2. Wait a small amount of time "to calm" the system.
3. Start CPU extensive operations on multi-node cluster.
4. Expect performance degradation during MR cache shrink operation.
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NEli Cohen <eli@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

There are several hits that WR buffer allocation(kmalloc) failed.
It failed at order 3 and/or 4 contigous pages allocation. At the same time
there are actually 100MB+ free memory but well fragmented.
So try vmalloc when kmalloc failed.
Signed-off-by: NWengang Wang <wen.gang.wang@oracle.com>
Acked-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>