ib_query_gid() fetches the GID from the software cache maintained in
ib_core for RoCE ports.

Therefore, simplify the provider drivers for RoCE to treat query_gid()
callback as never called for RoCE, and only require non-RoCE devices to
implement it.
Signed-off-by: NParav Pandit <parav@mellanox.com>
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

qedr driver's add_gid() and del_gid() callbacks are doing simple
checks which are already done by the ib core before invoking these
callback routines.

Therefore, code is simplified to skip implementing add_gid() and
del_gid() callback functions.
Signed-off-by: NParav Pandit <parav@mellanox.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Code includes wmb() followed by writel() in multiple places. writel()
already has a barrier on some architectures like arm64.

This ends up CPU observing two barriers back to back before executing the
register write.

Since code already has an explicit barrier call, changing writel() to
writel_relaxed().
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

iWARP does not support RDMA WRITE or SEND with immediate data.
Driver should check this before submitting to FW and return an
immediate error
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Race in qedr_poll_cq, lastest_cqe wasn't protected by lock,
leading to a case where two context's accessing poll_cq at
the same time lead to one of them having a pointer to an old
latest_cqe and reading an invalid cqe element
Signed-off-by: NAmit Radzi <Amit.Radzi@cavium.com>
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

There are races where can still get flush on CQEs before the QP enters
error state. This is not an error and should be treated as
debug information.
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The double swap matches what user space rdma-core does to imm_data.

wc->imm_data is not used in the kernel so this change has no practical
impact.
Acked-by: NMichal Kalderon <michal.kalderon@cavium.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Use dma_zalloc_coherent for allocating zeroed
memory and remove unnecessary memset function.

Done using Coccinelle.
Generated-by: scripts/coccinelle/api/alloc/kzalloc-simple.cocci
0-day tested with no failures.
Suggested-by: NLuis R. Rodriguez <mcgrof@kernel.org>
Signed-off-by: NHimanshu Jha <himanshujha199640@gmail.com>
Acked-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Signed-off-by: NBart Van Assche <bart.vanassche@wdc.com>
Cc: Ram Amrani <Ram.Amrani@cavium.com>
Cc: Michal Kalderon <Michal.Kalderon@cavium.com>
Cc: Ariel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NBart Van Assche <bart.vanassche@wdc.com>
Cc: Michal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

We should return -ENOMEM if the kzalloc() fails.  We currently return
success.

Fixes: 69ad0e7f ("RDMA/qedr: Add support for iWARP in user space")
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Trivial fix to spelling mistake in DP_ERR error message
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Reviewed-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Pass the second doorbell offset to userspace in create_qp response.
Pbl allocation is different for RoCE and iWARP and requires
different handling. RoCE allocated the pbl and passes the pointer
to qed, where-as in iWARP, qed allocates the pbl and qedr
populates it after it returns
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Implements the iWARP connection management functions:
connect, accept, create listener and destroy listener
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

This patch implements the following iWARP callbacks:
qp_add_ref
qp_rem_ref
get_qp
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

iWARP supports read with invalidate. There is an assumption
that read with invalidate will only be called on an iWARP device
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Make slight modifications to common RoCE/iWARP code.
Add additional doorbell for iWARP post_send.
iWARP QP pbl is allocated in qed and not in qedr.
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

There are slight differences between iWARP and RoCE in the ibdev
registration. This patch handles the changes.
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The main differences between iWARP and RoCE lay in the communication
 management functions. These will be placed in separate files.
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The number of supported WIDs, if they are supported at all, can be
limited due to resources. Notifying the user space application the
number of available WIDs allows it to utilize them correctly.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Direct Packet Mode support may be disabled, e.g, due to limited
resources. Notifying the user application prevents wasting cycles
on attempting to send these kind of packets.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Wrap ib_copy_to_udata with a function that ensures that the data
being copied over to user space isn't longer than the allowed.

Fixes: cecbcddf ("qedr: Add support for QP verbs")
Fixes: a7efd777 ("qedr: Add support for PD,PKEY and CQ verbs")
Fixes: ac1b36e5 ("qedr: Add support for user context verbs")
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

iWARP would require the chains to allocate/free their PBL memory
independently, so add the infrastructure to provide it externally.
Signed-off-by: NYuval Mintz <Yuval.Mintz@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add 64KB PAGE_SIZE support to user-space CQ, SQ and RQ queues.
De-facto it means that code was added to translate 64KB
pages to smaller 4KB pages that the FW can handle. Otherwise,
the FW would wrap (or jump to the next page)  when reaching 4KB
while the user space library will continue on the same large page.

Note that MR code remains as is since the FW supports larger pages
for MRs.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Initialize byte_len in work completion of RDMA_READ and RDMA_SEND.
Exposed by uDAPL application.
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

rdma_ah_attr can now be either ib or roce allowing
core components to use one type or the other and also
to define attributes unique to a specific type. struct
ib_ah is also initialized with the type when its first
created. This ensures that calls such as modify_ah
dont modify the type of the address handle attribute.
Reviewed-by: NIra Weiny <ira.weiny@intel.com>
Reviewed-by: NDon Hiatt <don.hiatt@intel.com>
Reviewed-by: NSean Hefty <sean.hefty@intel.com>
Reviewed-by: NNiranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: NDasaratharaman Chandramouli <dasaratharaman.chandramouli@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Modify core and driver components to use accessor functions
introduced to access individual fields of rdma_ah_attr
Reviewed-by: NIra Weiny <ira.weiny@intel.com>
Reviewed-by: NDon Hiatt <don.hiatt@intel.com>
Reviewed-by: NSean Hefty <sean.hefty@intel.com>
Reviewed-by: NNiranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: NDasaratharaman Chandramouli <dasaratharaman.chandramouli@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

This patch simply renames struct ib_ah_attr to
rdma_ah_attr as these fields specify attributes that are
not necessarily specific to IB.
Reviewed-by: NIra Weiny <ira.weiny@intel.com>
Reviewed-by: NDon Hiatt <don.hiatt@intel.com>
Reviewed-by: NNiranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: NSean Hefty <sean.hefty@intel.com>
Signed-off-by: NDasaratharaman Chandramouli <dasaratharaman.chandramouli@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Split the poll responder CQ into two functions.
Add support for send+invalidate in poll CQ.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Wait for all relevant CNQ interrupts before freeing the CQ.
Don't invoke completion handlers for a destroyed CQ.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Avoid attempting to release irrelevant (and unused) resources for GSI QP.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Size of pages are held by struct ib_umem in page_size field.

It is better to store it as an exponent, because page size by nature
is always power-of-two and used as a factor, divisor or ilog2's argument.

The conversion of page_size to be page_shift allows to have portable
code and avoid following error while compiling on ARM:

  ERROR: "__aeabi_uldivmod" [drivers/infiniband/core/ib_core.ko] undefined!

CC: Selvin Xavier <selvin.xavier@broadcom.com>
CC: Steve Wise <swise@chelsio.com>
CC: Lijun Ou <oulijun@huawei.com>
CC: Shiraz Saleem <shiraz.saleem@intel.com>
CC: Adit Ranadive <aditr@vmware.com>
CC: Dennis Dalessandro <dennis.dalessandro@intel.com>
CC: Ram Amrani <Ram.Amrani@Cavium.com>
Signed-off-by: NArtemy Kovalyov <artemyko@mellanox.com>
Signed-off-by: NLeon Romanovsky <leon@kernel.org>
Acked-by: NRam Amrani <Ram.Amrani@cavium.com>
Acked-by: NShiraz Saleem <shiraz.saleem@intel.com>
Acked-by: NSelvin Xavier <selvin.xavier@broadcom.com>
Acked-by: NSelvin Xavier <selvin.xavier@broadcom.com>
Acked-by: NAdit Ranadive <aditr@vmware.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

This patch advances the qed* drivers into using the newer firmware -
This solves several firmware bugs, mostly related [but not limited to]
various init/deinit issues in various offloaded protocols.

It also introduces a major 4-Cached SGE change in firmware, which can be
seen in the storage drivers' changes.

In addition, this firmware is required for supporting the new QL41xxx
series of adapters; While this patch doesn't add the actual support,
the firmware contains the necessary initialization & firmware logic to
operate such adapters [actual support would be added later on].

Changes from Previous versions:
-------------------------------
 - V2 - fix kbuild-test robot warnings
Signed-off-by: NTomer Tayar <Tomer.Tayar@cavium.com>
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NManish Rangankar <Manish.Rangankar@cavium.com>
Signed-off-by: NChad Dupuis <Chad.Dupuis@cavium.com>
Signed-off-by: NYuval Mintz <Yuval.Mintz@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

'qedr_alloc_pbl_tbl()' can not return NULL.

In qedr_init_user_queue():
 - simplify the test for the return value, no need to test for NULL
 - propagate the error pointer if needed, otherwise 0 (success) is returned.
   This is spurious.

In init_mr_info():
 - test the return value with IS_ERR
 - propagate the error pointer if needed instead of an exlictit -ENOMEM.
   This is a no-op as the only error pointer that we can have here is
   already -ENOMEM
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Acked-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Change the drivers to call ib_query_port in their get port
immutable handler instead of their own query port handler.

Doing this required to set the core cap flags of this device
before the ib_query_port call is made, since the IB core might
need these caps to serve the port query.

Drivers are ensured by the IB core that the port attributes passed
to the port query verb implementation are zero, and hence we
removed the zeroing from the drivers.

This patch doesn't add any new functionality.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Reviewed-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NLeon Romanovsky <leon@kernel.org>
Reviewed-by: NSteve Wise <swise@opengridcomputing.com>
Acked-by: NAdit Ranadive <aditr@vmware.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Free the PD if no internal resources were available. Move userspace
code under the relevant 'if'.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Remove standalone ';'.  List function's parameters in a single line.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

mark qedr_get_state_from_ibqp(), __qedr_alloc_mr() and __qedr_post_send()
as static since they are only used in the same file.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NAriel Elior <Ariel.Elior@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Fail QP state transition from error to reset if SQ/RQ are not empty
and still in the process of flushing out the queued work entries.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NMichal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>