Child_listen_node pointer is used in a debug print after kfree.
Move the print before kfree.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NShiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Fix size parameter passed to i40iw_reg_phys_mr and use it to
register memory.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NShiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Fix incorrect usage of ENOSYS and other return codes.
Signed-off-by: NShiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

And remove sysfs support in favor of the core version.
Reviewed-by: NDennis Dalessandro <dennis.dalessandro@intel.com>
Signed-off-by: NIra Weiny <ira.weiny@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

In practice, each RDMA device has a unique set of counters that the
hardware implements.  Having a central set of counters that they must
all adhere to is limiting and causes many useful counters to not be
available.

Therefore we create a dynamic counter registration infrastructure.

The driver must implement a stats structure allocation routine, in
which the driver must place the directory name it wants, a list of
names for all of the counters, an array of u64 counters themselves,
plus a few generic configuration options.

We then implement a core routine to create a sysfs file for each
of the named stats elements, and a core routine to retrieve the
stats when any of the sysfs attribute files are read.

To avoid excessive beating on the stats generation routine in the
drivers, the core code also caches the stats for a short period of
time so that someone attempting to read all of the stats in a
given device's directory will not result in a stats generation
call per file read.

Future work will attempt to standardize just the shared stats
elements, and possibly add a method to get the stats via netlink
in addition to sysfs.
Signed-off-by: NChristoph Lameter <cl@linux.com>
Signed-off-by: NMark Bloch <markb@mellanox.com>
Reviewed-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>
[ Add caching, make structure names more informative, add i40iw support,
  other significant rewrites from the original patch ]

passing hw_stats by value requires a 280 byte copy so instead
pass it by reference is much more efficient.
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Acked-by: NChien Tin Tung <chien.tin.tung@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Calling synchronize_irq() right before free_irq() is quite useless. On one
hand the IRQ can easily fire again before free_irq() is entered, on the
other hand free_irq() itself calls synchronize_irq() internally (in a race
condition free way), before any state associated with the IRQ is freed.

Patch was generated using the following semantic patch:
// <smpl>
@@
expression irq;
@@
-synchronize_irq(irq);
 free_irq(irq, ...);
// </smpl>
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Acked-by: Faisal Latif <faisal.latif#intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The i40iw_vf_cqp_ops structure is never modified, so declare it as const.

Done with the help of Coccinelle.
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

passing hw_stats by value requires a 280 byte copy so instead
pass it by reference is much more efficient.
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Acked-by: NChien Tin Tung <chien.tin.tung@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Calling synchronize_irq() right before free_irq() is quite useless. On one
hand the IRQ can easily fire again before free_irq() is entered, on the
other hand free_irq() itself calls synchronize_irq() internally (in a race
condition free way), before any state associated with the IRQ is freed.

Patch was generated using the following semantic patch:
// <smpl>
@@
expression irq;
@@
-synchronize_irq(irq);
 free_irq(irq, ...);
// </smpl>
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Acked-by: Faisal Latif <faisal.latif#intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The i40iw_vf_cqp_ops structure is never modified, so declare it as const.

Done with the help of Coccinelle.
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

passing hw_stats by value requires a 280 byte copy so instead
pass it by reference is much more efficient.
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Acked-by: NChien Tin Tung <chien.tin.tung@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Calling synchronize_irq() right before free_irq() is quite useless. On one
hand the IRQ can easily fire again before free_irq() is entered, on the
other hand free_irq() itself calls synchronize_irq() internally (in a race
condition free way), before any state associated with the IRQ is freed.

Patch was generated using the following semantic patch:
// <smpl>
@@
expression irq;
@@
-synchronize_irq(irq);
 free_irq(irq, ...);
// </smpl>
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Acked-by: Faisal Latif <faisal.latif#intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The i40iw_vf_cqp_ops structure is never modified, so declare it as const.

Done with the help of Coccinelle.
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The i40e_client_ops structure is never modified, so declare it as const.

Done with the help of Coccinelle.
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The SRP initiator allows to set max_sectors to a value that exceeds
the largest amount of data that can be mapped at once with an mlx4
HCA using fast registration and a page size of 4 KB. Hence modify
ib_map_mr_sg() such that it can map partial sg-elements. If an
sg-element has been mapped partially, let the caller know
which fraction has been mapped by adjusting *sg_offset.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Tested-by: NLaurence Oberman <loberman@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Tested-by: NSteve Wise <swise@opengridcomputing.com>
Reviewed-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The i40e_client_ops structure is never modified, so declare it as const.

Done with the help of Coccinelle.
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Fix for removing a quad hash entry when the
corresponding quad hash entry hasn't been added,
which is the case in loopback connections
Signed-off-by: NTatyana Nikolova <Tatyana.E.Nikolova@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Fix for checking if the QP associated with a completion
has been destroyed while processing CQ elements.
If that is the case, move the CQ head to the next element
and continue completion processing.
Signed-off-by: NTatyana Nikolova <tatyana.e.nikolova@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

A check is added to validate the requested sge number.
iWARP doesn't support multiple sg elements for
RDMA READ work requests.
Signed-off-by: NTatyana Nikolova <tatyana.e.nikolova@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Fix to calculate the SQ size based on the max
frag_count, requested by the application instead
of overwriting it with the max supported frag_count
Signed-off-by: NTatyana Nikolova <tatyana.e.nikolova@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Fix for filling in the WQE size for NOP
Signed-off-by: NTatyana Nikolova <tatyana.e.nikolova@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

STag index mask is calculated incorrectly, missing
the 14 bits minimum requirement. Add max macro to use
either # of MRs or 14 bits in the mask size calculation.
Signed-off-by: NTatyana Nikolova <Tatyana.E.Nikolova@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Invalidation after every WQE write is changed to invalidate
only if required. NOPs are padded so that WQE writes are
aligned to 64B boundary.
Signed-off-by: NTatyana Nikolova <tatyana.e.nikolova@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Adding sq and rq drain functions, which block until all
previously posted wr-s in the specified queue have completed.
A completion object is signaled to unblock the thread,
when the last cqe for the corresponding queue is processed.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Correct SD calculation by using base address returned from commit FPM.
This alleviates any assumptions on resource ordering and alignment
requirement. Also consolidate SD estimation code into i40iw_est_sd().
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Fix endian warnings and errors due to u32 stored to u16.
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Implement fast register mr, Local invalidate, send with
invalidate and RDMA read with invalidate.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Initialize max enabled vfs to max rdma vfs instead of 0.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Move return code check to immediately after i40iw_hmc_sd_one call
where it is set instead of outside the then statement.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Queue users of virtual channel on a waitqueue until the channel is
clear instead of failing the call when the channel is occupied.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Remove unused code and fix warning.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Populate PCI info fields from PCI device structure.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Add a check for cq_poll_info.error before setting vendor_err
instead of always setting it.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

QP may be freed during Async Event processing.
Add a lock around QP table to prevent it.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

iwqp->allocated_buffer is a self-referencing pointer to iwqp.
Do not set iwqp->allocated_buffer to NULL after freeing it.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Fix to correct max reported message size in query port.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Make sure cm_node is setup before sending SYN packet and
ORD/IRD negotiation.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Include inline data size as part of SQ size calculation.
RQ size calculation uses only number of SGEs and does not
support 96 byte WQE size.
Signed-off-by: NMustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>