T4 EQ entries are in multiples of 64 bytes.  Currently the RDMA SQ and
RQ use fixed sized entries composed of 4 EQ entries for the SQ and 2
EQ entries for the RQ.  For optimial latency with small IO, we need to
change this so the HW only needs to DMA the EQ entries actually used
by a given work request.

Implementation:

- add wq_pidx counter to track where we are in the EQ.  cidx/pidx are
  used for the sw sq/rq tracking and flow control.

- the variable part of work requests is the SGL.  Add new functions to
  build the SGL and/or immediate data directly in the EQ memory
  wrapping when needed.

- adjust the min burst size for the EQ contexts to 64B.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The alloc_skb() in various allocations are failable, so remove
__GFP_NOFAIL from their masks.
Signed-off-by: NDavid Rientjes <rientjes@google.com>
Acked-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The rest of the code seems to assume that ep->com.cm_id can't be NULL,
so remove an unneeded test.
Reported-by: NDan Carpenter <error27@gmail.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

We don't need to assign rpl here, we do that later on.
Signed-off-by: NDan Carpenter <error27@gmail.com>

[ Indeed this assignment makes no sense, since skb is set to NULL a
  couple of lines before.  - Roland ]
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Extract the microcode for the QLogic QLE7220 series IB HCA and use the
kernel microcode request facility to load the microcode.  This
supports Debian Linux's requirements to separate microcode which
doesn't have open source code available from the device driver.
Signed-off-by: NBen Hutchings <ben@decadent.org.uk>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

If qib_init() fails, the driver fails to free memory, unregister
device files, and unregister with the PCIe framework. The driver will
unload without error but a subsequent driver load will cause the
system to panic.  This was found by changing the 7220 code to load the
serdes microcode separately and not installing the microcode file.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Workqueues aren't exactly equivalent to tasklets since the callback
function may be called from multiple CPUs before the callback returns.
This causes completion notification callbacks to have MT bugs since
they weren't expecting this behavior. The fix is to use a single
threaded work queue.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The hardware error register needs to be cleared or another interrupt
will be generated, thus causing an infinite loop.  This is a
regression introduced when removing debug output.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The eager buffers are not being cleared before being mmapped into a
new user address space.  This is a potential security risk and should
be fixed.  Note that the eager header queue is already being cleared.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The HCA checks for certain hardware errors which can be falsely
triggered when the IB link is reset. The fix is to mask them rather
than report them.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Don't set write combining via PAT on the VL15 buffers to avoid a rare
problem with unaligned writes from interrupt-flushed store buffers.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The T4 IQ hw design assumes CIDX_INC credits will be returned on a
regular basis and always before the CIDX counter crosses over the PIDX
counter.  For RDMA CQs, however, returning CIDX_INC credits is only
needed and desired when and if the CQ is armed for notification.  This
can lead to a GTS write returning credits that causes the HW to reject
the credit update because it causes CIDX to pass PIDX.  Once this
happens, the CIDX/PIDX counters get out of whack and an application
can miss a notification and get stuck blocked awaiting a notification.

To avoid this, we allocate the HW IQ 2x times the requested size.
This seems to avoid the false overflow failures.  If we see more
issues with this, then we'll have to add code in the poll path to
return credits periodically like when the amount reaches 1/2 the queue
depth).  I would like to avoid this as it adds a PCI write transaction
for applications that never arm the CQ (like most MPIs).
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

This replace the PCI DMA state API (include/linux/pci-dma.h) with the
DMA equivalents since the PCI DMA state API will be obsolete.

No functional change.

For further information about the background:

http://marc.info/?l=linux-netdev&m=127037540020276&w=2Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

get_sb_single() calls fill_super with superblock locked; calling
deactivate_super() will deadlock immedately.  Moreover, if fill_super
callback returns an error, get_sb_single() will release the reference
to superblock itself just fine.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

The DCA code was left over from internal development to test the
hardware feature and allow performance testing.  The results were
mixed and will require some additional work to make full use of the
feature.  Therefore, it is being removed for now.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

By the previous modification, the cpu notifier can return encapsulate
errno value. This converts the cpu notifiers for ehca.
Signed-off-by: NAkinobu Mita <akinobu.mita@gmail.com>
Cc: Hoang-Nam Nguyen <hnguyen@de.ibm.com>
Cc: Christoph Raisch <raisch@de.ibm.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

As part of the earlier patches submitted and reviewed, it was agreed
to change the way serdes tuning parameters were specified to the
driver.  The updated patch got dropped by the linux-rdma email list so
the earlier version of qib_iba7322.c ended up being used.  This patch
updates qib_iab7322.c to the simpler, single parameter method of
setting the serdes parameters.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Some of the qib sysfs code passes a buffer pointer into 
simple_read_from_buffer() but relies on a function call in another 
parameter of the same call to initialize that pointer.  Since the order
of evaluation of function parameters is undefined, this will break if
gcc chooses the wrong order.

Fix this by splitting the code into two separate function calls.

This was noticed because of warnings like the following on ppc:

    drivers/infiniband/hw/qib/qib_fs.c: In function 'portcntrs_2_read':
    drivers/infiniband/hw/qib/qib_fs.c:203: warning: 'counters' is used uninitialized in this function
Reported-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

This patch fixes a compile error saying qib_init_iba6120_funcs() is
undefined when CONFIG_PCI_MSI is not defined.  Thanks to Randy Dunlap
<randy.dunlap@oracle.com> for finding this and suggesting the fix.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Commit ce6e74f2 ("RDMA/nes: Make nesadapter->phy_lock usage
consistent") introduced a problem where phy_lock was only unlocked
within an if statement and so nes_process_mac_intr() could return with
phy_lock still held.  Fix this.

This was discovered because of the sparse warning:

    drivers/infiniband/hw/nes/nes_hw.c:2643:9: warning: context imbalance in 'nes_process_mac_intr' - different lock contexts for basic block
Reported-by: NRoland Dreier <rdreier@cisco.com>
Signed-off-by: NChien Tung <chien.tin.tung@intel.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Under abnormal termination, modify_qp() closes the QP, and async event
(AE) handling also attempts to close the same QP, causing a crash.
Fix this by checking the state of the QP before processing the AE.
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Enhance ethtool to read hardware registers for rcv/tx error stats.
Also add support for free pbl resources.  Remove cq depth stats, which
are not used.
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

- wrap cq->cqidx_inc based on cq size.
- optimize t4_arm_cq logic.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

1) save the timestamp flit in the cq when we consume a CQE.

2) always compare the saved flit with the previous entry flit when
   reading the next CQE entry.  If the flits don't compare, then we
   have overflowed.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

We need 1 extra entry for the status page and 1 to always have 1 free
entry to detect when the queue is full.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The LLD now supports proper UP state change events, so move the RDMA
provider registration to UP path.

This fixes a crash when loading iw_cxgb4 _after_ the NFS/RDMA
transport is up and running.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

In the RDMA core unregister path, kernel users will be calling down
into the T4 provider to release resources.  So we cannot detach from
the LLD until this process completes.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The ib_qib driver is taking over support for QLogic PCIe QLE devices,
so remove support for them from ib_ipath.  The ib_ipath driver now
supports only the obsolete QLogic Hyper-Transport IB host channel
adapter (model QHT7140).
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Add a low-level IB driver for QLogic PCIe adapters.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

.name, .match_table and .owner are duplicated in both of_platform_driver
and device_driver.  This patch is a removes the extra copies from struct
of_platform_driver and converts all users to the device_driver members.

This patch is a pretty mechanical change.  The usage model doesn't change
and if any drivers have been missed, or if anything has been fixed up
incorrectly, then it will fail with a compile time error, and the fixup
will be trivial.  This patch looks big and scary because it touches so
many files, but it should be pretty safe.
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>
Acked-by: NSean MacLennan <smaclennan@pikatech.com>