The code to post UD sends tried to process work requests at the time
ib_post_send() is called without using a WQE queue.  This was fine as
long as HW resources were available for sending a packet.  This patch
changes UD to be handled more like RC and UC and shares more code.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Different processors have different ordering restrictions for write
combining.  By taking advantage of this, we can eliminate some write
barriers when writing to the send buffers.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

On iba6110 rev4, support for three more IB counters were added.  The
LocalLinkIntegrityError counter, the ExcessiveBufferOverrunErrors
counter and support for error counting of flow control packets on an
invalid VL.  These counters trigger GPIO interrupts and the sw keeps
track of the counts.  Since we also use GPIO interrupts to signal packet
reception, we need to turn off the fast interrupts, or we risk losing a
GPIO interrupt.
Signed-off-by: NArthur Jones <arthur.jones@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

On some system hardware, we are seeing moderately common cases of the
chip errormask register being overwritten due to a chip bug in iba6120
that is triggered by a vendor-specific PCIe broadcast message.  This
patch merely checks periodically, and corrects it if needed (the
overwrite can cause us to not get error and hardware error
interrupts).  Also, make dd->ipath_errormask the one, true canonical
source for kr_errormask, and remove references to ipath_ignorederrs as
it is currently unused.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NJohn Gregor <john.gregor@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

There was confused use of INFINIPATH_S_PIOBUFAVAILUPD (value) and
IPATH_S_PIOBUFAVAILUPD (bit position).  Also, some callers of
ipath_cancel_sends() need kr_sendctrl restored, and some want to do it
later.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

At one time the ipath_minrev field was initialized prior to the
ipath_init_iba6120_funcs call, but that is no longer the case, so the
slower put_tid routine was always being used.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The fastrcvint code's purpose was to avoid reading the interrupt
status if kernel packets were in the receive queue (to reduce
overhead).  Because intstatus was not read, we could miss the error
interrupt bit indicating freeze mode, since it only delivers a single
interrupt, even if still pending after intclear is written.

This patch removes that unsafe optimization.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Remove fs.h from mm.h. For this,
 1) Uninline vma_wants_writenotify(). It's pretty huge anyway.
 2) Add back fs.h or less bloated headers (err.h) to files that need it.

As result, on x86_64 allyesconfig, fs.h dependencies cut down from 3929 files
rebuilt down to 3444 (-12.3%).

Cross-compile tested without regressions on my two usual configs and (sigh):

alpha              arm-mx1ads        mips-bigsur          powerpc-ebony
alpha-allnoconfig  arm-neponset      mips-capcella        powerpc-g5
alpha-defconfig    arm-netwinder     mips-cobalt          powerpc-holly
alpha-up           arm-netx          mips-db1000          powerpc-iseries
arm                arm-ns9xxx        mips-db1100          powerpc-linkstation
arm-assabet        arm-omap_h2_1610  mips-db1200          powerpc-lite5200
arm-at91rm9200dk   arm-onearm        mips-db1500          powerpc-maple
arm-at91rm9200ek   arm-picotux200    mips-db1550          powerpc-mpc7448_hpc2
arm-at91sam9260ek  arm-pleb          mips-ddb5477         powerpc-mpc8272_ads
arm-at91sam9261ek  arm-pnx4008       mips-decstation      powerpc-mpc8313_rdb
arm-at91sam9263ek  arm-pxa255-idp    mips-e55             powerpc-mpc832x_mds
arm-at91sam9rlek   arm-realview      mips-emma2rh         powerpc-mpc832x_rdb
arm-ateb9200       arm-realview-smp  mips-excite          powerpc-mpc834x_itx
arm-badge4         arm-rpc           mips-fulong          powerpc-mpc834x_itxgp
arm-carmeva        arm-s3c2410       mips-ip22            powerpc-mpc834x_mds
arm-cerfcube       arm-shannon       mips-ip27            powerpc-mpc836x_mds
arm-clps7500       arm-shark         mips-ip32            powerpc-mpc8540_ads
arm-collie         arm-simpad        mips-jazz            powerpc-mpc8544_ds
arm-corgi          arm-spitz         mips-jmr3927         powerpc-mpc8560_ads
arm-csb337         arm-trizeps4      mips-malta           powerpc-mpc8568mds
arm-csb637         arm-versatile     mips-mipssim         powerpc-mpc85xx_cds
arm-ebsa110        i386              mips-mpc30x          powerpc-mpc8641_hpcn
arm-edb7211        i386-allnoconfig  mips-msp71xx         powerpc-mpc866_ads
arm-em_x270        i386-defconfig    mips-ocelot          powerpc-mpc885_ads
arm-ep93xx         i386-up           mips-pb1100          powerpc-pasemi
arm-footbridge     ia64              mips-pb1500          powerpc-pmac32
arm-fortunet       ia64-allnoconfig  mips-pb1550          powerpc-ppc64
arm-h3600          ia64-bigsur       mips-pnx8550-jbs     powerpc-prpmc2800
arm-h7201          ia64-defconfig    mips-pnx8550-stb810  powerpc-ps3
arm-h7202          ia64-gensparse    mips-qemu            powerpc-pseries
arm-hackkit        ia64-sim          mips-rbhma4200       powerpc-up
arm-integrator     ia64-sn2          mips-rbhma4500       s390
arm-iop13xx        ia64-tiger        mips-rm200           s390-allnoconfig
arm-iop32x         ia64-up           mips-sb1250-swarm    s390-defconfig
arm-iop33x         ia64-zx1          mips-sead            s390-up
arm-ixp2000        m68k              mips-tb0219          sparc
arm-ixp23xx        m68k-amiga        mips-tb0226          sparc-allnoconfig
arm-ixp4xx         m68k-apollo       mips-tb0287          sparc-defconfig
arm-jornada720     m68k-atari        mips-workpad         sparc-up
arm-kafa           m68k-bvme6000     mips-wrppmc          sparc64
arm-kb9202         m68k-hp300        mips-yosemite        sparc64-allnoconfig
arm-ks8695         m68k-mac          parisc               sparc64-defconfig
arm-lart           m68k-mvme147      parisc-allnoconfig   sparc64-up
arm-lpd270         m68k-mvme16x      parisc-defconfig     um-x86_64
arm-lpd7a400       m68k-q40          parisc-up            x86_64
arm-lpd7a404       m68k-sun3         powerpc              x86_64-allnoconfig
arm-lubbock        m68k-sun3x        powerpc-cell         x86_64-defconfig
arm-lusl7200       mips              powerpc-celleb       x86_64-up
arm-mainstone      mips-atlas        powerpc-chrp32
Signed-off-by: NAlexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The ipath_layer.[ch] code was an attempt to provide a single interface
for the ipath verbs and ipath_ether code to use.  As verbs
functionality increased, the layer's functionality became insufficient
and the verbs code broke away to interface directly to the driver.
The failed attempt to get ipath_ether upstream was the final nail in
the coffin and now it sits quietly in a dark kernel.org corner waiting
for someone to notice the smell and send it along to it's final
resting place.  Roland Dreier was that someone -- this patch expands
on his work...
Signed-off-by: NArthur Jones <arthur.jones@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

It has no callers and is completely dead code.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Make some functions that are only used in a single .c file static.  In
addition to being a cleanup, this shrinks the generated code.  On x86_64:

add/remove: 1/3 grow/shrink: 2/1 up/down: 4777/-4956 (-179)
function                                     old     new   delta
handle_errors                                  -    3994   +3994
__verbs_timer                                 42     710    +668
ipath_do_ruc_send                           2131    2246    +115
ipath_no_bufs_available                      136       -    -136
ipath_disarm_senderrbufs                     639       -    -639
ipath_ib_timer                               658       -    -658
ipath_intr                                  5878    2355   -3523
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Instead of all drivers reading pci config space to get the revision
ID, they can now use the pci_device->revision member.

This exposes some issues where drivers where reading a word or a dword
for the revision number, and adding useless error-handling around the
read. Some drivers even just read it for no purpose of all.

In devices where the revision ID is being copied over and used in what
appears to be the equivalent of hotpath, I have left the copy code
and the cached copy as not to influence the driver's performance.

Compile tested with make all{yes,mod}config on x86_64 and i386.
Signed-off-by: NAuke Kok <auke-jan.h.kok@intel.com>
Acked-by: NDave Jones <davej@redhat.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Return the PortGUID of the correct port when responding to a NodeInfo
query.  Returning the SystemImageGUID causes issues when there are
multiple HCAs in a single system.
Signed-off-by: NSean Hefty <sean.hefty@intel.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The changeset 3859e39d ("IB/ipath: Support larger IB_QP_MAX_DEST_RD_ATOMIC
and IB_QP_MAX_QP_RD_ATOMIC") added support for larger RD_ATOMIC values,
but it failed to take out the stricter checks that were before these and
hence had no effect.  This patch takes out the bogus checks....
Signed-off-by: NArthur Jones <arthur.jones@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

All too often, interrupts do not get enabled for our card due to BIOS
misconfiguration and other issues.  This patch checks for that
condition on startup and warns the user.  This patch is based on work
(check LID availability) by Robert Walsh.
Signed-off-by: NArthur Jones <arthur.jones@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Clean up some code by removing support for some older pre-production
HTX InfiniPath cards.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>

Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The default calculation for the number of send buffers to allocate to
the kernel was too high for the PCIe version of the chip thus leaving
fewer than desired send buffers for user MPI applications.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

We are more careful to be sure that we don't lose information about
changes that occurred while we were in freeze mode, when the chip will
not notify us, and try to avoid false error interrupts while doing
cleanup.  Put all of this logic in a new function ipath_clear_freeze().
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Add a barrier to make sure the CPU doesn't reorder writes to memory,
since user programs can be polling on the head index update and the
entry should be written before that.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Change Kconfig objects from "menu, config" into "menuconfig" so
that the user can disable the whole feature without having to
enter the menu first.
Signed-off-by: NJan Engelhardt <jengelh@gmx.de>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Now that it's June, it's about time to update
the copyright notices of files that have changed.
Signed-off-by: NJohn Gregor <john.gregor@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NRobert Walsh <robert.walsh@qlogic.com>
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Fix ipath_poll and enhance it so we can poll for urgent packets or
regular packets and receive notifications of when a header queue
overflows.
Signed-off-by: NRobert Walsh <robert.walsh@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The IB specification ch. 9.9.3 table 58 says that a QP which isn't set
up for the operation should return a NAK invalid request.
Signed-off-by: NRobert Walsh <robert.walsh@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

During compliance testing and when debugging some interconnect issues,
it is very useful to be able to send malformed packets, without having
the device signal them as malformed (drop, or terminate with EBP). The
hardware supports this, but the driver "diagnostic packet" interface
did not.

Extend capability to send specific malformed packets for testing.
Signed-off-by: NMichael Albaugh <Michael.Albaugh@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Previously the driver and userspace code handled the case of 1 subport
somewhat inconsistently.  The new interpretation of this situation is
that if one subport is requested, the driver turns on the subport
mechanism and arranges for the port to be "shared" by one process.  In
normal use the userspace library does not use this configuration and
instead arranges for the port not to be shared at all.  This
particular idiom can be useful for testing purposes.
Signed-off-by: NMark Debbage <mark.debbage@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

When subports are required to run a program, this patch checks that
the driver and the userspace library have compatible subport
implementations.  This is achieved through checks on the swminor
version field built into the driver and userspace library.  Bad
combinations are reported through syslog and result in an error when
opening the port.
Signed-off-by: NMark Debbage <mark.debbage@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

A duplicate RDMA read request can fool the responder into NAKing a new
RDMA read request because the responder wasn't keeping track of
whether the queue of RDMA read requests had been sent at least once.
For example, requester sends 4 2K byte RDMA read requests, times out,
and resends the first, then sees the 4 responses, then sends a 5th
RDMA read or atomic operation.  The responder sees the 4 requests,
sends 4 responses, sees the resent 1st request, rewinds the queue,
then sees the 5th request but thinks the queue is full and that the
requester is invalidly sending a 5th new request.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The code to copy data from the receive queue buffers to the IB SGEs
doesn't check the SGE length, only the memory region/page length when
copying data.  This could overwrite parts of the user's memory that
were not intended to be written.  It can only happen if multiple SGEs
are used to describe a receive buffer which almost never happens in
practice.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The send function is called when posting new send work requests.
There is no point in trying to send a packet if the QP is already
waiting for a HW send buffer so don't clear the busy bit until the
buffer available interrupt happens.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

A RDMA read response or atomic response can ACK earlier sends and RDMA
writes.  In this case, the wrong work request pointer was being used
to store the read first response or atomic result.  Also, if a RDMA
read request is retried, the code to compute which request to resend
was incorrect.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

This centralizes the use of the abort functionality, removes the
unneeded buffer cancel (abort does the same thing), sets up to ignore
launch errors after abort, same as cancel.  We need abort on exit from
freeze mode to avoid having buffers stuck in the busy state, if a user
process happened to complete the send while we were in freeze mode
doing the recovery.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The values passed have never been right for iba 6120 chips, but just
happened to work.  We needed to select the right buffer offset in the
chip (both are in same register), and the total length was wrong also,
but was covered by the rounding up.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Define pkt rcvd 'type' in a way consistent with HW spec and chips.

The hardware considers received packets of type 0 to be expected, and
type 1 to be eager. The driver was calling the ipath_f_put_tid
functions using a variable called 'type' set to 0 for eager and to 1
for expected packets.  Worse, the iba6110 and iba6120 drivers used
those values inconsistently.  This was quite confusing.  Now
everything is consistent with the hardware.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

According to chapter 17.2.8.1.1, QPs start in the migrated state and
should send packets with the M bit set in the BTH.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

This patch fixes a minor bug where the wrong QP was checked for a send
work request that should wait for an RNR timeout.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

This patch fixes a bug introduced when moving some code around for
readability.

Setting the wqe pointer at the end of the function is a NOP since it
isn't used.  Move it back to where it is used.
Signed-off-by: NRalph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

In ipath_query_device(), some of the struct ib_device_attr fields were
not being initialized.
Signed-off-by: NRobert Walsh <robert.walsh@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Although our chip supports 4K MTUs, our driver doesn't yet support
this feature, so limit the maximum MTU to 2K until we get support for
4K MTUs implemented.
Signed-off-by: NRobert Walsh <robert.walsh@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>