This is an addendum to commit 0e6e7416 ("IB/mlx4: Handle new FW
requirement for send request prefetching").  We also need to handle
prefetch marking properly for S/G segments, or else the HCA may end up
processing S/G segments that are not fully written and end up sending
the wrong data.  This can actually cause data corruption in practice,
especially on systems with relatively slow CPUs (where the HCA is more
likely to prefetch while the CPU is in the middle of writing a work
request into memory).

We write S/G segments in reverse order into the WQE, in order to
guarantee that the first dword of all cachelines containing S/G
segments is written last (overwriting the headroom invalidation
pattern).  The entire cacheline will thus contain valid data when the
invalidation pattern is overwritten.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Fix ehca SRQ support so that IPoIB connected mode works:

 - Report max_srq > 0 if SRQ is supported
 - Report "last wqe reached" asynchronous event when base QP dies;
   this is required by the IB spec and IPoIB CM relies on receiving it
   when cleaning up.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The new Small QP code had a few bugs that would also make it trigger
for non-Small QPs.  Fix them.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

cxgb3 used netdev_priv() and dev->priv for different purposes.
In 2.6.23, netdev_priv() == dev->priv, cxgb3 needs a fix.
This patch is a partial backport of Dave Miller's changes in the
net-2.6.24 git branch.

Without this fix, cxgb3 crashes on 2.6.23.
Signed-off-by: NDivy Le Ray <divy@chelsio.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

A stray semicolon makes us inadvertently ignore the value of err.
Signed-off-by: NIlpo Järvinen <ilpo.jarvinen@helsinki.fi>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Current code has a cut-and-paste error and returns IB_WC_SEND when it
should return IB_WC_RDMA_READ.
Signed-off-by: NVu Pham <vu@mellanox.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

This avoids deadlocks.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

One patch for two trivial typos of 'error' with three R's, appearing in message strings.

There's a bunch more of the same in comments, not dealt with here.
Signed-off-by: NEddy L O Jansson <eddy@klopper.net>
Cc: Roman Zippel <zippel@linux-m68k.org>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

On some system hardware, we are seeing moderately common cases of the
chip errormask register being overwritten due to a chip bug in iba6120
that is triggered by a vendor-specific PCIe broadcast message.  This
patch merely checks periodically, and corrects it if needed (the
overwrite can cause us to not get error and hardware error
interrupts).  Also, make dd->ipath_errormask the one, true canonical
source for kr_errormask, and remove references to ipath_ignorederrs as
it is currently unused.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NJohn Gregor <john.gregor@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

There was confused use of INFINIPATH_S_PIOBUFAVAILUPD (value) and
IPATH_S_PIOBUFAVAILUPD (bit position).  Also, some callers of
ipath_cancel_sends() need kr_sendctrl restored, and some want to do it
later.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

At one time the ipath_minrev field was initialized prior to the
ipath_init_iba6120_funcs call, but that is no longer the case, so the
slower put_tid routine was always being used.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The fastrcvint code's purpose was to avoid reading the interrupt
status if kernel packets were in the receive queue (to reduce
overhead).  Because intstatus was not read, we could miss the error
interrupt bit indicating freeze mode, since it only delivers a single
interrupt, even if still pending after intclear is written.

This patch removes that unsafe optimization.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Remove fs.h from mm.h. For this,
 1) Uninline vma_wants_writenotify(). It's pretty huge anyway.
 2) Add back fs.h or less bloated headers (err.h) to files that need it.

As result, on x86_64 allyesconfig, fs.h dependencies cut down from 3929 files
rebuilt down to 3444 (-12.3%).

Cross-compile tested without regressions on my two usual configs and (sigh):

alpha              arm-mx1ads        mips-bigsur          powerpc-ebony
alpha-allnoconfig  arm-neponset      mips-capcella        powerpc-g5
alpha-defconfig    arm-netwinder     mips-cobalt          powerpc-holly
alpha-up           arm-netx          mips-db1000          powerpc-iseries
arm                arm-ns9xxx        mips-db1100          powerpc-linkstation
arm-assabet        arm-omap_h2_1610  mips-db1200          powerpc-lite5200
arm-at91rm9200dk   arm-onearm        mips-db1500          powerpc-maple
arm-at91rm9200ek   arm-picotux200    mips-db1550          powerpc-mpc7448_hpc2
arm-at91sam9260ek  arm-pleb          mips-ddb5477         powerpc-mpc8272_ads
arm-at91sam9261ek  arm-pnx4008       mips-decstation      powerpc-mpc8313_rdb
arm-at91sam9263ek  arm-pxa255-idp    mips-e55             powerpc-mpc832x_mds
arm-at91sam9rlek   arm-realview      mips-emma2rh         powerpc-mpc832x_rdb
arm-ateb9200       arm-realview-smp  mips-excite          powerpc-mpc834x_itx
arm-badge4         arm-rpc           mips-fulong          powerpc-mpc834x_itxgp
arm-carmeva        arm-s3c2410       mips-ip22            powerpc-mpc834x_mds
arm-cerfcube       arm-shannon       mips-ip27            powerpc-mpc836x_mds
arm-clps7500       arm-shark         mips-ip32            powerpc-mpc8540_ads
arm-collie         arm-simpad        mips-jazz            powerpc-mpc8544_ds
arm-corgi          arm-spitz         mips-jmr3927         powerpc-mpc8560_ads
arm-csb337         arm-trizeps4      mips-malta           powerpc-mpc8568mds
arm-csb637         arm-versatile     mips-mipssim         powerpc-mpc85xx_cds
arm-ebsa110        i386              mips-mpc30x          powerpc-mpc8641_hpcn
arm-edb7211        i386-allnoconfig  mips-msp71xx         powerpc-mpc866_ads
arm-em_x270        i386-defconfig    mips-ocelot          powerpc-mpc885_ads
arm-ep93xx         i386-up           mips-pb1100          powerpc-pasemi
arm-footbridge     ia64              mips-pb1500          powerpc-pmac32
arm-fortunet       ia64-allnoconfig  mips-pb1550          powerpc-ppc64
arm-h3600          ia64-bigsur       mips-pnx8550-jbs     powerpc-prpmc2800
arm-h7201          ia64-defconfig    mips-pnx8550-stb810  powerpc-ps3
arm-h7202          ia64-gensparse    mips-qemu            powerpc-pseries
arm-hackkit        ia64-sim          mips-rbhma4200       powerpc-up
arm-integrator     ia64-sn2          mips-rbhma4500       s390
arm-iop13xx        ia64-tiger        mips-rm200           s390-allnoconfig
arm-iop32x         ia64-up           mips-sb1250-swarm    s390-defconfig
arm-iop33x         ia64-zx1          mips-sead            s390-up
arm-ixp2000        m68k              mips-tb0219          sparc
arm-ixp23xx        m68k-amiga        mips-tb0226          sparc-allnoconfig
arm-ixp4xx         m68k-apollo       mips-tb0287          sparc-defconfig
arm-jornada720     m68k-atari        mips-workpad         sparc-up
arm-kafa           m68k-bvme6000     mips-wrppmc          sparc64
arm-kb9202         m68k-hp300        mips-yosemite        sparc64-allnoconfig
arm-ks8695         m68k-mac          parisc               sparc64-defconfig
arm-lart           m68k-mvme147      parisc-allnoconfig   sparc64-up
arm-lpd270         m68k-mvme16x      parisc-defconfig     um-x86_64
arm-lpd7a400       m68k-q40          parisc-up            x86_64
arm-lpd7a404       m68k-sun3         powerpc              x86_64-allnoconfig
arm-lubbock        m68k-sun3x        powerpc-cell         x86_64-defconfig
arm-lusl7200       mips              powerpc-celleb       x86_64-up
arm-mainstone      mips-atlas        powerpc-chrp32
Signed-off-by: NAlexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Make sure declarations stay in sync with definitions by keeping all 
extern declarations in common .h files.
Signed-off-by: NHoang-Nam Nguyen <hnguyen@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Remove extra dumb-looking blank line that snuck in somehow.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Include <rdma/...> headers after <asm/...> headers.
Signed-off-by: NHoang-Nam Nguyen <hnguyen@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Fix a crash if the driver has to wait for a QP reference to be dropped
when destroying the QP.
Signed-off-by: NEthan Burns <eaburns@iol.unh.edu>
Acked-by: NTom Tucker <tom@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

eHCA2 supports QP queues that can be as small as 512 bytes. This
greatly reduces memory overhead for consumers that use lots of QPs
with small queues (e.g. RDMA-only QPs). Apart from dealing with
firmware, this code needs to manage bite-sized chunks of kernel pages,
making sure that no kernel page is shared between different protection
domains.
Signed-off-by: NHoang-Nam Nguyen <hnguyen@de.ibm.com>

They're only used in ehca_qp.c, so make them static to that file.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

ehca2ib_return_code() is not used in any fast path, and making it
non-inline saves ~1.5K of code.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Add support for MR pages larger than 4K on eHCA2. This reduces
firmware memory consumption.  If enabled via the mr_largepage module
parameter, the MR page size will be determined based on the MR length
and the hardware capabilities -- if the MR is >= 16M, 16M pages are
used, for example.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

    
The error handling code at err_wrid in create_qp_common() does not
handle a userspace QP attached to an SRQ correctly, since it ends up
in the else clause of the if statement.  This means it tries to
kfree() the uninitialized qp->sq.wrid and qp->rq.wrid pointers.  Fix
this so we only free the wrid arrays for kernel QPs.

Pointed out by Michael S. Tsirkin <mst@dev.mellanox.co.il>.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The FW command token is currently only updated on a command completion
event. This means that on command timeout, the same token will be
reused for new command, which results in a mess if the timed out
command *does* eventually complete.
Signed-off-by: NMichael S. Tsirkin <mst@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The ipath_layer.[ch] code was an attempt to provide a single interface
for the ipath verbs and ipath_ether code to use.  As verbs
functionality increased, the layer's functionality became insufficient
and the verbs code broke away to interface directly to the driver.
The failed attempt to get ipath_ether upstream was the final nail in
the coffin and now it sits quietly in a dark kernel.org corner waiting
for someone to notice the smell and send it along to it's final
resting place.  Roland Dreier was that someone -- this patch expands
on his work...
Signed-off-by: NArthur Jones <arthur.jones@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Temporarily allocated struct mlx4_qp_context *context is leaked by
several error paths.  The patch takes advantage of the return value
'err' being preinitialized to -EINVAL.

Spotted by Coverity (CID 1768).
Signed-off-by: NFlorin Malita <fmalita@gmail.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Slab destructors were no longer supported after Christoph's
c59def9f change. They've been
BUGs for both slab and slub, and slob never supported them
either.

This rips out support for the dtor pointer from kmem_cache_create()
completely and fixes up every single callsite in the kernel (there were
about 224, not including the slab allocator definitions themselves,
or the documentation references).
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Transform some calls to kmalloc/memset to a single kzalloc (or kcalloc).

Here is a short excerpt of the semantic patch performing
this transformation:

@@
type T2;
expression x;
identifier f,fld;
expression E;
expression E1,E2;
expression e1,e2,e3,y;
statement S;
@@

 x =
- kmalloc
+ kzalloc
  (E1,E2)
  ...  when != \(x->fld=E;\|y=f(...,x,...);\|f(...,x,...);\|x=E;\|while(...) S\|for(e1;e2;e3) S\)
- memset((T2)x,0,E1);

@@
expression E1,E2,E3;
@@

- kzalloc(E1 * E2,E3)
+ kcalloc(E1,E2,E3)

[akpm@linux-foundation.org: get kcalloc args the right way around]
Signed-off-by: NYoann Padioleau <padator@wanadoo.fr>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Acked-by: NRussell King <rmk@arm.linux.org.uk>
Cc: Bryan Wu <bryan.wu@analog.com>
Acked-by: NJiri Slaby <jirislaby@gmail.com>
Cc: Dave Airlie <airlied@linux.ie>
Acked-by: NRoland Dreier <rolandd@cisco.com>
Cc: Jiri Kosina <jkosina@suse.cz>
Acked-by: NDmitry Torokhov <dtor@mail.ru>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NMauro Carvalho Chehab <mchehab@infradead.org>
Acked-by: NPierre Ossman <drzeus-list@drzeus.cx>
Cc: Jeff Garzik <jeff@garzik.org>
Cc: "David S. Miller" <davem@davemloft.net>
Acked-by: NGreg KH <greg@kroah.com>
Cc: James Bottomley <James.Bottomley@steeleye.com>
Cc: "Antonino A. Daplas" <adaplas@pol.net>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

    
Current code sets size0 to 0 at the start of work request posting
functions and then handles size0 == 0 specially within the loop over
work requests.  Change this so size0 is set along with f0 the first
time through the loop (when nreq == 0).  This makes the code easier to
understand by making it clearer that f0 and size0 are always
initialized if nreq != 0 without having to know that size0 == 0
implies nreq == 0.

Also annotate size0 with uninitialized_var() so that this doesn't
introduce a new compiler warning.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

    
Factor code to set UD entries out of the work request posting
functions into inline functions set_tavor_ud_seg() and
set_arbel_ud_seg().  This doesn't change the generated code in any
significant way, and makes the source easier on the eyes.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

    
Factor code to set remote address and atomic segment entries out of the
work request posting functions into inline functions set_raddr_seg()
and set_atomic_seg().  This doesn't change the generated code in any
significant way, and makes the source easier on the eyes.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Factor code to set remote address, atomic and datagram segments out of
mlx4_ib_post_send() into small helper functions.  This doesn't change
the generated code in any significant way, and makes the source easier
on the eyes.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Factor code to set data segment entries out of mlx4_ib_post_send()
into set_data_seg().  This cleans up the code and lets the compiler do
a better job -- on x86_64:

add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-16 (-16)
function                                     old     new   delta
mlx4_ib_post_send                           1598    1582     -16
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Factor code to set data segment entries out of the work request
posting functions into inline functions mthca_set_data_seg() and
mthca_set_data_seg_inval().  This makes the code more readable and
also allows the compiler to do a better job -- on x86_64:

add/remove: 0/0 grow/shrink: 0/6 up/down: 0/-69 (-69)
function                                     old     new   delta
mthca_arbel_post_srq_recv                    373     369      -4
mthca_arbel_post_receive                     570     562      -8
mthca_tavor_post_srq_recv                    520     508     -12
mthca_tavor_post_send                       1344    1330     -14
mthca_arbel_post_send                       1481    1467     -14
mthca_tavor_post_receive                     792     775     -17
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Return the receive queue sizes for both userspace QPs and kernel Qps
(not just kernel QPs) from mlx4_ib_query_qp().  Also zero the send
queue sizes for userspace QPs to avoid a possible information leak,
and set the max_inline_data for kernel QPs to 0 since inline sends are
not supported for kernel QPs.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Commit 9db48926 ("drivers/infiniband/hw/mthca/mthca_qp: kill uninit'd
var warning") added "= 0" to the declarations of f0 to shut up gcc
warnings.  However, there's no point in making the code bigger by
initializing f0 to a random value just to get rid of a warning;
setting f0 to 0 is no safer than just using uninitialized_var(), which
documents the situation better and gives smaller code too.  For example, 
on x86_64:

add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-16 (-16)
function                                     old     new   delta
mthca_tavor_post_send                       1352    1344      -8
mthca_arbel_post_send                       1489    1481      -8
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

It has no callers and is completely dead code.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Make some functions that are only used in a single .c file static.  In
addition to being a cleanup, this shrinks the generated code.  On x86_64:

add/remove: 1/3 grow/shrink: 2/1 up/down: 4777/-4956 (-179)
function                                     old     new   delta
handle_errors                                  -    3994   +3994
__verbs_timer                                 42     710    +668
ipath_do_ruc_send                           2131    2246    +115
ipath_no_bufs_available                      136       -    -136
ipath_disarm_senderrbufs                     639       -    -639
ipath_ib_timer                               658       -    -658
ipath_intr                                  5878    2355   -3523
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

When warning about out-of-date firmware, current mthca code messes up
the formatting of the version if the subminor doesn't have three
digits.  It doesn't fill the field with 0s so we end up with:

    ib_mthca 0000:0b:00.0: HCA FW version 1.1.  0 is old (1.2.  0 is current).

Change the format from "%3d" to "%03d" to get the right thing printed.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The mthca driver supports both MSI and MSI-X.  However, MSI-X works with
all hardware that the driver handles, and provides a superset of what
MSI does, so there's no point in having code for both.  Schedule MSI
support for removal in 2008 to give anyone who actually needs MSI and
who can't use MSI time to speak up.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>