This is an addendum to commit 0e6e7416 ("IB/mlx4: Handle new FW
requirement for send request prefetching").  We also need to handle
prefetch marking properly for S/G segments, or else the HCA may end up
processing S/G segments that are not fully written and end up sending
the wrong data.  This can actually cause data corruption in practice,
especially on systems with relatively slow CPUs (where the HCA is more
likely to prefetch while the CPU is in the middle of writing a work
request into memory).

We write S/G segments in reverse order into the WQE, in order to
guarantee that the first dword of all cachelines containing S/G
segments is written last (overwriting the headroom invalidation
pattern).  The entire cacheline will thus contain valid data when the
invalidation pattern is overwritten.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Fix ehca SRQ support so that IPoIB connected mode works:

 - Report max_srq > 0 if SRQ is supported
 - Report "last wqe reached" asynchronous event when base QP dies;
   this is required by the IB spec and IPoIB CM relies on receiving it
   when cleaning up.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The new Small QP code had a few bugs that would also make it trigger
for non-Small QPs.  Fix them.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

cxgb3 used netdev_priv() and dev->priv for different purposes.
In 2.6.23, netdev_priv() == dev->priv, cxgb3 needs a fix.
This patch is a partial backport of Dave Miller's changes in the
net-2.6.24 git branch.

Without this fix, cxgb3 crashes on 2.6.23.
Signed-off-by: NDivy Le Ray <divy@chelsio.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

A stray semicolon makes us inadvertently ignore the value of err.
Signed-off-by: NIlpo Järvinen <ilpo.jarvinen@helsinki.fi>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

ipoib_transport_dev_init() calls ipoib_cm_dev_init(), so it needs to
call ipoib_cm_dev_cleanup() to unwind that on the error path.

Found by Dotan Barak of Mellanox.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Current code has a cut-and-paste error and returns IB_WC_SEND when it
should return IB_WC_RDMA_READ.
Signed-off-by: NVu Pham <vu@mellanox.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

New Cisco IB SRP targets use the Cisco OUI 00-1b-0d but still need the
Topspin workarounds.  Add this OUI to srp_target_is_topspin().
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Wrap the checking for Mellanox and Topspin OUIs to decide whether to
use a workaround into helper functions.  This will make it cleaner to
add a new OUI to check (as we need to do now that some targets with a
Cisco OUI still need the Topspin workarounds).
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

This avoids deadlocks.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

After moving the definition of struct ib_umem_chunk from ib_verbs.h to
ib_umem.h there isn't any reason for the macro IB_UMEM_MAX_PAGE_CHUNK
to stay in ib_verbs.h.  Move the macro to umem.c, the only place where
it is used.
Signed-off-by: NDotan Barak <dotanb@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The address handle associated with dual-sided RMPP direction switch
ACKs is never destroyed.  Free the AH for ACKs which fall into this
category.

Problem was reported by Dotan Barak <dotanb@dev.mellanox.co.il>.
Signed-off-by: NSean Hefty <sean.hefty@intel.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Nothing looks at the return value of agent_send_response(), so there's
no point in returning anything.
Signed-off-by: NHal Rosenstock <hal.rosenstock@gmail.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

If agent_send_response() returns an error, we shouldn't do anything
differently than if it succeeds; setting response to NULL just means
that the response buffer gets leaked.
Signed-off-by: NSuresh Shelvapille <suri@baymicrosystems.com>
Signed-off-by: NHal Rosenstock <hal.rosenstock@gmail.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

If ib_mad_recv_done_handler() fails to allocate response, then it just
printed a warning and continued, which leads to an oops if the MAD is
being handled for a switch device, because the switch code uses
response without checking for NULL.  Fix this by bailing out of the
function if the allocation fails.
Signed-off-by: NSuresh Shelvapille <suri@baymicrosystems.com>
Signed-off-by: NHal Rosenstock <hal.rosenstock@gmail.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Now that ib_find_pkey() ignores the membership bit of P_Keys, there's no
need for ib_sa to look for both 0x7fff and 0xffff in a port's P_Key table.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

ib_find_pkey() is used as a replacement for ib_find_cached_pkey(), and
the original function ignored the membership bit when searching for a
P_Key, so ib_find_pkey() should ignore the bit too.

In particular, IPoIB turns on the P_Key membership bit of limited
membership P_Keys when creating a child interface and looks for the
full membership P_key.  This broke if a port was a partial member of a
partition when IPoIB switched from ib_find_cached_pkey() to
ib_find_pkey(), and this change fixes things again.
Signed-off-by: NMoni Shoua <monis@voltaire.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

One patch for two trivial typos of 'error' with three R's, appearing in message strings.

There's a bunch more of the same in comments, not dealt with here.
Signed-off-by: NEddy L O Jansson <eddy@klopper.net>
Cc: Roman Zippel <zippel@linux-m68k.org>
Cc: Roland Dreier <rolandd@cisco.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

On some system hardware, we are seeing moderately common cases of the
chip errormask register being overwritten due to a chip bug in iba6120
that is triggered by a vendor-specific PCIe broadcast message.  This
patch merely checks periodically, and corrects it if needed (the
overwrite can cause us to not get error and hardware error
interrupts).  Also, make dd->ipath_errormask the one, true canonical
source for kr_errormask, and remove references to ipath_ignorederrs as
it is currently unused.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NJohn Gregor <john.gregor@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

There was confused use of INFINIPATH_S_PIOBUFAVAILUPD (value) and
IPATH_S_PIOBUFAVAILUPD (bit position).  Also, some callers of
ipath_cancel_sends() need kr_sendctrl restored, and some want to do it
later.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

At one time the ipath_minrev field was initialized prior to the
ipath_init_iba6120_funcs call, but that is no longer the case, so the
slower put_tid routine was always being used.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The fastrcvint code's purpose was to avoid reading the interrupt
status if kernel packets were in the receive queue (to reduce
overhead).  Because intstatus was not read, we could miss the error
interrupt bit indicating freeze mode, since it only delivers a single
interrupt, even if still pending after intclear is written.

This patch removes that unsafe optimization.
Signed-off-by: NDave Olson <dave.olson@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Remove fs.h from mm.h. For this,
 1) Uninline vma_wants_writenotify(). It's pretty huge anyway.
 2) Add back fs.h or less bloated headers (err.h) to files that need it.

As result, on x86_64 allyesconfig, fs.h dependencies cut down from 3929 files
rebuilt down to 3444 (-12.3%).

Cross-compile tested without regressions on my two usual configs and (sigh):

alpha              arm-mx1ads        mips-bigsur          powerpc-ebony
alpha-allnoconfig  arm-neponset      mips-capcella        powerpc-g5
alpha-defconfig    arm-netwinder     mips-cobalt          powerpc-holly
alpha-up           arm-netx          mips-db1000          powerpc-iseries
arm                arm-ns9xxx        mips-db1100          powerpc-linkstation
arm-assabet        arm-omap_h2_1610  mips-db1200          powerpc-lite5200
arm-at91rm9200dk   arm-onearm        mips-db1500          powerpc-maple
arm-at91rm9200ek   arm-picotux200    mips-db1550          powerpc-mpc7448_hpc2
arm-at91sam9260ek  arm-pleb          mips-ddb5477         powerpc-mpc8272_ads
arm-at91sam9261ek  arm-pnx4008       mips-decstation      powerpc-mpc8313_rdb
arm-at91sam9263ek  arm-pxa255-idp    mips-e55             powerpc-mpc832x_mds
arm-at91sam9rlek   arm-realview      mips-emma2rh         powerpc-mpc832x_rdb
arm-ateb9200       arm-realview-smp  mips-excite          powerpc-mpc834x_itx
arm-badge4         arm-rpc           mips-fulong          powerpc-mpc834x_itxgp
arm-carmeva        arm-s3c2410       mips-ip22            powerpc-mpc834x_mds
arm-cerfcube       arm-shannon       mips-ip27            powerpc-mpc836x_mds
arm-clps7500       arm-shark         mips-ip32            powerpc-mpc8540_ads
arm-collie         arm-simpad        mips-jazz            powerpc-mpc8544_ds
arm-corgi          arm-spitz         mips-jmr3927         powerpc-mpc8560_ads
arm-csb337         arm-trizeps4      mips-malta           powerpc-mpc8568mds
arm-csb637         arm-versatile     mips-mipssim         powerpc-mpc85xx_cds
arm-ebsa110        i386              mips-mpc30x          powerpc-mpc8641_hpcn
arm-edb7211        i386-allnoconfig  mips-msp71xx         powerpc-mpc866_ads
arm-em_x270        i386-defconfig    mips-ocelot          powerpc-mpc885_ads
arm-ep93xx         i386-up           mips-pb1100          powerpc-pasemi
arm-footbridge     ia64              mips-pb1500          powerpc-pmac32
arm-fortunet       ia64-allnoconfig  mips-pb1550          powerpc-ppc64
arm-h3600          ia64-bigsur       mips-pnx8550-jbs     powerpc-prpmc2800
arm-h7201          ia64-defconfig    mips-pnx8550-stb810  powerpc-ps3
arm-h7202          ia64-gensparse    mips-qemu            powerpc-pseries
arm-hackkit        ia64-sim          mips-rbhma4200       powerpc-up
arm-integrator     ia64-sn2          mips-rbhma4500       s390
arm-iop13xx        ia64-tiger        mips-rm200           s390-allnoconfig
arm-iop32x         ia64-up           mips-sb1250-swarm    s390-defconfig
arm-iop33x         ia64-zx1          mips-sead            s390-up
arm-ixp2000        m68k              mips-tb0219          sparc
arm-ixp23xx        m68k-amiga        mips-tb0226          sparc-allnoconfig
arm-ixp4xx         m68k-apollo       mips-tb0287          sparc-defconfig
arm-jornada720     m68k-atari        mips-workpad         sparc-up
arm-kafa           m68k-bvme6000     mips-wrppmc          sparc64
arm-kb9202         m68k-hp300        mips-yosemite        sparc64-allnoconfig
arm-ks8695         m68k-mac          parisc               sparc64-defconfig
arm-lart           m68k-mvme147      parisc-allnoconfig   sparc64-up
arm-lpd270         m68k-mvme16x      parisc-defconfig     um-x86_64
arm-lpd7a400       m68k-q40          parisc-up            x86_64
arm-lpd7a404       m68k-sun3         powerpc              x86_64-allnoconfig
arm-lubbock        m68k-sun3x        powerpc-cell         x86_64-defconfig
arm-lusl7200       mips              powerpc-celleb       x86_64-up
arm-mainstone      mips-atlas        powerpc-chrp32
Signed-off-by: NAlexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Make sure declarations stay in sync with definitions by keeping all 
extern declarations in common .h files.
Signed-off-by: NHoang-Nam Nguyen <hnguyen@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Remove extra dumb-looking blank line that snuck in somehow.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Include <rdma/...> headers after <asm/...> headers.
Signed-off-by: NHoang-Nam Nguyen <hnguyen@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Fix a crash if the driver has to wait for a QP reference to be dropped
when destroying the QP.
Signed-off-by: NEthan Burns <eaburns@iol.unh.edu>
Acked-by: NTom Tucker <tom@opengridcomputing.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

This prevents the iscsi modules from being unloaded while
there are active mounts from an iscsi target.
Signed-off-by: NOlaf Kirch <olaf.kirch@oracle.com>
Signed-off-by: NMike Christie <michaelc@cs.wisc.edu>
Signed-off-by: NJames Bottomley <James.Bottomley@SteelEye.com>

eHCA2 supports QP queues that can be as small as 512 bytes. This
greatly reduces memory overhead for consumers that use lots of QPs
with small queues (e.g. RDMA-only QPs). Apart from dealing with
firmware, this code needs to manage bite-sized chunks of kernel pages,
making sure that no kernel page is shared between different protection
domains.
Signed-off-by: NHoang-Nam Nguyen <hnguyen@de.ibm.com>

They're only used in ehca_qp.c, so make them static to that file.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

ehca2ib_return_code() is not used in any fast path, and making it
non-inline saves ~1.5K of code.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Add support for MR pages larger than 4K on eHCA2. This reduces
firmware memory consumption.  If enabled via the mr_largepage module
parameter, the MR page size will be determined based on the MR length
and the hardware capabilities -- if the MR is >= 16M, 16M pages are
used, for example.
Signed-off-by: NJoachim Fenkes <fenkes@de.ibm.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

    
The error handling code at err_wrid in create_qp_common() does not
handle a userspace QP attached to an SRQ correctly, since it ends up
in the else clause of the if statement.  This means it tries to
kfree() the uninitialized qp->sq.wrid and qp->rq.wrid pointers.  Fix
this so we only free the wrid arrays for kernel QPs.

Pointed out by Michael S. Tsirkin <mst@dev.mellanox.co.il>.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The FW command token is currently only updated on a command completion
event. This means that on command timeout, the same token will be
reused for new command, which results in a mess if the timed out
command *does* eventually complete.
Signed-off-by: NMichael S. Tsirkin <mst@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The ipath_layer.[ch] code was an attempt to provide a single interface
for the ipath verbs and ipath_ether code to use.  As verbs
functionality increased, the layer's functionality became insufficient
and the verbs code broke away to interface directly to the driver.
The failed attempt to get ipath_ether upstream was the final nail in
the coffin and now it sits quietly in a dark kernel.org corner waiting
for someone to notice the smell and send it along to it's final
resting place.  Roland Dreier was that someone -- this patch expands
on his work...
Signed-off-by: NArthur Jones <arthur.jones@qlogic.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Temporarily allocated struct mlx4_qp_context *context is leaked by
several error paths.  The patch takes advantage of the return value
'err' being preinitialized to -EINVAL.

Spotted by Coverity (CID 1768).
Signed-off-by: NFlorin Malita <fmalita@gmail.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Slab destructors were no longer supported after Christoph's
c59def9f change. They've been
BUGs for both slab and slub, and slob never supported them
either.

This rips out support for the dtor pointer from kmem_cache_create()
completely and fixes up every single callsite in the kernel (there were
about 224, not including the slab allocator definitions themselves,
or the documentation references).
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Transform some calls to kmalloc/memset to a single kzalloc (or kcalloc).

Here is a short excerpt of the semantic patch performing
this transformation:

@@
type T2;
expression x;
identifier f,fld;
expression E;
expression E1,E2;
expression e1,e2,e3,y;
statement S;
@@

 x =
- kmalloc
+ kzalloc
  (E1,E2)
  ...  when != \(x->fld=E;\|y=f(...,x,...);\|f(...,x,...);\|x=E;\|while(...) S\|for(e1;e2;e3) S\)
- memset((T2)x,0,E1);

@@
expression E1,E2,E3;
@@

- kzalloc(E1 * E2,E3)
+ kcalloc(E1,E2,E3)

[akpm@linux-foundation.org: get kcalloc args the right way around]
Signed-off-by: NYoann Padioleau <padator@wanadoo.fr>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Acked-by: NRussell King <rmk@arm.linux.org.uk>
Cc: Bryan Wu <bryan.wu@analog.com>
Acked-by: NJiri Slaby <jirislaby@gmail.com>
Cc: Dave Airlie <airlied@linux.ie>
Acked-by: NRoland Dreier <rolandd@cisco.com>
Cc: Jiri Kosina <jkosina@suse.cz>
Acked-by: NDmitry Torokhov <dtor@mail.ru>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NMauro Carvalho Chehab <mchehab@infradead.org>
Acked-by: NPierre Ossman <drzeus-list@drzeus.cx>
Cc: Jeff Garzik <jeff@garzik.org>
Cc: "David S. Miller" <davem@davemloft.net>
Acked-by: NGreg KH <greg@kroah.com>
Cc: James Bottomley <James.Bottomley@steeleye.com>
Cc: "Antonino A. Daplas" <adaplas@pol.net>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

    
Current code sets size0 to 0 at the start of work request posting
functions and then handles size0 == 0 specially within the loop over
work requests.  Change this so size0 is set along with f0 the first
time through the loop (when nreq == 0).  This makes the code easier to
understand by making it clearer that f0 and size0 are always
initialized if nreq != 0 without having to know that size0 == 0
implies nreq == 0.

Also annotate size0 with uninitialized_var() so that this doesn't
introduce a new compiler warning.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>