Remove some trailing whitespace that has snuck in despite the best
efforts of whitespace=error-all.  Also fix a few other whitespace
bogosities.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Documentation/infiniband/core_locking.txt says:

  All of the methods in struct ib_device exported by a low-level
  driver must be fully reentrant.  The low-level driver is required to
  perform all synchronization necessary to maintain consistency, even
  if multiple function calls using the same object are run
  simultaneously.

However, mthca's modify_qp, modify_srq and resize_cq methods are
currently not reentrant.  Add a mutex to the QP, SRQ and CQ structures
so that these calls can be properly serialized.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Memfree firmware is in rare cases reporting WQE index == base - 1 in
receive completion with error, instead of (rq size - 1); base is 0 in
mthca.  Here is a patch to avoid kernel crash and report a correct WR
id in this case.
Signed-off-by: NMichael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Fix races in in destroying various objects.  If a destroy routine
waits for an object to become free by doing

	wait_event(&obj->wait, !atomic_read(&obj->refcount));
	/* now clean up and destroy the object */

and another place drops a reference to the object by doing

	if (atomic_dec_and_test(&obj->refcount))
		wake_up(&obj->wait);

then this is susceptible to a race where the wait_event() and final
freeing of the object occur between the atomic_dec_and_test() and the
wake_up().  And this is a use-after-free, since wake_up() will be
called on part of the already-freed object.

Fix this in mthca by replacing the atomic_t refcounts with plain old
integers protected by a spinlock.  This makes it possible to do the
decrement of the reference count and the wake_up() so that it appears
as a single atomic operation to the code waiting on the wait queue.

While touching this code, also simplify mthca_cq_clean(): the CQ being
cleaned cannot go away, because it still has a QP attached to it.  So
there's no reason to be paranoid and look up the CQ by number; it's
perfectly safe to use the pointer that the callers already have.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Quite a few cleanup functions in mthca were marked as __devexit.
However, they could also be called from error paths during
initialization, so they cannot be marked that way.  Just delete all of
the incorrect annotations.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Add low-level driver support for resizing CQs (both kernel and
userspace) to mthca.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The might_sleep() annotations in mthca are silly -- they all occur
shortly before calls that will end up in core functions like kmalloc()
that will print the same warning in an unsafe context anyway.  In
fact, beyond cluttering the source, we're actually bloating text with
CONFIG_DEBUG_SPINLOCK_SLEEP and/or CONFIG_PREEMPT_VOLUNTARY set.

With both options set, getting rid of the might_sleep()s saves a lot:
add/remove: 0/0 grow/shrink: 0/7 up/down: 0/-171 (-171)
function                                     old     new   delta
mthca_pd_alloc                               132     109     -23
mthca_init_cq                                969     946     -23
mthca_mr_alloc                               592     568     -24
mthca_pd_free                                 67      42     -25
mthca_free_mr                                219     194     -25
mthca_free_cq                                570     545     -25
mthca_fmr_alloc                              742     716     -26
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The function mthca_free_err_wqe() can never fail, so get rid of its
return value.  That means handle_error_cqe() doesn't have to check
what mthca_free_err_wqe() returns, which means it can't fail either
and doesn't have to return anything either.  All this results in
simpler source code and a slight object code improvement:

add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-10 (-10)
function                                     old     new   delta
mthca_free_err_wqe                            83      81      -2
mthca_poll_cq                               1758    1750      -8
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Fill vendor_err field in completion with error.
Signed-off-by: NMichael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

When cleaning up a CQ for a QP attached to SRQ, need to free an SRQ
WQE only if the CQE is a receive completion.
Signed-off-by: NJack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Handle case where prod_index has wrapped around and become less than
cq->cons_index by checking that their difference as a signed int is
positive rather than comparing directly.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Implement reporting asynchronous CQ events in Mellanox HCA driver.
Signed-off-by: NMichael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Move the InfiniBand headers from drivers/infiniband/include to include/rdma.
This allows InfiniBand-using code to live elsewhere, and lets us remove the
ugly EXTRA_CFLAGS include path from the InfiniBand Makefiles.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Add mthca support for shared receive queues (SRQs),
including userspace SRQs.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Mem-free HCAs never generate error CQEs that complete multiple WQEs,
so just skip the call to mthca_free_err_wqe() for them rather than
having logic to handle the mem-free case in mthca_free_err_wqe().
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Clean up the allocation of memory for queues by factoring out the
common code into mthca_buf_alloc() and mthca_buf_free().  Now CQs and
QPs share the same queue allocation code, which we'll also use for SRQs.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Fix sparse warnings.  Use __be* where appropriate.
Signed-off-by: NSean Hefty <sean.hefty@intel.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Make some lawyers happy and add copyright notices for people who
forgot to include them when they actually touched the code.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Fix handling of error CQ entries on mem-free HCAs: the doorbell count
is never valid so we shouldn't look at it.  This fixes problems exposed
by new HCA firmware.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Add support for userspace completion queues (CQs) to mthca.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Future versions of Mellanox HCA firmware will require command mailboxes to be
aligned to 4K.  Support this by using a pci_pool to allocate all mailboxes.
This has the added benefit of shrinking the source and text of mthca.
Signed-off-by: NRoland Dreier <roland@topspin.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Make mthca_table_put() and mthca_table_put_range() NOPs if the device is not
mem-free, so that we don't have to have "if (mthca_is_memfree())" tests in the
callers of these functions.  This makes our code more readable and
maintainable, and saves a couple dozen bytes of text in ib_mthca.ko as well.
Signed-off-by: NRoland Dreier <roland@topspin.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Switch all allocations of coherent memory from pci_alloc_consistent() to
dma_alloc_coherent(), so that we can pass GFP_KERNEL.  This should help when
the system is low on memory.
Signed-off-by: NRoland Dreier <roland@topspin.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Clean up CQ debugging code: make dump_cqe print on one line, and only dump
error CQ entries for local operation errors.
Signed-off-by: NRoland Dreier <roland@topspin.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Add Sun copyright to files modified by Tom Duffy.
Signed-off-by: NTom Duffy <tduffy@sun.com>
Signed-off-by: NRoland Dreier <roland@topspin.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Clean up mem-free mode support by introducing mthca_is_memfree() function,
which encapsulates the logic of deciding if a device is mem-free.
Signed-off-by: NRoland Dreier <roland@topspin.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Fill in missing fields in send completions.
Signed-off-by: NItamar Rabenstein <itamar@mellanox.co.il>
Signed-off-by: NMichael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: NRoland Dreier <roland@topspin.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

On error path, only free doorbell records if we're in mem-free mode.
Signed-off-by: NRoland Dreier <roland@topspin.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!