Write MTT entries directly to ICM from the driver (eliminating use of
WRITE_MTT command).  This reduces the number of FW commands needed to
register an MR by at least a factor of 2 and speeds up memory
registration significantly.  This code will also be used to implement
FMRs.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NMichael S. Tsirkin <mst@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Everything that uses caps.reserved_mtts expects it to be a count of MTT
segments, not MTT entries.  So convert the value that the FW gives us to
a count of segments.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Taking ilog2(dev->caps.reserved_mtts) to find out the order to pass to
the MTT buddy allocator will do the wrong thing if reserved_mtts is ever
not a power of 2.  Be safe and use fls(dev->caps.reserved_mtts - 1).
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Enable having ICM tables in coherent memory, and use coherent memory
for the dMPT table.  This will allow writing MPT entries for MRs both
via the SW2HW_MPT command and also directly by the driver for FMR
remapping without needing to flush or worry about cacheline boundaries.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NMichael S. Tsirkin <mst@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

display the following device information under /sys/class/infiniband/mlx4_X:
board_id, fw_ver, hw_rev, hca_type.

This patch makes this information available to userspace utilities
such as ibstat and ibv_devinfo.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The SRC ("scalable RC") transport has been renamed to XRC ("extended 
RC"), to avoid having an abbreviation that is so easily confused with an 
abbreviation for "source."  Update the HCA capability decoding output to 
use the new name.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

mlx4_srq_query() returns a big-endian 16-bit value through an int *,
which screws up sparse checking.  Fix this so that a CPU-endian value
is returned.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Recover from MSI-X errors by automatically falling back on regular
interrupt, instead of asking the user to do this manually.  This makes
it possible to enable MSI-X by default, and will make it possible to
get rid of the msi_x module option in the future.
Signed-off-by: NMichael S. Tsirkin <mst@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Special QPs are not allocated using the regular QP number bitmap, so
when they are destroyed, their QP number should not be freed in the
bitmap.

Found by Dotan Barak of Mellanox.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Rename GO_BIT_TIMEOUT to GO_BIT_TIMEOUT_MSECS for clarity, and
actually use it as the go bit timeout (instead of having the define
but then ignoring it and using a hard-coded 10 * HZ for the actual
timeout).
Signed-off-by: NDotan Barak <dotanb@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Put a 1000 msec delay after resetting the device before attempting to
do config cycles on it.  Not waiting causes system hangs on some
chipsets, e.g. Intel E7520, when the driver is loaded.
Signed-off-by: NEli Cohen <eli@mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

mlx4_mr_alloc() doesn't actually allocate mr (it just initializes the
pointer that the caller passes in), so it shouldn't free it if an
error occurs.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

    
The FW command token is currently only updated on a command completion
event. This means that on command timeout, the same token will be
reused for new command, which results in a mess if the timed out
command *does* eventually complete.

This is the same change as the patch for mthca from Michael
S. Tsirkin <mst@dev.mellanox.co.il> that was just merged.  It seems
sensible to avoid gratuitous differences in FW command processing
between mthca and mlx4.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Change the maximum number of outstanding RDMA reads allowed as a
target from 4 to 16 to per QP.  This allows RDMA read operations to
pipeline better.

Pointed out by Dotan Barak and Sagi Rotem.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Reset the device when an internal error is detected.

Also, detect errors by polling the error buffer rather than using
interrupts.  This is more robust and doesn't depend on MSI-X.  Remove
the old interrupt handler entirely, since we don't want to support two
mechanisms for detecting internal errors.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NDotan Barak <dotanb@mellanox.co.il>
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Get the maximum message size from the device capabilities returned
from the QUERY_DEV_CAP firmware command, rather than hard-coding 2 GB.
Signed-off-by: NDotan Barak <dotanb@dev.mellanox.co.il>
Signed-off-by: NMichael S. Tsirkin <mst@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

mlx4.h uses struct mutex, so although <linux/mutex.h> seems to be pulled in
indirectly by one of the headers it includes, the right thing to do is
to include <linux/mutex.h> directly.
Signed-off-by: NMichael S. Tsirkin <mst@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Recent gcc versions emit warnings when unsigned variables are compared < 0 or >= 0.
Signed-off-by: NBill Nottingham <notting@redhat.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Add new IDs for PCIe gen2 devices.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Upcoming firmware introduces command interface revision 3, which
changes the way port capabilities are queried and set.  Update the
driver to handle both the new and old command interfaces by adding a
new MLX4_FLAG_OLD_PORT_CMDS that it is set after querying the firmware
interface revision and then using the correct interface based on the
setting of the flag.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

New ConnectX firmware introduces FW command interface revision 2,
which requires that for each QP, a chunk of send queue entries (the
"headroom") is kept marked as invalid, so that the HCA doesn't get
confused if it prefetches entries that haven't been posted yet.  Add
code to the driver to do this, and also update the user ABI so that
userspace can request that the prefetcher be turned off for userspace
QPs (we just leave the prefetcher on for all kernel QPs).

Unfortunately, marking send queue entries this way is confuses older
firmware, so we change the driver to allow only FW command interface
revisions 2.  This means that users will have to update their firmware
to work with the new driver, but the firmware is changing quickly and
the old firmware has lots of other bugs anyway, so this shouldn't be too
big a deal.

Based on a patch from Jack Morgenstein <jackm@dev.mellanox.co.il>.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

If a dMPT entry has the PA flag (direct physical address) set, then
the (unused) MTT base address field has to be set to 0.
Signed-off-by: NJack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

HCA firmware with incompatible changes to the FW commmand interface is
coming soon.  Add a check of the interface revision during
initialization and bail out if the firmware advertises a revision that
the driver doesn't know about.  This will avoid strange failures later
if the driver goes on using the wrong interface revision. 
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

s/signifant/significant/
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

We need to pass the same dev_id to free_irq() and request_irq().  When
using MSI-X, the MLX4_EQ_CATAS interrupt uses a different dev_id from
the other interrupts.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

We may call mlx4_dispatch_event() before mlx4_register_device() is
called for a device, because for example a catastrophic error happens
immediately after we enable interrupts.  Therefore priv->ctx_list and
priv->ctx_lock need to be initialized earlier.

This bug was actually exposed by the MSI-X bug that returned IRQ numbers 
to drivers in reverse order, so that the first FW command 
interrupt looked to mlx4 like a catastrophic error.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

The reserved6 field should be 64 bits, not just 16 bits.  Without
this, the structure does not match the hardware layout on 32-bit
architectures: the db_rec_addr field ends up at offset 52 instead of
offset 56.  The bug slipped by because the alignment of __be64 members
ends up putting it in the right place on x86-64.
Signed-off-by: NEli Cohen <eli@mellanox.co.il>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Set last allocated object to the object after the one just allocated
before ORing in the extra top bits.  Also handle the case where this
wraps around.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Don't overrun fname[] array when decoding device flags.

This was spotted by the Coverity checker (CID 1642).
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
Acked-by: NJeff Garzik <jeff@garzik.org>
Acked-by: NRoland Dreier <rolandd@cisco.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

struct mlx4_priv.doorbell_lock is never used, so delete it.
Signed-off-by: NRoland Dreier <rolandd@cisco.com>

Cc: Jeff Garzik <jeff@garzik.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Add an InfiniBand driver for Mellanox ConnectX adapters.  Because
these adapters can also be used as ethernet NICs and Fibre Channel 
HBAs, the driver is split into two modules: 
 
  mlx4_core: Handles low-level things like device initialization and 
    processing firmware commands.  Also controls resource allocation 
    so that the InfiniBand, ethernet and FC functions can share a 
    device without stepping on each other. 
 
  mlx4_ib: Handles InfiniBand-specific things; plugs into the 
    InfiniBand midlayer. 
Signed-off-by: NRoland Dreier <rolandd@cisco.com>