This code adds multiple Tx queue infrastructure much like we
previously did in ixgbe. The MSI-X vector mapping is the bulk of
the change.

IAM can now be safely enabled and we've verified that it does
work correctly. We can also eliminate the tx ring lock.
Signed-off-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Signed-off-by: NAuke Kok <auke-jan.h.kok@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

network maintainers suggest NAPI only drivers are the only way to go.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

cleaned up some spacing in defines
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

calls to kfree_skb_any are only required when calling kfree
from interrupt context.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

rx cleanup should look more like our other drivers that have evolved
to nicer performance levels over time.  Changes consist of refilling
tx buffers to hardware more often, some minor assignment cleanups.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

this patch is trivial but because I want to have everything be nice and
tidy I'm updating it.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

copybreak code was already in the driver, allow the user to turn it
off if they don't like it.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

a) kernel developers suggest LLTX is broken and unsafe to use, remove it.
b) remember to pre-stop the queue if we won't have room
c) removing lltx means we can remove our tx lock
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

ixgb needs to call flush scheduled work to flush any timers before
unregistering the netdev.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

some random coverage testing found that when changing mtu
under heavy traffic load, NAPI would use the rx_buffer_len variable
after it had been changed by change_mtu.

Similar to e1000 bugs found long ago.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

a user pointed out that setting variables out of order with respect
to the checks we make for tx timeout handling could result in a race
where ->dma was set but ->time_stamp was set to the old value.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

in order to prevent the case where poll_disable is waiting
on our device to permanently, check the flag to make sure we're not
down or closing down before re-enabling interrupts.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

ixgb hardware (not ixgbe) has a problem where it might dma past the
end of a buffer in certain cases.  leave 8 bytes extra room.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

there was one more bug hidden in the prefetch routines in ixgb hardware
that force us to remove it completely.  Writebacks were being done on
descriptors with stale data due to internal hardware fifo corruption.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

this patch has been made to many other drivers in kernel to fix
the storage of 64 bit resources in 32 bit variables.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

back when maybe stop tx was added to the ixgb driver some mistakes
were made and the driver
a) didn't remove the tx lock, which is now un-necessary
b) didn't change the restart code to be compliant with maybe_stop
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NFrancois Romieu <romieu@fr.zoreil.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Compile-tested only.
Signed-off-by: NFrancois Romieu <romieu@fr.zoreil.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Compile-tested only.
Signed-off-by: NFrancois Romieu <romieu@fr.zoreil.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Compile-tested only.
Signed-off-by: NFrancois Romieu <romieu@fr.zoreil.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

- Updated version number
Signed-off-by: NSantosh Rastapur <santosh.rastapur@neterion.com>
Signed-off-by: NRamkrishna Vepa <ram.vepa@neterion.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

- Enable msi-x link interrupts because the timer based scheduler was getting
cancelled causing the link state to be lost with repetitive card up/downs
when changing the mtu.
- Unmask mac_rmac_link interrupts only for Xframe I and prevent a spurious
link interrupt in Xframe II.
- Stop the tx queue and indicate link down when card is down
Signed-off-by: NSantosh Rastapur <santosh.rastapur@neterion.com>
Signed-off-by: NRamkrishna Vepa <ram.vepa@neterion.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

- Fix IOMMU overflow checking. As reported by Andi Kleen <ak@linux.intel.com>
removed check for zero dma address.
Signed-off-by: NSantosh Rastapur <santosh.rastapur@neterion.com>
Signed-off-by: NRamkrishna Vepa <ram.vepa@neterion.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

It removes a dependancy from velocity_init_rd_ring to dev->mtu.
Signed-off-by: NFrancois Romieu <romieu@fr.zoreil.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Updates of the RBRDU have two different meanings depending on their
context:
1. the receiving process has not started - the value which is written
   into the RBRDU register is supposed to be the free rx descriptors
   count (rounded to a multiple of 4)
2. the receiving process is running - the value increments the count
   above (sic)

The update is currently issued deep inside the rx replenish chain (see
velocity_give_many_rx_descs).

Let's propagate enough information to the caller so that the rx
replenish functions do not depend on hardware any more.

It is needed to perform the Rx/Tx buffers housekeeping when MTU changes.
Signed-off-by: NFrancois Romieu <romieu@fr.zoreil.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

- PCI consistent areas need no memset
- use dev_err instead of plain printk
- avoid a few casts
Signed-off-by: NFrancois Romieu <romieu@fr.zoreil.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Executive summary: the bounce buffers are in my way

- they use something like a 64 * 1500 bytes area of PCI
  consistent area
- they are not resized when the MTU changes
- they are used
  - to hand-pad undersized packets. skb_pad anyone ?
  - to linearize fragmented skbs whose fragment count
    goes beyond the 7 fragments hardware limit in order
    to claim scatter-gather support

Actually the SG code is commented out and I wonder if it
could not be implemented (ab-)using the large send feature
of the chipset since the latter should support some
multi-descriptor packet transmitting.
Signed-off-by: NFrancois Romieu <romieu@fr.zoreil.com>
Fixed-by: NSéguier Régis <rseguier@e-teleport.net>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Multiple TX queue support is a core networking feature.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Accesses are mostly structured such that when there are multiple TX
queues the code transformations will be a little bit simpler.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>