This change cleans up the use of rsc_count and changes it to a boolean since
the actual numerical value is used nowhere in the Rx cleanup path.  I am
also moving the skb count into the RSC_CB path since it is much easier to
track it there than when it is passed as a parameter to various function
calls.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change adds a set of state flags to the rings that allow them to
independently function allowing for features like RSC, packet split, and
TX hang detection to be done per ring instead of for the entire device.

This is accomplished by re-purposing the flow director reinit_state member
and making it a global state instead since a long for a single bit flag is
a bit wasteful.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This is the start of work to sort out what belongs in the rings and what
belongs in the q_vector. Items like the CPU variable for make much more
sense in the q_vector since the CPU is a per-interrupt thing rather than a
per ring thing.
I also added a back-pointer from the ring to the q_vector.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change places a netdev pointer directly into the ring structure. This
way we can avoid having to determine which netdev we are supposed to be
using and can just access the one on the ring directly.
As a result of this change further collapse of the code is possible by
dropping the adapter from ixgbe_alloc_rx_buffers, and the netdev pointer
from ixgbe_xmit_frame_ring_adv and ixgbe_maybe_stop_tx.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change moved some of the RX and TX stats into separate structures and
them placed those structures in a union in order to help reduce the size of
the ring structure.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change is meant to simplify DMA map/unmap by providing a device
pointer. As a result the adapter pointer can be dropped from many of
the calls.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change drops ring->head since it is not used in any hot-path and can
easily be determined using IXGBE_[RT]DH(ring->reg_idx).

It also changes ring->tail into a true pointer so we can avoid unnecessary
pointer math to find the location of the tail.

In addition I also dropped the setting of head and tail in
ixgbe_clean_[rx|tx]_ring. The only location that should be setting the head
and tail values is ixgbe_configure_[rx|tx]_ring and that is only while the
queue is disabled.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change re-orders alloc_rx_buffers to make better use of the packet
split enabled flag.  The new setup should require less branching in the
code since now we are down to fewer if statements since we either are
handling packet split or aren't.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change simplifies the work being done by the TX interrupt handler and
pushes it into the tx_map call. This allows for fewer cache misses since
the TX cleanup now accesses almost none of the skb members.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Currently the high and low water marks for PFC are being set
conservatively for jumbo frames. This means the RX buffers
are being underutilized in the default 1500 MTU. This patch
fixes this so that the water marks are set as described in
the data sheet considering the MTU size.

The equation used is,

RTT * 1.44 + MTU * 1.44 + MTU

Where RTT is the round trip time and MTU is the max frame size
in KB. To avoid floating point arithmetic FC_HIGH_WATER is
defined

((((RTT + MTU) * 144) + 99) / 100) + MTU

This changes how the hardware field fc.low_water and
fc.high_water are used. With this change they are no longer
storing the actual low water and high water markers but are
storing the required head room in the buffer. This simplifies
the logic and we do not need to account for the size of the
buffer when setting the thresholds.

Testing with iperf and 16 threads showed a slight uptick in
throughput over a single traffic class .1-.2Gbps and a reduction
in pause frames. Without the patch a 30 second run would show
~10-15 pause frames being transmitted with the patch ~2-5 are
seen. Test were run back to back with 82599.

Note RXPBSIZE is in KB and low and high water marks fields are
also in KB. However the FCRT* registers are 32B granularity and
right shifted 5 into the register,

(((rx_pbsize - water_mark) * 1024) / 32) << 5

is the most explicit conversion here we simplify

(rx_pbsize - water_mark) * 32 << 5 = (rx_pbsize - water_mark) << 10

This patch updates the PFC thresholds and legacy FC thresholds.
Signed-off-by: NJohn Fastabend <john.r.fastabend@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

"cat /proc/net/dev" uses RCU protection only.

Its quite possible we call a driver get_stats() method while device is
dismantling and freeing its data structures.

So get_stats() methods must be very careful not accessing driver private
data without appropriate locking.

In ixgbe case, we access rx_ring pointers. These pointers are freed in
ixgbe_clear_interrupt_scheme() and set to NULL, this can trigger NULL
dereference in ixgbe_get_stats64()

A possible fix is to use RCU locking in ixgbe_get_stats64() and defer
rx_ring freeing after a grace period in ixgbe_clear_interrupt_scheme()
Signed-off-by: NEric Dumazet <eric.dumazet@gmail.com>
Reported-by: NTantilov, Emil S <emil.s.tantilov@intel.com>
Tested-by: NRoss Brattain <ross.b.brattain@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Current ixgbe stats have following problems :

- Not 64 bit safe (on 32bit arches)

- Not safe in ixgbe_clean_rx_irq() :
   All cpus dirty a common location (netdev->stats.rx_bytes &
netdev->stats.rx_packets) without proper synchronization.
   This slow down a bit multiqueue operations, and possibly miss some
updates.

Fixes :

Implement ndo_get_stats64() method to provide accurate 64bit rx|tx
bytes/packets counters, using 64bit safe infrastructure.

ixgbe_get_ethtool_stats() also use this infrastructure to provide 64bit
safe counters.
Signed-off-by: NEric Dumazet <eric.dumazet@gmail.com>
Acked-by: NDon Skidmore <donald.c.skidmore@intel.com>
Tested-by: NStephen Ko <stephen.s.ko@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Make the ixgbe driver use the new vlan accleration model.
Signed-off-by: NJesse Gross <jesse@nicira.com>
CC: Peter Waskiewicz <peter.p.waskiewicz.jr@intel.com>
CC: Emil Tantilov <emil.s.tantilov@intel.com>
CC: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use the new infrastructure to balance interrupts for flow
alignment when ATR or Flow Director are enabled.
Signed-off-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Tested-by: NStephen Ko <stephen.s.ko@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This change makes it so that the ethtool loopback test uses the standard
ring configuration and allocation functions.  As a result the loopback test
will be much more effective at testing core driver functionality.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

All of the DESC_ADV macros are currently needing the pointers to be
de-referenced before accessing the ring.  Instead of having to add all of
the asterisks it is easier to just update the macro to expect a pointer to
the ring.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Bump the header size for packet split to 512 bytes since this makes the
best use of the 1k buffer that is allocated for any skb 512 bytes or
smaller.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This change corrects issues where macvlan was not correctly triggering
promiscuous mode on ixgbe due to the filters not being correctly set.  It
also corrects the fact that VF rar filters were being overwritten when the
PF was reset.

CC: Shirley Ma <xma@us.ibm.com>
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NEmil Tantilov <emil.s.tantilov@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch is alternative to a previous patch submitted by Joe Perches.

Create common macros e_<level> and e_dev_<level> that use netdev_<level> and
dev_<level> similar to e1000e.
Redefined pr_fmt for driver messages.
Use %pM to display MAC address.
Aligned text to better match the new format.

CC: Joe Perches <joe@perches.com>
Signed-off-by: NEmil Tantilov <emil.s.tantilov@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This adds support for a new copper device for 82599, device id 0x151c.
This 82599 10GBase-T device uses the PHY's internal temperature sensor
to guard against over-temp conditions. In this scenario the PHY will be
put in a low power mode and link will no longer be able to transmit or
receive any data. When this occurs, the over-temp interrupt is latched
and driver logs this error message. A HW reset or power cycle is
required to clear this status.
Signed-off-by: NMallikarjuna R Chilakala <mallikarjuna.chilakala@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add support for the "ip link set" and "ip link show" commands that allow
configuration of the virtual functions' MAC and port VLAN via user space
command line.
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This reverts commit d5ffd75a.

As requested by Jeff Kircher.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Due to an errata in 82598 parts MSI-X needs to be disabled
in certain ixgbe devices designed to transfer peer-to-peer
traffic on the PCIe bus. This patch sets the default
interrupt type to MSI rather than MSI-X for specific Cisco
ixgbe adapters.
Signed-off-by: NNicholas Nunley <nicholasx.d.nunley@intel.com>
Acked-by: NJohn Ronciak <john.ronciak@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In the last patch I missed an unecessary min_t comparison.
This patch removes it, the path allocates at most
72 tx queues for 82599 and 24 for 82598 there is no need
for this check.

Additionally this sets MAX_[TX|RX]_QUEUES to 72.  Which is
used as the size for the tx/rx_ring arrays. There is no
reason to have more tx_rings/rx_rings then num_tx_queues.
Signed-off-by: NJohn Fastabend <john.r.fastabend@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Acked-by: NEric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds n-tuple filter programming to 82599.
Signed-off-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch allocates the ring structures themselves on each
NUMA node along with the buffer_info structures.  This way we
don't allocate the entire ring memory on a single node in one
big block, thus reducing NUMA node memory crosstalk.
Signed-off-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The default policy for the current driver is to do all its memory
allocation on whatever processor is running insmod/modprobe.  This
is less than optimal.

This driver's default mode of operation will be to use each node for each
subsequent transmit/receive queue.  The most efficient allocation will be
to then have the interrupts bound in such a way as to match up the
interrupt of the queue to the cpu where its memory was allocated.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Processing multiple ixgbe_watchdog_task calls may cause
the link_up variable and IXGBE_FLAG_NEED_LINK_UPDATE flag
to be set incorrectly.  In the worse case this is causing
the netif_carrier_off to be called inappropriately which
results in an interface that can't be brought up.

Although schedule_work() will only schedule the task if
it is not already on the work queue the WORK_STRUCT_PENDING
bits are cleared just before calling the work function.
This allows WORK_STRUCT_PENDING to be cleared, the work
function to start and meanwhile schedule another task.

This patch adds a mutex to the watchdog task. This bug is
actualized by changing DCB settings or doing extended
cable pull or reset tests.
Signed-off-by: NJohn Fastabend <john.r.fastabend@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This data storage for SW emulated MAC addresses is unlikely to ever be used
so pull it.
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds register definitions, bit definitions and structures used by
the driver to support SR-IOV features of the 82599 controller.
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

drop variables that had cache lines modified in simultaneous hot paths.
keep some variables modified on hot paths but make their storage per queue.
cache align DMA data buffer start addresses.
cache align (padding) some structures that end within a cacheline.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Acked-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

decrease the memory utilization of the tx / rx queue allocation
by changing the default ring size to 512 (from 1024).  At
1024 rx entries of 2KB each (from 4kB slab) with 16 queues
ixgbe was using 64 MB of memory per port, which is not
necessary.

Users can still change queue lengths with ethtool -k.
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Acked-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch removes skb_dma_map/unmap calls from the ixgbe driver due to the
fact that the calls don't work with HW IOMMU enabled systems.  The problem
is that multiple mappings will give different results when HW IOMMU is
enabled and the skb_dma_map/unmap calls only have one location to store
mappings.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Acked-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Divide 82599 HWRSC counters into aggregated and flushed to count number of
packets getting coalesced per TCP connection.
Signed-off-by: NMallikarjuna R Chilakala <mallikarjuna.chilakala@intel.com>
Acked-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Implements the netdev_ops.ndo_fcoe_get_wwn in 82599 if it finds valid
prefix for the World Wide Node Name (WWNN) or World Wide Port Name (WWPN),
as well as valid SAN MAC address.
Signed-off-by: NYi Zou <yi.zou@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Since net_device has an instance of net_device_stats,
we can remove the instance of this from the private adapter structure.
Signed-off-by: NAjit Khaparde <ajitk@serverengines.com>
Acked-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The data center bridging ops structure can be const
Signed-off-by: NStephen Hemminger <shemminger@vyatta.com>
Acked-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Allow the user to set Tx specific itr values.  This only makes sense
when there are separate vectors for Tx and Rx.  When the queues are
doubled up RxTx on the vectors, we still only use the rx itr value.
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add support for dcbnl_rtnl_ops.setapp/getapp to set or get the current user
priority bitmap for the given application protocol. Currently, 82599 only
supports setapp/getapp for Fiber Channel over Ethernet (FCoE) protocol.
Signed-off-by: NYi Zou <yi.zou@intel.com>
Acked-by: NPeter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>