The rmb in the Tx cleanup path is a much stronger barrier than we really need.
All that is really needed is a read_barrier_depends since the location of the
EOP descriptor is dependent on the eop_desc value.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add a check against possible Rx timestamp freezing in the hardware via
watchdog mechanism. This situation can occur when an Rx timestamp has been
latched, but the packet has been dropped because the Rx ring is full.

Whenever a packet comes in that should be timestamped, the Rx timestamp
gets latched into the hardware registers and we will store the jiffy value
in the rx_ring. The watchdog will keep track of his own jiffy timer
whenever there is no valid timestamp in the registers.

If the watchdog detects a valid timestamp in the registers, meaning that no
Rx packet has consumed it yet, it will check which time is most recent: the
last time in the watchdog or any time in the rx_rings. If the most recent
"event" was more than 5 seconds ago, it will flush the Rx timestamp and
print a warning message to the syslog.

Cc: Richard Cochran <richardcochran@gmail.com>
Signed-off-by: NMatthew Vick <matthew.vick@intel.com>
Acked-by: NRichard Cochran <richardcochran@gmail.com>
Acked-by: NJacob Keller <Jacob.e.keller@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

When transmitting a packet that must return a Tx timestamp, a work item
gets scheduled to poll for the Tx timestamp being completed in hardware.
Add a timeout on this work item of 15 seconds from when the driver gets the
skb, after which it will stop polling. Report via stats and system log if
this occurs.

Cc: Richard Cochran <richardcochran@gmail.com>
Signed-off-by: NMatthew Vick <matthew.vick@intel.com>
Acked-by: NJacob Keller <Jacob.e.keller@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Enable SW timestamping for situations where the user may prefer it over HW
timestamping or there may not be HW timestamping.

Cc: Richard Cochran <richardcochran@gmail.com>
Signed-off-by: NMatthew Vick <matthew.vick@intel.com>
Acked-by: NJacob Keller <Jacob.e.keller@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Some of our adapters have internal sensors that report thermal data.  This
patch enables reporting of that data via sysfs.
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NJeff Pieper <jeffrey.e.pieper@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Some of our adapters have sensors on them accessible via i2c and a private
interface.  This patch implements the kernel interface for i2c to those sensors.
Subsequent patches will provide functions to export that data.
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NJeff Pieper <jeffrey.e.pieper@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Implement callback in the driver for the new PCI bus driver
interface that allows the user to enable/disable SR-IOV
virtual functions in a device via the sysfs interface.
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

perm_addr is initialized correctly in register_netdevice() so to init it in
drivers is no longer needed.
Signed-off-by: NJiri Pirko <jiri@resnulli.us>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The __dev* removal patches for the network drivers ended up messing up
the function prototypes for a bunch of drivers.  This patch fixes all of
them back up to be properly aligned.

Bonus is that this almost removes 100 lines of code, always a nice
surprise.
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

During MSI-X setup the system might run out of vectors. If this happens the
already assigned vectors for this NIC should be freed before trying the
disable MSI-X. Failing to do so results in the following oops.

kernel BUG at drivers/pci/msi.c:341!
[...]
Call Trace:
 [<ffffffff8128f39d>] pci_disable_msix+0x3d/0x60
 [<ffffffffa037d1ce>] igb_reset_interrupt_capability+0x27/0x5c [igb]
 [<ffffffffa037d229>] igb_clear_interrupt_scheme+0x26/0x2d [igb]
 [<ffffffffa0384268>] igb_request_irq+0x73/0x297 [igb]
 [<ffffffffa0384554>] __igb_open+0xc8/0x223 [igb]
 [<ffffffffa0384815>] igb_open+0x13/0x15 [igb]
 [<ffffffff8144592f>] __dev_open+0xbf/0x120
 [<ffffffff81443e51>] __dev_change_flags+0xa1/0x180
 [<ffffffff81445828>] dev_change_flags+0x28/0x70
 [<ffffffff814af537>] devinet_ioctl+0x5b7/0x620
 [<ffffffff814b01c8>] inet_ioctl+0x88/0xa0
 [<ffffffff8142e8a0>] sock_do_ioctl+0x30/0x70
 [<ffffffff8142ecf2>] sock_ioctl+0x72/0x270
 [<ffffffff8118062c>] do_vfs_ioctl+0x8c/0x340
 [<ffffffff81180981>] sys_ioctl+0xa1/0xb0
 [<ffffffff815161a9>] system_call_fastpath+0x16/0x1b
Code: 48 89 df e8 1f 40 ed ff 4d 39 e6 49 8b 45 10 75 b6 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f c9 c3 48 8b 7b 20 e8 3e 91 db ff eb ae <0f> 0b eb fe 0f 1f 84 00 00 00 00 00 55 48 89 e5 0f 1f 44 00 00
RIP  [<ffffffff8128e144>] free_msi_irqs+0x124/0x130
 RSP <ffff880037503bd8>
Signed-off-by: NStefan Assmann <sassmann@kpanic.de>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Given a small change to igb_init_interrupt_scheme() the function fits
igb_request_irq() for MSI/legacy interrupts initialization as well, instead of
duplicating most of its code there.

Also adding a missing igb_configure() to igb_request_irq() for MSI fallback
to work properly.
Signed-off-by: NStefan Assmann <sassmann@kpanic.de>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

CONFIG_HOTPLUG is going away as an option.  As result the __dev*
markings will be going away.

Remove use of __devinit, __devexit_p, __devinitdata, __devinitconst,
and __devexit.
Signed-off-by: NBill Pemberton <wfp5p@virginia.edu>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Cc: Jesse Brandeburg <jesse.brandeburg@intel.com>
Cc: Bruce Allan <bruce.w.allan@intel.com>
Cc: Carolyn Wyborny <carolyn.wyborny@intel.com>
Cc: Don Skidmore <donald.c.skidmore@intel.com>
Cc: Greg Rose <gregory.v.rose@intel.com>
Cc: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Cc: Alex Duyck <alexander.h.duyck@intel.com>
Cc: John Ronciak <john.ronciak@intel.com>
Cc: Tushar Dave <tushar.n.dave@intel.com>
Cc: e1000-devel@lists.sourceforge.net
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

For some devices, the result of the flow control high watermark gets
truncated when programming it into the registers because of the mask used.
Switch the mask to 32-bit to prevent this from happening.
Signed-off-by: NMatthew Vick <matthew.vick@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change makes it so that only the first fragment in a series of fragments
will have the L4 header pulled.  Previously we were always pulling the L4
header as well and in the case of UDP this can harm performance since only the
first fragment will have the header, the rest just contain data which should
be left in the paged portion of the packet.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Historically, we've been using the APME bit to determine whether a device
supports wake on a given port or not. However, this bit specifies the
default wake setting, rather than the wake support. Change the behavior so
that we use a flag to keep the capabilities separate from the enablement
while meeting customer requirements.
Signed-off-by: NMatthew Vick <matthew.vick@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change is meant to both improve the performance and reduce the size of
igb_tx_map.  To do this I have expanded the work done in the main loop by
pushing first into tx_buffer.  This allows us to pull in the dma_mapping_error
check, the tx_buffer value assignment, and the initial DMA value assignment to
the Tx descriptor.  The net result is that the function reduces in size by a
little over a 100 bytes and is about 1% or 2% faster.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This change is meant to improve the efficiency of the Tx flags in igb by
aligning them with the values that will later be written into either the
cmd_type or olinfo.  By doing this we are able to reduce most of these
functions to either just a simple shift followed by an or in the case of
cmd_type, or an and followed by an or in the case of olinfo.

In order to avoid type conversion errors I also adjusted the locations
where we were switching between CPU and little endian.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This change is meant to reduce the overhead for workloads that are not
using either TSO or checksum offloads.  Most of the time the compiler
should jump ahead after failing this check to the VLAN check since in the
igb_tx_csum call we start with that check as well.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch provides ability to enable or disable UDP RSS hashing. It gives
users option of generating RSS hash based on the UDP source and destination
ports numbers. Currently, UDP flow hash is always disabled in igb-driver.
Signed-off-by: NAkeem G Abodunrin <akeem.g.abodunrin@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Where a PTP clock driver is associated with a net or PHY driver, it
should be enabled automatically whenever that driver is enabled.
Therefore:

- Make PTP clock drivers select rather than depending on PTP_1588_CLOCK
- Remove separate boolean options for PTP clock drivers that are built
  as part of net driver modules.  (This also fixes cases where the PTP
  subsystem is wrongly forced to be built-in.)
- Set 'default y' for PTP clock drivers that depend on specific net
  drivers but are built separately
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There are multiple places in our device nvm where the version is stored.
This update fixes some output errors with some types of images and
refactors the way the version data is gathered and stored for ethtool output.
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch updates the igb driver version to 4.0.17.
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change makes it so that igb_update_dca is broken into two halves, one
for Rx and one for Tx.  The advantage to this is primarily readability.

In addition I am enabling relaxed ordering for reads from hardware since
this is supported on all of the igb parts.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change helps to address locking issues seen with
netif_set_real_num_tx_queues and netif_set_real_num_rx_queues when used in
the igb_set_interrupt_capability function.  To resolve these locking issues
I have moved the two function calls into __igb_open so that they can be
called while the RTNL lock is held.

An added advantage to this is that the number of queues is not updated
until the last possible moment so if there are any issues in allocating
MSI-X interrupts or resources for the rings we have time to change the
values prior to updating the netdev.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change combines the the allocation of q_vectors and rings into a single
function.  The advantage of this is that we are guaranteed we will avoid
overlap in the L1 cache sets.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change locks us in at 2K buffers even on a system that supports larger
frames.  The reason for this change is to make better use of pages and to
reduce the overall truesize of frames generated by igb.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

In order to try and isolate things a bit further I am moving the code
related to retrieving data from the rx_buffer_info structure into a
separate function.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change makes it so that we map the entire page and just sync half of
it for the device at a time.  The advantage to this approach is that we can
avoid the locking on map/unmap seen in many IOMMU implementations.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change is meant to just clean-up a number of function calls that were
made at the end of the Rx clean-up path by combining them into a single
function call.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change makes it so that we no longer use header split.  The idea is to
reduce partial cache line writes by hardware when handling frames larger
then header size.  We can compensate for the extra overhead of having to
memcpy the header buffer by avoiding the cache misses seen by leaving an
full skb allocated and sitting on the ring.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Current implementation mess up the tail pointer. This patch sets skb->tail
correctly.
Also, the small packet check and padding is optimized by using unlikely and
calling skb_pad directly.
Signed-off-by: NTushar Dave <tushar.n.dave@intel.com>
Tested-by: NJeff Pieper <jeffrey.e.pieper@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change is meant to improve performance on systems that do not require
the DMA unmap calls.  On those systems we do not need to make use of the
unmap address for Tx or the unmap length so we can drop both thereby
reducing the size of the Tx buffer info structure.

In addition I have changed the logic to check for unmap length instead of
unmap address when checking to see if a buffer needs to be unmapped from
DMA use.  The reasons for this change is that on some platforms it is
possible to receive a valid DMA address of 0 from an IOMMU.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Instead of storing the RSS key as a character array we can simplify the
configuration by making it a u32 array.  This allows us to just write one
value per register without any unnecessary operations to construct the
value.

This change will produce the same exact key, the only difference is that I
translated the u8 array to a u32 array which will be correctly ordered on
writes to hardware by the cpu_to_le32 operations that are built into the
writel calls.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch cleans up our RSS indirection table configuration so that we
generate the same table regardless of CPU endianness.  In addition it
changes the table setup so that instead of doing a modulo based setup it is
instead a divisor based setup.  The advantage to this is that we should be
able to take the Rx hash and compute the Rx queue with very little CPU
overhead if needed.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change makes it so that Tx cleanup is done in a do/while loop instead
of a for loop.  The main motivation behind this is the fact that we should
never be invoked with a budget less than 1 so we can skip checking the
budget before processing the first descriptor.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This change removes the code that was doing the NUMA allocations for the
q_vectors, rings, and ring resources.  The problem is the logic used assumed
that the NUMA nodes were always interleved and that is not always the case.

At some point I hope to add this functionality back in a more controlled
manner in the future.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Due to a hardware issue, on i210 and i211 parts, the TNCRS statistic
provides an invalid value.  This patch changes the update stats function
to increment the stat only for non-i210/i211 parts.
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NJeff Pieper <jeffrey.e.pieper@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Adapt the pre-existing and assigned VFs code to the ixgbe way introduced
in commit 9297127b.

Instead of searching the enabled VFs we use pci_num_vf to determine enabled VFs.
By comparing to which PF an assigned VF is owned it's possible to decide
whether to leave it enabled or not.
Signed-off-by: NStefan Assmann <sassmann@kpanic.de>
Acked-by: NGreg Rose <gregory.v.rose@intel.com>
Tested-by: NRobert Garrett <robertx.e.garrett@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

For some reason the reading of the RQDPC register was being artificially
limited to 4K.  Instead of limiting the value we should read the value and
add the full amount.  Otherwise this can lead to a misleading number of
dropped packets when the actual value is in fact much higher.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Tested-by: NJeff Pieper   <jeffrey.e.pieper@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>