If transmit VLAN HW offloads are disabled then the network stack sends up
an skb with the protocol set to 8021q. In that case to get the correct
checksum offloads we have to reset the skb protocol to the encapsulated
ethertype.

Change-ID: I903d78533de09b1c5d3ec695ee1990dd0fa5dd0d
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If the skb allocation fails we should not continue using the skb
pointer.  Breaking out at the point of failure means that at the next
RX interrupt the driver will try the allocation again.

Change-ID: Iefaad69856ced7418bfd92afe55322676341f82e
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Joe Perches pointed out that we were inconsistent in the use of
PF vs pf or VF vs vf in our driver code.  Since acronyms are usually
capitalized to denote that it is an acronym, changed all references to
be consistent throughout the code.
Reported-by: NJoe Perches <joe@perches.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Fix a bug introduced in the force writeback code, where the interrupt
rate was set to 0 (maximum) by accident.

The driver must correctly set the NOITR fields to avoid ITR update
as a side effect of triggering the software interrupt.

Change-ID: I290851ae04ef3811c43aab5ee33242029f26c1a3
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Performance can be improved a bit by imitating ixgbe and using
prefetch to get us the next Tx descriptor.

Change-ID: Ice7ffd4cd0ce87c35295059bdb7972a7f53723aa
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Use l4_tunnel type generically to keep code flow simple.

Change-ID: Ic52287e3b1ca4204e6b6e13431890c1a6ae9c422
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Since GLQF_FDCNT_0 register now has the right offset, use it to simplify our
FD flush flow.
If the filter add error happens to be for SB we just auto disable SB.

If filter error happens to be for ATR, auto disable ATR and mark
the state to FD_FLUSH_REQUESTED. Which gets cleared when flush completes.

If we are entering flush too quickly (< 30 seconds) and we have quite
a few SB rules, its time to disable ATR for good. Since SB + ATR rules
is most likely making the FD table unstable.

ATR can be re-enabled by turning ntuple off (ethtool -K ntuple off)
and will remain off after turning ntuple on till it gets unstable again.

Change-ID: I2154a2e0a5d44851a2f0eb8731e2f1d4a4d1acbc
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

It is not necessary to print FD filter add/delete log with
normal debug settings because ethtool -n ethx shows all the FD-SB
filters on an interface. The log can still be turned on through higher
debug levels and it will continue to print a log if there was an error
in the add/delete process.

Change-ID: I67db2baf49e2075d2f537de40f7895e5b02cd610
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

A sparse complaint in i40e_debug_aq in a funky buffer write goes away by
straightening out the code out to something less convoluted.

Also fix some other sparse warnings while we are at it, making some
functions static and using NULL instead of 0.

Change-ID: I93907534fe1f1f675830774b3d14ecf1c6ffc9a0
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

We were missing a few packet types for VXLAN offload. This patch fixes
that.

Change-ID: I4b23aa0b08e40ed49d0df6c49a5ed9f2009b44ce
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If DCB is not enabled priority tagging is not needed
so skip over that section.

Change-ID: Ia3f3fa07945b421259a9ca38329d6d1cbd6c6bcc
Signed-off-by: NNeerav Parikh <neerav.parikh@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The driver was having some issues with false Tx hang detection. This
makes the driver a little more direct with the checks for progress
forward by directly checking the head write back address and tail register
when determining progress.  This avoids Tx hangs where the software
gets behind, because we are directly checking hardware state when
determining hang state.

Change-ID: I774f0e861c9e8ab5ccb213634100fe15440ae24a
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The hardware has some limitations the driver needs to adhere to,
that we found in extended testing.
  1) no more than 8 descriptors per packet on the wire
  2) no header can span more than 3 descriptors

If one of these events occurs, the hardware will generate an internal
error and freeze the Tx queue.

This patch linearizes the skb to avoid these situations.

Change-ID: I37dab7d3966e14895a9663ec4d0aaa8eb0d9e115
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Split the receive hot path code into two, one for packet split and one
for single buffer. This improves receive performance since we only need
to check if the ring is in packet split mode once per NAPI poll time,
not several times per packet. The single buffer code is further improved
by the removal of a bunch of code and several variables that are not
needed. On a receive-oriented test this can improve single-threaded
throughput.

Also refactor the packet split receive path to use a fixed buffer for
headers, like ixgbe does. This vastly reduces the number of DMA mappings
and unmappings we need to do, allowing for much better performance in
the presence of an IOMMU.

Lastly, correct packet split descriptor types now that we are actually
using them.

Change-ID: I3a194a93af3d2c31e77ff17644ac7376da6f3e4b
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch fixes indentation issue and error found in argument
reported by static analysis.  Without this patch, sparse and other
static analysis errors will be found.
Reported-by: NFengguang Wu <fengguang.wu@intel.com>
Reported-by: NJulia Lawall <julia.lawall@lip6.fr>
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds a call to u64_stats_init to Rx setup.
This done in order to avoid lockdep errors with seqcount on newer kernels.

Change-ID: Ia8ba8f0bcbd1c0e926f97d70aeee4ce4fd055e93
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch forces Tx descriptor writebacks on ITR by kicking
off the SWINT interrupt when we notice that there are non-cache-aligned
Tx descriptors waiting in the ring while interrupts are disabled
under NAPI.

Change-ID: dd6d9675629bf266c7515ad7a201394618c35444
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

We should not be doing Tx or Rx timestamps if we do not have PTP
enabled. Add checks to ensure that we don't attempt to handle any PTP
related timestamping code if we have not enabled PTP on that PF.

Change-ID: I4335942ae2d5c5f91abfdbeeea02bcace49e7677
Signed-off-by: NJacob Keller <jacob.e.keller@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The same macros are used for rx as well. So rename it.
Signed-off-by: NJiri Pirko <jiri@resnulli.us>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The driver was examining the outer protocol layer to set the inner protocol
layer checksum offload.  In the case of TCP over IPV6 over an IPv4 based
VXLAN the inner checksum offloads would be set to look for IPv4/UDP instead
of IPv6/TCP.  This code fixes that so that the driver will look at the
proper layer for encapsulation offload settings.
Signed-off-by: NAnjali Singhai <anjali.singhai@intel.com>
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The Rx port checksum error counter was incrementing incorrectly with
UDP encapsulated tunneled traffic.  This patch fixes the problem so that
the port_rx_csum counter will show accurate statistics.
Signed-off-by: NAnjali Singhai <anjali.singhai@intel.com>
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

When the driver was polling with interrupts disabled the hardware
will occasionally not write back descriptors.  This patch causes
the driver to detect this situation and force an interrupt to
fire which will flush the stuck descriptor.  Does not conflict
with napi because if we are already polling the napi_schedule is
ignored.  Additionally the extra interrupts are rate limited, so
don't cause a burden to the CPU.

Change-ID: Iba4616d2a71288672a5f08e4512e2704b97335e8
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Update the Intel Ethernet drivers to use eth_skb_pad() and skb_put_padto
instead of doing their own implementations of the function.

Also this cleans up two other spots where skb_pad was called but the length
and tail pointers were being manipulated directly instead of just having
the padding length added via __skb_put.

Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NAlexander Duyck <alexander.h.duyck@redhat.com>
Acked-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Support skb->xmit_more in i40e is straightforward : we need to move
around i40e_maybe_stop_tx() call to correctly test netif_xmit_stopped()
before taking the decision to not kick the NIC.
Signed-off-by: NEric Dumazet <edumazet@google.com>
Signed-off-by: NDaniel Borkmann <dborkman@redhat.com>
Acked-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Convert two more Intel NIC drivers to dev_consume_skb_any() to help
make dropped packet profiling sane.
Signed-off-by: NRick Jones <rick.jones2@hp.com>
Tested-by: NAaron Brown <aaron.f.brown@intel.com>
Tested-by: NJim Young <jamesx.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Set skb->csum_level instead of skb->encapsulation when indicating
CHECKSUM_UNNECESSARY for an encapsulated checksum.
Signed-off-by: NTom Herbert <therbert@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We are seeing situations where the driver sees a hang with less than 4
desc pending, if the driver chooses to ignore it the queue progresses
forward and the stack never experiences a real hang.
With this patch we will log a stat when this situation happens
"tx_sluggish" will increment and we can see some more details
at a higher debug level. Other than that we will ignore this
particular case of Tx hang.

Change-ID: I7d1d1666d990e2b12f4f6bed0d17d22e1b6410d5
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Instead of disabling ATR when we get a programming error, we now
will wait it out to see if some room gets created by ATR rule deletion.
If we still have too many errors and ATR filter count did not change
much, its time to flush and replay. We no more auto-disable ATR when
we have errors in programming.
The disabling of ATR when we get programming error was buggy and
was still adding new rules and causing continuous errors. With this
policy change we flush instead when we see too many errors.
ATR is still disabled if we add a SB rule for TCP/IPv4 flow type,
more logic is added to re-enable it once all SB TCP/IPv4 rules are gone.

Change-ID: I77edcbeab9500c72a7e0bd7b5c5b113ced133a9c
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <jamesx.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Change the message that gets printed when adding/deleting a filter to
the SB, so that user can tell if a filter was added or deleted.
Print filter add failures only in case of SB filters. For ATR the
information is not useful to the user and hence suppress it unless in
higher debug mode.

Change-ID: I78d7a7a6ecfa82a38a582b0d7b4da038355e3735
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <jamesx.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch changes the wording of the flow director add/remove and
asynchronous failure messages to include fd_id to try and add some
way to track the operations on a given fd_id.  Its not perfect, but
its better than what we had as PCTYPE can apply to several different
filter requests.

This patch also removes a redundant message when filter
addition fails due to full condition.

Change-ID: Icf58b0603d4f162d9fc542f11a74866a907049f2
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NJim Young <jamesx.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This device claims TSO and checksum support for vlans.  It also
allows a user to control vlan acceleration offloading.  As such,
it is possible to turn off vlan acceleration and configure a vlan
which will continue to support TSO and hw checksums.

In such situation the packet passed down the the device will contain
a vlan header and skb->protocol will be set to ETH_P_8021Q.
The device assumes that skb->protocol contains network protocol
value and uses that value to set up TSO and checksum information.
This results in corrupted frames sent on the wire.

This patch extract the protocol value correctly and corrects TSO
and checksums for non-accelerated traffic.

Fix this by using vlan_get_protocol() helper.

CC: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
CC: Jesse Brandeburg <jesse.brandeburg@intel.com>
CC: Bruce Allan <bruce.w.allan@intel.com>
CC: Carolyn Wyborny <carolyn.wyborny@intel.com>
CC: Don Skidmore <donald.c.skidmore@intel.com>
CC: Greg Rose <gregory.v.rose@intel.com>
CC: Alex Duyck <alexander.h.duyck@intel.com>
CC: John Ronciak <john.ronciak@intel.com>
CC: Mitch Williams <mitch.a.williams@intel.com>
CC: Linux NICS <linux.nics@intel.com>
CC: e1000-devel@lists.sourceforge.net
Signed-off-by: NVladislav Yasevich <vyasevic@redhat.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds FCoE specific code to existing i40e core driver to:-

1. have separate FCoE VSI with additional FCoE queues pairs.
2. have FCoE related hash defines.
3. have additional FCoE related stats code.
4. export and then re-use existing functions required by FCoE build.
Signed-off-by: NVasu Dev <vasu.dev@intel.com>
Tested-by: Jack Morgan<jack.morgan@intel.com>
Signed-off-by: NAaron Brown <aaron.f.brown@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The hardware design requires that the driver avoid indicating
checksum offload success on some ipv6 frames with extension
headers.

The code needs to just check for the IPV6EXADD bit and if
it is set punt the checksum to the stack.  I don't know why
the code was checking TCP on inner protocol, as that code
doesn't make any sense to me but seems wrong, so remove it.

Change-ID: I10d3aacdbb1819fb60b4b0eb80e6cc67ef2c9599
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-By: NJim Young <jamesx.m.young@intel.com>
Tested-by: NSibai Li <sibai.li@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

1. Remove some break statements that will never get touched.
2. Remove an extra space.
3. Remove a comment for a parameter that doesn't exist
4. Move the assignment of a variable up to get rid of an else case.

Change-ID: I308a4b5ec070b1f0601f13b041ba4375aaad4b06
Signed-off-by: NPaul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Tested-by: NJim Young <jamesx.m.young@intel.com>
Tested-by: NSibai Li <sibai.li@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

i40e has a single set of TX time stamping resources per NIC.
Use a simple bit lock to avoid race conditions and leaking skbs
when multiple TX rings try to claim time stamping.
Signed-off-by: NJakub Kicinski <kubakici@wp.pl>
Tested-By: NJim Young <jamesx.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

skb_tx_timestamp() does not report software time stamp
if SKBTX_IN_PROGRESS is set. According to timestamping.txt
software time stamps are a fallback and should not be
generated if hardware time stamp is provided.

Move call to skb_tx_timestamp() after setting
SKBTX_IN_PROGRESS.
Signed-off-by: NJakub Kicinski <kubakici@wp.pl>
Tested-by: NJim Young <jamesx.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The HW still needs to consume it and freeing it in the function
that created it would mean we will be racing with the HW. The
i40e_clean_tx_ring() routine will free up the buffer attached once
the HW has consumed it.  The clean_fdir_tx_irq function had to be fixed
to handle the freeing correctly.

Cases where we program more than one filter per flow (Ipv4), the
code had to be changed to allocate dummy buffer multiple times
since it will be freed by the clean routine.  This also fixes an issue
where the filter program routine was not checking if there were
descriptors available for programming a filter.

Change-ID: Idf72028fd873221934e319d021ef65a1e51acaf7
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <jamesx.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Driver needs to initialize all members of context descriptor. Stale
data is possible otherwise.

Change-ID: Idc6b53af45583509da42d5ec0824cbaf78aee64f
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

With the auto_disable flags added there was a bug that was causing the
replay logic to not work correctly.
This patch fixes the issue so that we call a replay after a sideband
reset correctly.

Change-ID: I005fe1ac361188ee5b19517a83c922038cba1b00
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

There were a couple of fields in the fdir descriptor setup that
were not being reprogrammed, which left the opportunity for stale
data to be pushed as part of the descriptor next time it was used.

Change-ID: Ieee5c96a7d4713d469693f086c4854de949a7633
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>