Support packet split receive on VFs. This is off by default but can be
enabled using ethtool private flags. Because we need to trigger a reset
from outside of i40evf_main.c, create a new function to do so, and
export it.

Also update copyright year in file headers.

Change-ID: I721aa5d70113d3d6d94102e5f31526f6fc57cbbb
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Bump version to i40e-1.4.13 and i40evf-1.4.9

Change-ID: I9db37f9d4899141c3e5455dfb456d45465b8c035
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Get rid of the unused hsplit field in the ring struct and use the
existing macro to detect packet split enablement. This allows debugfs
dumps of the VSI to properly show which Rx routine is in use.

Change-ID: Ic4e9589e6a788ab196ed0850703f704e30c03781
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Mr. Spock would certainly raise an eyebrow to see us using bitwise
operators, when we should clearly be relying on logic. Fascinating.

Change-ID: Ie338010c016f93e9faa2002c07c90b15134b7477
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Refactor the packet split Rx code to properly use half-pages for
receives. The previous code was doing way more mapping and unmapping
than it needed to, and wasn't properly using half-pages.

Increment the page use count each time we give a half-page to an skb,
knowing that the stack will probably process and release the page before
we need it again. Only free and reallocate pages if the count shows that
both half-pages are in use. Add counters to track reallocations and page
reuse.

Change-ID: I534b299196036b64be82b4861a0a4036310a8f22
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The i40e and i40evf drivers now cleanly handle allocation
failures and can avoid kernel log spew from the memory allocator
when allocations fail, so set __GFP_NOWARN on Rx buffer alloc.

Change-ID: Ic9e1b83c495e2a3ef6b069ba7fb6e52ce134cd23
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This is the "Don't Give Up" patch.  Previously the
driver could fail an allocation, and then possibly stall
a queue forever, by never coming back to continue receiving
or allocating buffers.

With this patch, the driver will keep polling trying to allocate
receive buffers until it succeeds.  This should keep all receive
queues running even in the face of memory pressure.

Also update copyright year in file header.

Change-ID: I2b103d1ce95b9831288a7222c3343ffa1988b81b
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

While re-enabling interrupts the driver would clear all pending
causes. This meant that if an interrupt was generated while the driver
was cleaning or polling with interrupts disabled, then that interrupt
was lost.  This could cause a queue to become dead, especially for
receive.  Refactored the enable_icr0 function in order to allow
it to be decided by the caller whether the CLEARPBA (clear pending
events) bit will be set while re-enabling the interrupt.

Also update copyright year in file headers.

Change-ID: Ic1db100a05e13c98919057696db147a258ca365a
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Change the driver string to 40-10 Gigabit instead of XL710/X710 for X722
and all future products.

Also update copyright year in file header.

Change-ID: I57fae656b36dc4eb682b2b7a054f8f48f3589149
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Now that the Force-WriteBack functionality in X710/XL710 devices
has been moved out of the clean routine and into the service task,
we need to make sure WriteBack-On-ITR is separated out since it
is still called from clean.

In the X722 devices, Force-WriteBack implies WriteBack-On-ITR but
without the interrupt, which put the driver into a missed
interrupt scenario and a potential tx-timeout report.

With this patch, we break the two functions out, and call the
appropriate ones at the right place. This will avoid creating missed
interrupt like scenarios for X722 devices.

Also update copyright year in file headers.

Change-ID: Iacbde39f95f332f82be8736864675052c3583a40
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Bump driver versions to i40e-1.4.12 and i40evf-1.4.8

Change-ID: I0ad82668c4ded04250391fda396ce191a42ab754
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Don't bother trying to set up a TSO if the skb->ip_summed is not
set to CHECKSUM_PARTIAL.

Change-ID: I6495b3568e404907a2965b48cf3e2effa7c9ab55
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Driver was using an offset based off a DMA handle while mapping and
unmapping using sync_single_range_for[cpu|device], where it should
be using DMA handle (returned from alloc_coherent) and the offset of the
memory to be sync'd.

Change-ID: I208256565b1595ff0e9171ab852de06b997917c6
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Reviewed-by: NNelson, Shannon <shannon.nelson@intel.com>
Reviewed-by: NWilliams, Mitch A <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Bump.

Change-ID: I21aa520a3c8c5f4f562a98019bf8b76b3706c480
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If the VF is reset via VFLR, the device will be knocked out of bus
master mode, and the driver will fail to recover from the reset. Fix
this by enabling bus mastering after every reset. In a non-VFLR case,
the bus master bit will not be disabled, and this call will have no effect.

Change-ID: Id515859ac7a691db478222228add6d149e96801a
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

We were not doing write-back on interrupt throttle for Legacy case in X722.
This patch fixes that, so we do WB_ON_ITR for Legacy as well. Plus the issue
that we should still be setting NO_ITR if we are touching the DYN_CTLN register
since we do not want to change ITR setting here.

Change-ID: I5db8491ee1544118a389db839cecc93e1bbc480e
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Bump AQ minor version to 1.5 for new FW features.

Change-ID: I5a790f7f519a2a8921aaa1c5663727dd1897ffec
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NKevin Scott <kevin.c.scott@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add the new AQ command and struct for managing a thermal sensor.

Change-ID: I6f5631839a0f3dca352a6c222f1269a960e2310a
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add the new Cisco VXLAN-GPE cloud tunnel type for the Add Cloud Filter
and UDP tunnel AQ commands.

Change-ID: I2c093c7d79726c7fca08a36e5c63581a905da3d2
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NKevin Scott <kevin.c.scott@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add the new Set Switch Config AdminQ command, and mark the L2 Filter
bit as deprecated in the Add VEB command.

Change-ID: I5b24790f14c56f0ddf3f70df1e486844146b039f
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NKevin Scott <kevin.c.scott@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add flags to MAC allocation requests to signify that the MAC VLAN filters
should come from the shared resource pool rather than the dedicated PF
resource pools.

Change-ID: I4c2da64c01856edcb0982bc4aab75c5a91047a7a
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NKevin Scott <kevin.c.scott@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add the new External Device Power Ability field to the get_link_status data
structure, using space from the reserved field at the end of the struct.
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NKevin Scott <kevin.c.scott@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Fix the name of the new cloud tunnel type from the place-holder NGE
name to the official Geneve.  Also fix the spelling of the VXLAN type.
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NKevin Scott <kevin.c.scott@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>

Add the AQ opcode and struct definitions for the Run PHY Activity command
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NKevin Scott <kevin.c.scott@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add the new proxy-wake-on-lan capability bit available with the
new X722 device.
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

As done per ixgbe, use a private workqueue to avoid blocking the
system workqueue.  This avoids some strange side effects when
some other entity is depending on the system work queue.

Change-ID: Ic8ba08f5b03696cf638b21afd25fbae7738d55ee
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add write-back on interrupt throttle rate timer expiration support
for the i40evf driver, when running on X722 devices.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The PCTYPES for the X710 and X722 families are different. This patch
makes adjustments for that.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Since we check these ring pointers to make sure we don't double-allocate
or double-free the rings, we had better null them out after we free
them. In very rare cases this can cause a panic if the driver is removed
during reset recovery.

Change-ID: Ib06eb4910a3058275c8f7ec5ef7f45baa4674f96
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The device capabilities were defined in two places, and neither had all
the definitions.  It really belongs with the AQ API definition, so this
patch removes the other set of definitions and fills out the missing item.

Change-ID: I273ba7d79a476cd11d2e0ca5825fec1716740de2
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

In some modes, bonding would not enslave VF interfaces. This is due to
bonding calling change_mtu and the immediately calling open. Because of
the asynchronous nature of the admin queue mechanism, the VF returns
-EBUSY to the open call, because it knows the previous operation hasn't
finished yet. This causes bonding to fail with a less-than-useful error
message.

To fix this, remove the check for pending operations at the beginning of
open. But this introduces a new bug where the driver will panic on a
quick close/open cycle. To fix that, we add a new driver state,
__I40EVF_DOWN_PENDING, that the driver enters when down is called. The
driver finally transitions to a fully DOWN state when it receives
confirmation from the PF driver that all the queues are disabled. This
allows open to complete even if there is a pending mtu change, and
bonding is finally happy.

Change-ID: I06f4c7e435d5bacbfceaa7c3f209e0ff04be21cc
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch fixes the Hash PCTYPE enable for X722 since it supports
a broader selection of PCTYPES for TCP and UDP.

This patch also fixes a bug in XL710, X710, X722 support for RSS,
as of now we cannot reduce the (4)tuple for RSS for TCP/IPv4/IPV6 or
UDP/IPv4/IPv6 packets since this requires a product feature change
that comes in a later release.

A VF should never be allowed to change the tuples for RSS for any
PCTYPE since that's a global setting for the device in case of i40e
devices.

Change-ID: I0ee7203c9b24813260f58f3220798bc9d9ac4a12
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Somehow an ID that has never been productized is in the
code.  There are no plans to use it, so just get
rid of it.

Change-ID: I59117d48ea9ee0360b0fe33833ac8092f8a24b4c
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If the driver calls skb_set_hash even with a zero hash, that
indicates to the stack that the hash calculation is offloaded
in hardware. So the Stack doesn't do a SW hash which is required
for load balancing if the user decides to turn of rx-hashing
on our device.

This patch fixes the path so that we do not call skb_set_hash
if the feature is disabled.

Change-ID: Ic4debfa4ff91b5a72e447348a75768ed7a2d3e1b
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The SCTP checksum is really a CRC and is very different from the
standards 1's complement checksum that serves as the checksum
for IP protocols. This offload interface is also very different.
Rename NETIF_F_SCTP_CSUM to NETIF_F_SCTP_CRC to highlight these
differences. The term CSUM should be reserved in the stack to refer
to the standard 1's complement IP checksum.
Signed-off-by: NTom Herbert <tom@herbertland.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Bump.

Change-ID: I2b8976bde070244de144e2ed8990b083de39f332
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Generate version strings like the PF driver does. This gives us more
flexibility to add suffixes to the version string at build time.

Change-ID: I0a5ca0783dd8fb849516bfc1e37ea070127847bd
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

X722 supports Expanded version of TCP, UDP PCTYPES for RSS.
Add a Virtchnl offload to support this.

Without this patch with X722 devices, driver will set wrong PCTYPES
for VF and UDP flows will not fan out.

Change-ID: I04fe4988253b7cd108c9179a643c969764efcb76
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This is a part of implementation which contains data structures and
opcode for new AQ command. There's a new ARQ message that gets sent
near the end of the NVM update process that the driver should recognize
and ignore, rather than printing an Unknown Event error.

Change-ID: I04830a5bcae14823e16b9424cc4165e169336c1f
Signed-off-by: NMichal Kosiarz <michal.kosiarz@intel.com>
Acked-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If the driver gets unloaded during reset recovery, it's possible
that it will attempt to free resources when they're already free.

Add a check to make sure that the Tx and Rx rings actually exist
before dereferencing them to free resources.

Change-ID: I4d2b7e9ede49f634d421a4c5deaa5446bc755eee
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>