X722 uses the admin queue to configure RSS. This patch adds the necessary
flow changes to configure RSS through AQ. It also adds the separate VMDQ2
lookup tables and hash key programming for X722.

X722 also exposes a different set of PCTYPES for RSS, this patch
accommodates those changes.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Update the i40e and i40evf register.h file with the registers for X722.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch does the firmware API update to support the new X722 device.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add capabilities flags specific to X722.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Adding device ids for new hardware X722
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Use macros for abstracting (1 << foo) to BIT(foo)
and (1ULL << foo64) to BIT_ULL(foo64) in order to match
better with kernel requirements.

NOTE: the adminq_cmd.h file was not modified on purpose because
of the dependency upon firmware for that file.

Change-ID: I73ee2e48c880d671948aad19bd53ca6b2ac558fc
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Clean up a little confusion in reporting error status in phy and fcoe
setup error reports by separating the return status from the AQ error.

Add two decoder functions to make this easier.

Change-ID: I960bcdeef3978a15fec1cdb5eff781d5cbae42fb
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Store off the PF's API version, then use it to determine whether or not
to send it our capabilities. Change the version checking to allow for PF
drivers with lower API versions than our current version, so we can
still talk to PF drivers over the 1.0 API.

Change-ID: I8edc55d1229c7decf0ed3f285a63032694007c2e
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The most common type of reset that the VF will encounter is a PF reset
that cascades down into a VF reset for each VF. In this case, the VF
will always be assigned the same VSI and recovery is fairly simple.

However, in the case of 'bigger' resets, such as a Core or EMP reset,
when the device is reinitialized, it's probable that the VF will NOT get
the same VSI. When this happens, the VF will not be able to recover, as
it will continue to request resources for its original VSI.

Add an extra state to the admin queue state machine so that the driver
can re-request its configuration information at runtime. During reset
recovery, set this bit in the aq_required field, and fetch the (possibly
new) configuration information before attempting to bring the driver
back up. Since the driver doesn't know what kind of reset it has
encountered, this step is done even for a PF reset, but it doesn't hurt
anything - it just gets the same VSI back.

Change-ID: I915d59ffb40375215117362f4ac7a37811aba748
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Now that we've rolled the virtual channel API version to 1.1, add some
macros to test what version is being used by our partner in crime. For the
VF, add some macros to determine what our device capabilities are.

Change-ID: I79f6683d4c23bd76a8ad9fd492776fcc1208e1dc
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

To prepare for the changes coming up in the X722 device and future
devices, the virtual channel interface has to change slightly. The VF
driver can now report what its capable of supporting, which then informs
the PF driver when it sends the configuration information back to the
VF.

A 1.1 VF driver on a 1.0 PF driver should not send its capabilities.
Likewise, a 1.1 PF driver controlling a 1.0 VF driver should not expect
or depend upon receiving the VF capabilities.

All other aspects of the API are unchanged.

Change-ID: I530cc55f107edd1ee8bdf95830aa90b87854058a
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Acked-by: NShannon Nelson <shannon.nelson@intel.com>
Acked-by: NAnjali Singhai <anjali.singhai@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch changes the switch statement for dynamic interrupt throttling
and adds a default case. With this patch, we check the latency setting
instead of the current ITR settings and the included refactor improves
performance.

Without this patch, the ITR setting would never change dynamically, and
there was no default.

Change-ID: Idb5a8a14c7109ec47c90f6e94bd43baa17d7ee37
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Bump.

Change-ID: I84573d9fa51effc5b29bf5b8c74e3cc8b2673f48
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The i40e_add_pd_table_entry() routine is being modified to handle both
cases where a backing page is passed and where backing page is allocated
in i40e_add_pd_table_entry().

For PBLE resource management, it is more efficient for it to manage its
backing pages. For VF, PBLE backing page addresses will be send to PF
driver for PBLE resource.

The i40e_remove_pd_bp() is also modified to not free pre-allocated pages and
free only ones which were allocated in i40e_add_pd_table_entry().

Change-ID: Ie673f0403f22979e9406f5a94048dceb91bcf9a8
Signed-off-by: NFaisal Latif <faisal.latif@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

During close, all of the MAC filters are cleared, so the driver would be
unable to receive unicast packets after being closed and reopened.

Add the adapter's "hardware" MAC address filter in open, not init. This
ensures that the correct filter is present each time.

Change-ID: I51a11e9c1200139dab6f66a5353bd38c7d26f875
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Due to an inverted conditional, the driver was marking all of its MAC
filters for deletion every time set_rx_mode was called. Depending upon
the timing of the calls to set_rx_mode and the processing of the admin
queue, the driver would (accidentally) end up with a varying number of
functional filters.

Correct this logic so that MAC filters are added and removed correctly.
Add a check for the driver's "hardware" MAC address so that this filter
doesn't get removed incorrectly.

Change-ID: Ib3e7c4a5b53df6835f164fe44cb778cb71f8aff8
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Make the necessary updates to i40e_adminq_cmd.h.

Change-ID: Ib031c86cc6cab78e5aa44c64d8ce5474be8d7e42
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The driver currently only maps TX and RX queues to a single MSI-X vector
per queue pair if there are exactly enough vectors for this.
Unfortunately, if we have too many vectors it will fail and allocate
queues to vectors in a suboptimal manner. Change the condition check to
allow for excess vectors. In this case, the extras just won't be used.

Change-ID: I23e1e2955c64739c86612db88a25583e6a7e0b17
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add a prefetch for the next Tx descriptor to be used when we know
there are more coming.

Change-ID: Ibb9acab11d508eec2db7da795df74debc16eeacb
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The Flex10 device/function capability has been upgraded to include
information needed to support Flex-10 configurations. This patch adds new
fields to the i40e_hw_capabilities structure and updates
i40e_parse_discover_capabilities functions to extract them from the AQ
response. Naming convention has changed to use flex10 mode instead of
existing mfp_mode_1.

Change-ID: I305dd888866985a30293acb3fb14fa43ca6b79ea
Signed-off-by: NPawel Orlowski <pawel.orlowski@intel.com>
Signed-off-by: NAkeem G Abodunrin <akeem.g.abodunrin@intel.com>
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Since the driver can dynamically enable/disable FD ATR and SB features,
these stats help keep track of the current state and along with
fd_flush count provide a means to debug what could be going on
with the flow director filters. This will take away the need for
being verbose in our debug logs with respect to FD.

Change-ID: I29224f750fe6602391043655d18996570720377d
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The driver will only configure as many queues as there are available
CPUs, up the maximum number of queues. However, it always configures
RSS as though it is using the maximum number of queues. This can cause
the device to drop a lot of RX traffic, as the packets get assigned to
nonfunctional queues.

Fix this by only configuring RSS with the number of active queues.
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Down was requesting queue disables, but then exited immediately
without waiting for the queues to actually disable.  This could
allow any function called after i40evf_down to run immediately,
including i40evf_up, and causes a memory leak.

Removing the whole reinit_locked function is the best way
to go about this, and allows for the driver to handle the
state changes by requesting reset from the periodic timer.

Also, add a couple WARN_ONs in slow path to help us recognize
if we re-introduce this issue or missed any cases.
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch fixes a bug where the i40e Tx queue will hang if this
skb is passed to the driver.

With mixed size fragments while using TSO there was a corner case
where we needed to linearize but we were not. This was seen with
iSCSI traffic and could be reproduced with a frag list that looks
like this:

num_frags = 17, gso_segs = 17, hdr_len = 66,
skb_shinfo(skb)->gso_size = 1448
size = 3002, j = 1, frag_size = 2936, num_frags = 17
size = 4268, j = 1, frag_size = 4096, num_frags = 16
size = 5534, j = 1, frag_size = 4096, num_frags = 15
size = 5352, j = 1, frag_size = 4096, num_frags = 14
size = 5170, j = 1, frag_size = 4096, num_frags = 13
size = 3468, j = 1, frag_size = 2576, num_frags = 12
size = 750, j = 1, frag_size = 112, num_frags = 11
size = 862, j = 2, frag_size = 112, num_frags = 10
size = 974, j = 3, frag_size = 112, num_frags = 9
size = 1126, j = 4, frag_size = 152, num_frags = 8
size = 1330, j = 5, frag_size = 204, num_frags = 7
size = 1534, j = 6, frag_size = 204, num_frags = 6
size = 356, j = 1, frag_size = 204, num_frags = 5
size = 560, j = 2, frag_size = 204, num_frags = 4
size = 764, j = 3, frag_size = 204, num_frags = 3
size = 968, j = 4, frag_size = 204, num_frags = 2
size = 1140, j = 5, frag_size = 172, num_frags = 1
result: linearize = 0, j = 6

Change-ID: I79bb1aeab0af255fe2ce28e93672a85d85bf47e8
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The driver doesn't use the time_stamp member to determine if there is a
tx_hang any more. There really isn't any point to the variable at all
so just remove it. It was left over from a previous tx_hang design.

Change-ID: I4c814827e1bcb46e45118fe37acdcfa814fb62a0
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Inlining these functions gives us about 15% more 64 byte packets per
second when using pktgen. 13.3 million to 15 million with a single
queue.

Also fix the function names in i40evf to i40evf not i40e while we are
touching the function header.

Change-ID: I3294ae9b085cf438672b6db5f9af122490ead9d0
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Eric added support for skb->xmit_more in i40e, this ports that into
i40evf as well.

Support skb->xmit_more in i40evf is straightforward; we need to move
around i40e_maybe_stop_tx() call to correctly test netif_xmit_stopped()
before taking the decision to not kick the NIC.

Change-ID: Idddda6a2e4a7ab335631c91ced51f55b25eb8468
Signed-off-by: NEric Dumazet <edumazet@google.com>
Signed-off-by: NDaniel Borkmann <dborkman@redhat.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

There's no need for a counter so remove the TODO comment.

Change-ID: I3321dda04934c4f5fda9b279ab666192bda44214
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add a 3rd dynamic filter counter to track Tunneled ATR hits separately.
Ethtool port stat "fdir_atr_tunnel_match"

Change-ID: Idd978b6db2a462b5722397cd2ffd04ef055f8655
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Without this, RSS would have done inner header load balancing. Now we can
get the benefits of ATR for tunneled packets to better align TX and RX
queues with the right core/interrupt.

Change-ID: I07d0e0a192faf28fdd33b2f04c32b2a82ff97ddd
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

None of those drivers uses last_rx for its own needs.

See 4dc89133 ("net: add a comment on
netdev->last_rx") for reference.
Signed-off-by: NFrancois Romieu <romieu@fr.zoreil.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Zhangfei Gao <zhangfei.gao@linaro.org>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Cc: Wingman Kwok <w-kwok2@ti.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Chris Metcalf <cmetcalf@tilera.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Store the 8 bytes of the WR_CSR_PROT field returned as part of the get
device/function capabilities AQ command.

Change-ID: Ifcaeea2ff29885fa769e4f384c7db88a25e8afd0
Signed-off-by: NKevin Scott <kevin.c.scott@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The aq_pending field in the adapter structure is actually redundant with
the current_op field. Remove the aq_pending field and expunge all traces
of it from the official record. This simplifies the code significantly,
especially in the virtual channel completion routine.

Change-ID: Ib2957c8c19882bd0cecc6fcd133912c24b46a1ff
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Not sure how this slipped through. Cosmetic change only.
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Update i40e and i40evf to use dma_rmb.  This should improve performance by
decreasing the barrier overhead on strong ordered architectures.

Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NAlexander Duyck <alexander.h.duyck@redhat.com>
Acked-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Refactor VF RSS code to allow RSS on a single queue and eliminate
the need for the next_queue function.

Change-ID: I9253bad96b7f542ee7036e15636db0e5d58d8ef2
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The MAC filter list is protected by a critical task bit, and the VLAN
list should be protected as well. This prevents list corruption if the
watchdog happens to run at the same time as a VLAN filter is being added
or deleted.

Change-ID: Ia4867cebbbb046a1f38012771b288a634ca5882b
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If transmit VLAN HW offloads are disabled then the network stack sends up
an skb with the protocol set to 8021q. In that case to get the correct
checksum offloads we have to reset the skb protocol to the encapsulated
ethertype.

Change-ID: I903d78533de09b1c5d3ec695ee1990dd0fa5dd0d
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If the skb allocation fails we should not continue using the skb
pointer.  Breaking out at the point of failure means that at the next
RX interrupt the driver will try the allocation again.

Change-ID: Iefaad69856ced7418bfd92afe55322676341f82e
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Inner protocol being UDP should not stop us from verifying Outer UDP
checksum correctness.

If the Outer protocol is not UDP (NVGRE) we should not be doing a UDP
checksum check. If the packet has zero checksum, skip checksum check.

Change-ID: Ie7f153feb276a59f66a54a0938901b2c0a8100fa
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>