Instead of having our own custom symbol, we can just rely
on whether or not the kernel has the feature enabled.

In this case use IS_ENABLED(CONFIG_VXLAN) in order to handle
built-in or module in the current BKM way.

Change-ID: I5890fbb518ff8ed6bb07c3362fb0a8a829f9b241
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Ethtool priv flags implementation to enable or disable packet split, which
is a hardware feature that inspects headers and will put headers in a
separate DMA buffer from the payload data.  The driver was automatically
choosing to enable packet split in some cases and this gives the user the
ability to turn it off/on explicitly.

to query state:
ethtool --show-priv-flags ethx

to enable:
ethtool --set-priv-flags ethx packet-split on
to disable:
ethtool --set-priv-flags ethx packet-split off

Why would anyone want this?
	Because some environments benefit from header/data split in the receive
	buffer, and the driver defaults to one or the other depending on
	environment/kernel parameters.

Why didn't you implement a generic ethtool control for this feature?
	Because Intel hardware is the only hardware that supports header/data
	split.

Change-ID: I803121e1eecc9ccb2884031fd85dd1110b3af66d
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch contains following changes:
   - detection and recovery logic (issue SW interrupt) has been moved to
     service_task from timeout function.
   - added some more debug info from tx_timeout.

Logic to detect and recover TX queue hung is now two step process:
  - service_task detects TX queue hung and sets a bit(hung_detected) if
    it was not set.
  - if bit was set (means this is back-back hung condition detected),
    issue SW interrupt and clear the bit.
  - napi_poll clears the bit unconditionally since it cleans TX/RX queues.

Change-ID: Ieed03a48927c845a988b3ff375090bf37caeb903
Signed-off-by: NKiran Patil <kiran.patil@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

With the final edition of the patches to remove sleeps from
the driver's entry points, the grab_rtnl argument is no
longer needed, so partially revert the commit that added it.

Change-ID: Ib9778476242586cc9e58b670f5f48d415cb59003
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch renames rss_size to alloc_rss_size in i40e_pf, which is
clearer and avoids confusion. It also adds comments to the other
related structure members to help clarify usage.

Change-ID: Ia90090609d006ab589cb639975bb8a0af795d16f
Signed-off-by: NHelin Zhang <helin.zhang@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch adds new fields to i40e_vsi to store user configured
RSS config data and code to use it.
Signed-off-by: NHelin Zhang <helin.zhang@intel.com>
Change-ID: I73886469dca9e9f6b16d842182a87f3f4009f95d
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch renames the old pf-specific function in order to clarify
its scope. This patch also creates a more generic configure RSS
function with the old name.

This patch also creates a new more generic function to get RSS
configuration, using the appropriate method.

Change-ID: Ieddca2707b708ef19f1ebccdfd03a0a0cd63d3af
Signed-off-by: NHelin Zhang <helin.zhang@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Adjust the RSS configure functions so that there is a generic way to
hook to ethtool hooks.

Change-ID: If446e34fcfaf1bc3320d9d319829a095b5976e67
Signed-off-by: NHelin Zhang <helin.zhang@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

When in NAPI with interrupts disabled, the HW needs to be forced to do a
write back on TX if the number of descriptors pending are less than a
cache line.

This stat helps keep track of how many times we get into this situation.

Change-ID: I76c1bcc7ebccd6bffcc5aa33bfe05f2fa1c9a984
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch fixes a problem where the driver output of the OEM
version string varied from the other tools.  The mask value
and the order of operations were incorrect, per the original
change request.  Without this patch, the version string will
appear incorrect from the driver.

Change-ID: Ie1ca6485284b4ce3b57e5a99b18b7641617c7ef7
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The adaptive ITR (interrupt throttle rate) algorithm was adjusting
the hardware's interrupt rate too frequently.  This caused a lot
of variation in the interrupt rate for fairly constant workloads.

Change the code to have a counter and adjust only once every N
number of interrupts.

Change-ID: I0460f1f86571037484eca5aca36ac4d889cb8389
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch introduces a spinlock which is to be used for synchronizing
access to VSI's MAC filter list.

This patch also synchronizes execution of other codepaths which are
accessing VSI's MAC filter list with execution of
service_task:sync_vsi_filters.

In function i40e_add_vsi, copied out LAA MAC address instead of cloning
MAC filter entry because only MAC address is needed to remove MAC VLAN
filter from FW/HW.

Change-ID: I0e10ac7c715d44aa994239642aa4d57c998573a2
Signed-off-by: NKiran Patil <kiran.patil@intel.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

With 64 VFs, we can easily overwhelm the AQ on the PF if we have too low
a limit on the number of AQ requests. This leads to ARQ overflow errors,
and occasionally VFs that fail to initialize.

Since we really only hit this condition on initial VF driver load, the
requests that we process are lightweight, so this extra work doesn't
cause problems for the PF driver.

Change-ID: I620221520d8af987df6ace9ba938ffaf22107681
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The XL710 hardware has a different interrupt moderation design
that can support a limit of total interrupts per second per
vector, in addition to the "number of interrupts per second"
controls already established in the driver.  This combination
of hardware features allows us to set very low default latency
settings but minimize the total CPU utilization by not
making too many interrupts, should the user desire.

The current driver implementation is still enabling the dynamic
moderation in the driver, and only using the rx/tx-usecs
limit in ethtool to limit the interrupt rate per second, by default.

The new code implemented in this patch
2) adds init/use of the new "Interrupt Limit" register
3) adds ethtool knob to control/report the limits above

Usage is ethtool -C ethx rx-usecs-high <value> Where <value> is number
of microseconds to create a rate of 1/N interrupts per second,
regardless of rx-usecs or tx-usecs values. Since there is a credit based
scheme in the hardware, the rx-usecs and tx-usecs can be configured for
very low latency for short bursts, but once the credit runs out the
refill rate on the credits is limited by rx-usecs-high.

Change-ID: I3a1075d3296123b0f4f50623c779b027af5b188d
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add an ethtool priv flag to enable and disable printing
the VEB statistics.

Change-ID: I7654054a3a73b08aa8310d94ee8fce6219107dd8
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Cut down on the number of startup log entries by putting a couple behind
debug flags and combining a couple others into a single line.

Change-ID: I708089f086308f84d43f8b6f0e8a634a02d058fb
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The code in i40e and i40evf is using an "IN_NETPOLL" flag that has never
added any value due to the fact that the Rx clean-up is handled in NAPI.
As such the flag was set, the queue was scheduled via NAPI, and then polled
from the netpoll controller and if any Rx packets were processed the were
processed in the wrong context.

In addition the flag itself just added an unneeded conditional to the
hot-path so it can safely be dropped and save us a few instructions.
Signed-off-by: NAlexander Duyck <aduyck@mirantis.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

X722 will report Gen 1x1 in the PCI config space as it is on
IOSF bus, so skip the PCI bus link/speed check.

Change-ID: Icd5f5751dc7fb00dccf0d5dc5a0a644948e7062e
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Some customers wish to be able to control our hardware specific
feature called flow director, at runtime.  This patch enables
ethtool priv flags to control this driver/hardware specific feature.

ethtool --set-priv-flags ethX flow-director-atr off

NOTE: the ethtool ntuple interface controls the flow-director
      sideband rules.

Change-ID: Iba156350b07fa2ce66f53ded51739f9a3781fe0e
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch moves the internal fw version and fw api version info to be
output in probe.  The nvm version, etrack and oem version info are now
configured for output via ethtool -i.

Change-ID: I05d490093a7137dbefcdef263d014d1e5c9e83d0
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

There's been some need for controlling the periodic link polling for
debugging link issues.  This patch enables switching it off and on
through an ethtool private flag.  The link poll remains on by default,
but can be turned off with
	ethtool --set-priv-flags p261p1 LinkPolling off
and later turned back on with
	ethtool --set-priv-flags p261p1 LinkPolling on

To check the current status, use
	ethtool --show-priv-flags p261p1

Change-ID: I32e4ab654ff3eec90a06cf144899971b82d71c40
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch addresses an issue where multiple link up messages can be logged
resulting from aq link status timing when link properties are changed (fc,
speed, etc.); solved by using a single function to handle status printing
and adding a mechanism to track whether link state (up or down) has
actually changed.

Change-ID: Ied6ed6e49dc397c77d992adc0bc9ed3767152b9d
Signed-off-by: NMatt Jared <matthew.a.jared@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch fixes a problem where the PF's fdir filter table would have an
entry that the hw was unable to add. This notification happens in the hot
path, so instead of trying to fix it then, we note the location in the
failure case and delete it during regular fdir subtask callback. Without
this patch, a case can occur where an invalid entry gets replayed and a
valid one is not.

Change-ID: I67831c183b5d0309876de807cc434809b74c9cb7
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The interrupt enable function can be inlined by moving it to the header
file, which decreases the function call overhead for a frequently called
function.

Change-ID: I3214cc99593725768642680e7b8ce7e9bba7e44d
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Keep track of how many times we ask the stack to linearize the
skb because the HW cannot handle skbs with more than 8 frags per
segment/single packet.

Change-ID: If455452060963a769bbe6112cba952e79e944b52
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch extends the size of the text available for the interrupt names.
Without this patch, all the descriptive data available for the Flow
Director interrupts is truncated.

Change-ID: I2ac458f23ac3b4ea8f1edf73edc283b1d3704c7f
Signed-off-by: NCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

i40e_init_pf_fcoe() didn't return anything except 0, it prints enough
error info already, and no driver logic depends on the return value,
so this can be void.

Change-ID: Ie6afad849857d87a7064c42c3cce14c74c2f29d8
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Due to performance reasons, VEB stats have been disabled in the hw. This
patch adds code to check for that condition before accumulating these
stats.

Change-ID: I7d805669476fedabb073790403703798ae5d878e
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Don't let the debugfs register read and write commands try to access
outside of the ioremapped space.  While we're at it, remove the use of
a misleading constant.

Change-ID: Ifce2893e232c65c7a76c23532c658f298218a81b
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The sync_vsi_filters function can be called directly under RTNL
or through the timer subtask without one. This was causing a deadlock.

If sync_vsi_filters is called from a thread which held the lock,
and in another thread the PROMISC setting got changed we would
be executing the PROMISC change in the thread which already held
the lock alongside the other filter update. The PROMISC change
requires a reset if we are on a VEB, which requires it to be called
under RTNL.

Earlier the driver would call reset for PROMISC change without
checking if we were already under RTNL and would try to grab it
causing a deadlock. This patch changes the flow to see if we are
already under RTNL before trying to grab it.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NKiran Patil <kiran.patil@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch modifies the driver timeout logic by issuing a writeback
request via a software interrupt to the hardware the first time the
driver detects a hang. The driver was too aggressive in resetting a hung
queue, so back that off by removing logic to down the netdevice after
too many hangs, and move the function to the service task.

Change-ID: Ife100b9d124cd08cbdb81ab659008c1b9abbedea
Signed-off-by: NKiran Patil <kiran.patil@intel.com>
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This fixes bugs where the port is not receiving multicast or VLAN tagged
packets when in promiscuous mode. This can occur when a SW bridge is
created on top of the device.

This also fixes issues where the promiscuous behavior setting was not
being preserved across a reset caused by features being enabled or
disabled.

We are using defport instead of doing a true promiscuous mode because we do
not need to receive the SRIOV or VMDq VSI directed traffic which would suck
up bandwidth and is really not intended for the SW bridge.

In addition, with defport we get VLAN promiscuous behavior which is not
possible from the VSI level promiscuous setting.

Change-ID: Ie21985eac32d5af1c02e9d71c6430a90d5bab40f
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch adds capability to update per VEB per TC statistics and dump
it via ethtool. It also adds a structure to hold VEB per TC statistics.
The fields can be filled by reading the GLVEBTC_* counters.

Change-ID: I28b4759b9ab6ad5a61f046a1bc9ef6b16fe31538
Signed-off-by: NNeerav Parikh <neerav.parikh@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

X722 fixes an issue from X710 where TX descriptor WB would not happen if
the interrupts were disabled. In order for the write backs to happen a
bit needs to be set in the dynamic interrupt control register called
WB_ON_ITR. With this feature, the SW driver need not arm SW interrupts to
work around the issue in X710.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

X722 uses the admin queue to configure RSS. This patch adds the necessary
flow changes to configure RSS through AQ. It also adds the separate VMDQ2
lookup tables and hash key programming for X722.

X722 also exposes a different set of PCTYPES for RSS, this patch
accommodates those changes.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add capabilities flags specific to X722.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Use macros for abstracting (1 << foo) to BIT(foo)
and (1ULL << foo64) to BIT_ULL(foo64) in order to match
better with kernel requirements.

NOTE: the adminq_cmd.h file was not modified on purpose because
of the dependency upon firmware for that file.

Change-ID: I73ee2e48c880d671948aad19bd53ca6b2ac558fc
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The patch fixes a bug in the default configuration which
prevented a software bridge loaded on the PF interface from
working correctly because broadcast packets are incorrectly
looped back.

Fix the general case, by loading the driver in VEPA mode Until a
VF or VMDq VSI is added. This way loopback on the Main VSI is
turned off until needed and can resolve the issue of unnecessary
reflection for users that do not have VF or VMDq VSIs setup.

The driver must now coordinate the loopback setting for the Flow
Director (FDIR) VSI to make sure it is in sync with the current
VEB or VEPA mode setting.

The user can still switch bridge modes from the bridge commands and
choose to be in VEPA mode with VF VSIs. Because of hardware
requirements, the call to switch to VEB mode when no VF/VMDqs are
present will be rejected.

NOTE: This patch uses BIT_ULL as that is preferred going forward,
a followup patch in the lower priority queue to net-next will fix
up the remaining 1 << usages.

Change-ID: Ib121ddb18fe4b3c4f52e9deda6fcbeb9105683d1
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

We can use the stat index macro directly, a variable is not required.

Change-ID: I19f08ac16353dc0cd87a1a8248d714e15a54aa8a
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add a 3rd dynamic filter counter to track Tunneled ATR hits separately.
Ethtool port stat "fdir_atr_tunnel_match"

Change-ID: Idd978b6db2a462b5722397cd2ffd04ef055f8655
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>