If we reset a VF, its VSI goes away, and it gets a new one. So don't
hang on to the now-stale local VSI pointer. It just leads to suffering
and kernel panics.

Change-ID: Ia8823b4e85893e95e963acee284968022b29177a
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Use the new AdminQ functions for safely accessing the Rx control
registers that may be affected by heavy small packet traffic.

We can't use AdminQ calls in i40e_clear_hw() because the HW is being
initialized and the AdminQ is not alive.  We recently added an AQ
related replacement for reading PFLAN_QALLOC, and this patch puts
back the original register read.

Change-ID: Ib027168c954a5733299aa3a4ce5f8218c6bb5636
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Explain why we cannot remove this code, even though it works differently
than any of our other interrupt cause handling code.

Change-ID: Ie66203bd037a466066036611c31d44f759ec5176
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

We have a define for this, use it. No functional change.

Change-ID: Ic0e3ea4f562e46de63b2a8de07f291ccc10205fd
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

While re-enabling interrupts the driver would clear all pending
causes. This meant that if an interrupt was generated while the driver
was cleaning or polling with interrupts disabled, then that interrupt
was lost.  This could cause a queue to become dead, especially for
receive.  Refactored the enable_icr0 function in order to allow
it to be decided by the caller whether the CLEARPBA (clear pending
events) bit will be set while re-enabling the interrupt.

Also update copyright year in file headers.

Change-ID: Ic1db100a05e13c98919057696db147a258ca365a
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

When calculating TX bandwidth for VFs, we need to know the link speed to
make sure we don't allocate more bandwidth than is available. Add 20G
link speed to the switch statement so we can support devices that link
at that speed.

Change-ID: I5409f6139d549e5832777db9c22ca0664e0c5f8b
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add write-back on interrupt throttle rate timer expiration support
for the i40evf driver, when running on X722 devices.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The PCTYPES for the X710 and X722 families are different. This patch
makes adjustments for that.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Allow the user to specify a zero MAC address for VFs. This removes the
existing MAC address and allows the VF to use a random address. Libvirt
does this normally when removing a VF from a VM.
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

When VFs are being reset, there is a brief window of time when they
cannot be configured because they don't have a VSI to configure. If
a script is quick, it can fall through that window. To avoid
defenestration, log a useful error message and return -EAGAIN.
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

When VFs are created, the MAC address defaults to all zeros, indicating
to the VF driver that it should use a random MAC address. However, the
PF driver was incorrectly adding this zero MAC to the filter table,
along with the VF's randomly generated MAC address.

Check for a good address before adding the default filter. While we're
at it, make the error message a bit more useful.

Change-ID: Ia100947d68140e0f73a19ba755cbffc3e79a8fcf
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The virtual channel interface was using incorrect semantics to remove
MAC addresses, which would leave incorrect filters active when using
VLANs. To correct this, add a new function that unconditionally removes
MAC addresses from all VLANs, and call this function when the VF
requests a MAC filter removal.

Change-ID: I69826908ae4f6c847f5bf9b32f11faa760189c74
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Operators should have spaces around them.

Change-ID: I64735e9aa8618b9a5059a87ace1c999d6d3bfcfb
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

i40e_sync_vsi_filters() is the surly teenager of this driver. It says
it's going to report errors, but it doesn't actually do that most of the
time. And when it does, it leaves a mess.

Change this function to have a common exit point so it will properly
release the busy lock on the VSI. Propagate errors to the callers.
Finally, adjust a few callers to check for and deal with errors from
this function.

Change-ID: Ic6af4956491e72402ebb3c538a3c31a0ad7f8667
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

With the final edition of the patches to remove sleeps from
the driver's entry points, the grab_rtnl argument is no
longer needed, so partially revert the commit that added it.

Change-ID: Ib9778476242586cc9e58b670f5f48d415cb59003
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If we get an invalid message from a VF, we should tell the user which VF
is being naughty, rather than making them guess.

Change-ID: I9252cef7baea3d8584043ed6ff12619a94e2f99c
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Clean up and enhance error messages related to VF MAC/VLAN filters.
Indicate which VF is having issues, and if possible indicate the MAC
address or VLAN involved.

Also, when an error is returned from the PF driver, print useful
information about what went wrong, for the most likely cases.

Change-ID: Ib3d15eef9e3369a78fd142948671e5fa26d921b8
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If a call to enable SR-IOV in the kernel failed, we need to disable
I40E_FLAG_VEB_MODE_ENABLED, so that bridge mode could fall back to
VEPA, which is a default.

Change-ID: I12b6f776769506db85b29bea94b9c88d0b5ee65e
Signed-off-by: NAkeem G Abodunrin <akeem.g.abodunrin@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This patch introduces a spinlock which is to be used for synchronizing
access to VSI's MAC filter list.

This patch also synchronizes execution of other codepaths which are
accessing VSI's MAC filter list with execution of
service_task:sync_vsi_filters.

In function i40e_add_vsi, copied out LAA MAC address instead of cloning
MAC filter entry because only MAC address is needed to remove MAC VLAN
filter from FW/HW.

Change-ID: I0e10ac7c715d44aa994239642aa4d57c998573a2
Signed-off-by: NKiran Patil <kiran.patil@intel.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

When the number of invalid messages from a VF is exceeded, the VF
will be disabled, due to the invalid messages.  This happens if
other VF drivers (like DPDK) send a message through the driver's
mailbox (aka virtchannel) interface, but the message is not
supported by the i40e pf driver, such as CONFIG_PROMISCUOUS_MODE.

This patch changes the num_invalid_msgs in struct i40e_vf to record
the continuous invalid msgs, and it will be reset when a valid msg
is received.

Change-ID: Iaec42fd3dcdd281476b3518be23261dd46fc3718
Signed-off-by: NJingjing Wu <jingjing.wu@intel.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

X722 has a way to work around the descriptor WB issue,
this offload helps turn that feature on.

Change-ID: I7ffa67622426bfca5a651417b63e3afcfeb60412
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

In some circumstances, the firmware may fail to allocate a VSI for a VF.
When this happens, the driver does not react well to the bad news and
has a panic attack.

To fix this problem, check the return value from i40e_alloc_vf_res and
don't try to configure the device further if it failed. Additionally,
explicitly clear the INIT bit when we free VF resources, so that this
bit will be in the proper state in the failure case, and won't blow up
elsewhere.

Change-ID: I6a20ce2b59c3458fd832032e88fa28cd42500189
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This function call isn't needed here; the same function is already
called by i40e_reset_vf.

Change-ID: I96ccbf91b752965c9e28fe895d4c7d4c46e3ba44
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The VF really doesn't care about the QOS handle but it will in the
future. Since the VF only uses TC0, send it that handle. On the VF
side, save the handle and use it to populate the QOS params when we call
into the client interface.

Change-ID: I76f41b070baeaa09b19383e9168bc677837e0761
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add missings spaces after declarations, remove another __func__ use,
remove uncessary braces, remove unneeded breaks, and useless returns,
and generally fix up some code.

Change-ID: Ie715d6b64976c50e1c21531685fe0a2bd38c4244
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NShannon Nelson <shannon.nelson@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Under certain circumstances, the device may not have enough resources to
enable all of the VFs that it advertises in config space. Although the
number of supported VFs is reported upon driver init, it is not obvious
when this is different from the number reported in config space. To
eliminate this confusion, add an error message explaining the problem.
Additionally, move the 'Allocating VFs' message down below the error
checks so as to prevent further confusion.

Change-ID: I45b7efca53a7aebf7777be33a8bc9d615ae48ea1
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Explicitly stop the rings belonging to each VF when disabling SR-IOV.
Even though the VFs were gone, and the associated VSIs were removed, the
rings were not stopped, and in some circumstances the hardware would
continue to access the memory formerly used by the rings, causing memory
corruption or DMAR errors, both of which would lead to general malaise
of the kernel.

To relieve this condition, explicitly stop all the rings associated with
each VF before releasing its resources.

Change-ID: I78c05d562c66e7b594b7e48d67860f49b3e5b6ec
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

In i40e_ndo_set_vf_port_vlan, we were using the QOS value
inconsistently, sometimes shifting it, sometimes not. Do the shift-and-
or operation correctly, once, and use the result consistently everywhere
in the function.

Change-ID: I46f062f3edc90a8a017ecec9137f4d1ab0ab9e41
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The sync_vsi_filters function can be called directly under RTNL
or through the timer subtask without one. This was causing a deadlock.

If sync_vsi_filters is called from a thread which held the lock,
and in another thread the PROMISC setting got changed we would
be executing the PROMISC change in the thread which already held
the lock alongside the other filter update. The PROMISC change
requires a reset if we are on a VEB, which requires it to be called
under RTNL.

Earlier the driver would call reset for PROMISC change without
checking if we were already under RTNL and would try to grab it
causing a deadlock. This patch changes the flow to see if we are
already under RTNL before trying to grab it.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NKiran Patil <kiran.patil@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Add virtual channel offload capability to support RX polling mode in the
VF.

Change-ID: Ib643ae2a7506dfc75fc489fc207493fabefa4832
Signed-off-by: NJingjing Wu <jingjing.wu@intel.com>
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The kernel notifies all VXLAN capable registered drivers, i.e. any
driver that implements ndo_add_vxlan_port(), of the addition of a
port so that the driver can track which ports are in use.  There's
no need to log this - it just fills the system log with useless and
irksome noise.

Also, when failing to init SR-IOV interfaces the driver was printing the
same message twice. Just remove the inner printk and let the outer message
catch enable as well as the other failures.

Change-ID: Id5ecb1d425c2a357ee2bc1635dab24553831dade
Signed-off-by: NGreg Rose <gregory.v.rose@intel.com>
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Sparse cries when we compare an __le16 to a u16, almost like it cares
about architectures other than x86. Weird. Use the le16_to_cpu macro to
make it stop crying.

Change-ID: Id068f4d7868a2d3df234a791a76d15938f37db35
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

X722 uses the admin queue to configure RSS. This patch adds the necessary
flow changes to configure RSS through AQ. It also adds the separate VMDQ2
lookup tables and hash key programming for X722.

X722 also exposes a different set of PCTYPES for RSS, this patch
accommodates those changes.
Signed-off-by: NAnjali Singhai Jain <anjali.singhai@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Use macros for abstracting (1 << foo) to BIT(foo)
and (1ULL << foo64) to BIT_ULL(foo64) in order to match
better with kernel requirements.

NOTE: the adminq_cmd.h file was not modified on purpose because
of the dependency upon firmware for that file.

Change-ID: I73ee2e48c880d671948aad19bd53ca6b2ac558fc
Signed-off-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NCatherine Sullivan <catherine.sullivan@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

This driver fully supports VF drivers using both the 1.0 and 1.1
versions of the virtual channel API. However, VF drivers using
version 1.0 get upset if we provide them with a version other than
that, and refuse to play with us.

Correct this by checking the VFs API version at the time that we
store it off, and provide the correct version number back to the VF
so we can all get along.

Change-ID: I86dfe02e67b2bef336b4b49a1bb072f3e7229abc
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Store off the VF API version for use when figuring out the VF driver
capabilities. Add support for the VF driver handing its capabilities to
the PF driver and then use this information when sending VF resource
information back to the VF driver.

Change-ID: Ic00d0eeeb5b8118085e12f068ef857089a8f7c2d
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

When a VF is disabled, there is no way for it to recover until either
the PF driver is reloaded or SR-IOV is disabled and enabled. To correct
this, enable the VF after a successful reset.

Change-ID: I9e0788476c4d53d5407961b503febdfff2b8a7c6
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

The VF disable code was just whanging on the reset bit without properly
cleaning up the VF, which would leave the VF in an indeterminate state
from which it could not recover. Fix this by notifying the VF and then
by calling the normal VF reset routine.

Change-ID: I862b9dfa919368773cbdc212b805b520db2f7430
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

MAC filters for VFs were being programmed with 0 for the VLAN value when
there was no VLAN assigned. This is incorrect and actually assigns the
VF to VLAN 0. Instead, we must use -1 to indicate that no VLAN is in
use. This change programs the filters correctly and gets rid of a bogus
error message when setting a port VLAN on an active VF.

Change-ID: Ica9a9906d768405377ff3308e27f7d0b5b2ea96e
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

If user attempts to set a port VLAN on a VF that already has the same
port VLAN configured, the driver will go through a completely
unnecessary flurry of filter removals and filter adds. Just check for
this condition and return success instead of doing a bunch of busywork.

Change-ID: Ia1a9e83e6ed48b3f4658bc20dfc6af0cf525d54a
Signed-off-by: NMitch Williams <mitch.a.williams@intel.com>
Tested-by: NJim Young <james.m.young@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>