Implement and export UDP segmentation offload (USO) support for both
NIC and MQPRIO QoS offload Tx path. Update appropriate logic in Tx to
parse GSO info in skb and configure FW_ETH_TX_EO_WR request needed to
perform USO.

v2:
- Remove inline keyword from write_eo_udp_wr() in sge.c. Let the
  compiler decide.
Signed-off-by: NRahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>

Add TC-MATCHALL classifier ingress offload support. The same actions
supported by existing TC-FLOWER offload can be applied to all incoming
traffic on the underlying interface.

Ensure the rule priority doesn't conflict with existing rules in the
TCAM. Only 1 ingress matchall rule can be active at a time on the
underlying interface.

v5:
- No change.

v4:
- Added check to ensure the matchall rule's prio doesn't conflict with
  other rules in TCAM.
- Added logic to fill default mask for VIID, if none has been
  provided, to prevent conflict with duplicate VIID rules.
- Used existing variables in private structure to fill VIID info,
  instead of extracting the info manually.

v3:
- No change.

v2:
- Removed logic to fetch free index from end of TCAM. Must maintain
  same ordering as in kernel.
Signed-off-by: NRahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add TC-MATCHALL classifier offload with TC-POLICE action applied for
all outgoing traffic on the underlying interface. Split flow block
offload to support both egress and ingress classification.

For example, to rate limit all outgoing traffic to 1 Gbps:

$ tc qdisc add dev enp2s0f4 clsact
$ tc filter add dev enp2s0f4 egress matchall skip_sw \
	action police rate 1Gbit burst 8Kbit

Note that skip_sw is important. Otherwise, both stack and hardware
will end up doing policing. Policing can't be shared across flow
blocks. Only 1 egress matchall rule can be active at a time on the
underlying interface.

v5:
- No change.

v4:
- Removed check to reject police offload if prio is not 1.
- Moved TC_SETUP_BLOCK code to separate function.

v3:
- Added check to reject police offload if prio is not 1.
- Assign block_shared variable only for TC_SETUP_BLOCK.

v2:
- Added check to reject flow block sharing for policers.
Signed-off-by: NRahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add support for configuring and managing ETHOFLD hardware queues.
Keep the queue count and MSI-X allocation scheme same as NIC queues.
ETHOFLD hardware queues are dynamically allocated/destroyed as
TC-MQPRIO Qdisc offload is enabled/disabled on the corresponding
interface, respectively.
Signed-off-by: NRahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add logic for validation and configuration of TC-MQPRIO Qdisc
offload. Also, add support to manage EOSW_TXQ, which have 1-to-1
mapping with EOTIDs, and expose them to network stack.

Move common skb validation in Tx path to a separate function and
add minimal Tx path for ETHOFLD. Update Tx queue selection to return
normal NIC Txq to send traffic pattern that can't go through ETHOFLD
Tx path.
Signed-off-by: NRahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Simplify queue configuration and MSI-X allocation logic. Use a single
MSI-X information table for both NIC and ULDs. Remove hard-coded
MSI-X indices for firmware event queue and non data interrupts.
Instead, use the MSI-X bitmap to obtain a free MSI-X index
dynamically. Save each Rxq's index into the MSI-X information table,
within the Rxq structures themselves, for easier cleanup.
Signed-off-by: NRahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

QoS offload needs Ethernet Offload (ETHOFLD) resources present in the
NIC. These resources are shared with other ULDs. So, query firmware
for the available number of traffic classes, as well as, start and
end indices (EOTID) of the ETHOFLD region.
Signed-off-by: NRahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch implements reset_prepare and reset_done, which are used
for handling FLR.
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The "chip" variable is an enum, and it's treated as unsigned int by GCC
in this context so the error handling isn't triggered.

Fixes: e8d45292 ("cxgb4: clean up init_one")
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When calling debugfs functions, there is no need to ever check the
return value.  The function can work or not, but the code logic should
never do something different based on this.

If a debugfs call fails, it will properly warn in the syslog, there's no
need for all individual drivers to also print a message, so that is one
more reason to not care about checking the return values.

Cc: Vishal Kulkarni <vishal@chelsio.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Casey Leedom <leedom@chelsio.com>
Cc: netdev@vger.kernel.org
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Commit 8c0d3a02 ("PCI: Add accessors for PCI Express Capability")
added accessors for the PCI Express Capability so that drivers didn't
need to be aware of differences between v1 and v2 of the PCI
Express Capability.

Replace pci_read_config_word() and pci_write_config_word() calls with
pcie_capability_read_word() and pcie_capability_write_word().
Signed-off-by: NFrederick Lawler <fred@fredlawl.com>
Reviewed-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

And any other existing fields in this structure that refer to tc.
Specifically:

* tc_cls_flower_offload_flow_rule() to flow_cls_offload_flow_rule().
* TC_CLSFLOWER_* to FLOW_CLS_*.
* tc_cls_common_offload to tc_cls_common_offload.
Signed-off-by: NPablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch updates flow_block_cb_setup_simple() to use the flow block API.
Several drivers are also adjusted to use it.

This patch introduces the per-driver list of flow blocks to account for
blocks that are already in use.

Remove tc_block_offload alias.
Signed-off-by: NPablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Most drivers do the same thing to set up the flow block callbacks, this
patch adds a helper function to do this.

This preparation patch reduces the number of changes to adapt the
existing drivers to use the flow block callback API.

This new helper function takes a flow block list per-driver, which is
set to NULL until this driver list is used.

This patch also introduces the flow_block_command and
flow_block_binder_type enumerations, which are renamed to use
FLOW_BLOCK_* in follow up patches.

There are three definitions (aliases) in order to reduce the number of
updates in this patch, which go away once drivers are fully adapted to
use this flow block API.
Signed-off-by: NPablo Neira Ayuso <pablo@netfilter.org>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds reference counting support for
alloc/free mac filters
Signed-off-by: NRaju Rangoju <rajur@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds TCAM reference counting
support for cxgb4 change mac path
Signed-off-by: NRaju Rangoju <rajur@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Remove existing mps refcounting code which was
added only for encap filters and add necessary
data structures/functions to support mps reference
counting for all the mac filters. Also add wrapper
functions for allocating and freeing encap mac
filters.
Signed-off-by: NRaju Rangoju <rajur@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Page pods are used for direct data placement, this patch
enables eDRAM page pods if firmware supports this feature.
Signed-off-by: NVarun Prakash <varun@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Spread initial IRQ affinity hints across the device node CPUs,
for nic queue and uld queue IRQs, to load balance and avoid
all interrupts on CPU0.
Signed-off-by: NNirranjan Kirubaharan <nirranjan@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Hash (exact-match) filters used for offloading flows share the
same active region resources on the chip with upper layer drivers,
like iw_cxgb4, chcr, etc. Currently, only either Hash filters
or ULDs can use the active region resources, but not both. Hence,
use the new firmware configuration parameters (when available)
to allow both the Hash filters and ULDs to share the
active region simultaneously.
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

BUG: unable to handle kernel paging request at ffffffffa016a270
PGD 3270067 P4D 3270067 PUD 3271063 PMD 230bbd067 PTE 0
Oops: 0000 [#1
CPU: 0 PID: 6134 Comm: modprobe Not tainted 5.1.0+ #33
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org 04/01/2014
RIP: 0010:atomic_notifier_chain_register+0x24/0x60
Code: 1f 80 00 00 00 00 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 ae b4 38 01 48 8b 53 38 48 8d 4b 38 48 85 d2 74 20 45 8b 44 24 10 <44> 3b 42 10 7e 08 eb 13 44 39 42 10 7c 0d 48 8d 4a 08 48 8b 52 08
RSP: 0018:ffffc90000e2bc60 EFLAGS: 00010086
RAX: 0000000000000292 RBX: ffffffff83467240 RCX: ffffffff83467278
RDX: ffffffffa016a260 RSI: ffffffff83752140 RDI: ffffffff83467240
RBP: ffffc90000e2bc70 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000000000000 R11: 00000000014fa61f R12: ffffffffa01c8260
R13: ffff888231091e00 R14: 0000000000000000 R15: ffffc90000e2be78
FS:  00007fbd8d7cd540(0000) GS:ffff888237a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffffffa016a270 CR3: 000000022c7e3000 CR4: 00000000000006f0
Call Trace:
 register_inet6addr_notifier+0x13/0x20
 cxgb4_init_module+0x6c/0x1000 [cxgb4
 ? 0xffffffffa01d7000
 do_one_initcall+0x6c/0x3cc
 ? do_init_module+0x22/0x1f1
 ? rcu_read_lock_sched_held+0x97/0xb0
 ? kmem_cache_alloc_trace+0x325/0x3b0
 do_init_module+0x5b/0x1f1
 load_module+0x1db1/0x2690
 ? m_show+0x1d0/0x1d0
 __do_sys_finit_module+0xc5/0xd0
 __x64_sys_finit_module+0x15/0x20
 do_syscall_64+0x6b/0x1d0
 entry_SYSCALL_64_after_hwframe+0x49/0xbe

If pci_register_driver fails, register inet6addr_notifier is
pointless. This patch fix the error path in cxgb4_init_module.

Fixes: b5a02f50 ("cxgb4 : Update ipv6 address handling api")
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

During driver unload, hash/TCAM filter deletion doesn't wait for
completion.This patch deletes all the filters with completion before
clearing the resources.
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

After the previous patch, all the callers of ndo_select_queue()
provide as a 'fallback' argument netdev_pick_tx.
The only exceptions are nested calls to ndo_select_queue(),
which pass down the 'fallback' available in the current scope
- still netdev_pick_tx.

We can drop such argument and replace fallback() invocation with
netdev_pick_tx(). This avoids an indirect call per xmit packet
in some scenarios (TCP syn, UDP unconnected, XDP generic, pktgen)
with device drivers implementing such ndo. It also clean the code
a bit.

Tested with ixgbe and CONFIG_FCOE=m

With pktgen using queue xmit:
threads		vanilla 	patched
		(kpps)		(kpps)
1		2334		2428
2		4166		4278
4		7895		8100

 v1 -> v2:
 - rebased after helper's name change
Signed-off-by: NPaolo Abeni <pabeni@redhat.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Some of these macros were conflicting with global namespace,
hence prefixing them with CXGB4.
Signed-off-by: NArjun Vynipadath <arjun@chelsio.com>
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use ndo_set_vf_link_state to control the link states associated
with the virtual interfaces.
Signed-off-by: NArjun Vynipadath <arjun@chelsio.com>
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

T6 adapters support outer UDP checksum offload for
encapsulated packets, hence enabling netdev feature flag
NETIF_F_GSO_UDP_TUNNEL_CSUM.
Signed-off-by: NArjun Vynipadath <arjun@chelsio.com>
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

GRO is done by cxgb4/cxgb4vf. Hence set NETIF_F_GRO flag for
both cxgb4/cxgb4vf.
Cleaned up VLAN netdev features in cxgb4vf. Also fixed
NETIF_F_HIGHDMA being set unconditionally for vlan netdev
features.
Signed-off-by: NArjun Vynipadath <arjun@chelsio.com>
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch gets/sets SGE Doorbell Queue timer ticks via ethtool

Original work by: Casey Leedom <leedom@chelsio.com>
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

T6 introduced a Timer Mechanism in SGE called the
SGE Doorbell Queue Timer. With this we can now configure
TX Queues to get CIDX Updates when:

    Time(CIDX == PIDX) >= Timer

Previously we rely on TX Queue Status Page updates by hardware
for DMA completions. This will make Hardware/Firmware actually
deliver the CIDX Updates as Ingress Queue messages with
commensurate Interrupts.

So we now have a new RX Path component for processing CIDX Updates
and reclaiming TX Descriptors faster.

Original work by: Casey Leedom <leedom@chelsio.com>
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

{t4/t4_vf}_change_mac() API's were only doing additions to MPS_TCAM.
This will fail, when the number of tcam entries is limited particularly
in vf's.
This fix programs hash region with the mac address, when TCAM
addtion fails for {t4/t4vf}_change_mac(). Since the locally maintained
driver list for hash entries is shared across mac_{sync/unsync}(),
added an extra parameter if_mac to track the address added thorugh
{t4/t4vf}_change_mac()
Signed-off-by: NArjun Vynipadath <arjun@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Enable Inline TLS record by default
Signed-off-by: NAtul Gupta <atul.gupta@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Total number of VFs supported by PF is used to determine the last
byte of VF's mac address. Number of VFs supported is not always
16, use the variable nvfs to get the number of VFs supported
rather than hard coding it to 16.
Signed-off-by: NCasey Leedom <leedom@chelsio.com>
Signed-off-by: NGanesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

If the fw supports returning VIN/VIVLD in FW_VI_CMD save it
in port_info structure else retrieve these from viid and save
them  in port_info structure. Do the same for smt_idx from
FW_VI_MAC_CMD
Signed-off-by: NSantosh Rastapur <santosh@chelsio.com>
Signed-off-by: NGanesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Null pointer dereference seen when cxgb4vf driver is unloaded
without bringing up any interfaces, moving mac_hlist initialization
to driver probe and free the mac_hlist in remove to fix the issue.

Fixes: 24357e06 ("cxgb4vf: fix memleak in mac_hlist initialization")
Signed-off-by: NArjun Vynipadath <arjun@chelsio.com>
Signed-off-by: NCasey Leedom <leedom@chelsio.com>
Signed-off-by: NGanesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

with CONFIG_THERMAL=m and cxgb4 as built-in build fails, and
'commit e70a57fa ("cxgb4: fix thermal configuration dependencies")'
tries to fix it but when cxgb4i is made built-in build fails again,
use IS_REACHABLE instead of IS_ENABLED to fix the issue.

Fixes: e70a57fa (cxgb4: fix thermal configuration dependencies)
Reported-by: NRandy Dunlap <rdunlap@infradead.org>
Signed-off-by: NGanesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The locally maintained list for tracking hash mac table was
not freed during driver remove.
Signed-off-by: NArjun Vynipadath <arjun@chelsio.com>
Signed-off-by: NGanesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

With CONFIG_THERMAL=m, we get a build error:

drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.c: In function 'cxgb4_thermal_get_trip_type':
drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.c:48:11: error: 'struct adapter' has no member named 'ch_thermal'

Once that is fixed by using IS_ENABLED() checks, we get a link error
against the thermal subsystem when cxgb4 is built-in:

drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.o: In function `cxgb4_thermal_init':
cxgb4_thermal.c:(.text+0x180): undefined reference to `thermal_zone_device_register'
drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.o: In function `cxgb4_thermal_remove':
cxgb4_thermal.c:(.text+0x1e0): undefined reference to `thermal_zone_device_unregister'

Finally, since CONFIG_THERMAL can be =m, the Makefile fails to pick up the
extra file into built-in.a, and we get another link failure against the
cxgb4_thermal_init/cxgb4_thermal_remove files, so the Makefile has to
be adapted as well to work for both CONFIG_THERMAL=y and =m.

Fixes: b1871915 ("cxgb4: Add thermal zone support")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add thermal zone support to monitor ASIC's temperature.
Signed-off-by: NGanesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

After bfcb79fc ("PCI/ERR: Run error recovery callbacks for all affected
devices"), AER errors are always cleared by the PCI core and drivers don't
need to do it themselves.

Remove calls to pci_cleanup_aer_uncorrect_error_status() from device
driver error recovery functions.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
[bhelgaas: changelog, remove PCI core changes, remove unused variables]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Remove duplicated include.
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>