Calling ethtool changing the RX ring parameters like this:

  $ ethtool -G eth0 rx 1024

on igc triggers kernel warnings like this:

[  225.198467] ------------[ cut here ]------------
[  225.198473] Missing unregister, handled but fix driver
[  225.198485] WARNING: CPU: 7 PID: 959 at net/core/xdp.c:168
xdp_rxq_info_reg+0x79/0xd0
[...]
[  225.198601] Call Trace:
[  225.198604]  <TASK>
[  225.198609]  igc_setup_rx_resources+0x3f/0xe0 [igc]
[  225.198617]  igc_ethtool_set_ringparam+0x30e/0x450 [igc]
[  225.198626]  ethnl_set_rings+0x18a/0x250
[  225.198631]  genl_family_rcv_msg_doit+0xca/0x110
[  225.198637]  genl_rcv_msg+0xce/0x1c0
[  225.198640]  ? rings_prepare_data+0x60/0x60
[  225.198644]  ? genl_get_cmd+0xd0/0xd0
[  225.198647]  netlink_rcv_skb+0x4e/0xf0
[  225.198652]  genl_rcv+0x24/0x40
[  225.198655]  netlink_unicast+0x20e/0x330
[  225.198659]  netlink_sendmsg+0x23f/0x480
[  225.198663]  sock_sendmsg+0x5b/0x60
[  225.198667]  __sys_sendto+0xf0/0x160
[  225.198671]  ? handle_mm_fault+0xb2/0x280
[  225.198676]  ? do_user_addr_fault+0x1eb/0x690
[  225.198680]  __x64_sys_sendto+0x20/0x30
[  225.198683]  do_syscall_64+0x38/0x90
[  225.198687]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  225.198693] RIP: 0033:0x7f7ae38ac3aa

igc_ethtool_set_ringparam() copies the igc_ring structure but neglects to
reset the xdp_rxq_info member before calling igc_setup_rx_resources().
This in turn calls xdp_rxq_info_reg() with an already registered xdp_rxq_info.

Make sure to unregister the xdp_rxq_info structure first in
igc_setup_rx_resources.

Fixes: 73f1071c ("igc: Add support for XDP_TX action")
Reported-by: NLennert Buytenhek <buytenh@arista.com>
Signed-off-by: NCorinna Vinschen <vinschen@redhat.com>
Acked-by: NVinicius Costa Gomes <vinicius.gomes@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

{__,}napi_alloc_skb() allocates and reserves additional NET_SKB_PAD
+ NET_IP_ALIGN for any skb.
OTOH, igc_construct_skb_zc() currently allocates and reserves
additional `xdp->data_meta - xdp->data_hard_start`, which is about
XDP_PACKET_HEADROOM for XSK frames.
There's no need for that at all as the frame is post-XDP and will
go only to the networking stack core.
Pass the size of the actual data only (+ meta) to
__napi_alloc_skb() and don't reserve anything. This will give
enough headroom for stack processing.
Also, net_prefetch() xdp->data_meta and align the copy size to
speed-up memcpy() a little and better match igc_construct_skb().

Fixes: fc9df2a0 ("igc: Enable RX via AF_XDP zero-copy")
Signed-off-by: NAlexander Lobakin <alexandr.lobakin@intel.com>
Reviewed-by: NMichal Swiatkowski <michal.swiatkowski@linux.intel.com>
Tested-by: NNechama Kraus <nechamax.kraus@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

As stated in [1], dma_set_mask() with a 64-bit mask never fails if
dev->dma_mask is non-NULL.
So, if it fails, the 32 bits case will also fail for the same reason.

So, if dma_set_mask_and_coherent() succeeds, 'pci_using_dac' is known to be
1.

Simplify code and remove some dead code accordingly.

[1]: https://lkml.org/lkml/2021/6/7/398Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Time synchronization was not properly enabled on non-MSI-X platforms.

Fixes: 2c344ae2 ("igc: Add support for TX timestamping")
Signed-off-by: NJames McLaughlin <james.mclaughlin@qsc.com>
Reviewed-by: NVinicius Costa Gomes <vinicius.gomes@intel.com>
Tested-by: NNechama Kraus <nechamax.kraus@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

napi_build_skb() reuses per-cpu NAPI skbuff_head cache in order
to save some cycles on freeing/allocating skbuff_heads on every
new Rx or completed Tx.
igc driver runs Tx completion polling cycle right before the Rx
one and uses napi_consume_skb() to feed the cache with skbuff_heads
of completed entries, so it's never empty and always warm at that
moment. Switch to the napi_build_skb() to relax mm pressure on
heavy Rx.
Signed-off-by: NAlexander Lobakin <alexandr.lobakin@intel.com>
Reviewed-by: NMichal Swiatkowski <michal.swiatkowski@linux.intel.com>
Tested-by: NNechama Kraus <nechamax.kraus@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

In non trivial scenarios, the action id alone is not sufficient to
identify the program causing the warning. Before the previous patch,
the generated stack-trace pointed out at least the involved device
driver.

Let's additionally include the program name and id, and the relevant
device name.

If the user needs additional infos, he can fetch them via a kernel
probe, leveraging the arguments added here.
Signed-off-by: NPaolo Abeni <pabeni@redhat.com>
Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
Acked-by: NToke Høiland-Jørgensen <toke@redhat.com>
Link: https://lore.kernel.org/bpf/ddb96bb975cbfddb1546cf5da60e77d5100b533c.1638189075.git.pabeni@redhat.com

Enabling the XDP bpf_prog access to data_meta area is a very small
change. Hint passing 'true' to xdp_prepare_buff().

The SKB layers can also access data_meta area, which required more
driver changes to support. Reviewers, notice the igc driver have two
different functions that can create SKBs, depending on driver config.

Hint for testers, ethtool priv-flags legacy-rx enables
the function igc_construct_skb()

 ethtool --set-priv-flags DEV legacy-rx on
Signed-off-by: NJesper Dangaard Brouer <brouer@redhat.com>
Tested-by: NNechama Kraus <nechamax.kraus@linux.intel.com>
Reviewed-by: NAlexander Lobakin <alexandr.lobakin@intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Driver already implicitly supports XDP metadata access in AF_XDP
zero-copy mode, as xsk_buff_pool's xp_alloc() naturally set xdp_buff
data_meta equal data.

This works fine for XDP and AF_XDP, but if a BPF-prog adjust via
bpf_xdp_adjust_meta() and choose to call XDP_PASS, then igc function
igc_construct_skb_zc() will construct an invalid SKB packet. The
function correctly include the xdp->data_meta area in the memcpy, but
forgot to pull header to take metasize into account.

Fixes: fc9df2a0 ("igc: Enable RX via AF_XDP zero-copy")
Signed-off-by: NJesper Dangaard Brouer <brouer@redhat.com>
Tested-by: NNechama Kraus <nechamax.kraus@linux.intel.com>
Acked-by: NMaciej Fijalkowski <maciej.fijalkowski@intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Add new device ID for the next step of the silicon and
reflect the I226_LMVP part.
Signed-off-by: NSasha Neftin <sasha.neftin@intel.com>
Tested-by: NNechama Kraus <nechamax.kraus@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Convert all Ethernet drivers from memcpy(... dev->addr_len)
to eth_hw_addr_set():

  @@
  expression dev, np;
  @@
  - memcpy(dev->dev_addr, np, dev->addr_len)
  + eth_hw_addr_set(dev, np)

In theory addr_len may not be ETH_ALEN, but we don't expect
non-Ethernet devices to live under this directory, and only
the following cases of setting addr_len exist:
 - cxgb4 for mgmt device,
and the drivers which set it to ETH_ALEN: s2io, mlx4, vxge.
Signed-off-by: NJakub Kicinski <kuba@kernel.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Checking tunnel offloading, it turns out that offloading doesn't work
as expected.  The following script allows to reproduce the issue.
Call it as `testscript DEVICE LOCALIP REMOTEIP NETMASK'

=== SNIP ===
if [ $# -ne 4 ]
then
  echo "Usage $0 DEVICE LOCALIP REMOTEIP NETMASK"
  exit 1
fi
DEVICE="$1"
LOCAL_ADDRESS="$2"
REMOTE_ADDRESS="$3"
NWMASK="$4"
echo "Driver: $(ethtool -i ${DEVICE} | awk '/^driver:/{print $2}') "
ethtool -k "${DEVICE}" | grep tx-udp
echo
echo "Set up NIC and tunnel..."
ip addr add "${LOCAL_ADDRESS}/${NWMASK}" dev "${DEVICE}"
ip link set "${DEVICE}" up
sleep 2
ip link add vxlan1 type vxlan id 42 \
		   remote "${REMOTE_ADDRESS}" \
		   local "${LOCAL_ADDRESS}" \
		   dstport 0 \
		   dev "${DEVICE}"
ip addr add fc00::1/64 dev vxlan1
ip link set vxlan1 up
sleep 2
rm -f vxlan.pcap
echo "Running tcpdump and iperf3..."
( nohup tcpdump -i any -w vxlan.pcap >/dev/null 2>&1 ) &
sleep 2
iperf3 -c fc00::2 >/dev/null
pkill tcpdump
echo
echo -n "Max. Paket Size: "
tcpdump -r vxlan.pcap -nnle 2>/dev/null \
| grep "${LOCAL_ADDRESS}.*> ${REMOTE_ADDRESS}.*OTV" \
| awk '{print $8}' | awk -F ':' '{print $1}' \
| sort -n | tail -1
echo
ip link del vxlan1
ip addr del ${LOCAL_ADDRESS}/${NWMASK} dev "${DEVICE}"
=== SNAP ===

The expected outcome is

  Max. Paket Size: 64904

This is what you see on igb, the code igc has been taken from.
However, on igc the output is

  Max. Paket Size: 1516

so the GSO aggregate packets are segmented by the kernel before calling
igc_xmit_frame.  Inside the subsequent call to igc_tso, the check for
skb_is_gso(skb) fails and the function returns prematurely.

It turns out that this occurs because the feature flags aren't set
entirely correctly in igc_probe.  In contrast to the original code
from igb_probe, igc_probe neglects to set the flags required to allow
tunnel offloading.

Setting the same flags as igb fixes the issue on igc.

Fixes: 34428dff ("igc: Add GSO partial support")
Signed-off-by: NPaolo Abeni <pabeni@redhat.com>
Tested-by: NCorinna Vinschen <vinschen@redhat.com>
Acked-by: NSasha Neftin <sasha.neftin@intel.com>
Tested-by: NNechama Kraus <nechamax.kraus@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Implement support for Credit-based shaper(CBS) Qdisc hardware
offload mode in the driver. There are two sets of IEEE802.1Qav
(CBS) HW logic in i225 controller and this patch supports
enabling them in the top two priority TX queues.

Driver implemented as recommended by Foxville External
Architecture Specification v0.993. Idleslope and Hi-credit are
the CBS tunable parameters for i225 NIC, programmed in TQAVCC
and TQAVHC registers respectively.

In-order for IEEE802.1Qav (CBS) algorithm to work as intended
and provide BW reservation CBS should be enabled in highest
priority queue first. If we enable CBS on any of low priority
queues, the traffic in high priority queue does not allow low
priority queue to be selected for transmission and bandwidth
reservation is not guaranteed.
Signed-off-by: NAravindhan Gunasekaran <aravindhan.gunasekaran@intel.com>
Signed-off-by: NMallikarjuna Chilakala <mallikarjuna.chilakala@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Separates the procedure done during reset from applying a
configuration, knowing when the code is executing allow us to
separate the better what changes the hardware state from what
changes only the driver state.

Introduces a flag for bookkeeping the driver state of TSN
features. When Qav and frame-preemption is also implemented
this flag makes it easier to keep track on whether a TSN feature
driver state is enabled or not though controller state changes,
say, during a reset.
Signed-off-by: NVinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: NAravindhan Gunasekaran <aravindhan.gunasekaran@intel.com>
Signed-off-by: NMallikarjuna Chilakala <mallikarjuna.chilakala@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Sets default values for each queue cycle start and cycle end.
This allows some simplification in the handling of these
configurations as most TSN features in i225 require a cycle
to be configured.

In i225, cycle start and end time is required to be programmed
for CBS to work properly.
Signed-off-by: NVinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: NAravindhan Gunasekaran <aravindhan.gunasekaran@intel.com>
Signed-off-by: NMallikarjuna Chilakala <mallikarjuna.chilakala@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Enables PCIe PTM (Precision Time Measurement) support in the igc
driver. Notifies the PCI devices that PCIe PTM should be enabled.

PCIe PTM is similar protocol to PTP (Precision Time Protocol) running
in the PCIe fabric, it allows devices to report time measurements from
their internal clocks and the correlation with the PCIe root clock.

The i225 NIC exposes some registers that expose those time
measurements, those registers will be used, in later patches, to
implement the PTP_SYS_OFFSET_PRECISE ioctl().
Signed-off-by: NVinicius Costa Gomes <vinicius.gomes@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Use num_tx_queues rather than the IGC_MAX_TX_QUEUES fixed number 4 when
iterating over tx_ring queue since instantiated queue count could be
less than 4 where on-line cpu count is less than 4.

Fixes: ec50a9d4 ("igc: Add support for taprio offloading")
Signed-off-by: NToshiki Nishioka <toshiki.nishioka@intel.com>
Signed-off-by: NMuhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
Tested-by: NMuhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
Acked-by: NSasha Neftin <sasha.neftin@intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

After unplug thunderbolt dock with i225, pciehp interrupt is triggered,
remove call will read/write mmio address which is already disconnected,
then cause page fault and make system hang.

Check PCI state to remove device safely.

Trace:
BUG: unable to handle page fault for address: 000000000000b604
Oops: 0000 [#1] SMP NOPTI
RIP: 0010:igc_rd32+0x1c/0x90 [igc]
Call Trace:
igc_ptp_suspend+0x6c/0xa0 [igc]
igc_ptp_stop+0x12/0x50 [igc]
igc_remove+0x7f/0x1c0 [igc]
pci_device_remove+0x3e/0xb0
__device_release_driver+0x181/0x240

Fixes: 13b5b7fd ("igc: Add support for Tx/Rx rings")
Fixes: b03c49cd ("igc: Save PTP time before a reset")
Signed-off-by: NAaron Ma <aaron.ma@canonical.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Most users of ndo_do_ioctl are ethernet drivers that implement
the MII commands SIOCGMIIPHY/SIOCGMIIREG/SIOCSMIIREG, or hardware
timestamping with SIOCSHWTSTAMP/SIOCGHWTSTAMP.

Separate these from the few drivers that use ndo_do_ioctl to
implement SIOCBOND, SIOCBR and SIOCWANDEV commands.

This is a purely cosmetic change intended to help readers find
their way through the implementation.

Cc: Doug Ledford <dledford@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Jay Vosburgh <j.vosburgh@gmail.com>
Cc: Veaceslav Falico <vfalico@gmail.com>
Cc: Andy Gospodarek <andy@greyhouse.net>
Cc: Andrew Lunn <andrew@lunn.ch>
Cc: Vivien Didelot <vivien.didelot@gmail.com>
Cc: Florian Fainelli <f.fainelli@gmail.com>
Cc: Vladimir Oltean <olteanv@gmail.com>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: linux-rdma@vger.kernel.org
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NJason Gunthorpe <jgg@nvidia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

As the cycle time is set to maximum of 1s, the TX Hang timeout need to
be increase to avoid possible TX Hang.

There is no dedicated number specific in data sheet for the timeout factor.
Timeout factor was determined during the debugging to solve the "Tx Hang"
issues that happen in some cases mainly during ETF(Earliest TxTime First).

This can be test by using TSN Schedule Tx Tools udp_tai sample application.
Signed-off-by: NMuhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
Acked-by: NSasha Neftin <sasha.neftin@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

i225 devices have only one phy->type: copper. There is no point checking
phy->type during the igc_has_link method from the watchdog that
invoked every 2 seconds.
This patch comes to clean up these pointless checkings.
Signed-off-by: NSasha Neftin <sasha.neftin@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

i225 devices have only one PHY vendor. There is no point checking
_I_PHY_ID during the link establishment and auto-negotiation process.
This patch comes to clean up these pointless checkings.
Signed-off-by: NSasha Neftin <sasha.neftin@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Ensure that the adapter->q_vector[MAX_Q_VECTORS] array isn't accessed
beyond its size. It was fixed by using a local variable num_q_vectors
as a limit for loop index, and ensure that num_q_vectors is not bigger
than MAX_Q_VECTORS.
Suggested-by: NAleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: NSasha Neftin <sasha.neftin@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

This reverts commit cf833182.

There are better Linux interfaces to export the different LED modes
and blinking reasons.

Revert this patch for now and come up with better solution later.
Suggested-by: NAndrew Lunn <andrew@lunn.ch>
Signed-off-by: NKurt Kanzenbach <kurt@linutronix.de>
Link: https://lore.kernel.org/r/20210719101640.16047-1-kurt@linutronix.deSigned-off-by: NJakub Kicinski <kuba@kernel.org>

Each i225 has three LEDs. Export them via the LED class framework.

Each LED is controllable via sysfs. Example:

$ cd /sys/class/leds/igc_led0
$ cat brightness      # Current Mode
$ cat max_brightness  # 15
$ echo 0 > brightness # Mode 0
$ echo 1 > brightness # Mode 1

The brightness field here reflects the different LED modes ranging
from 0 to 15.
Signed-off-by: NKurt Kanzenbach <kurt@linutronix.de>
Reviewed-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Currently flex filters are only used for filters containing user data.
However, it makes sense to utilize them also for filters having
multiple conditions, because that's not supported by the driver at the
moment. Add it.
Signed-off-by: NKurt Kanzenbach <kurt@linutronix.de>
Reviewed-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NVinicius Costa Gomes <vinicius.gomes@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Use the flex filter mechanism to extend the current ethtool filter
operations by intercoperating the user data. This allows to match
eight more bytes within a Ethernet frame in addition to macs, ether
types and vlan.

The matching pattern looks like this:

 * dest_mac [6]
 * src_mac [6]
 * tpid [2]
 * vlan tci [2]
 * ether type [2]
 * user data [8]

This can be used to match Profinet traffic classes by FrameID range.
Signed-off-by: NKurt Kanzenbach <kurt@linutronix.de>
Reviewed-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

The Intel i225 NIC has the possibility to add flex filters which can
match up to the first 128 byte of a packet. These filters are useful
for all kind of packet matching. One particular use case is Profinet,
as the different traffic classes are distinguished by the frame id
range which cannot be matched by any other means.

Add code to configure and enable flex filters.
Signed-off-by: NKurt Kanzenbach <kurt@linutronix.de>
Reviewed-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NVinicius Costa Gomes <vinicius.gomes@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

If an error occurs after a 'pci_enable_pcie_error_reporting()' call, it
must be undone by a corresponding 'pci_disable_pcie_error_reporting()'
call, as already done in the remove function.

Fixes: c9a11c23 ("igc: Add netdev")
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Acked-by: NSasha Neftin <sasha.neftin@intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Cleans the next descriptor to watch (next_to_watch) when cleaning the
TX ring.

Failure to do so can cause invalid memory accesses. If igc_poll() runs
while the controller is being reset this can lead to the driver try to
free a skb that was already freed.

Log message:

 [  101.525242] refcount_t: underflow; use-after-free.
 [  101.525251] WARNING: CPU: 1 PID: 646 at lib/refcount.c:28 refcount_warn_saturate+0xab/0xf0
 [  101.525259] Modules linked in: sch_etf(E) sch_mqprio(E) rfkill(E) intel_rapl_msr(E) intel_rapl_common(E)
 x86_pkg_temp_thermal(E) intel_powerclamp(E) coretemp(E) binfmt_misc(E) kvm_intel(E) kvm(E) irqbypass(E) crc32_pclmul(E)
 ghash_clmulni_intel(E) aesni_intel(E) mei_wdt(E) libaes(E) crypto_simd(E) cryptd(E) glue_helper(E) snd_hda_codec_hdmi(E)
 rapl(E) intel_cstate(E) snd_hda_intel(E) snd_intel_dspcfg(E) sg(E) soundwire_intel(E) intel_uncore(E) at24(E)
 soundwire_generic_allocation(E) iTCO_wdt(E) soundwire_cadence(E) intel_pmc_bxt(E) serio_raw(E) snd_hda_codec(E)
 iTCO_vendor_support(E) watchdog(E) snd_hda_core(E) snd_hwdep(E) snd_soc_core(E) snd_compress(E) snd_pcsp(E)
 soundwire_bus(E) snd_pcm(E) evdev(E) snd_timer(E) mei_me(E) snd(E) soundcore(E) mei(E) configfs(E) ip_tables(E) x_tables(E)
 autofs4(E) ext4(E) crc32c_generic(E) crc16(E) mbcache(E) jbd2(E) sd_mod(E) t10_pi(E) crc_t10dif(E) crct10dif_generic(E)
 i915(E) ahci(E) libahci(E) ehci_pci(E) igb(E) xhci_pci(E) ehci_hcd(E)
 [  101.525303]  drm_kms_helper(E) dca(E) xhci_hcd(E) libata(E) crct10dif_pclmul(E) cec(E) crct10dif_common(E) tsn(E) igc(E)
 e1000e(E) ptp(E) i2c_i801(E) crc32c_intel(E) psmouse(E) i2c_algo_bit(E) i2c_smbus(E) scsi_mod(E) lpc_ich(E) pps_core(E)
 usbcore(E) drm(E) button(E) video(E)
 [  101.525318] CPU: 1 PID: 646 Comm: irq/37-enp7s0-T Tainted: G            E     5.10.30-rt37-tsn1-rt-ipipe #ipipe
 [  101.525320] Hardware name: SIEMENS AG SIMATIC IPC427D/A5E31233588, BIOS V17.02.09 03/31/2017
 [  101.525322] RIP: 0010:refcount_warn_saturate+0xab/0xf0
 [  101.525325] Code: 05 31 48 44 01 01 e8 f0 c6 42 00 0f 0b c3 80 3d 1f 48 44 01 00 75 90 48 c7 c7 78 a8 f3 a6 c6 05 0f 48
 44 01 01 e8 d1 c6 42 00 <0f> 0b c3 80 3d fe 47 44 01 00 0f 85 6d ff ff ff 48 c7 c7 d0 a8 f3
 [  101.525327] RSP: 0018:ffffbdedc0917cb8 EFLAGS: 00010286
 [  101.525329] RAX: 0000000000000000 RBX: ffff98fd6becbf40 RCX: 0000000000000001
 [  101.525330] RDX: 0000000000000001 RSI: ffffffffa6f2700c RDI: 00000000ffffffff
 [  101.525332] RBP: ffff98fd6becc14c R08: ffffffffa7463d00 R09: ffffbdedc0917c50
 [  101.525333] R10: ffffffffa74c3578 R11: 0000000000000034 R12: 00000000ffffff00
 [  101.525335] R13: ffff98fd6b0b1000 R14: 0000000000000039 R15: ffff98fd6be35c40
 [  101.525337] FS:  0000000000000000(0000) GS:ffff98fd6e240000(0000) knlGS:0000000000000000
 [  101.525339] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 [  101.525341] CR2: 00007f34135a3a70 CR3: 0000000150210003 CR4: 00000000001706e0
 [  101.525343] Call Trace:
 [  101.525346]  sock_wfree+0x9c/0xa0
 [  101.525353]  unix_destruct_scm+0x7b/0xa0
 [  101.525358]  skb_release_head_state+0x40/0x90
 [  101.525362]  skb_release_all+0xe/0x30
 [  101.525364]  napi_consume_skb+0x57/0x160
 [  101.525367]  igc_poll+0xb7/0xc80 [igc]
 [  101.525376]  ? sched_clock+0x5/0x10
 [  101.525381]  ? sched_clock_cpu+0xe/0x100
 [  101.525385]  net_rx_action+0x14c/0x410
 [  101.525388]  __do_softirq+0xe9/0x2f4
 [  101.525391]  __local_bh_enable_ip+0xe3/0x110
 [  101.525395]  ? irq_finalize_oneshot.part.47+0xe0/0xe0
 [  101.525398]  irq_forced_thread_fn+0x6a/0x80
 [  101.525401]  irq_thread+0xe8/0x180
 [  101.525403]  ? wake_threads_waitq+0x30/0x30
 [  101.525406]  ? irq_thread_check_affinity+0xd0/0xd0
 [  101.525408]  kthread+0x183/0x1a0
 [  101.525412]  ? kthread_park+0x80/0x80
 [  101.525415]  ret_from_fork+0x22/0x30

Fixes: 13b5b7fd ("igc: Add support for Tx/Rx rings")
Reported-by: NErez Geva <erez.geva.ext@siemens.com>
Signed-off-by: NVinicius Costa Gomes <vinicius.gomes@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

The Intel drivers all have rcu_read_lock()/rcu_read_unlock() pairs around
XDP program invocations. However, the actual lifetime of the objects
referred by the XDP program invocation is longer, all the way through to
the call to xdp_do_flush(), making the scope of the rcu_read_lock() too
small. This turns out to be harmless because it all happens in a single
NAPI poll cycle (and thus under local_bh_disable()), but it makes the
rcu_read_lock() misleading.

Rather than extend the scope of the rcu_read_lock(), just get rid of it
entirely. With the addition of RCU annotations to the XDP_REDIRECT map
types that take bh execution into account, lockdep even understands this to
be safe, so there's really no reason to keep it around.
Signed-off-by: NToke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
Tested-by: Jesper Dangaard Brouer <brouer@redhat.com> # i40e
Cc: Jesse Brandeburg <jesse.brandeburg@intel.com>
Cc: Tony Nguyen <anthony.l.nguyen@intel.com>
Cc: intel-wired-lan@lists.osuosl.org
Link: https://lore.kernel.org/bpf/20210624160609.292325-12-toke@redhat.com

Add HW VLAN acceleration protocol handling. In case of HW VLAN tagging,
we need that protocol available in the ndo_start_xmit(), so that it will be
stored in a new fields in the skb.

HW offloading is set to OFF by default.
Users are allow to turn on/off Rx/Tx HW VLAN acceleration via ethtool.
Signed-off-by: NMuhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Acked-by: NSasha Neftin <sasha.neftin@intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Add missing exception tracing to XDP when a number of different
errors can occur. The support was only partial. Several errors
where not logged which would confuse the user quite a lot not
knowing where and why the packets disappeared.

Fixes: 73f1071c ("igc: Add support for XDP_TX action")
Fixes: 4ff32036 ("igc: Add support for XDP_REDIRECT action")
Reported-by: NJesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: NMagnus Karlsson <magnus.karlsson@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Add support for transmitting packets via AF_XDP zero-copy mechanism.

The packet transmission itself is implemented by igc_xdp_xmit_zc() which
is called from igc_clean_tx_irq() when the ring has AF_XDP zero-copy
enabled. Likewise i40e and ice drivers, the transmission budget used is
the number of descriptors available on the ring.

A new tx buffer type is introduced to 'enum igc_tx_buffer_type' to
indicate the tx buffer uses memory from xsk pool so it can be properly
cleaned after transmission or when the ring is cleaned.

The I225 controller has only 4 Tx hardware queues so the main difference
between igc and other Intel drivers that support AF_XDP zero-copy is
that there is no tx ring dedicated exclusively to XDP. Instead, tx
rings are shared between the network stack and XDP, and netdev queue
lock is used to ensure mutual exclusion. This is the same approach
implemented to support XDP_TX and XDP_REDIRECT actions.
Signed-off-by: NAndre Guedes <andre.guedes@intel.com>
Signed-off-by: NVedang Patel <vedang.patel@intel.com>
Signed-off-by: NJithu Joseph <jithu.joseph@intel.com>
Reviewed-by: NMaciej Fijalkowski <maciej.fijalkowski@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Add support for receiving packets via AF_XDP zero-copy mechanism.

Add a new flag to 'enum igc_ring_flags_t' to indicate the ring has
AF_XDP zero-copy enabled so proper ring setup is carried out during ring
configuration in igc_configure_rx_ring().

RX buffers can now be allocated via the shared pages mechanism (default
behavior of the driver) or via xsk pool (when AF_XDP zero-copy is
enabled) so a union is added to the 'struct igc_rx_buffer' to cover both
cases.

When AF_XDP zero-copy is enabled, rx buffers are allocated from the xsk
pool using the new helper igc_alloc_rx_buffers_zc() which is the
counterpart of igc_alloc_rx_buffers().

Likewise other Intel drivers that support AF_XDP zero-copy, in igc we
have a dedicated path for cleaning up rx irqs when zero-copy is enabled.
This avoids adding too many checks within igc_clean_rx_irq(), resulting
in a more readable and efficient code since this function is called from
the hot-path of the driver.
Signed-off-by: NAndre Guedes <andre.guedes@intel.com>
Signed-off-by: NVedang Patel <vedang.patel@intel.com>
Signed-off-by: NJithu Joseph <jithu.joseph@intel.com>
Reviewed-by: NMaciej Fijalkowski <maciej.fijalkowski@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Up to this point, Tx buffers are associated with either a skb or a xdpf,
and the IGC_TX_FLAGS_XDP flag was enough to distinguish between these
two case. However, with upcoming patches that will add AF_XDP zero-copy
support, a third case will be introduced so this flag-based approach
won't fit well.

In preparation to land AF_XDP zero-copy support, replace the
IGC_TX_FLAGS_XDP flag by an enum which will be extended once zero-copy
support is introduced to the driver.
Signed-off-by: NAndre Guedes <andre.guedes@intel.com>
Signed-off-by: NVedang Patel <vedang.patel@intel.com>
Signed-off-by: NJithu Joseph <jithu.joseph@intel.com>
Reviewed-by: NMaciej Fijalkowski <maciej.fijalkowski@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

In preparation for AF_XDP zero-copy support, encapsulate the code that
unmaps Tx buffers into its own local helper so we can reuse it, avoiding
code duplication.
Signed-off-by: NAndre Guedes <andre.guedes@intel.com>
Signed-off-by: NVedang Patel <vedang.patel@intel.com>
Signed-off-by: NJithu Joseph <jithu.joseph@intel.com>
Reviewed-by: NMaciej Fijalkowski <maciej.fijalkowski@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

In preparation for AF_XDP zero-copy support, encapsulate the code that
updates the driver RX stats in its own local helper so it can be reused
in the zero-copy path. Likewise, encapsulate TX stats code as well.
Signed-off-by: NAndre Guedes <andre.guedes@intel.com>
Signed-off-by: NVedang Patel <vedang.patel@intel.com>
Signed-off-by: NJithu Joseph <jithu.joseph@intel.com>
Reviewed-by: NMaciej Fijalkowski <maciej.fijalkowski@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Refactor XDP rxq info registration code, preparing the driver for AF_XDP
zero-copy support which is added by upcoming patches.

Currently, xdp_rxq and memory model are both registered during RX
resource setup time by igc_xdp_register_rxq_info() helper. With AF_XDP,
we want to register the memory model later on while configuring the ring
because we will know which memory model type to register
(MEM_TYPE_PAGE_SHARED or MEM_TYPE_XSK_BUFF_POOL).

The helpers igc_xdp_register_rxq_info() and igc_xdp_unregister_rxq_
info() are not useful anymore so they are removed.
Signed-off-by: NAndre Guedes <andre.guedes@intel.com>
Signed-off-by: NVedang Patel <vedang.patel@intel.com>
Signed-off-by: NJithu Joseph <jithu.joseph@intel.com>
Reviewed-by: NMaciej Fijalkowski <maciej.fijalkowski@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Refactor igc_clean_rx_ring() helper, preparing the code for AF_XDP
zero-copy support which is added by upcoming patches.

The refactor consists of encapsulating page-shared specific code into
its own helper, leaving common code that will be shared by both
page-shared and xsk pool in igc_clean_rx_ring().
Signed-off-by: NAndre Guedes <andre.guedes@intel.com>
Signed-off-by: NVedang Patel <vedang.patel@intel.com>
Signed-off-by: NJithu Joseph <jithu.joseph@intel.com>
Reviewed-by: NMaciej Fijalkowski <maciej.fijalkowski@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>

Refactor __igc_xdp_run_prog() helper from igc_xdp_run_prog(),
preparing the code for AF_XDP zero-copy support which is added
by upcoming patches.

The existing igc_xdp_run_prog() caters to regular XDP rx path
which has to verify if bpf_prog is not NULL. Zero-copy
path assumes that bpf_prog is not NULL and hence this check is
not required. Therefore it makes sense to refactor the common
code into a helper function, to avoid code duplication.
Signed-off-by: NAndre Guedes <andre.guedes@intel.com>
Signed-off-by: NVedang Patel <vedang.patel@intel.com>
Signed-off-by: NJithu Joseph <jithu.joseph@intel.com>
Reviewed-by: NMaciej Fijalkowski <maciej.fijalkowski@intel.com>
Tested-by: NDvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
Signed-off-by: NTony Nguyen <anthony.l.nguyen@intel.com>