Rearrange RX buffer page_pool recycling logics into dma_recycle_rx_skbufs,
so that we prepare stmmac_reinit_rx_buffers() for XSK pool expansion.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch restructures the per RX queue buffer allocation from page_pool
to stmmac_alloc_rx_buffers().

We also rearrange dma_free_rx_skbufs() so that it can be used in
init_dma_rx_desc_rings() during freeing of RX buffer in the event of
page_pool allocation failure to replace the more efficient method earlier.
The replacement is needed to make the RX buffer alloc and free method
scalable to XDP ZC xsk_pool alloc and free later.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

of_get_mac_address() returns a "const void*" pointer to a MAC address.
Lately, support to fetch the MAC address by an NVMEM provider was added.
But this will only work with platform devices. It will not work with
PCI devices (e.g. of an integrated root complex) and esp. not with DSA
ports.

There is an of_* variant of the nvmem binding which works without
devices. The returned data of a nvmem_cell_read() has to be freed after
use. On the other hand the return of_get_mac_address() points to some
static data without a lifetime. The trick for now, was to allocate a
device resource managed buffer which is then returned. This will only
work if we have an actual device.

Change it, so that the caller of of_get_mac_address() has to supply a
buffer where the MAC address is written to. Unfortunately, this will
touch all drivers which use the of_get_mac_address().

Usually the code looks like:

  const char *addr;
  addr = of_get_mac_address(np);
  if (!IS_ERR(addr))
    ether_addr_copy(ndev->dev_addr, addr);

This can then be simply rewritten as:

  of_get_mac_address(np, ndev->dev_addr);

Sometimes is_valid_ether_addr() is used to test the MAC address.
of_get_mac_address() already makes sure, it just returns a valid MAC
address. Thus we can just test its return code. But we have to be
careful if there are still other sources for the MAC address before the
of_get_mac_address(). In this case we have to keep the
is_valid_ether_addr() call.

The following coccinelle patch was used to convert common cases to the
new style. Afterwards, I've manually gone over the drivers and fixed the
return code variable: either used a new one or if one was already
available use that. Mansour Moufid, thanks for that coccinelle patch!

<spml>
@a@
identifier x;
expression y, z;
@@
- x = of_get_mac_address(y);
+ x = of_get_mac_address(y, z);
  <...
- ether_addr_copy(z, x);
  ...>

@@
identifier a.x;
@@
- if (<+... x ...+>) {}

@@
identifier a.x;
@@
  if (<+... x ...+>) {
      ...
  }
- else {}

@@
identifier a.x;
expression e;
@@
- if (<+... x ...+>@e)
-     {}
- else
+ if (!(e))
      {...}

@@
expression x, y, z;
@@
- x = of_get_mac_address(y, z);
+ of_get_mac_address(y, z);
  ... when != x
</spml>

All drivers, except drivers/net/ethernet/aeroflex/greth.c, were
compile-time tested.
Suggested-by: NAndrew Lunn <andrew@lunn.ch>
Signed-off-by: NMichael Walle <michael@walle.cc>
Reviewed-by: NAndrew Lunn <andrew@lunn.ch>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds the support of XDP_REDIRECT to another remote cpu for
further action. It also implements ndo_xdp_xmit ops, enabling the driver
to transmit packets forwarded to it by XDP program running on another
interface.

This patch has been tested using "xdp_redirect_cpu" for XDP_REDIRECT
+ drop testing. It also been tested with "xdp_redirect" sample app
which can be used to exercise ndo_xdp_xmit ops. The burst traffics are
generated using pktgen_sample03_burst_single_flow.sh in samples/pktgen
directory.

v4: Move xdp_do_flush() processing into stmmac_finalize_xdp_rx() and
    combined the XDP verdict of XDP TX and REDIRECT together.

v3: Added 'nq->trans_start = jiffies' to avoid TX time-out as we are
    sharing TX queue between slow path and XDP. Thanks to Jakub Kicinski
    for point out.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds support for XDP_TX action which enables XDP program to
transmit back received frames.

This patch has been tested with the "xdp2" app located in samples/bpf
dir. The DUT receives burst traffic packet generated using pktgen script
'pktgen_sample03_burst_single_flow.sh'.

v4: Moved stmmac_tx_timer_arm() to be done once at the end of NAPI RX.
    Fixed stmmac_xdp_xmit_back() to return STMMAC_XDP_CONSUMED if
    XDP buffer to frame conversion fails. Thanks to Jakub's input.

v3: Added 'nq->trans_start = jiffies' to avoid TX time-out as we are
    sharing TX queue between slow path and XDP. Thanks to Jakub Kicinski
    for pointing out.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds the initial XDP support to stmmac driver. It supports
XDP_PASS, XDP_DROP and XDP_ABORTED actions. Upcoming patches will add
support for XDP_TX and XDP_REDIRECT.

To support XDP headroom, this patch adds page_offset into RX buffer and
change the dma_sync_single_for_device|cpu(). The DMA address used for
RX operation are changed to take into page_offset too. As page_pool
can handle dma_sync_single_for_device() on behalf of driver with
PP_FLAG_DMA_SYNC_DEV flag, we skip doing that in stmmac driver.

Current stmmac driver supports split header support (SPH) in RX but
the flexibility of splitting header and payload at different position
makes it very complex to be supported for XDP processing. In addition,
jumbo frame is not supported in XDP to keep the initial codes simple.

This patch has been tested with the sample app "xdp1" located in
samples/bpf directory for both SKB and Native (XDP) mode. The burst
traffic generated using pktgen_sample03_burst_single_flow.sh in
samples/pktgen directory.

Changes in v3:
 - factor in xdp header and tail adjustment done by XDP program.
   Thanks to Jakub Kicinski for pointing out the gap in v2.

Changes in v2:
 - fix for "warning: variable 'len' set but not used" reported by lkp.
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch organizes TX tail pointer update into a new function called
stmmac_flush_tx_descriptors() so that we can reuse it in stmmac_xmit(),
stmmac_tso_xmit() and up-coming XDP implementation.

Changes to v2:
 - Fix for warning: unused variable ‘desc_size’
   https://patchwork.hopto.org/static/nipa/457321/12170149/build_32bit/stderrSigned-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

SPH functionality splits header and payload according to split mode and
offsef fields (SPLM and SPLOFST). It is beneficials for Linux network
stack RX processing however it adds a lot of complexity in XDP
processing.

So, this patch makes the split-header (SPH) capability of the controller
is stored in "priv->sph_cap" and the enabling/disabling of SPH is decided
by "priv->sph".

This is to prepare initial XDP enabling for stmmac to disable the use of
SPH whenever XDP is enabled.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Certain platform likes Intel mGBE has independent hardware IRQ resources
for TX and RX DMA operation. In preparation to support XDP TX, we add IRQ
affinity hint to group both RX and TX queue of the same queue ID to the
same CPU.

Changes in v2:
 - IRQ affinity hint need to set to null before IRQ is released.
   Thanks to issue reported by Song, Yoong Siang.
Reported-by: NSong, Yoong Siang <yoong.siang.song@intel.com>
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add the missing unlock before return from stmmac_suspend()
in the error handling case.

Fixes: 5ec55823 ("net: stmmac: add clocks management for gmac driver")
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYang Yingliang <yangyingliang@huawei.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In this patch, "net: stmmac: support FPE link partner hand-shaking
procedure", priv->plat->fpe_cfg wouldn`t be "devm_kzalloc"ed if
dma_cap->frpsel is 0 (Flexible Rx Parser is not supported in SoC) in
tc_init(). So, fpe_cfg will be remain as NULL and accessing it will cause
kernel panic.

To fix this, move the "devm_kzalloc"ing of priv->plat->fpe_cfg before
dma_cap->frpsel checking in tc_init(). Additionally, checking of
priv->dma_cap.fpesel is added before calling stmmac_fpe_link_state_handle()
as only FPE supported SoC is allowed to call the function.

Below is the kernel panic dump reported by Marek Szyprowski
<m.szyprowski@samsung.com>:

meson8b-dwmac ff3f0000.ethernet eth0: PHY [0.0:00] driver [RTL8211F Gigabit Ethernet] (irq=35)
meson8b-dwmac ff3f0000.ethernet eth0: No Safety Features support found
meson8b-dwmac ff3f0000.ethernet eth0: PTP not supported by HW
meson8b-dwmac ff3f0000.ethernet eth0: configuring for phy/rgmii link mode
Unable to handle kernel NULL pointer dereference at virtual address 0000000000000001
Mem abort info:
...
user pgtable: 4k pages, 48-bit VAs, pgdp=00000000044eb000
[0000000000000001] pgd=0000000000000000, p4d=0000000000000000
Internal error: Oops: 96000004 [#1] PREEMPT SMP
Modules linked in: dw_hdmi_i2s_audio dw_hdmi_cec meson_gxl realtek meson_gxbb_wdt snd_soc_meson_axg_sound_card dwmac_generic axg_audio meson_dw_hdmi crct10dif_ce snd_soc_meson_card_utils snd_soc_meson_axg_tdmout panfrost rc_odroid gpu_sched reset_meson_audio_arb meson_ir snd_soc_meson_g12a_tohdmitx snd_soc_meson_axg_frddr sclk_div clk_phase snd_soc_meson_codec_glue dwmac_meson8b snd_soc_meson_axg_fifo stmmac_platform meson_rng meson_drm stmmac rtc_meson_vrtc rng_core meson_canvas pwm_meson dw_hdmi mdio_mux_meson_g12a pcs_xpcs snd_soc_meson_axg_tdm_interface snd_soc_meson_axg_tdm_formatter nvmem_meson_efuse display_connector
CPU: 1 PID: 7 Comm: kworker/u8:0 Not tainted 5.12.0-rc4-next-20210325+
Hardware name: Hardkernel ODROID-C4 (DT)
Workqueue: events_power_efficient phylink_resolve
pstate: 20400009 (nzCv daif +PAN -UAO -TCO BTYPE=--)
pc : stmmac_mac_link_up+0x14c/0x348 [stmmac]
lr : stmmac_mac_link_up+0x284/0x348 [stmmac] ...
Call trace:
 stmmac_mac_link_up+0x14c/0x348 [stmmac]
 phylink_resolve+0x104/0x420
 process_one_work+0x2a8/0x718
 worker_thread+0x48/0x460
 kthread+0x134/0x160
 ret_from_fork+0x10/0x18
Code: b971ba60 350007c0 f958c260 f9402000 (39400401)
---[ end trace 0c9deb6c510228aa ]---

Fixes: 5a558611 ("net: stmmac: support FPE link partner hand-shaking
procedure")
Reported-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: NMohammad Athari Bin Ismail <mohammad.athari.ismail@intel.com>
Tested-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For interrupt mode INTM=0, TX/RX transfer complete will trigger signal
not only on sbd_perch_[tx|rx]_intr_o (Transmit/Receive Per Channel) but
also on the sbd_intr_o (Common).

As for multi-MSI implementation, setting interrupt mode INTM=1 is more
efficient as each TX intr and RX intr (TI/RI) will be handled by TX/RX ISR
without the need of calling the common MAC ISR.

Updated the TX/RX NORMAL interrupts status checking process as the
NIS status bit is not asserted for any RI/TI events for INTM=1.
Signed-off-by: NWong, Vee Khee <vee.khee.wong@intel.com>
Co-developed-by: NVoon Weifeng <weifeng.voon@intel.com>
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Now we introduce MSI interrupt service routines and hook these routines
up if stmmac_open() sees valid irq line being requested:-

stmmac_mac_interrupt()    :- MAC (dev->irq), WOL (wol_irq), LPI (lpi_irq)
stmmac_safety_interrupt() :- Safety Feat Correctible Error (sfty_ce_irq)
                             & Uncorrectible Error (sfty_ue_irq)
stmmac_msi_intr_rx()      :- For all RX MSI irq (rx_irq)
stmmac_msi_intr_tx()      :- For all TX MSI irq (tx_irq)

Each of IRQs will have its unique name so that we can differentiate
them easily under /proc/interrupts.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Refactor stmmac_interrupt() by introducing stmmac_common_interrupt()
so that we prepare the ISR operation to be friendly to MSI later.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In preparation to make stmmac support multi-vector MSI, we introduce the
interrupt status masking according to RX, TX or RXTX. Default to use RXTX
inside stmmac_dma_interrupt(), so there is no run-time logic difference
now.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In order to discover whether remote station supports frame preemption,
local station sends verify mPacket and expects response mPacket in
return from the remote station.

So, we add the functions to send and handle event when verify mPacket
and response mPacket are exchanged between the networked stations.

The mechanism to handle different FPE states between local and remote
station (link partner) is implemented using workqueue which starts a
task each time there is some sign of verify & response mPacket exchange
as check in FPE IRQ event. The task retries couple of times to try to
spot the states that both stations are ready to enter FPE ON. This allows
different end points to enable FPE at different time and verify-response
mPacket can happen asynchronously. Ultimately, the task will only turn
FPE ON when local station have both exchange response in both directions.

Thanks to Voon Weifeng for implementing the core functions for detecting
FPE events and send mPacket and phylink related change.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Co-developed-by: NVoon Weifeng <weifeng.voon@intel.com>
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Co-developed-by: NTan Tee Min <tee.min.tan@intel.com>
Signed-off-by: NTan Tee Min <tee.min.tan@intel.com>
Co-developed-by: NMohammad Athari Bin Ismail <mohammad.athari.ismail@intel.com>
Signed-off-by: NMohammad Athari Bin Ismail <mohammad.athari.ismail@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Below EST errors are added into ethtool statistic:
1) Constant Gate Control Error (CGCE):
   The counter "mtl_est_cgce" increases everytime CGCE interrupt is
   triggered.

2) Head-of-Line Blocking due to Scheduling (HLBS):
   The counter "mtl_est_hlbs" increases everytime HLBS interrupt is
   triggered.

3) Head-of-Line Blocking due to Frame Size (HLBF):
   The counter "mtl_est_hlbf" increases everytime HLBF interrupt is
   triggered.

4) Base Time Register error (BTRE):
   The counter "mtl_est_btre" increases everytime BTRE interrupt is
   triggered but BTRL not reaches maximum value of 15.

5) Base Time Register Error Loop Count (BTRL) reaches maximum value:
   The counter "mtl_est_btrlm" increases everytime BTRE interrupt is
   triggered and BTRL value reaches maximum value of 15.

Please refer to MTL_EST_STATUS register in DesignWare Cores Ethernet
Quality-of-Service Databook for more detail explanation.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Co-developed-by: NMohammad Athari Bin Ismail <mohammad.athari.ismail@intel.com>
Signed-off-by: NMohammad Athari Bin Ismail <mohammad.athari.ismail@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Enabled EST related interrupts as below:
1) Constant Gate Control Error (CGCE)
2) Head-of-Line Blocking due to Scheduling (HLBS)
3) Head-of-Line Blocking due to Frame Size (HLBF).
4) Base Time Register error (BTRE)
5) Switch to S/W owned list Complete (SWLC)

For HLBS, the user will get the info of all the queues that shows this
error. For HLBF, the user will get the info of all the queue with the
latest frame size which causes the error. Frame size 0 indicates no
error.

The ISR handling takes place when EST feature is enabled by user.
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Co-developed-by: NMohammad Athari Bin Ismail <mohammad.athari.ismail@intel.com>
Signed-off-by: NMohammad Athari Bin Ismail <mohammad.athari.ismail@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Extending the driver to support per-queue RX and TX coalesce settings in
order to support below commands:

To show per-queue coalesce setting:-
 $ ethtool --per-queue <DEVNAME> queue_mask <MASK> --show-coalesce

To set per-queue coalesce setting:-
 $ ethtool --per-queue <DEVNAME> queue_mask <MASK> --coalesce \
     [rx-usecs N] [rx-frames M] [tx-usecs P] [tx-frames Q]
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Acked-by: NJakub Kicinski <kuba@kernel.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

According to Synopsis DesignWare EQoS Databook, the Clock Domain Cross
synchronization error is introduced tue to the clock(GMII Tx/Rx clock)
being different at the capture as compared to the PTP
clock(clk_ptp_ref_i) that is used to generate the time.

The CDC synchronization error is almost equal to 2 times the clock
period of the PTP clock(clk_ptp_ref_i).

On a Intel Tigerlake platform (with Marvell 88E2110 external PHY):

Before applying this patch (with CDC synchronization error):
ptp4l[64.044]: rms    8 max   13 freq +30877 +/-  11 delay   216 +/-   0
ptp4l[65.047]: rms   13 max   20 freq +30869 +/-  17 delay   213 +/-   0
ptp4l[66.050]: rms   12 max   20 freq +30857 +/-  11 delay   213 +/-   0
ptp4l[67.052]: rms   11 max   22 freq +30849 +/-  10 delay   215 +/-   0
ptp4l[68.055]: rms   10 max   16 freq +30853 +/-  13 delay   215 +/-   0
ptp4l[69.057]: rms    7 max   13 freq +30848 +/-   9 delay   216 +/-   0
ptp4l[70.060]: rms    8 max   13 freq +30846 +/-  10 delay   216 +/-   0
ptp4l[71.063]: rms    9 max   15 freq +30836 +/-   8 delay   218 +/-   0

After applying this patch (CDC syncrhonization error is taken care of):
ptp4l[61.516]: rms  773 max  824 freq +31526 +/- 158 delay   200 +/-   0
ptp4l[62.519]: rms  427 max  596 freq +31668 +/-  39 delay   198 +/-   0
ptp4l[63.522]: rms  113 max  206 freq +31482 +/-  57 delay   198 +/-   0
ptp4l[64.525]: rms   40 max   56 freq +31316 +/-  29 delay   200 +/-   0
ptp4l[65.528]: rms   47 max   56 freq +31255 +/-  17 delay   200 +/-   0
ptp4l[66.531]: rms   26 max   36 freq +31246 +/-   9 delay   200 +/-   0
ptp4l[67.534]: rms   12 max   18 freq +31254 +/-  12 delay   202 +/-   0
ptp4l[68.537]: rms    7 max   12 freq +31263 +/-  10 delay   202 +/-   0
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Signed-off-by: NWong Vee Khee <vee.khee.wong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch intends to add platform level clocks management. Some
platforms may have their own special clocks, they also need to be
managed dynamically. If you want to manage such clocks, please implement
clks_config callback.
Reviewed-by: NAndrew Lunn <andrew@lunn.ch>
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch intends to add clocks management for stmmac driver:

If CONFIG_PM enabled:
1. Keep clocks disabled after driver probed.
2. Enable clocks when up the net device, and disable clocks when down
the net device.

If CONFIG_PM disabled:
Keep clocks always enabled after driver probed.

Note:
1. It is fine for ethtool, since the way of implementing ethtool_ops::begin
in stmmac is only can be accessed when interface is enabled, so the clocks
are ticked.
2. The MDIO bus has a different life cycle to the MAC, need ensure
clocks are enabled when _mdio_read/write() need clocks, because these
functions can be called while the interface it not opened.
Reviewed-by: NAndrew Lunn <andrew@lunn.ch>
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

As the support for MAC-side SGMII C37 AN is added to pcs-xpcs, phydev
should be attached to phylink during driver's open(). So, we change the
condition to "Not C73 AN" instead.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Not all platform uses DT, so phylink_parse_mode() will skip in-band setup
of pl->supported and pl->link_config.advertising entirely. So, we add the
setting of ovr_an_inband flag to make it works for non-DT platform.
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For Intel mGbE controller, MAC VLAN filter delete operation will time-out
if serdes power-down sequence happened first during driver remove() with
below message.

[82294.764958] intel-eth-pci 0000:00:1e.4 eth2: stmmac_dvr_remove: removing driver
[82294.778677] intel-eth-pci 0000:00:1e.4 eth2: Timeout accessing MAC_VLAN_Tag_Filter
[82294.779997] intel-eth-pci 0000:00:1e.4 eth2: failed to kill vid 0081/0
[82294.947053] intel-eth-pci 0000:00:1d.2 eth1: stmmac_dvr_remove: removing driver
[82295.002091] intel-eth-pci 0000:00:1d.1 eth0: stmmac_dvr_remove: removing driver

Therefore, we delay the serdes power-down to be after unregister_netdev()
which triggers the VLAN filter delete.

Fixes: b9663b7c ("net: stmmac: Enable SERDES power up/down sequence")
Signed-off-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

During suspend/resume stress test, we found descriptor write back by DMA
could exhibit unusual behavior, e.g.:
	003 [0xc4310030]: 0x0 0x40 0x0 0xb5010040

We can see that desc3 write back is 0xb5010040, it is still ownd by DMA,
so application would not recycle this buffer. It will trigger fatal bus
error when DMA try to use this descriptor again. To fix this issue, we
should re-init all rx buffers when mac resume back.
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

In current driver, buffer2 available only when hardware supports split
header. Wrongly set buffer2 valid in stmmac_rx_refill when refill buffer
address. You can see that desc3 is 0x81000000 after initialization, but
turn out to be 0x83000000 after refill.

Fixes: 67afd6d1 ("net: stmmac: Add Split Header support and enable it in XGMAC cores")
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Driver uses dma_alloc_coherent to allocate dma memory for descriptors,
dma_alloc_coherent will return both the virtual address and physical
address. AFAIK, virt_to_phys could not convert virtual address to
physical address, for which memory is allocated by dma_alloc_coherent.

dwmac4_display_ring() function is broken for various descriptor, it only
support normal descriptor(struct dma_desc) now, this patch also extends to
support all descriptor types.
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

stmmac_xmit() call stmmac_tx_timer_arm() at the end to modify tx timer to
do the transmission cleanup work. Imagine such a situation, stmmac enters
suspend immediately after tx timer modified, it's expire callback
stmmac_tx_clean() would not be invoked. This could affect BQL, since
netdev_tx_sent_queue() has been called, but netdev_tx_completed_queue()
have not been involved, as a result, dql_avail(&dev_queue->dql) finally
always return a negative value.

__dev_queue_xmit->__dev_xmit_skb->qdisc_run->__qdisc_run->qdisc_restart->dequeue_skb:
	if ((q->flags & TCQ_F_ONETXQUEUE) &&
		netif_xmit_frozen_or_stopped(txq)) // __QUEUE_STATE_STACK_XOFF is set

Net core will stop transmitting any more. Finillay, net watchdong would timeout.
To fix this issue, we should call netdev_tx_reset_queue() in stmmac_resume().

Fixes: 54139cf3 ("net: stmmac: adding multiple buffers for rx")
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Since the original mtu is not used when the mtu is updated,
the mtu is aligned with cache, this will get an incorrect.
For example, if you want to configure the mtu to be 1500,
but mtu 1536 is configured in fact.

Fixed: eaf4fac4 ("net: stmmac: Do not accept invalid MTU values")
Signed-off-by: NDavid Wu <david.wu@rock-chips.com>
Link: https://lore.kernel.org/r/20210113034109.27865-1-david.wu@rock-chips.comSigned-off-by: NJakub Kicinski <kuba@kernel.org>

Use of __napi_schedule_irqoff() is not safe with PREEMPT_RT in which
hard interrupts are not disabled while running the threaded interrupt.

Using __napi_schedule() works for both PREEMPT_RT and mainline Linux,
just at the cost of an additional check if interrupts are disabled for
mainline (since they are already disabled).

Similar to the fix done for enetc commit 215602a8 ("enetc: use
napi_schedule to be compatible with PREEMPT_RT")
Signed-off-by: NSeb Laveze <sebastien.laveze@nxp.com>
Link: https://lore.kernel.org/r/20210112140121.1487619-1-sebastien.laveze@oss.nxp.comSigned-off-by: NJakub Kicinski <kuba@kernel.org>

The current IP register MAC_HW_Feature1[ADDR64] only defines
32/40/64 bit width, but some SOCs support others like i.MX8MP
support 34 bits but it maps to 40 bits width in MAC_HW_Feature1[ADDR64].
So overwrite dma_cap.addr64 according to HW real design.

Fixes: 94abdad6 ("net: ethernet: dwmac: add ethernet glue logic for NXP imx8 chip")
Signed-off-by: NFugang Duan <fugang.duan@nxp.com>
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There have chance to re-enable the eee_ctrl_timer and fire the timer
in napi callback after delete the timer in .stmmac_release(), which
introduces to access eee registers in the timer function after clocks
are disabled then causes system hang. Found this issue when do
suspend/resume and reboot stress test.

It is safe to delete the timer after napi disabled and disable lpi mode.

Fixes: d765955d ("stmmac: add the Energy Efficient Ethernet support")
Signed-off-by: NFugang Duan <fugang.duan@nxp.com>
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When do suspend/resume test, there have WARN_ON() log dump from
stmmac_xmit() funciton, the code logic:
	entry = tx_q->cur_tx;
	first_entry = entry;
	WARN_ON(tx_q->tx_skbuff[first_entry]);

In normal case, tx_q->tx_skbuff[txq->cur_tx] should be NULL because
the skb should be handled and freed in stmmac_tx_clean().

But stmmac_resume() reset queue parameters like below, skb buffers
may not be freed.
	tx_q->cur_tx = 0;
	tx_q->dirty_tx = 0;

So free tx skb buffer in stmmac_resume() to avoid warning and
memory leak.

log:
[   46.139824] ------------[ cut here ]------------
[   46.144453] WARNING: CPU: 0 PID: 0 at drivers/net/ethernet/stmicro/stmmac/stmmac_main.c:3235 stmmac_xmit+0x7a0/0x9d0
[   46.154969] Modules linked in: crct10dif_ce vvcam(O) flexcan can_dev
[   46.161328] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G           O      5.4.24-2.1.0+g2ad925d15481 #1
[   46.170369] Hardware name: NXP i.MX8MPlus EVK board (DT)
[   46.175677] pstate: 80000005 (Nzcv daif -PAN -UAO)
[   46.180465] pc : stmmac_xmit+0x7a0/0x9d0
[   46.184387] lr : dev_hard_start_xmit+0x94/0x158
[   46.188913] sp : ffff800010003cc0
[   46.192224] x29: ffff800010003cc0 x28: ffff000177e2a100
[   46.197533] x27: ffff000176ef0840 x26: ffff000176ef0090
[   46.202842] x25: 0000000000000000 x24: 0000000000000000
[   46.208151] x23: 0000000000000003 x22: ffff8000119ddd30
[   46.213460] x21: ffff00017636f000 x20: ffff000176ef0cc0
[   46.218769] x19: 0000000000000003 x18: 0000000000000000
[   46.224078] x17: 0000000000000000 x16: 0000000000000000
[   46.229386] x15: 0000000000000079 x14: 0000000000000000
[   46.234695] x13: 0000000000000003 x12: 0000000000000003
[   46.240003] x11: 0000000000000010 x10: 0000000000000010
[   46.245312] x9 : ffff00017002b140 x8 : 0000000000000000
[   46.250621] x7 : ffff00017636f000 x6 : 0000000000000010
[   46.255930] x5 : 0000000000000001 x4 : ffff000176ef0000
[   46.261238] x3 : 0000000000000003 x2 : 00000000ffffffff
[   46.266547] x1 : ffff000177e2a000 x0 : 0000000000000000
[   46.271856] Call trace:
[   46.274302]  stmmac_xmit+0x7a0/0x9d0
[   46.277874]  dev_hard_start_xmit+0x94/0x158
[   46.282056]  sch_direct_xmit+0x11c/0x338
[   46.285976]  __qdisc_run+0x118/0x5f0
[   46.289549]  net_tx_action+0x110/0x198
[   46.293297]  __do_softirq+0x120/0x23c
[   46.296958]  irq_exit+0xb8/0xd8
[   46.300098]  __handle_domain_irq+0x64/0xb8
[   46.304191]  gic_handle_irq+0x5c/0x148
[   46.307936]  el1_irq+0xb8/0x180
[   46.311076]  cpuidle_enter_state+0x84/0x360
[   46.315256]  cpuidle_enter+0x34/0x48
[   46.318829]  call_cpuidle+0x18/0x38
[   46.322314]  do_idle+0x1e0/0x280
[   46.325539]  cpu_startup_entry+0x24/0x40
[   46.329460]  rest_init+0xd4/0xe0
[   46.332687]  arch_call_rest_init+0xc/0x14
[   46.336695]  start_kernel+0x420/0x44c
[   46.340353] ---[ end trace bc1ee695123cbacd ]---

Fixes: 47dd7a54 ("net: add support for STMicroelectronics Ethernet controllers.")
Signed-off-by: NFugang Duan <fugang.duan@nxp.com>
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Start phylink instance and resume back the PHY to supply
RX clock to MAC before MAC layer initialization by calling
.stmmac_hw_setup(), since DMA reset depends on the RX clock,
otherwise DMA reset cost maximum timeout value then finally
timeout.

Fixes: 74371272 ("net: stmmac: Convert to phylink and remove phylib logic")
Signed-off-by: NFugang Duan <fugang.duan@nxp.com>
Signed-off-by: NJoakim Zhang <qiangqing.zhang@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This driver uses a normal timer for TX coalescing, which means that the
with the default tx-usecs of 1000 microseconds the cleanups actually
happen 10 ms or more later with HZ=100.  This leads to very low
througput with TCP when bridged to a slow link such as a 4G modem.  Fix
this by using an hrtimer instead.

On my ARM platform with HZ=100 and the default TX coalescing settings
(tx-frames 25 tx-usecs 1000), with "tc qdisc add dev eth0 root netem
delay 60ms 40ms rate 50Mbit" run on the server, netperf's TCP_STREAM
improves from ~5.5 Mbps to ~100 Mbps.
Signed-off-by: NVincent Whitchurch <vincent.whitchurch@axis.com>
Link: https://lore.kernel.org/r/20201120150208.6838-1-vincent.whitchurch@axis.comSigned-off-by: NJakub Kicinski <kuba@kernel.org>

Fix an issue where dump stack is printed on suspend resume flow due to
netif_set_real_num_rx_queues() is not called with rtnl_lock held().

Fixes: 686cff3d ("net: stmmac: Fix incorrect location to set real_num_rx|tx_queues")
Reported-by: NChristophe ROULLIER <christophe.roullier@st.com>
Tested-by: NChristophe ROULLIER <christophe.roullier@st.com>
Cc: Alexandre TORGUE <alexandre.torgue@st.com>
Reviewed-by: NOng Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: NWong Vee Khee <vee.khee.wong@intel.com>
Link: https://lore.kernel.org/r/20201115074210.23605-1-vee.khee.wong@intel.comSigned-off-by: NJakub Kicinski <kuba@kernel.org>

This patch enables the HW LPI Timer which controls the automatic entry
and exit of the LPI state.
The EEE LPI timer value is configured through ethtool. The driver will
auto select the LPI HW timer if the value in the HW timer supported range.
Else, the driver will fallback to SW timer.
Signed-off-by: NVineetha G. Jaya Kumaran <vineetha.g.jaya.kumaran@intel.com>
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Link: https://lore.kernel.org/r/20201027160051.22898-1-weifeng.voon@intel.comSigned-off-by: NJakub Kicinski <kuba@kernel.org>

Commit 0366f7e0 ("net: stmmac: add ethtool support for get/set
channels") refactored channel initialization, but during that operation,
the spinlock initialization got lost. Fix this. This fixes the following
lockdep warning:

meson8b-dwmac ff3f0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
INFO: trying to register non-static key.
the code is fine but needs lockdep annotation.
turning off the locking correctness validator.
CPU: 1 PID: 331 Comm: kworker/1:2H Not tainted 5.9.0-rc3+ #1858
Hardware name: Hardkernel ODROID-N2 (DT)
Workqueue: kblockd blk_mq_run_work_fn
Call trace:
 dump_backtrace+0x0/0x1d0
 show_stack+0x14/0x20
 dump_stack+0xe8/0x154
 register_lock_class+0x58c/0x590
 __lock_acquire+0x7c/0x1790
 lock_acquire+0xf4/0x440
 _raw_spin_lock_irqsave+0x80/0xb0
 stmmac_tx_timer+0x4c/0xb0 [stmmac]
 call_timer_fn+0xc4/0x3e8
 run_timer_softirq+0x2b8/0x6c0
 efi_header_end+0x114/0x5f8
 irq_exit+0x104/0x110
 __handle_domain_irq+0x60/0xb8
 gic_handle_irq+0x58/0xb0
 el1_irq+0xbc/0x180
 _raw_spin_unlock_irqrestore+0x48/0x90
 mmc_blk_rw_wait+0x70/0x160
 mmc_blk_mq_issue_rq+0x510/0x830
 mmc_mq_queue_rq+0x13c/0x278
 blk_mq_dispatch_rq_list+0x2a0/0x698
 __blk_mq_do_dispatch_sched+0x254/0x288
 __blk_mq_sched_dispatch_requests+0x190/0x1d8
 blk_mq_sched_dispatch_requests+0x34/0x70
 __blk_mq_run_hw_queue+0xcc/0x148
 blk_mq_run_work_fn+0x20/0x28
 process_one_work+0x2a8/0x718
 worker_thread+0x48/0x460
 kthread+0x134/0x160
 ret_from_fork+0x10/0x1c

Fixes: 0366f7e0 ("net: stmmac: add ethtool support for get/set channels")
Signed-off-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Link: https://lore.kernel.org/r/20201029185011.4749-1-m.szyprowski@samsung.comSigned-off-by: NJakub Kicinski <kuba@kernel.org>

Ethtool manual stated that the tx-timer is the "the amount of time the
device should stay in idle mode prior to asserting its Tx LPI". The
previous implementation for "ethtool --set-eee tx-timer" sets the LPI TW
timer duration which is not correct. Hence, this patch fixes the
"ethtool --set-eee tx-timer" to configure the EEE LPI timer.

The LPI TW Timer will be using the defined default value instead of
"ethtool --set-eee tx-timer" which follows the EEE LS timer implementation.

Changelog V2
*Not removing/modifying the eee_timer.
*EEE LPI timer can be configured through ethtool and also the eee_timer
module param.
*EEE TW Timer will be configured with default value only, not able to be
configured through ethtool or module param. This follows the implementation
of the EEE LS Timer.

Fixes: d765955d ("stmmac: add the Energy Efficient Ethernet support")
Signed-off-by: NVineetha G. Jaya Kumaran <vineetha.g.jaya.kumaran@intel.com>
Signed-off-by: NVoon Weifeng <weifeng.voon@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>