Instead of allocating page fragments via the network stack,
use the page allocator directly. For now, we consume one page
for each Rx buffer.

With the new memory model we are free to consider adding more
XDP support.

Performance decreases slightly in some IP forwarding cases.
No visible effect on termination traffic. The driver memory
footprint increases as a result of this change, but it is
still small enough to not really matter.

Another side effect is that now Rx buffer alignment requirements
are naturally satisfied without any additional actions needed.
Remove alignment related code, except in the buffer layout
information conveyed to MC, as hardware still needs to know the
alignment value we guarantee.
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Export detailed driver counters through debugfs.

Statistics already available in ethtool are presented in a
structured manner. Includes per-core, per-FQ and per-channel statistics.

Also transition from module_fsl_mc_driver to explicit module_init/exit
in order to create the debugfs directory besides registering the driver.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In the current implementation, on interface down we disabled NAPI and
then manually drained any remaining ingress frames. This could lead
to a situation when, under heavy traffic, the data availability
notification for some of the channels would not get rearmed correctly.

Change the implementation such that we let all remaining ingress frames
be processed as usual and only disable NAPI once the hardware queues
are empty.

We also add a wait on the Tx side, to allow hardware time to process
all in-flight Tx frames before issueing the disable command.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add comments in the switch statement for XDP action to indicate
fallthrough is intended.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add counters for xdp processed frames to the channel statistics.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Reviewed-by: NDavid Ahern <dsahern@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Remove unused counter. Reorder fields in channel stats structure
to match the ethtool strings order and make it easier to print them
with ethtool -S.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Send frames back on the same port for XDP_TX action.
Since the frame buffers have been allocated by us, we can recycle
them directly into the Rx buffer pool instead of requesting a
confirmation frame upon transmission complete.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In order to support enqueueing Rx FDs back to hardware, we need to
DMA map Rx buffers as bidirectional.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Instead of freeing the RX buffers, release them back into the pool.
We wait for the maximum number of buffers supported by a single
release command to accumulate before issuing the command.

Also, don't unmap the Rx buffers at the beginning of the Rx routine
anymore, since that would require remapping them before release.
Instead, just do a DMA sync at first and only unmap if the frame is
meant for the stack.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We'll use function free_bufs() on the XDP path as well, so move
it higher in order to avoid a forward declaration.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Reserve XDP_PACKET_HEADROOM bytes in Rx buffers to allow XDP
programs to increase frame header size.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We keep one XDP program reference per channel. The only actions
supported for now are XDP_DROP and XDP_PASS.

Until now we didn't enforce a maximum size for Rx frames based
on MTU value. Change that, since for XDP mode we must ensure no
scatter-gather frames can be received.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Acked-by: NCamelia Groza <camelia.groza@nxp.com>
Reviewed-by: NDavid Ahern <dsahern@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add support for byte queue limit.

On NAPI poll, we save the total number of Tx confirmed frames/bytes
and register them with bql at the end of the poll function.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Change the frame consume callback signature:
* the entire FQ structure is passed to the callback instead
of just the queue index
* the NAPI structure can be easily obtained from the channel
it is associated to, so we don't need to pass it explicitly
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The DPNI object on which we build a network interface has a
certain number of {Rx, Tx, Tx confirmation} frame queues as
resources. The default hardware setup offers one queue of each
type, as well as one DPCON channel, for each core available
in the system.

There are however cases where the number of queues is greater
than the number of cores or channels. Until now, we configured
and used all the frame queues associated with a DPNI, even if it
meant assigning multiple queues of one type to the same channel.

Update the driver to only use a number of queues equal to the
number of channels, ensuring each channel will contain exactly
one Rx and one Tx confirmation queue.

>From the user viewpoint, this change is completely transparent.
Performance wise there is no impact in most scenarios. In case
the number of queues is larger than and not a multiple of the
number of channels, Rx hash distribution offers now better load
balancing between cores, which can have a positive impact on
overall system performance.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The fsl_mc_object_allocate function can fail because not all allocatable
objects are probed by the fsl_mc_allocator at the call time. Defer the
dpaa2-eth probe when this happens.
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Reviewed-by: NAndrew Lunn <andrew@lunn.ch>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

According to the hardware ArchDef, the PTV1 field in FD[CTRL]
is ignored by WRIOP, so setting it for Tx FDs is pointless.

Remove all references to it from the code.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The ch parameter is never used in the dpaa2_eth_tx_conf function but
since its prototype must match the type defined in the consume field of
struct dpaa2_eth_fq, just mark it as __always_unused.
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The priv parameter is never used in the build_linear_skb and
drain_channel function. Remove it from the function definitions.
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

All 3 cases of possible uninitialized variables are false
positives since they are used only as output parameters.
Nonetheless, fix the warnings.
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The dpaa2_eth_set_dist_key function is only used in a single file.
Make it static.
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Until now, both Rx and Tx confirmation frames handled during
NAPI poll were counted toward the NAPI budget. However, Tx
confirmations are lighter to process than Rx frames, which can
skew the amount of work actually done inside one NAPI cycle.

Update the code to only count Rx frames toward the NAPI budget
and set a separate threshold on how many Tx conf frames can be
processed in one poll cycle.

The NAPI poll routine stops when either the budget is consumed
by Rx frames or when Tx confirmation frames reach this threshold.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add support for inserting and deleting Rx flow classification
rules through ethtool.

We support classification based on some header fields for
flow-types ether, ip4, tcp4, udp4 and sctp4.

Rx queues are core affine, so the action argument effectively
selects on which cpu the matching frame will be processed.
Discarding the frame is also supported.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For firmware versions that support it, configure an Rx flow
classification key at probe time.

Hardware expects all rules in the classification table to share
the same key. So we setup a key containing all supported fields
at driver init and when a user adds classification rules through
ethtool, we will just mask out the unused header fields.

Since the key composition process is the same for flow
classification and hashing, reuse existing code where possible.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Since the array of supported header fields will be used for
Rx flow classification as well, rename it from "hash_fields" to
the more inclusive "dist_fields".
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The Management Complex (MC) firmware initially allowed the
configuration of a single key to be used both for Rx flow hashing
and flow classification. This prevented us from supporting
Rx flow classification through ethtool.

Starting with version 10.7.0, the Management Complex(MC) offers
a new set of APIs for separate configuration of Rx hashing and
classification keys.

Update the Rx flow hashing support to use the new API, if available.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Until now, the Rx flow hash key was a 5-tuple (IP src, IP dst,
IP nextproto, L4 src port, L4 dst port) fixed value that we
configured at probe.

Add support for configuring this hash key at runtime.
We support all standard header fields configurable through ethtool,
but cannot differentiate between flow types, so the same hash key
is applied regardless of protocol.

We also don't support the discard option.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The DPAA2 Ethernet driver supports Freescale/NXP SoCs with DPAA2
(DataPath Acceleration Architecture v2). The driver manages
network objects discovered on the fsl-mc bus.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Only call netdev_register() at the end of the probe function,
once all other necessary bits and pieces are properly initialized.

We keep the rest of the netdevice initialization code in place,
at the earlier point of the probing sequence, including the
settings previously done in ndo_init.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Suggested-by: NAndrew Lunn <andrew@lunn.ch>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The DPAA2 Ethernet driver files use a GPL-2.0+ OR BSD-3-Clause
license. Add SPDX tags and delete the full license text,
keeping the existing licenses for each file.
Add a GPL-2.0 tag for the Makefile.
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

We never really used the driver version, so no point
in keeping it around.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

In our documentation, we claim to use a 5-tuple key for Rx hash
distribution of flows. The code however configures a key composed
of all supported header fields.

Update the Rx hash key to contain only the documented fields:
{IP src, IP dst, IP nextproto, L4 src, L4 dst}, which was the
original intention and makes most sense as a default.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Previous commits:
commit 6e2387e8 ("staging: fsl-dpaa2/eth: Add Freescale DPAA2
Ethernet driver")
commit 39163c0c ("staging: fsl-dpaa2/eth: Errors checking update")

have added bits that are not specific to the WRIOP accelerator.

Move these where they belong (in DPIO) such that other accelerators
can make use of them.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Acked-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NLi Yang <leoyang.li@nxp.com>

There's no need to explicitly cast DPAA2_ETH_MFL to u16,
so remove it.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Suggested-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Most Ethernet drivers don't enforce the MTU value as upper limit
for ingress frames. We too support receiving frames larger than
MTU, so allow that.

Remove our ndo_change_mtu implementation, letting the default
stack implementation handle things. Also, set the max frame length
allowed by hardware only once at probe time, with the largest
possible value.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Don't set the lower MTU limit explicitly, since we use
the default value anyway.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

We don't need to call dev_set_drvdata(dev, NULL) on driver
remove since core kernel code also performs this step.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Commit 2b7c86eb ("staging: fsl-dpaa2/eth: Don't enable FAS on Tx")
removed the status field from the TX confirm frame annotation,
but a reference to it remained in the description of free_tx_fd().
Remove it.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

We are using DMA_FROM_DEVICE when mapping RX frame buffers,
but DMA_BIDIRECTIONAL for unmap. Fix the direction for DMA
unmapping operation.

Fixes: 87eb55e4 ("staging: fsl-dpaa2/eth: Fix potential endless loop")
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Hardware timestamping is supported both on Rx and Tx paths.
On Rx, timestamping is enabled for all frames. On Tx, we
only instruct the hardware to timestamp the frames marked
accordingly by the stack.
Signed-off-by: NIoana Radulescu <ruxandra.radulescu@nxp.com>
Signed-off-by: NYangbo Lu <yangbo.lu@nxp.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>