Implement support for transferring XDP meta data into skb for
nfp driver; before calling into the program, xdp.data_meta points
to xdp.data, where on program return with pass verdict, we call
into skb_metadata_set().
Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
Acked-by: NAlexei Starovoitov <ast@kernel.org>
Acked-by: NJohn Fastabend <john.fastabend@gmail.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This work enables generic transfer of metadata from XDP into skb. The
basic idea is that we can make use of the fact that the resulting skb
must be linear and already comes with a larger headroom for supporting
bpf_xdp_adjust_head(), which mangles xdp->data. Here, we base our work
on a similar principle and introduce a small helper bpf_xdp_adjust_meta()
for adjusting a new pointer called xdp->data_meta. Thus, the packet has
a flexible and programmable room for meta data, followed by the actual
packet data. struct xdp_buff is therefore laid out that we first point
to data_hard_start, then data_meta directly prepended to data followed
by data_end marking the end of packet. bpf_xdp_adjust_head() takes into
account whether we have meta data already prepended and if so, memmove()s
this along with the given offset provided there's enough room.

xdp->data_meta is optional and programs are not required to use it. The
rationale is that when we process the packet in XDP (e.g. as DoS filter),
we can push further meta data along with it for the XDP_PASS case, and
give the guarantee that a clsact ingress BPF program on the same device
can pick this up for further post-processing. Since we work with skb
there, we can also set skb->mark, skb->priority or other skb meta data
out of BPF, thus having this scratch space generic and programmable
allows for more flexibility than defining a direct 1:1 transfer of
potentially new XDP members into skb (it's also more efficient as we
don't need to initialize/handle each of such new members). The facility
also works together with GRO aggregation. The scratch space at the head
of the packet can be multiple of 4 byte up to 32 byte large. Drivers not
yet supporting xdp->data_meta can simply be set up with xdp->data_meta
as xdp->data + 1 as bpf_xdp_adjust_meta() will detect this and bail out,
such that the subsequent match against xdp->data for later access is
guaranteed to fail.

The verifier treats xdp->data_meta/xdp->data the same way as we treat
xdp->data/xdp->data_end pointer comparisons. The requirement for doing
the compare against xdp->data is that it hasn't been modified from it's
original address we got from ctx access. It may have a range marking
already from prior successful xdp->data/xdp->data_end pointer comparisons
though.
Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
Acked-by: NAlexei Starovoitov <ast@kernel.org>
Acked-by: NJohn Fastabend <john.fastabend@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The control process (NSP) may take some time to complete its
initialization.  This is not a problem on most servers, but
on very fast-booting machines it may not be ready for operation
when driver probes the device.  There is also a version of the
flash in the wild where NSP tries to train the links as part
of init.  To wait for NSP initialization we should make sure
its resource has already been added to the resource table.
NSP adds itself there as last step of init.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Board state informs us which low-level initialization stages the card
has completed.  We should wait for the card to be fully initialized
before trying to communicate with it, not only before we configure
passing traffic.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Previously we did not check the flow dissector against a list of allowed
and supported flow key dissectors. This patch introduces such a list and
correctly rejects unsupported flow keys.

Fixes: 43f84b72 ("nfp: add metadata to each flow offload")
Signed-off-by: NPieter Jansen van Vuuren <pieter.jansenvanvuuren@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When we moved to updating representors from a workqueue grabbing
the RTNL somehow got lost in the process.  Restore it, and make
sure RCU lock is not held while we are grabbing the RTNL.  RCU
protects the representor table, so since we will be under RTNL
we can drop RCU lock as soon as we find the netdev pointer.
RTNL is needed for the dev_set_mtu() call.

Fixes: 2dff1962 ("nfp: process MTU updates from firmware flower app")
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use dev_consume_skb_any() in place of dev_kfree_skb_any()
when control frame has been successfully processed in flower
and on the driver's main TX completion path.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Since representors are now created with a separate callback
start/stop app callbacks can be moved again to their original
location.  They are intended to app-specific init/clean up
over the control channel.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Create representors after lower vNIC is registered and destroy
them before it is destroyed.  Move the code out of start/stop
callbacks directly into vnic_init/clean callbacks.  Make sure
SR-IOV callbacks don't try to create representors when lower
device does not exist.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We currently only have one app callback for vNIC creation
and destruction.  This is insufficient, because some actions
have to be taken before netdev is registered, after it's
registered and after it's unregistered.  Old callbacks
were really corresponding to alloc/free actions.  Rename
them and add proper init/clean.  Apps using representors
will be able to use new callbacks to manage lifetime of
upper devices.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Both the nfp_net_pf_app_start() and the nfp_net_pci_probe() functions
call nfp_net_pf_app_stop_ctrl(pf) so there is a double free.  The free
should be done from the probe function because it's allocated there so
I have removed the call from nfp_net_pf_app_start().

Fixes: 02082701 ("nfp: create control vNICs and wire up rx/tx")
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Acked-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Previously the vlan tci field was incorrectly exact matched. This patch
fixes this by using the flow dissector to populate the vlan tci field.

Fixes: 5571e8c9 ("nfp: extend flower matching capabilities")
Signed-off-by: NPieter Jansen van Vuuren <pieter.jansenvanvuuren@netronome.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Previously when calculating the supported key layers MPLS, IPv4/6
TTL and TOS were not considered. This patch checks that the TTL and
TOS fields are masked out before offloading. Additionally this patch
checks that MPLS packets are correctly handled, by not offloading them.

Fixes: af9d842c ("nfp: extend flower add flow offload")
Signed-off-by: NPieter Jansen van Vuuren <pieter.jansenvanvuuren@netronome.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Previously flow dissectors were referenced without first checking that
they are in use and correctly populated by TC. This patch fixes this by
checking each flow dissector key before referencing them.

Fixes: 5571e8c9 ("nfp: extend flower matching capabilities")
Signed-off-by: NPieter Jansen van Vuuren <pieter.jansenvanvuuren@netronome.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add basic ndo_set/get_vf to support SR-IOV on all types
of port representors.
Signed-off-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDirk van der Merwe <dirk.vandermerwe@netronome.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add basic ndo_set/get_vf to support SR-IOV.

VF to egress phy static mapping by now.

Use vfcfg ABI version 2 to write the info to the FW and collect
the return value from the mailbox.
Signed-off-by: NPablo Cascón <pablo.cascon@netronome.com>
Signed-off-by: NJimmy Kizito <jimmy.kizito@netronome.com>
Signed-off-by: NRami Tomer <rami.tomer@netronome.com>
Signed-off-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

TX completion may happen any time after HW queue was kicked.
We can't access the skb afterwards.  Move the time stamping
before ringing the doorbell.

Fixes: 4c352362 ("net: add driver for Netronome NFP4000/NFP6000 NIC VFs")
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When driver receives a muxed frame, but it can't find the representor
netdev it is destined to it will try to "drop" that frame, i.e. reuse
the buffer.  The issue is that the replacement buffer has already been
allocated at this point, and reusing the buffer from received frame
will leak it.  Change the code to put the new buffer on the ring
earlier and not reuse the old buffer (make the buffer parameter
to nfp_net_rx_drop() a NULL).

Fixes: 91bf82ca ("nfp: add support for tx/rx with metadata portid")
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

App start/stop callbacks can perform application initialization.
Unfortunately, flower app started using them for creating and
destroying representors.  This can lead to a situation where
lower vNIC netdev is destroyed while representors still try
to pass traffic.  This will most likely lead to a NULL-dereference
on the lower netdev TX path.

Move the start/stop callbacks, so that representors are created/
destroyed when vNICs are fully initialized.

Fixes: 5de73ee4 ("nfp: general representor implementation")
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Enabling SR-IOV VFs will cause the PCI subsystem to schedule a
work and flush its workqueue.  Since the nfp driver schedules its
own work we can't enable VFs while holding driver load.  Commit
6d48ceb2 ("nfp: allocate a private workqueue for driver work")
tried to avoid this deadlock by creating a separate workqueue.
Unfortunately, due to the architecture of workqueue subsystem this
does not guarantee a separate thread of execution.  Luckily
we can simply take pci_enable_sriov() from under the driver lock.

Take pci_disable_sriov() from under the lock too for symmetry.

Fixes: 6d48ceb2 ("nfp: allocate a private workqueue for driver work")
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We were reusing skb pointer when reading page frag, since ring
entries contain a union of a skb and frag pointer.  This can
be confusing to people reading the code.  Refactor the code
to read frag pointer directly.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Data and control vNICs currently use the same area name and
error message.  This could lead to confusion.  Make sure
the error message says "ctrl" in case of control and the
data area is called "nfp.bar0".
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Representors may be associated with both VFs or more importantly
with physical ports.  Allow vNIC and MAC statistics to be read
with ethtool -S on representors.  In case of vNICs we reuse
the vNIC statistic helper, we just need to swap RX and TX to
give statistics the "switch perspective."
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Simplify the statistics handling code by keeping pointer to vNIC's
config memory in nfp_port.  Note that this is referring to the
representor side of vNICs, vNIC side has the pointer in nfp_net.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add reporting of MAC statistics in ethtool.  MAC statistics
are read out from the MAC IP and accumulated by application
FW, therefore their presence depends on the application FW.

Add missing defines and string names for the statistics and
dump them in ethtool -S.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Store pointer to device memory containing MAC statistics
in nfp_port.  This simplifies representor code and will
be used to dump those statistics in ethtool as well.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In preparation for reporting vNIC HW stats on representors
split handling of the SW and HW stats in ethtool -S.
Representors don't have SW stats (since vNIC is assigned
to the VM).

Remove the questionable defines which assume nn variable
exists in the scope.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add a helper for printing ethtool strings and advancing the
pointer correctly.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We have been recently called out as a bad example for reporting
standard netdev statistics as part of ethtool.  Fix that :)

Removing standard statistics allows us to simplify the structure
holding definitions since we no longer have to mux different types
of statistics.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Users should be able to dump the management FW logs on any
of the driver's netdevs.  Make the code only depend on the
nfp_app and share it between vNICs and representors.

Storing the dump flag is simply dropped for now, since we
only support the argument being set to 0.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Extend representors' ethtool ops to show basic info like firmware
version, driver version, and driver name.

While at it don't set drvinfo.n_stats and drvinfo.regdump_len,
core will invoke appropriate handlers to get those.

A helper is added to turn a netdev into nfp_app for convenience.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Start linking ethtool ops to representors.  Begin by adding
a separate ops structure and providing link state.  Next
patches will convert appropriate functions to only use nfp_port,
which will make them reusable on representors.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: NSimon Horman <simon.horman@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The offending commit used a newly added helper function.
But the logic is wrong. Without this fix, the affected NICs
can't do HW offload. Error -EOPNOTSUPP will be returned directly.

Fixes: a2e8da93 ("net/sched: use newly added classid identity helpers")
Signed-off-by: NChris Mi <chrism@mellanox.com>
Acked-by: NJiri Pirko <jiri@mellanox.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The while loop that performs the dma page unmapping never decrements
index counter f and hence loops forever. Fix this with a pre-decrement
on f.

Detected by CoverityScan, CID#1357309 ("Infinite loop")

Fixes: 4c352362 ("net: add driver for Netronome NFP4000/NFP6000 NIC VFs")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Acked-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Now that control message processing occurs in a workqueue rather than a BH
handler MTU updates received from the firmware may be safely processed.
Signed-off-by: NSimon Horman <simon.horman@netronome.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Processing of control messages is not time-critical and future processing
of some messages will require taking the RTNL which is not possible
in a BH handler. It seems simplest to move all control message processing
to a workqueue.
Signed-off-by: NSimon Horman <simon.horman@netronome.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The firmware expects a MAC_REPR control message when a MAC representor
is created. The driver should expect a PORTMOD message to follow which
will provide the link states of the physical port associated with the MAC
representor.
Signed-off-by: NSimon Horman <simon.horman@netronome.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The Flower app may receive a request to update the MTU of a representor
netdev upon receipt of a control message from the firmware. This requires
the RTNL lock which needs to be taken outside of the packet processing
path.

As a handling of this correctly seems a little to invasive for a fix simply
skip setting the MTU for now.

Relevant backtrace:
 [ 1496.288489] BUG: scheduling while atomic: kworker/0:3/373/0x00000100
 [ 1496.294911]  dca syscopyarea sysfillrect sysimgblt fb_sys_fops ptp drm mxm_wmi ahci pps_core libahci i2c_algo_bit wmi [last unloaded: nfp]
 [ 1496.294918] CPU: 0 PID: 373 Comm: kworker/0:3 Tainted: G           OE   4.13.0-rc3+ #3
 [ 1496.294919] Hardware name: Supermicro X10DRi/X10DRi, BIOS 2.0 12/28/2015
 [ 1496.294923] Workqueue: events work_for_cpu_fn
 [ 1496.294924] Call Trace:
 [ 1496.294927]  <IRQ>
 [ 1496.294931]  dump_stack+0x63/0x82
 [ 1496.294935]  __schedule_bug+0x54/0x70
 [ 1496.294937]  __schedule+0x62f/0x890
 [ 1496.294941]  ? intel_unmap_sg+0x90/0x90
 [ 1496.294942]  schedule+0x36/0x80
 [ 1496.294943]  schedule_preempt_disabled+0xe/0x10
 [ 1496.294945]  __mutex_lock.isra.2+0x445/0x4a0
 [ 1496.294947]  ? device_is_rmrr_locked+0x12/0x50
 [ 1496.294950]  ? kfree+0x162/0x170
 [ 1496.294952]  ? device_is_rmrr_locked+0x12/0x50
 [ 1496.294953]  ? iommu_should_identity_map+0x50/0xe0
 [ 1496.294954]  __mutex_lock_slowpath+0x13/0x20
 [ 1496.294955]  ? iommu_no_mapping+0x48/0xd0
 [ 1496.294956]  ? __mutex_lock_slowpath+0x13/0x20
 [ 1496.294957]  mutex_lock+0x2f/0x40
 [ 1496.294960]  rtnl_lock+0x15/0x20
 [ 1496.294979]  nfp_flower_cmsg_rx+0xc8/0x150 [nfp]
 [ 1496.294986]  nfp_ctrl_poll+0x286/0x350 [nfp]
 [ 1496.294989]  tasklet_action+0xf6/0x110
 [ 1496.294992]  __do_softirq+0xed/0x278
 [ 1496.294993]  irq_exit+0xb6/0xc0
 [ 1496.294994]  do_IRQ+0x4f/0xd0
 [ 1496.294996]  common_interrupt+0x89/0x89

Fixes: 948faa46 ("nfp: add support for control messages for flower app")
Signed-off-by: NSimon Horman <simon.horman@netronome.com>
Reviewed-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Instead of checking handle, which does not have the inner class
information and drivers wrongly assume clsact->egress as ingress, use
the newly introduced classid identification helpers.
Signed-off-by: NJiri Pirko <jiri@mellanox.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This work implements jiting of BPF_J{LT,LE} instructions with
BPF_X/BPF_K variants for the nfp eBPF JIT. The two BPF_J{SLT,SLE}
instructions have not been added yet given BPF_J{SGT,SGE} are
not supported yet either.
Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
Acked-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>