Get the link/pci speed query and logic into a single function.
Unify the heuristics and use a single PCI threshold (16G) for all.
Signed-off-by: NTariq Toukan <tariqt@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

We support outer VLAN offload in driver and HW regardless of whether
an encapsulation is present in the next headers.

Exposing this in hw_enc_features will allow us to offload outer VLANs
in cases where encapsulation protocols like VXLAN and IPsec are used.
Signed-off-by: NAviv Heller <avivh@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Add a new macro to prevent copy-pasting the same code for each new
feature.
Signed-off-by: NGal Pressman <galp@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

The function is used in en_main.c only, we can make it static and remove
its declaration from en.h
Signed-off-by: NGal Pressman <galp@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

ethtool core code makes sure data isn't NULL before calling
get_ethtool_stats, testing it again in the driver is redundant.
Signed-off-by: NGal Pressman <galp@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

The system with CONFIG_UBSAN enabled on produces the following error
during driver initialization. The reason to it that max_reg_cmds can be
larger enough to cause to "1 << max_reg_cmds" overflow the unsigned long.

================================================================================
UBSAN: Undefined behaviour in drivers/net/ethernet/mellanox/mlx5/core/cmd.c:1805:42
signed integer overflow:
-2147483648 - 1 cannot be represented in type 'int'
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.15.0-rc2-00032-g06cda2358d9b-dirty #724
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
Call Trace:
 dump_stack+0xe9/0x18f
 ? dma_virt_alloc+0x81/0x81
 ubsan_epilogue+0xe/0x4e
 handle_overflow+0x187/0x20c
 mlx5_cmd_init+0x73a/0x12b0
 mlx5_load_one+0x1c3d/0x1d30
 init_one+0xd02/0xf10
 pci_device_probe+0x26c/0x3b0
 driver_probe_device+0x622/0xb40
 __driver_attach+0x175/0x1b0
 bus_for_each_dev+0xef/0x190
 bus_add_driver+0x2db/0x490
 driver_register+0x16b/0x1e0
 __pci_register_driver+0x177/0x1b0
 init+0x6d/0x92
 do_one_initcall+0x15b/0x270
 kernel_init_freeable+0x2d8/0x3d0
 kernel_init+0x14/0x190
 ret_from_fork+0x24/0x30
================================================================================
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Currently, we are emulating the offload of vlan push/pop actions using
global setup as done by commit f5f82476 ("net/mlx5: E-Switch, Support
VLAN actions in the offloads mode"). With newer NICs, we can apply a flow
action for that matter, do that while keeping the emulated path for the
older HW brands.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Reviewed-by: NMark Bloch <markb@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Newer NICs (ConnectX-5 and onward) can apply vlan pop or push as an
action taking place during flow steering. Add the core bits for that.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Reviewed-by: NMark Bloch <markb@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Align the checks for modify header and encap actions with the
rest of the code.
Signed-off-by: NOr Gerlitz <ogerlitz@mellanox.com>
Reviewed-by: NMark Bloch <markb@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Added the following packets drop counter:
Rx interface down dropped packets - counts packets which were received
while the ETH interface was down.
This counter will be shown on ethtool as a new counter called
rx_if_down_packets.

The implementation allocates a q_counter for drop rq which gets all the
received traffic while the interface is down.
Signed-off-by: NMoshe Shemesh <moshe@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Added the following packets dropped while vport down statistics:

Rx dropped while vport down - counts packets which were steered by
e-switch to a vport, but dropped since the vport was down. This counter
will be shown on ip link tool as part of the vport rx_dropped counter.

Tx dropped while vport down - counts packets which were transmitted by
a vport, but dropped due to vport logical link down. This counter
will be shown on ip link tool as part of the vport tx_dropped counter.

The counters are read from FW by command QUERY_VNIC_ENV.
Signed-off-by: NMoshe Shemesh <moshe@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Added the following packets drop counter:
Rx steering missed dropped packets - counts packets which were dropped
due to miss on NIC rx steering rules.
This counter will be shown on ethtool as a new counter called
rx_steer_missed_packets.
Signed-off-by: NMoshe Shemesh <moshe@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Add support for new FW command QUERY_VNIC_ENV.
The command is used by the driver to query vnic diagnostic statistics
from FW.
Signed-off-by: NMoshe Shemesh <moshe@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Implement set/get functions to configure PFC stall prevention
timeout by tunables api through ethtool.
By default the stall prevention timeout is configured to 8 sec.
Timeout range is: 80-8000 msec.

Enabling stall prevention with the auto timeout will set
the timeout to 100 msec.
Signed-off-by: NInbar Karmy <inbark@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Add the needed capability bit and counters to device spec description.
Expose the following two counters in ethtool:

tx_pause_storm_warning_events: when the device is stalled for a period
longer than a pre-configured watermark, the counter increase, allowing
the debug utility an insight into current device status.

tx_pause_storm_error_events: when the device is stalled for a period
longer than a pre-configured timeout, the pause transmission is disabled,
and the counter increase.
Signed-off-by: NInbar Karmy <inbark@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

_rule_ is being freed and then dereferenced by accessing rule->ctx

Fix this by copying the value returned by PTR_ERR(rule->ctx) into a local
variable for its safe use after freeing _rule_

Addresses-Coverity-ID: 1466041 ("Read from pointer after free")
Fixes: 05564d0a ("net/mlx5: Add flow-steering commands for FPGA IPSec implementation")
Reviewed-by: NYuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>
Acked-by: NSaeed Mahameed <saeedm@mellanox.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The mlx5 driver calls ida_pre_get() in a loop for no readily apparent
reason.  The driver uses ida_simple_get() which will call ida_pre_get()
by itself and there's no need to use ida_pre_get() unless using
ida_get_new().
Signed-off-by: NMatthew Wilcox <mawilcox@microsoft.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The kernel compiled with CONFIG_REFCOUNT_FULL produces the following
error. The reason to it that initial value of refcount_t is supposed
to be more than 0, change it.

[    3.106634] ------------[ cut here ]------------
[    3.107756] refcount_t: increment on 0; use-after-free.
[    3.109130] WARNING: CPU: 0 PID: 1 at lib/refcount.c:153 refcount_inc+0x27/0x30
[    3.110085] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.16.0-rc1-00028-gf683e04bdccc #137
[    3.110085] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
[    3.110085] RIP: 0010:refcount_inc+0x27/0x30
[    3.110085] RSP: 0000:ffffaa620000fba0 EFLAGS: 00010286
[    3.110085] RAX: 0000000000000000 RBX: ffff9a6d1a1821c8 RCX: ffffffff98a50f48
[    3.110085] RDX: 0000000000000001 RSI: 0000000000000086 RDI: 0000000000000246
[    3.110085] RBP: ffff9a6d1ac800a0 R08: 0000000000000289 R09: 000000000000000a
[    3.110085] R10: fffff03bc0682840 R11: ffffffff9949856d R12: ffff9a6d1b4a4000
[    3.110085] R13: 0000000000000000 R14: ffff9a6d1a0a6c00 R15: ffffaa620000fc5c
[    3.110085] FS:  0000000000000000(0000) GS:ffff9a6d1fc00000(0000) knlGS:0000000000000000
[    3.110085] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    3.110085] CR2: 0000000000000000 CR3: 000000000ba0a000 CR4: 00000000000006b0
[    3.110085] Call Trace:
[    3.110085]  mlx5_core_create_cq+0xde/0x250
[    3.110085]  ? __kmalloc+0x1ce/0x1e0
[    3.110085]  mlx5e_create_cq+0x15c/0x1e0
[    3.110085]  mlx5e_open_drop_rq+0xea/0x190
[    3.110085]  mlx5e_attach_netdev+0x53/0x140
[    3.110085]  mlx5e_attach+0x3d/0x60
[    3.110085]  mlx5e_add+0x11d/0x2f0
[    3.110085]  mlx5_add_device+0x77/0x170
[    3.110085]  mlx5_register_interface+0x74/0xc0
[    3.110085]  ? set_debug_rodata+0x11/0x11
[    3.110085]  init+0x67/0x72
[    3.110085]  ? mlx4_en_init_ptys2ethtool_map+0x346/0x346
[    3.110085]  do_one_initcall+0x98/0x147
[    3.110085]  ? set_debug_rodata+0x11/0x11
[    3.110085]  kernel_init_freeable+0x164/0x1e0
[    3.110085]  ? rest_init+0xb0/0xb0
[    3.110085]  kernel_init+0xa/0x100
[    3.110085]  ret_from_fork+0x35/0x40
[    3.110085] Code: 00 00 00 00 e8 ab ff ff ff 84 c0 74 02 f3 c3 80 3d 3b c3 64 01 00 75 f5 48 c7 c7 68 0b 81 98 c6 05 2b c3 64 01 01 e8 79 d7 a3 ff <0f> ff c3 66 0f 1f 44 00 00 8b 06 83 f8 ff 74 39 31 c9 39 f8 89
[    3.110085] ---[ end trace a0068e1c68438a74 ]---

Fixes: f105b45b ("net/mlx5: CQ hold/put API")
Signed-off-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Currently ESN is not supported with IPSec device offload.

This patch adds ESN support to IPsec device offload.
Implementing new xfrm device operation to synchronize offloading device
ESN with xfrm received SN. New QP command to update SA state at the
following:

           ESN 1                    ESN 2                  ESN 3
|-----------*-----------|-----------*-----------|-----------*
^           ^           ^           ^           ^           ^

^ - marks where QP command invoked to update the SA ESN state
    machine.
| - marks the start of the ESN scope (0-2^32-1). At this point move SA
    ESN overlap bit to zero and increment ESN.
* - marks the middle of the ESN scope (2^31). At this point move SA
    ESN overlap bit to one.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NYossef Efraim <yossefe@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

New function for getting driver internal sa entry from xfrm state.
All checks are done in one function.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

In order to add a context to the FPGA, we need to get both the software
transform context (which includes the keys, etc) and the
source/destination IPs (which are included in the steering
rule). Therefore, we register new set of firmware like commands for
the FPGA. Each time a rule is added, the steering core infrastructure
calls the FPGA command layer. If the rule is intended for the FPGA,
it combines the IPs information with the software transformation
context and creates the respective hardware transform.
Afterwards, it calls the standard steering command layer.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

The current code has one layer that executed FPGA commands and
the Ethernet part directly used this code. Since downstream patches
introduces support for IPSec in mlx5_ib, we need to provide some
abstractions. This patch refactors the accel code into one layer
that creates a software IPSec transformation and another one which
creates the actual hardware context.
The internal command implementation is now hidden in the FPGA
core layer. The code also adds the ability to share FPGA hardware
contexts. If two contexts are the same, only a reference count
is taken.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Currently our device requires additional metadata in packet
to perform ipsec crypto offload.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

We will need that for ipsec verbs.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

This patch adds V2 command support.
New fpga devices support extended features (udp encap, esn etc...), this
features require new hardware sadb format therefore we have a new version
of commands to manipulate it.
Signed-off-by: NYossef Efraim <yossefe@mellanox.com>
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Current hardware decrypts and authenticates incoming ESP packets.
Subsequently, the software extracts the nexthdr field, truncates the
trailer and adjusts csum accordingly.

With this patch and a capable device, the trailer is being removed
by the hardware and the nexthdr field is conveyed via PET. This way
we avoid both the need to access the trailer (cache miss) and to
compute its relative checksum, which significantly improve
the performance.

Experiment shows that trailer removal improves the performance by
2Gbps, (netperf). Both forwarding and host-to-host configurations.
Signed-off-by: NYossi Kuperman <yossiku@mellanox.com>
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

The current code assume only SA QP commands.
Refactor in order to pave the way for new QP commands:
1. Generic cmd response format.
2. SA cmd checks are in dedicated functions.
3. Aligned debug prints.
Signed-off-by: NYossi Kuperman <yossiku@mellanox.com>
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Fix build break of mlx5_accel_ipsec_device_caps is not defined when
MLX5_ACCEL is not selected, use MLX5_IPSEC_DEV instead which handles
such case.
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>
Reported-by: NDoug Ledford <dledford@redhat.com>

Previously, deleting a flow steering entry only got the index.
Since the FPGA implementation of FTE's deletion might need to dig
inside the FTE itself, we would like to get the FTE's context.
Changing the interface to pass the FTE context.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

fte objects contain the match value and action. Currently, extending
the actions require in adding them both to the API and fs_fte.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Currently, we don't support egress flow steering namespace in mlx5
flow steering core implementation. However, when we want to encrypt
a packet, we model it as a flow steering rule in the egress path.
To overcome this, we add an empty egress namespace to flow steering.
This namespace is initialized only when ipsec support exists.
In the future, this will grow to a full blown full steering
implementation, resembling the ingress path.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

The shim layer allows each namespace to define possibly different
functionality for add/delete/update commands. The shim layer
introduced here, will be used to support flow steering with the FPGA.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NBoris Pismenny <borisp@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

The has_tag member will indicate whether a tag action was specified
in flow specification.

A flow tag 0 = MLX5_FS_DEFAULT_FLOW_TAG is assumed a valid flow tag
that is currently used by mlx5 RDMA driver, whereas in HW flow_tag = 0
means that the user doesn't care about flow_tag.  HW always provide
a flow_tag = 0 if all flow tags requested on a specific flow are 0.

So we need a way (in the driver) to differentiate between a user really
requesting flow_tag = 0 and a user who does not care, in order to be
able to report conflicting flow tags on a specific flow.
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Reviewed-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Some flow steering namespace initialization (i.e. egress namespace)
might depend on FPGA capabilities. Changing the initialization order
such that the FPGA will be initialized before flow steering.

Flow steering fs cmds initialization might depend on
IPSec capabilities. Changing the initialization order such
that the IPSec will be initialized before flow steering as well.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NMatan Barak <matanb@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

This is already done by xfrm layer between state_dev_del callback
to state_dev_free callback.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

We can't allocate with GFP_KERNEL inside spinlock.
Actually ida_simple doesn't require spinlock so remove it.

Fixes: 547eede0 ("net/mlx5e: IPSec, Innova IPSec offload infrastructure")
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

Generally, FPGA IPSec commands must always complete.
We want to wait for one minute for them to complete gracefully also
when killing a process.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

IPSec init and cleanup functions also depends on linux/mlx5/driver.h.
Signed-off-by: NAviad Yehezkel <aviadye@mellanox.com>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>

All other mlx5_events report the port number as 1 based, which is how FW
reports it in the port event EQE. Reporting 0 for this event causes
mlx5_ib to not raise a fatal event notification to registered clients
due to a seemingly invalid port.

All switch cases in mlx5_ib_event that go through the port check are
supposed to set the port now, so just do it once at variable
declaration.

Fixes: 89d44f0a("net/mlx5_core: Add pci error handlers to mlx5_core driver")
Reviewed-by: NMajd Dibbiny <majd@mellanox.com>
Signed-off-by: NDaniel Jurgens <danielj@mellanox.com>
Signed-off-by: NLeon Romanovsky <leon@kernel.org>
Signed-off-by: NJason Gunthorpe <jgg@mellanox.com>

Up until this point it wasn't possible to activate IB representors
when switching to switchdev mode, remove this limitation.

We trigger reload of the PF IB interface in order to make sure that
already allocated resources are invalid and new resources will be opened
correctly with all the limitations of switchdev mode applied (only raw
packet capabilities, without RoCE). We also move the remove/add to a
place where the E-Switch mode is set/unset to better control when to
trigger this action, this will allow the IB side to start in the correct
mode.

For better code reuse, create a function which reloads an interface and
export it.
Signed-off-by: NMark Bloch <markb@mellanox.com>
Reviewed-by: NOr Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NLeon Romanovsky <leon@kernel.org>
Signed-off-by: NSaeed Mahameed <saeedm@mellanox.com>