Added tracepoints in mailbox code so that
the mailbox operations like message allocation,
sending message and message interrupts are traced.
Also the mailbox errors occurred like timeout
or wrong responses are traced.
These will help in debugging mailbox issues.

Here's an example output showing one of the mailbox
messages sent by PF to AF and AF responding to it:

~# mount -t tracefs none /sys/kernel/tracing/
~# echo 1 > /sys/kernel/tracing/events/rvu/enable
~# ifconfig eth0 up
~# cat /sys/kernel/tracing/trace

~# cat /sys/kernel/tracing/trace
 tracer: nop

		      _-----=> irqs-off
		     / _----=> need-resched
		    | / _---=> hardirq/softirq
		    || / _--=> preempt-depth
		    ||| /     delay
   TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
      | |       |   ||||       |         |
ifconfig-2382  [002] ....   756.161892: otx2_msg_alloc: [0002:02:00.0] msg:(0x400) size:40

ifconfig-2382  [002] ...1   756.161895: otx2_msg_send: [0002:02:00.0] sent 1 msg(s) of size:48

 <idle>-0     [000] d.h1   756.161902: otx2_msg_interrupt: [0002:01:00.0] mbox interrupt PF(s) to AF (0x2)

kworker/u49:0-1165  [000] ....   756.162049: otx2_msg_process: [0002:01:00.0] msg:(0x400) error:0

kworker/u49:0-1165  [000] ...1   756.162051: otx2_msg_send: [0002:01:00.0] sent 1 msg(s) of size:32

kworker/u49:0-1165  [000] d.h.   756.162056: otx2_msg_interrupt: [0002:02:00.0] mbox interrupt AF to PF (0x1)
Signed-off-by: NSubbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Precision Timestamping block found on Octeontx2
platform is an independent coprocessor and has
internal PTP hardware clock. Once configured PTP
runs independently and when a packet arrives
CGX hardware block gets the current timestamp
from PTP block and forwards the packet to NIX
by prepending timestamp to the packet.
This patch adds the pci driver for PTP block.
The driver gets registered by AF driver and does
initial configuration and exposes a mailbox function to
read and adjust PTP hardware clock. The mailbox function
is called by AF consumers like netdev drivers or
userspace drivers. Since PTP being a single block
in platform this driver helps in accessing PTP
block by any AF consumer.
Co-developed-by: NSubbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: NSubbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: NAleksey Makarov <amakarov@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Acked-by: NRichard Cochran <richardcochran@gmail.com>
Acked-by: NJakub Kicinski <kuba@kernel.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Removed MODULE_VERSION and fixed MODULE_AUTHOR.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Reviewed-by: NLeon Romanovsky <leonro@mellanox.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Currently on the first check if the operation is still not
finished, the poll goes to sleep for 2-5 usecs. But if for
some reason (due to other priority stuff like interrupts etc) by
the time the poll wakes up the 10ms time is expired then we don't
check if operation is finished or not and return failure.

This patch modifies poll logic to check HW operation after sleep so
that the status is checked atleast twice.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Bus mastering is enabled by firmware, but when this driver
is unbinded bus mastering gets disabled by the PCI subsystem
which results interrupts not working when driver is reloaded.
Hence set bus mastering everytime in probe().

Also
- Converted pci_set_dma_mask() and pci_set_consistent_dma_mask()
  to dma_set_mask_and_coherent().
- Cleared transaction pending bit which gets set during
  driver unbind due to clearing of bus mastering (ME bit).
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Currently there is no way for AF dependent drivers in
any domain to check if the AF driver is loaded. This
patch sets an ID for RVUM block which will automatically
reflects in PF/VFs discovery register which they can
check and defer their probe until AF is up.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For retrieving info like interface MAC addresses, packet
parser key extraction config etc currently a command
is sent to firmware and firmware which periodically polls
for commands, processes these and returns the info.

This is resulting in interface initialization taking lot
of time. To optimize this a memory region is shared between
firmware and this driver, firmware while booting puts
static info like these into that region for driver to
read directly without using commands.

With this
- Logic for retrieving packet parser extraction config
  via commands is removed and repalced with using the
  shared 'fwdata' structure.
- Now RVU MSIX vector address is also retrieved from this fwdata struct
  instead of from CSR. Otherwise when kexec/kdump crash kernel loads
  CSR will have a IOVA setup by primary kernel which impacts
  RVU PF/VF's interrupts.
- Also added a mbox handler for PF/VF interfaces to retrieve their MAC
  addresses from AF.
Signed-off-by: NLinu Cherian <lcherian@marvell.com>
Signed-off-by: NChristina Jacob <cjacob@marvell.com>
Signed-off-by: NRakesh Babu <rsaladi2@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Traffic for a CGX mapped NIXLF can be stopped by disabling entries
in NPC MCAM or by configuring CGX and mailbox messages exist for the
two options. If traffic is stopped at CGX then VFs of that PF are
also effected hence CGX traffic should be started/stopped by
tracking all the users of it. This patch implements that CGX users
tracking. CGX is also configured along with NPC if required.

Also removed a check which mandates even number of LBK VFs.
Signed-off-by: NSubbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Ingress packet replication support has been added to 96xx B0
silicon. This patch enables using that feature to replicate
ingress broadcast packets to PF and it's VFs.

Also fixed below issues
- VFs can also install NPC MCAM entry to forward broadcast pkts.
  Otherwise, unless PF's interface is UP, VFs will not receive
  bcast packets.
- NPC MCAM entry is disabled when PF and all it's VFs are down.
- Few corner cases in installing multicast entry list.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

CN96xx initial silicon doesn't support all features pertaining to
NIX transmit scheduling and shaping.
- It supports a fixed topology of 1:1 mapped transmit
  limiters at all levels.
- Supports DWRR only at SMQ/MDQ and TL1.
- Doesn't support shaping and coloring.

This patch adds HW capability structure by which each variant
and skew of silicon can be differentiated by their supported
features. And adds support for A0 silicon's transmit scheduler
capabilities or rather limitations.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For every mailbox handler added to rvu, we are adding a function
declaration in rvu header file. Cleaned this up by adding a macro
to generate these declarations automatically.
Signed-off-by: NSubbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Added a new mailbox API which goes through all responses
to check their IDs and response codes.

Also added logic to prevent queuing multiple works to
process the same mailbox message. This scenario happens
when AF is processing a PF's request and menawhile PF
sends ACK to AF sent UP message, then mbox_hdr->num_msgs
in the PF->AF DOWN mbox region will be nonzero and AF
will end up processing PF's request again. This is fixed
by taking a backup of num_msgs counter and clearing the
same in the mbox region before scheduling work.
Signed-off-by: NSubbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

NDC is a data cache unit which caches NPA and NIX block's
aura/pool/RQ/SQ/CQ/etc contexts to reduce number of costly
DRAM accesses.

This patch adds support to dump cache's performance stats
like cache line hit/miss counters, average cycles taken for
accessing cached and non-cached data. This will help in
checking if NPA/NIX context reads/writes are having NDC cache
misses which inturn might effect performance.

Also changed NDC enums to reflect correct NDC hardware instance.
Signed-off-by: NPrakash Brahmajyosyula <bprakash@marvell.com>
Signed-off-by: NLinu Cherian <lcherian@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Added support to dump current resource provisioning status
of all resource virtualization unit (RVU) block's
(i.e NPA, NIX, SSO, SSOW, CPT, TIM) local functions attached
to a PF_FUNC into a debugfs file.

'cat /sys/kernel/debug/octeontx2/rsrc_alloc'
will show the current block LF's allocation status.
Signed-off-by: NChristina Jacob <cjacob@marvell.com>
Signed-off-by: NPrakash Brahmajyosyula <bprakash@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The following set of NPC registers allow the driver to configure NPC
to generate different key value schemes to compare against packet
payload in MCAM search.

NPC_AF_INTF(0..1)_KEX_CFG
NPC_AF_KEX_LDATA(0..1)_FLAGS_CFG
NPC_AF_INTF(0..1)_LID(0..7)_LT(0..15)_LD(0..1)_CFG
NPC_AF_INTF(0..1)_LDATA(0..1)_FLAGS(0..15)_CFG

Currently, the AF driver populates these registers to
configure the default values to address the most common
use cases such as key generation for channel number + DMAC.

The secure firmware stores different configuration
value of these registers to enable different NPC use case
along with the name for the lookup.

Patch loads profile binary from secure firmware over
the exiting CGX mailbox interface and apply the profile.

AF driver shall fall back to the default configuration
in case of any errors.

The AF consumer driver can know the selected profile
on response to NPC_GET_KEX_CFG mailbox by introducing
mkex_pfl_name in the struct npc_get_kex_cfg_rsp.
Signed-off-by: NVamsi Attunuru <vamsi.attunuru@marvell.com>
Signed-off-by: NJerin Jacob <jerinj@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

* Do CGX init before NIX init
  This would add consistency in NIX code that depends on cgx ports

* Few other misc cleanups
  - rvu_cgx_probe renamed as rvu_cgx_init for consistency
  - rvu_cgx_exit wrapper added to take care of the exit path
  - Added error check on cgx_lmac_event_handler_init
  - Minor cleanups in cgx.h related to tab alignment
  - Removed redundant ids from enum cgx_cmd_id
Signed-off-by: NLinu Cherian <lcherian@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

- Add interrupt handlers for Master Enable events from PFs
  and Master Enable events from VFs of AF
- Master Enable is required for the MSIX delivery to work
- Master Enable bit trap handler doesn't have to do any anything
  other than clearing the TRPEND bit, since the enable/disable
  requirements are already taken care using mbox requests/flr handler.
Signed-off-by: NLinu Cherian <lcherian@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Added support to handle FLR for AF's VFs (i.e LBK VFs).
Just the FLR interrupt enable/disable, handler registration
etc, actual HW resource cleanup or LFs teardown logic is
already there.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Enable all AF VFs during probe. Since AF's VFs work in pairs
(eg: Pkts sent on VF0 are received by VF1 and viceversa),
enable only even number of VFs out of totalVFs, which should
again be less than number of loopback (LBK) channels.

Also enable VF's mailbox interrupts.
Signed-off-by: NTomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

VFs attached to PFs other than AF can not communicate with AF
directly. Instead they are supposed to first send message to
the PF they are residing on and PF forwards it to the AF.
Responses to messages are handled in the reverse order.

On the other hand if VFs are on AF (PF0) itself then direct mailbox
communication is possible since there's no other PF in the way.

This patch addresses this particular case and adds support for
handling it.
Signed-off-by: NTomasz Duszynski <tduszynski@marvell.com>
Signed-off-by: NMarko Kallio <mkallio@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Upon receiving FLR IRQ for a RVU PF, teardown or cleanup
resources held by that PF_FUNC. This patch cleans up,
NIX LF
 - Stop ingress/egress traffic
 - Disable NPC MCAM entries being used.
 - Free Tx scheduler queues
 - Disable RQ/SQ/CQ HW contexts
NPA LF
 - Disable Pool/Aura HW contexts
In future teardown of SSO/SSOW/TIM/CPT will be added.

Also added a mailbox message for a RVU PF to request
AF, to perform FLR for a RVU VF under it.
Signed-off-by: NGeetha sowjanya <gakula@marvell.com>
Signed-off-by: NStanislaw Kardach <skardach@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

RVU admin function (AF) has all the priviliges to cleanup
HW state when VFIO triggers a PCIe function level reset (FLR)
due to either reset or a VM crash. FLR for RVU PF1-PFn will
trigger an IRQ to AF.

This patch enables all RVU PF's FLR interrupts and registers a
handler. Upon receiving an interrupt, a workqueue is scheduled
to cleanup all hardware blocks being used by the PF which
received the FLR.
Signed-off-by: NGeetha sowjanya <gakula@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

While mapping a NIX LF to a NPA LF attached PF_FUNC or
SSO LF attached PF_FUNC, verify if PF_FUNC is valid and
if that PF_FUNC has a LF of that block attached to it or not.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Mailbox message handling is done in a workqueue context scheduled
from interrupt handler. So resource locks does not need to be a spinlock.
Therefore relax them into a mutex so that later on we may use them
in routines that might sleep.
Signed-off-by: NStanislaw Kardach <skardach@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch converts all mailbox message handler API
names to lowercase.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch configures all 16 KPUs and iKPU (pkinds) with
the KPU parser profile defined in npc_profile.h. Each KPU
engine has a 128 entry CAM, only CAM entries which are listed
in the profile are enabled and rest are left disabled.

Also
- Memory is allocated for pkind's bitmap and PFFUNC, interface
  channel mapping.
- Added all CSR offsets of NPC HW block.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Added support for a PF/VF to allocate or free NIX transmit
scheduler queues via mbox. For setting up pkt transmission
priorities between queues, the scheduler queues have to be
contiguous w.r.t their HW indices. So both contiguous and
non-contiguous allocations are supported.

Upon receiving NIX_TXSCH_FREE mbox msg all scheduler queues
allocated to sending PFFUNC (PF/VF) will be freed. Selective
free is not supported.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NNithin Dabilpuram <ndabilpuram@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Fixes gcc '-Wunused-but-set-variable' warning:

drivers/net/ethernet/marvell/octeontx2/af/rvu.c: In function 'rvu_detach_rsrcs':
drivers/net/ethernet/marvell/octeontx2/af/rvu.c:855:6: warning:
 variable 'devnum' set but not used [-Wunused-but-set-variable]

drivers/net/ethernet/marvell/octeontx2/af/rvu.c:853:7: warning:
 variable 'is_pf' set but not used [-Wunused-but-set-variable]

drivers/net/ethernet/marvell/octeontx2/af/rvu.c: In function 'rvu_mbox_handler_ATTACH_RESOURCES':
drivers/net/ethernet/marvell/octeontx2/af/rvu.c:1054:7: warning:
 variable 'is_pf' set but not used [-Wunused-but-set-variable]

drivers/net/ethernet/marvell/octeontx2/af/rvu.c:1053:6: warning:
 variable 'devnum' set but not used [-Wunused-but-set-variable]

It never used since introduction in commit
746ea742 ("octeontx2-af: Add RVU block LF provisioning support")
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Initialize NIX admin queue (AQ) i.e alloc memory for
AQ instructions and for the results. All NIX LFs will submit
instructions to AQ to init/write/read RQ/SQ/CQ/RSS contexts
and in case of read, get context from result memory.

Also before configuring/using NIX block calibrate X2P bus
and check if NIX interfaces like CGX and LBK are in active
and working state.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Upon receiving NPA_LF_ALLOC mbox message allocate memory for
NPALF's aura, pool and qint contexts and configure the same
to HW. Enable caching of contexts into NPA NDC.

Return pool related info like stack size, num pointers per
stack page e.t.c to the mbox msg sender.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Initialize NPA admin queue (AQ) i.e alloc memory for
AQ instructions and for the results. All NPA LFs will submit
instructions to AQ to init/write/read Aura/Pool contexts
and in case of read, get context from result memory.

Added some common APIs for allocating memory for a queue
and get IOVA in return, these APIs will be used by
NIX AQ and for other purposes.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Upon receiving notification from firmware the CGX event handler
in the AF driver gets the current link info such as status, speed,
duplex etc from CGX driver and sends it across to PFs who have
registered to receive such notifications.

To support above
 - Mbox messaging support for sending msgs from AF to PF has been added.
 - Added mbox msgs so that PFs can register/unregister for link events.
 - Link notifications are sent to PF under two scenarioss.
  1. When a asynchronous link change notification is received from
     firmware with notification flag turned on for that PF.
  2. Upon notification turn on request, the current link status is
     send to the PF.

Also added a new mailbox msg using which RVU PF/VF can retrieve
their mapped CGX LMAC's current link info. Link info includes
status, speed, duplex and lmac type.
Signed-off-by: NLinu Cherian <lcherian@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Instead of looping on a integer timeout, use time_before(jiffies),
so that maximum poll time is capped.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Suggested-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Added support in RVU AF driver to register for
CGX LMAC link status change events from firmware
and managing them. Processing part will be added
in followup patches.

- Introduced eventqueue for posting events from cgx lmac.
  Queueing mechanism will ensure that events can be posted
  and firmware can be acked immediately and hence event
  reception and processing are decoupled.
- Events gets added to the queue by notification callback.
  Notification callback is expected to be atomic, since it
  is called from interrupt context.
- Events are dequeued and processed in a worker thread.
Signed-off-by: NLinu Cherian <lcherian@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Each of the enabled CGX LMAC is considered a physical
interface and RVU PFs are mapped to these. VFs of these
SRIOV PFs will be virtual interfaces and share CGX LMAC
along with PF.

This mapping info will be used later on for Rx/Tx pkt steering.
Signed-off-by: NLinu Cherian <lcherian@marvell.com>
Signed-off-by: NGeetha sowjanya <gakula@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds basic template for Marvell OcteonTX2's
CGX ethernet interface driver. Just the probe.
RVU AF driver will use APIs exported by this driver
for various things like PF to physical interface mapping,
loopback mode, interface stats etc. Hence marged both
drivers into a single module.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
create a IOMMU mapping for the physcial address configured by
firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.
Signed-off-by: NGeetha sowjanya <gakula@marvell.com>
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Firmware configures a certain number of MSIX vectors to each of
enabled RVU PF/VF. When a block LF is attached to a PF/VF, number
of MSIX vectors needed by that LF are set aside (out of PF/VF's
total MSIX vectors) and LF's msix_offset is configured in HW.

Also added support for a RVU PF/VF to retrieve that block LF's
MSIX vector offset information from AF via mbox.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Added support for a RVU PF/VF to request AF via mailbox
to attach or detach NPA/NIX/SSO/SSOW/TIM/CPT block LFs.
Also supports partial detachment and modifying current
LF attached count of a certian block type.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Scan all RVU blocks to find any 'LF to RVU PF/VF' mapping done by
low level firmware. If found any, mark them as used in respective
block's LF bitmap and also save mapped PF/VF's PF_FUNC info.

This is done to avoid reattaching a block LF to a different RVU PF/VF.
Signed-off-by: NSunil Goutham <sgoutham@marvell.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>