Add support to thunderx to report bpf_prog ID during XDP_QUERY_PROG.
Signed-off-by: NMartin KaFai Lau <kafai@fb.com>
Cc: Sunil Goutham <sgoutham@cavium.com>
Acked-by: NAlexei Starovoitov <ast@fb.com>
Acked-by: NDaniel Borkmann <daniel@iogearbox.net>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Driver follows a method of taking one extra reference on the
page for recycling which is fine in usual packet path where
each 64KB page is segmented into multiple receive buffers.

But in XDP mode since there is just one receive buffer per
page taking extra page reference itself becomes big bottleneck
consuming ~50% of CPU cycles due to atomic operations.

This patch adds a internal ref count in pgcache for each
page and additional page references are taken in a batch
instead of just one at a time. Internal i.e 'pgcache->ref_count'
and page's i.e 'page->_refcount' counters are compared to check
page's recyclability.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When in XDP mode reserve XDP_PACKET_HEADROOM bytes at the start
of receive buffer for XDP program to modify headers and adjust
packet start. Additional code changes done to handle such packets.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds support for XDP_TX i.e transmits packet out of
the XDP TX queue mapped to the corresponding Rx queue
on which packet is received.

Since SQ for XDP TX will be used only on a single cpu i.e
SQ description creation and freeing, using atomic free count
is not necessary and will become a bottleneck. Hence added
a separate 'xdp_free_cnt' used for SQs designated for XDP
to track descriptor free count.

Changes also include
- A new entry 'xdp_page' is added to save transmitted packet's
  page pointer for later cleanup.
- XDP Tx SQ's doorbell is ringed once per NAPI instance.
- Retrieving designated SQ for packets being sent out by stack
  via 'nicvf_xmit'.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds support for XDP_DROP.
Also since in XDP mode there is just a single buffer per page,
made changes to recycle DMA mapping info as well along with pages.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds basic XDP support i.e attaching a BPF program to an
interface. Also takes care of allocating separate Tx queues
for XDP path and for network stack packet transmission.

This patch doesn't support handling of any of the XDP actions,
all are treated as XDP_PASS i.e packets will be handed over to
the network stack.

Changes also involve allocating one receive buffer per page in XDP
mode and multiple in normal mode i.e when no BPF program is attached.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Get rid of unnecessary double pointer references and type casting
in receive buffer allocation code.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Optimized CQE handling with below changes
- Feeing descriptors back to SQ in bulk i.e once per NAPI
  instance instead for every CQE_TX, this will reduce number
  of atomic updates to 'sq->free_cnt'.
- Checking errors in CQE_TX and CQE_RX before calling appropriate
  fn()s to update error stats i.e reduce branching.

Also removed debug messages in packet handling path which otherwise
causes issues if DEBUG is enabled.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Receive buffer's physical address or iova will anyway not
go beyond 49bits, since it is the max supported HW address.
As per perf, updating bitfields i.e buf_addr:42 in RBDR
descriptor entry consumes lots of cpu cycles, hence changed
it to a 64bit field with alignment requirements taken care of.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds support for page recycling for allocating receive buffers
to reduce cost of refilling RBDR ring. Also got rid of using
compound pages when pagesize is 4K, only order-0 pages now.

Only page is recycled, DMA mappings still needs to be done for
every receive buffer allocated due to following constraints
- Cannot have just one receive buffer per 64KB page.
- There is just one buffer ring shared across 8 Rx queues, so
  buffers of same page can go to any Rx queue.
- HW gives buffer address where packet has been DMA'ed and not
  the index into buffer ring.
This makes it not possible to resue DMA mapping info. So unfortunately
have to go through costly mapping route for every buffer.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add the PCI_SUBSYS_DEVID_81XX_RGX and use the same to set
the max bgx per node count.

This fixes the issue intoduced by following commit
78aacb6f net: thunderx: Fix invalid mac addresses for node1 interfaces
With this commit the max_bgx_per_node for 81xx is set as 2 instead of 3
because of which num_vfs is always calculated as zero.
Signed-off-by: NGeorge Cherian <george.cherian@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Remove deprecated pci_enable_msix API in favour of its
successor pci_alloc_irq_vectors.
Signed-off-by: NThanneeru Srinivasulu <tsrinivasulu@cavium.com>
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adding support for TSO and checksum hardware offloads for ipv6.
Signed-off-by: NThanneeru Srinivasulu <tsrinivasulu@cavium.com>
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Do not consider IPv6 frames with zero UDP checksum as frames
with bad checksum and drop them.
Signed-off-by: NThanneeru Srinivasulu <tsrinivasulu@cavium.com>
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When booted with ACPI, random mac addresses are being
assigned to node1 interfaces due to mismatch of bgx_id
in BGX driver and ACPI tables.

This patch fixes this issue by setting maximum BGX devices
per node based on platform/soc instead of a macro. This
change will set the bgx_id appropriately.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When BGX/LMACs are in QSGMII mode, for some LMACs, mode info is
not being printed. This patch will fix that. With changes already
done to not do any sort of serdes 2 lane mapping config calculation
in kernel driver, we can get rid of this logic.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

ACPI support has been added to ARM IOMMU driver in 4.10 kernel
and that has resulted in VNIC interfaces throwing translation
faults when kernel is booted with ACPI as driver was not using
DMA API. This patch fixes the issue by using DMA API which inturn
will create translation tables when IOMMU is enabled.

Also VNIC doesn't have a seperate receive buffer ring per receive
queue, so there is no 1:1 descriptor index matching between CQE_RX
and the index in buffer ring from where a buffer has been used for
DMA'ing. Unlike other NICs, here it's not possible to maintain dma
address to virt address mappings within the driver. This leaves us
no other choice but to use IOMMU's IOVA address conversion API to
get buffer's virtual address which can be given to network stack
for processing.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch fixes the case where there is no phydev attached
to a LMAC in DT due to non-existance of a PHY driver or due
to usage of non-stanadard PHY which doesn't support autoneg.
Changes dependeds on firmware to send correct info w.r.t
PHY and autoneg capability.

This patch also covers a case where a 10G/40G interface is used
as a 1G with convertors with Cortina PHY in between.
Signed-off-by: NThanneeru Srinivasulu <tsrinivasulu@cavium.com>
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This fixes the following smatch and coccinelle warnings:

  drivers/net/ethernet/cavium/thunder/thunder_xcv.c:119 xcv_setup_link() error: we previously assumed 'xcv' could be null (see line 118) [smatch]
  drivers/net/ethernet/cavium/thunder/thunder_xcv.c:119:16-20: ERROR: xcv is NULL but dereferenced. [coccinelle]

Fixes: 6465859a ("net: thunderx: Add RGMII interface type support")
Signed-off-by: NVincent Stehlé <vincent.stehle@laposte.net>
Cc: Sunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

napi_complete_done() allows to opt-in for gro_flush_timeout,
added back in linux-3.19, commit 3b47d303
("net: gro: add a per device gro flush timer")

This allows for more efficient GRO aggregation without
sacrifying latencies.
Signed-off-by: NEric Dumazet <edumazet@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For DLMs and SLMs on 80/81/83xx, many lane configurations
across different boards are coming up. Also kernel doesn't have
any way to identify board type/info and since firmware does,
just get rid of figuring out lane to serdes config and take
whatever has been programmed by low level firmware.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds support to set Rx/Tx queue sizes from ethtool. Fixes
an issue with retrieving queue size. Also sets SQ's CQ_LIMIT
based on configured Tx queue size such that HW doesn't process
SQEs when there is no sufficient space in CQ.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

While probing BGX we requesting appropriate QLM for it's configuration
and get LMAC count by that request. Then, while reading configured
MAC values from SSDT table we need to save them in proper mapping:
  BGX[i]->lmac[j].mac = <MAC value>
to later provide for initialization stuff. In order to fill
such mapping properly we need to add lmac index to be used while
acpi initialization since at this moment bgx->lmac_count already contains
actual value.
Signed-off-by: NVadim Lomovtsev <Vadim.Lomovtsev@caviumnetworks.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

>From struct ethtool_ops:

        int     (*set_rxfh)(struct net_device *, const u32 *indir,
                            const u8 *key, const u8 hfunc);

Change function arg of hfunc to const type.

V2: Fixed indentation.
Signed-off-by: NRobert Richter <rrichter@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Fix to return a negative error code from the error handling
case instead of 0, as done elsewhere in this function.

Fixes: 712c3185 ("net: thunderx: Program LMAC credits based on MTU")
Signed-off-by: NWei Yongjun <weiyongjun1@huawei.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The network device operation for reading statistics is only called
in one place, and it ignores the return value. Having a structure
return value is potentially confusing because some future driver could
incorrectly assume that the return value was used.

Fix all drivers with ndo_get_stats64 to have a void function.
Signed-off-by: NStephen Hemminger <sthemmin@microsoft.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The ethtool api {get|set}_settings is deprecated.
We move this driver to new api {get|set}_link_ksettings.
Signed-off-by: NPhilippe Reynes <tremyfr@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Transmit queue timeout issue is seen in two cases
- Due to a race condition btw setting stop_queue at xmit()
  and checking for stopped_queue in NAPI poll routine, at times
  transmission from a SQ comes to a halt. This is fixed
  by using barriers and also added a check for SQ free descriptors,
  incase SQ is stopped and there are only CQE_RX i.e no CQE_TX.
- Contrary to an assumption, a HW errata where HW doesn't stop transmission
  even though there are not enough CQEs available for a CQE_TX is
  not fixed in T88 pass 2.x. This results in a Qset error with
  'CQ_WR_FULL' stalling transmission. This is fixed by adjusting
  RXQ's  RED levels for CQ level such that there is always enough
  space left for CQE_TXs.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Enable pause frames on both Rx and Tx side, configure pause
interval e.t.c. Also support for enable/disable pause frames
on Rx/Tx via ethtool has been added.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch enables moving average calculation of Rx pkt's resources
and configures RED and backpressure levels for both CQ and RBDR.
Also initialize SQ's CQ_LIMIT properly.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NThanneeru Srinivasulu <tsrinivasulu@cavium.com>
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

On 80xx only one lane of DLM0 and DLM1 (of BGX0) can be used
, so even though lmac count may be 2 but LMAC1 should use
serdes lane of DLM1. Since it's not possible to distinguish
80xx from 81xx as PCI devid are same, this patch adds this
config support by replying on what firmware configures the
lmacs with.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch fixes the following
1. When interface is being teardown and queues are being cleaned up,
   free pending SKBs that are in SQ which are either not transmitted
   or freed as NAPI is disabled by that time.
2. While interface initialization, delay CFG_DONE notification till
   the end to avoid corner cases where TXQs are enabled but CQ
   interrupts are not which results blocking transmission and kicking
   off watchdog.
3. Check for IFF_UP while re-enabling RBDR interrupts from tasklet.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch fixes multiple issues
1. Convert all driver statistics to percpu counters for accuracy.
2. To avoid multiple CQEs posted by a TSO packet appended to HW,
   TSO pkt's SQE has 'post_cqe' not set but a dummy SQE is added
   for getting HW transmit completion notification. This dummy
   SQE has 'dont_send' set and HW drops the pkt pointed to in this
   thus Tx drop counter increases. This patch fixes this by subtracting
   SW tx tso counter from HW Tx drop counter for actual packet drop counter.
3. Reset all individual queue's and VNIC HW stats when interface is going down.
4. Getrid off unnecessary counters in hot path.
5. Bringout all CQE error stats i.e both Rx and Tx.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch fixes enabling of HW verification of L3/L4 length and
TCP/UDP checksum which is currently being cleared. Also fixed VLAN
stripping config which is being cleared when multiqset is enabled.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Programming LMAC credits taking 9K frame size by default is incorrect
as for an interface which is one of the many on the same BGX/QLM
no of credits available will be less as Tx FIFO will be divided
across all interfaces. So let's say a BGX with 40G interface and another
BGX with multiple 10G, bandwidth of 10G interfaces will be effected when
traffic is running on both 40G and 10G interfaces simultaneously.

This patch fixes this issue by programming credits based on netdev's MTU.
Also fixed configuring MTU to HW and added CQE counter for pkts which
exceed this value.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch fixes the 'bgx_id' determination on 83xx where there are
4 BGX blocks instead of 2 on other platforms.
Signed-off-by: NRadha Mohan Chintakuntla <rchintakuntla@cavium.com>
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

liquidio: min_mtu 68, max_mtu 16000

thunder: min_mtu 64, max_mtu 9200

CC: netdev@vger.kernel.org
CC: Sunil Goutham <sgoutham@cavium.com>
CC: Robert Richter <rric@kernel.org>
CC: Derek Chickles <derek.chickles@caviumnetworks.com>
CC: Satanand Burla <satananda.burla@caviumnetworks.com>
CC: Felix Manlunas <felix.manlunas@caviumnetworks.com>
CC: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: NJarod Wilson <jarod@redhat.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds support for byte queue limits
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch fixes a regression caused by previous commit
when irq name exceeds 20 byte array if interface's name
size is large.

Fixes: e4126213 ("net: thunderx: Use netdev's name for naming VF's interrupts")
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>