Add support to thunderx to report bpf_prog ID during XDP_QUERY_PROG.
Signed-off-by: NMartin KaFai Lau <kafai@fb.com>
Cc: Sunil Goutham <sgoutham@cavium.com>
Acked-by: NAlexei Starovoitov <ast@fb.com>
Acked-by: NDaniel Borkmann <daniel@iogearbox.net>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

A common pattern with skb_put() is to just want to memcpy()
some data into the new space, introduce skb_put_data() for
this.

An spatch similar to the one for skb_put_zero() converts many
of the places using it:

    @@
    identifier p, p2;
    expression len, skb, data;
    type t, t2;
    @@
    (
    -p = skb_put(skb, len);
    +p = skb_put_data(skb, data, len);
    |
    -p = (t)skb_put(skb, len);
    +p = skb_put_data(skb, data, len);
    )
    (
    p2 = (t2)p;
    -memcpy(p2, data, len);
    |
    -memcpy(p, data, len);
    )

    @@
    type t, t2;
    identifier p, p2;
    expression skb, data;
    @@
    t *p;
    ...
    (
    -p = skb_put(skb, sizeof(t));
    +p = skb_put_data(skb, data, sizeof(t));
    |
    -p = (t *)skb_put(skb, sizeof(t));
    +p = skb_put_data(skb, data, sizeof(t));
    )
    (
    p2 = (t2)p;
    -memcpy(p2, data, sizeof(*p));
    |
    -memcpy(p, data, sizeof(*p));
    )

    @@
    expression skb, len, data;
    @@
    -memcpy(skb_put(skb, len), data, len);
    +skb_put_data(skb, data, len);

(again, manually post-processed to retain some comments)
Reviewed-by: NStephen Hemminger <stephen@networkplumber.org>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NWeilin Chang <weilin.chang@cavium.com>
Signed-off-by: NDerek Chickles <derek.chickles@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Disallow enabling firmware debug from a VF.  Only PF is allowed to do that.
Signed-off-by: NDerek Chickles <derek.chickles@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NSatanand Burla <satananda.burla@cavium.com>
Signed-off-by: NDerek Chickles <derek.chickles@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Set initialization state variable to (reflect interrupt initialization) at
correct time (immediately after having configured interrupts).  This fixes
problem of inconsistent IRQ allocation in case of [obscure] failure when
negotiating with PF driver during init.

Clean-up of interrupt enablement during initialization & avoid potential
race condition with chip-specific code (i.e. perform interrupt control in
main driver module).  Added explanatory comments regarding interrupt
enablement.
Signed-off-by: NRick Farrington <ricardo.farrington@cavium.com>
Signed-off-by: NSatanand Burla <satananda.burla@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

lio_enable_irq (called by napi poll) is reporting to Octeon an inaccurate
count of processed rx packets causing Octeon to eventually stop forwarding
packets to the host.  Fix it by using this formula for an accurate count:

    processed rx packets = droq->pkt_count - droq->pkts_pending

Also increase SOFT_COMMAND_BUFFER_SIZE to match what the firmware expects.
Signed-off-by: NPrasad Kanneganti <prasad.kanneganti@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There's a rare pci_driver.probe failure of the VF driver that's caused by
PF/VF handshake going out of sync.  The culprit is octeon_mbox_write() who
ignores an ack timeout condition; it just keeps unconditionally writing all
elements of mbox_cmd->data[] even when the other side is not ready for
them.  Fix it by making each write of mbox_cmd->data[i] conditional to
having previously received an ack.

Also fix the octeon_mbox_state enum such that each state gets a unique
value.  Also add ULL suffix to numeric literals in macro definitions.
Signed-off-by: NPrasad Kanneganti <prasad.kanneganti@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Include HWTSTAMP_FILTER_NTP_ALL in net_hwtstamp_validate() as a valid
filter and update drivers which can timestamp all packets, or which
explicitly list unsupported filters instead of using a default case, to
handle the filter.

CC: Richard Cochran <richardcochran@gmail.com>
CC: Willem de Bruijn <willemb@google.com>
Signed-off-by: NMiroslav Lichvar <mlichvar@redhat.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

octeon_devices_lock can be made static as it does not need to be
in global scope.

Cleans up sparse warning: "warning: symbol 'octeon_devices_lock'
was not declared. Should it be static?"
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In the function assigned to .ndo_set_vf_mac, check the validity of the
vfidx argument before proceeding to tell the firmware to set the VF MAC
address.
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDerek Chickles <derek.chickles@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When multiple liquidio NICs are plugged in, the first insmod of the PF
driver succeeds.  But after an rmmod, a subsequent insmod fails.  Reason is
during rmmod, the PF driver resets the Octeon of only one of the NICs; it
neglects to reset the Octeons of the other NICs.

Fix the insmod failure by adding the missing Octeon resets at rmmod.  Keep
a per-NIC refcount that indicates the number of active PFs in a given NIC.
When the refcount goes to zero, then reset the Octeon of that NIC.
Signed-off-by: NRick Farrington <ricardo.farrington@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Tested-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Driver follows a method of taking one extra reference on the
page for recycling which is fine in usual packet path where
each 64KB page is segmented into multiple receive buffers.

But in XDP mode since there is just one receive buffer per
page taking extra page reference itself becomes big bottleneck
consuming ~50% of CPU cycles due to atomic operations.

This patch adds a internal ref count in pgcache for each
page and additional page references are taken in a batch
instead of just one at a time. Internal i.e 'pgcache->ref_count'
and page's i.e 'page->_refcount' counters are compared to check
page's recyclability.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When in XDP mode reserve XDP_PACKET_HEADROOM bytes at the start
of receive buffer for XDP program to modify headers and adjust
packet start. Additional code changes done to handle such packets.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds support for XDP_TX i.e transmits packet out of
the XDP TX queue mapped to the corresponding Rx queue
on which packet is received.

Since SQ for XDP TX will be used only on a single cpu i.e
SQ description creation and freeing, using atomic free count
is not necessary and will become a bottleneck. Hence added
a separate 'xdp_free_cnt' used for SQs designated for XDP
to track descriptor free count.

Changes also include
- A new entry 'xdp_page' is added to save transmitted packet's
  page pointer for later cleanup.
- XDP Tx SQ's doorbell is ringed once per NAPI instance.
- Retrieving designated SQ for packets being sent out by stack
  via 'nicvf_xmit'.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds support for XDP_DROP.
Also since in XDP mode there is just a single buffer per page,
made changes to recycle DMA mapping info as well along with pages.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds basic XDP support i.e attaching a BPF program to an
interface. Also takes care of allocating separate Tx queues
for XDP path and for network stack packet transmission.

This patch doesn't support handling of any of the XDP actions,
all are treated as XDP_PASS i.e packets will be handed over to
the network stack.

Changes also involve allocating one receive buffer per page in XDP
mode and multiple in normal mode i.e when no BPF program is attached.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Get rid of unnecessary double pointer references and type casting
in receive buffer allocation code.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Optimized CQE handling with below changes
- Feeing descriptors back to SQ in bulk i.e once per NAPI
  instance instead for every CQE_TX, this will reduce number
  of atomic updates to 'sq->free_cnt'.
- Checking errors in CQE_TX and CQE_RX before calling appropriate
  fn()s to update error stats i.e reduce branching.

Also removed debug messages in packet handling path which otherwise
causes issues if DEBUG is enabled.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Receive buffer's physical address or iova will anyway not
go beyond 49bits, since it is the max supported HW address.
As per perf, updating bitfields i.e buf_addr:42 in RBDR
descriptor entry consumes lots of cpu cycles, hence changed
it to a 64bit field with alignment requirements taken care of.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds support for page recycling for allocating receive buffers
to reduce cost of refilling RBDR ring. Also got rid of using
compound pages when pagesize is 4K, only order-0 pages now.

Only page is recycled, DMA mappings still needs to be done for
every receive buffer allocated due to following constraints
- Cannot have just one receive buffer per 64KB page.
- There is just one buffer ring shared across 8 Rx queues, so
  buffers of same page can go to any Rx queue.
- HW gives buffer address where packet has been DMA'ed and not
  the index into buffer ring.
This makes it not possible to resue DMA mapping info. So unfortunately
have to go through costly mapping route for every buffer.
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Presumably we never hit this return, but static checkers complain that
we need to unlock so we may as well fix that.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Acked-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

gcc points out an useless assignment that was added during code refactoring:

drivers/net/ethernet/cavium/liquidio/lio_ethtool.c: In function 'octnet_intrmod_callback':
drivers/net/ethernet/cavium/liquidio/lio_ethtool.c:1315:59: error: parameter 'oct_dev' set but not used [-Werror=unused-but-set-parameter]

This is harmless but can clearly be remove to avoid the warning.

Fixes: 50c0add5 ("liquidio: refactor interrupt moderation code")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add the PCI_SUBSYS_DEVID_81XX_RGX and use the same to set
the max bgx per node count.

This fixes the issue intoduced by following commit
78aacb6f net: thunderx: Fix invalid mac addresses for node1 interfaces
With this commit the max_bgx_per_node for 81xx is set as 2 instead of 3
because of which num_vfs is always calculated as zero.
Signed-off-by: NGeorge Cherian <george.cherian@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Remove deprecated pci_enable_msix API in favour of its
successor pci_alloc_irq_vectors.
Signed-off-by: NThanneeru Srinivasulu <tsrinivasulu@cavium.com>
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For security reasons, NIC firmware does not allow VF to set its VLAN if PF
set it already.  Firmware allows VF to set its VLAN if PF did not set it.
After the VF instructs the firmware to set the VLAN, VF always indicates
(via return 0) that the operation is successful--even for the times when it
isn't.

Put in a mechanism for the VF's set VLAN function to receive the firmware
response code, then make that function return -EPERM if the firmware
forbids the operation.

Make that mechanism available for other functions that may, in the future,
be interested in receiving the response code from the firmware.  That
mechanism involves adding new fields to struct octnic_ctrl_pkt, so make all
users of struct octnic_ctrl_pkt initialize the struct to zero before using
it; otherwise, the mechanism might act on uninitialized garbage.
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDerek Chickles <derek.chickles@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adding support for TSO and checksum hardware offloads for ipv6.
Signed-off-by: NThanneeru Srinivasulu <tsrinivasulu@cavium.com>
Signed-off-by: NSunil Goutham <sgoutham@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Detection of watchdog timeout of Octeon cores is flawed and susceptible to
false alarms.  Refactor by removing the detection code, and in its place,
leverage existing code that monitors for an indication from the NIC
firmware that an Octeon core crashed; expand the meaning of the indication
to "an Octeon core crashed or its watchdog timer expired".  Detection of
watchdog timeout is now delegated to an exception handler in the NIC
firmware; this is free of false alarms.

Also if there's an Octeon core crash or watchdog timeout:
(1) Disable VF Ethernet links.
(2) Decrement the module refcount by an amount equal to the number of
    active VFs of the NIC whose Octeon core crashed or had a watchdog
    timeout.  The refcount will continue to reflect the active VFs of
    other liquidio NIC(s) (if present) whose Octeon cores are faultless.

Item (2) is needed to avoid the case of not being able to unload the driver
because the module refcount is stuck at some non-zero number.  There is
code that, in normal cases, decrements the refcount upon receiving a
message from the firmware that a VF driver was unloaded.  But in
exceptional cases like an Octeon core crash or watchdog timeout, arrival of
that particular message from the firmware might be unreliable.  That normal
case code is changed to not touch the refcount in the exceptional case to
avoid contention (over the refcount) with the liquidio_watchdog kernel
thread who will carry out item (2).
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDerek Chickles <derek.chickles@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There is a cut and paste bug here so we accidentally clear the first
few bytes of "resp" a second time instead clearing "ctx".

Fixes: 50c0add5 ("liquidio: refactor interrupt moderation code")
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Acked-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

These files all use functions declared in interrupt.h, but currently rely
on implicit inclusion of this file (via netns/xfrm.h).

That won't work anymore when the flow cache is removed so include that
header where needed.
Signed-off-by: NFlorian Westphal <fw@strlen.de>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Refactor interrupt moderation code for flexibility because parameters are
different for 10G and 25G cards.  Currently parameters (for 10G only) come
from macros compiled-in to the PF and VF drivers; fix it so that parameters
suitable for the card (10G or 25G) come from the NIC firmware via response
to a command.

Also bump up driver version to 1.5.1 to match newer NIC firmware version.
Signed-off-by: NPrasad Kanneganti <prasad.kanneganti@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDerek Chickles <derek.chickles@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The PF driver is incorrectly resetting Octeon when the module parameter
"fw_type=none" is there.  "fw_type=none" means the PF should not load any
firmware to the NIC because Octeon is already running preloaded firmware.

Fix it by putting an if (fw_type != none) around the reset code.

Because the Octeon reset is now conditionally gone, when unloading the
driver, conditionally send the RESET_PF command to the firmware who will
then free up PF-related data structures.
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NSatanand Burla <satananda.burla@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

If there are no egress packets pending, then don't look for tx completions
in napi poll.  Also, fix broken tx queue wakeup logic.
Signed-off-by: NVSR Burru <veerasenareddy.burru@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NSatanand Burla <satananda.burla@cavium.com>
Signed-off-by: NDerek Chickles <derek.chickles@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add workqueue that is periodically run to try to allocate RX buffers in OOM
conditions in PF and VF.
Signed-off-by: NSatanand Burla <satananda.burla@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Fix Coverity scan errors by not dereferencing lio->glists_dma_base pointer
if it's NULL.

See http://marc.info/?l=linux-netdev&m=149002294305614&w=2Reported-by: NStephen Hemminger <stephen@networkplumber.org>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NVSR Burru <veerasenareddy.burru@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Change to support host<->firmware command return value.
Fix for vf mac addr state command.
1. Added support for firmware commands to return a value:
   - previously, the returned code overlapped with host codes, thus
     commands were only returning 0 (success) or -1 (interpreted as
     timeout)
   - per 'response_manager.h', the error codes are split into two fields
     (major/minor) now, firmware commands are grouped into their own
     'major' group, separate from the host's 'major' group, which allow f/w
     commands to return any 16-bit value
2. The command to set vf mac addr was logging a success message even if
   command failed.  Now command uses a callback function to log the status
   message.
3. The command to set vf mac addr was not logging a message when set via
   the host 'ip' command.  Now, the callback function will log an
   appropriate message.
Signed-off-by: NRick Farrington <ricardo.farrington@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDerek Chickles <derek.chickles@cavium.com>
Signed-off-by: NSatanand Burla <satananda.burla@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add timeout error message in lio_process_ordered_list().  Add host failure
status in existing error message in if_cfg_callback().
Signed-off-by: NRick Farrington <ricardo.farrington@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Remove code duplicated in PF and VF; define that code once only in a common
header file included by PF and VF.
Signed-off-by: NSatanand Burla <satananda.burla@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Information reported to ethtool about link modes is wrong for 25G NIC.  Fix
it by checking for presence of 25G NIC, checking the link speed reported by
NIC firmware, and then assigning proper values to the
ethtool_link_ksettings struct.
Signed-off-by: NManish Awasthi <manish.awasthi@cavium.com>
Signed-off-by: NFelix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>