Supports hardware and software time stamping via the
Linux SO_TIMESTAMPING socket option.

Cc: Richard Cochran <richardcochran@gmail.com>
Signed-off-by: NAtul Gupta <atul.gupta@chelsio.com>
Signed-off-by: NGanesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In linux-4.5, busy polling was implemented in core
NAPI stack, meaning that all custom implementation can
be removed from drivers.

Not only we remove lot of code, we also remove one spin_lock()
from driver fast path.
Signed-off-by: NEric Dumazet <edumazet@google.com>
Cc: Ganesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Without any uld being loaded, uld_txq_info[] will be NULL. uld_send()
is also used for sending control work requests(for eg: setting filter)
that dont require any ulds to be loaded. Hence move uld_txq_info[]
assignment after ctrl_xmit().

Also added a NULL check for uld_txq_info[].

Fixes: 94cdb8bb (cxgb4: Add support for dynamic allocation
       of resources for ULD).
Signed-off-by: NArjun V <arjun@chelsio.com>
Signed-off-by: NCasey Leedom <leedom@chelsio.com>
Signed-off-by: NGanesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

t6fw-1.15.15.0 enabled compressed error vector in cpl_rx_pkt for T6.
Updating driver to take care of these changes.
Signed-off-by: NSantosh Rastapur <santosh@chelsio.com>
Signed-off-by: NArjun V <arjun@chelsio.com>
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NGanesh Goudar <ganeshgr@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Allocate resources dynamically for Upper layer driver's (ULD) like
cxgbit, iw_cxgb4, cxgb4i and chcr. The resources allocated include Tx
queues which are allocated when ULD register with cxgb4 driver and freed
while un-registering. The Tx queues which are shared by ULD shall be
allocated by first registering driver and un-allocated by last
unregistering driver.
Signed-off-by: NAtul Gupta <atul.gupta@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Calling napi_hash_del() before netif_napi_del() is dangerous
if a synchronize_rcu() is not enforced before NAPI struct freeing.

Lets leave this detail to core networking stack and feel
more comfortable.
Signed-off-by: NEric Dumazet <edumazet@google.com>
Cc: Hariprasad S <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Allocate resources dynamically to cxgb4's Upper layer driver's(ULD) like
cxgbit, iw_cxgb4 and cxgb4i. Allocate resources when they register with
cxgb4 driver and free them while unregistering. All the queues and the
interrupts for them will be allocated during ULD probe only and freed
during remove.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add a new commmon infrastructure to allocate reosurces dynamically to
Upper layer driver's(ULD) when they register with cxgb4 driver and free
them during unregistering. All the queues and the interrupts for
them will be allocated during ULD probe only and freed during remove.
Signed-off-by: NAtul Gupta <atul.gupta@chelsio.com>
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The commit 637d3e99 ("cxgb4: Discard the packet if the length is
greater than mtu") introduced a regression in the VLAN interface
performance when Tx VLAN offload is disabled.

Check if skb is tagged, regardless of whether it is hardware accelerated
or not. Presently we were checking only for hardware acclereated one,
which caused performance to drop to ~0.17Mbps on a 10GbE adapter for
VLAN interface, when tx vlan offload is turned off using ethtool.
The ethernet head length calculation was going wrong in this case, and
driver ended up dropping packets.

Fixes: 637d3e99 ("cxgb4: Discard the packet if the length is greater than mtu")
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There could be race between t4_eth_xmit() and t4_free_sge_resources() while
freeing tx descriptors, take txq lock in t4_free_sge_resources(). We need
to stop the xmit frame path which runs in bottom half context while
unloading the driver using _bh variant of the lock. This is to prevent
the deadlock between xmit and driver unload.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Stop all Ethernet RX Queues before freeing up various Ingress/Egress
Queues, etc. We were seeing cases of Ingress Queues not getting serviced
during the shutdown process leading to Ingress Paths jamming up through
the chip and blocking the shutdown effort itself.

One such case involved the Firmware sending a "Flush Token" through the
ULP-TX -> ULP-RX path for an Ethernet TX Queue being freed in order to
make sure there weren't any remaining TX Work Requests in the pipeline.
But the return path was stalled by Ingress Data unable to be delivered to
the Host because those Ingress Queues were no longer being serviced.

Based on original work by Casey Leedom <leedom@chelsio.com>
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

add large receive offload(LRO) support
for upper layer drivers.
Signed-off-by: NVarun Prakash <varun@chelsio.com>
Acked-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

allocate rxqs for non T4 adapters,
dump rxqs sge qinfo through debugfs.
Signed-off-by: NVarun Prakash <varun@chelsio.com>
Acked-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

T4 and T5 hardware will not coalesce Free List PCI-E Fetch Requests if
the Host Driver provides more Free List Pointers than the Fetch Burst
Minimum value.  So if we set FBMIN to 64 bytes and the Host Driver
supplies 128 bytes of Free List Pointer data, the hardware will issue two
64-byte PCI-E Fetch Requests rather than a single coallesced 128-byte
Fetch Request. T6 fixes this. So, for T4/T5 we set the FBMIN value to 128
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use freelist capacity instead of freelist size while checking, if
freelist needs to be refilled
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

SGE context congestion map changed from 4 to 8 priority per port
in T6 as there are only 2 channels.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Ingress padding boundary values got changed for T6.
    T5: 0=32B 1=64B 2=128B 3=256B 4=512B 5=1024B 6=2048B 7=4096B
    T6: 0=8B  1=16B 2=32B  3=64B  4=128B 5=128B  6=256B  7=512B

Updating the driver to set the correct boundary values in SGE_CONTROL to
32B.
Also, need to take care of this fl alignment change when calculating the
next packet offset.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

All the upper level protocols like rdma, iscsi have their own offload rx
queues, so instead of using the generic naming convention be specific
while naming them. Improves code readability
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add Free List DMA Mapping Errors to SGE Queue info for
Free Lists. Add Free List "Low" counter to count the number of times we
see the number of pointers that we _think_ the hardware sees in the
Free List below the Egress Threshold.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The WR headers  may not fit within one descriptor.
So we need to deal with wrap-around here.

Based on original patch by Pranjal Joshi <pjoshi@chelsio.com>
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Change mutual exclusion mechanism to prevent multiple threads of
execution from running in service_ofldq() at the same time.  The old
mechanism used an implicit guard on the down-call path and none on the
restart path and wasn't working. This checking makes the mechanism
explicit and is much easier to understand as a result.

Based on original work by Casey Leedom <leedom@chelsio.com>
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use helper macro ACCESS_ONCE() to load from the SGE status page
to prevent the compiler loading multiple times.

Based on original work by Mike Werner <werner@chelsio.com>
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

NAPI drivers no longer need to observe a particular protocol
to benefit from busy polling (CONFIG_NET_RX_BUSY_POLL=y)

napi_hash_add() and napi_hash_del() are automatically called
from core networking stack, respectively from
netif_napi_add() and netif_napi_del()

This patch depends on free_netdev() and netif_napi_del() being
called from process context, which seems to be the norm.

Drivers might still prefer to call napi_hash_del() on their
own, since they might combine all the rcu grace periods into
a single one, knowing their NAPI structures lifetime, while
core networking stack has no idea of a possible combining.

Once this patch proves to not bring serious regressions,
we will cleanup drivers to either remove napi_hash_del()
or provide appropriate rcu grace periods combining.
Signed-off-by: NEric Dumazet <edumazet@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We would like to automatically provide busy polling support
to all NAPI drivers, without them having to implement anything.

skb_mark_napi_id() can be called from napi_gro_receive() and
napi_get_frags().

Few drivers are still calling skb_mark_napi_id() because
they use netif_receive_skb(). They should eventually call
napi_gro_receive() instead. I will leave this to drivers
maintainers.
Signed-off-by: NEric Dumazet <edumazet@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds support for ethtool get time stamp ioctl, which is used by
tcpdump to get the supported time stamp types

eg: tcpdump -i eth5 -J
Time stamp types for eth5 (use option -j to set):
  host (Host)
  adapter_unsynced (Adapter, not synced with system time)

Adds support for adapter unsynced mode, by adding SIOCSHWTSTAMP support
in driver.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In commit 0aac3f56 ("cxgb4: Add comment for calculate tx flits
and sge length code") introduced a regression where tx flit calculation
is going wrong, which can lead to data corruption, hang, stall and
write-combining failure. Fixing it.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add debugfs support to dump tid info like stid, sftid, tids, atid and
hwtids
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NAnish Bhatt <anish@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Enhance cxgb4_t4_bar2_sge_qregs() and cxgb4_bar2_sge_qregs() to support T4
user mode mappings.  Update all the current users as well.
Signed-off-by: NSteve Wise <swise@opengridcomputing.com>
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

Adds NIC driver related changes for T6 adapter. Register related
changes, MC related changes, VF related changes, doorbell related
changes, debugfs changes, etc
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch changes variable name 'fn' to 'pf' of structure adapter.
A 'fn' usually stands for PCI function which could be a PF or a VF.
However, the use of this particular variable is explicitly limited to PF
only. So, be specific about it in the variable name.

Also corrects arguments passed for fn t4_ofld_eq_free, t4_ctrl_eq_free,
t4_eth_eq_free, t4_iq_free, t4_alloc_vi, t4_fw_hello, t4_wr_mbox and
t4_cfg_pfvf function.

Also renames cxgb4_t4_bar2_sge_qregs to t4_bar2_sge_qregs and renames
the latter function name in cxgb4vf driver to t4vf_bar2_sge_qregs to
avoid conflicts. Also fixes alignment for these function.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add debugfs entry to inject Freelist starvation, used only for debugging
purpose.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Cleanup few MACROS left out in t4_hw.h to be consistent with the
existing ones. Also replace few hardcoded values with MACROS. Also
update comments for some code
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

pktgen sends raw udp packets and bypasses most of the
linux networking stack. User can specify different packet sizes.
Hence we need to discard the packet if the length is greater than mtu
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Adds device node to ULD info. Use the node info to alloc_ring() for ctrl
TX queues
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Passes a Congestion Channel Map to t4_sge_alloc_rxq()
for the Ethernet RX Queues based on the MPS Buffer Group Map
of the TX Channel rather than just the TX Channel Map.
Also, in t4_sge_alloc_rxq() for T5, setting up the
Congestion Manager values of the new RX Ethernet Queue is
done by firmware now.
Signed-off-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>