Now that we have reworked the way we perform the PHY initialization, we
no longer need to differentiate between init time vs. non-init time
calls, just use a dev_info_once() print to print the PHY type.
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We are currently doing a full PHY initialization and even starting the
pHY state machine during bcmgenet_mii_init() which is executed in the
driver's probe function. This is convenient to determine whether we can
attach to a proper PHY device but comes at the expense of spending up to
10ms per MDIO transactions (to reach the waitqueue timeout), which slows
things down.

This also creates a sitaution where we end-up attaching twice to the
PHY, which is not quite correct either.

Fix this by moving bcmgenet_mii_probe() into bcmgenet_open() and update
its error path accordingly.

Avoid printing the message "attached PHY at address 1 [...]" every time
we bring up/down the interface and remove this print since it duplicates
what the PHY driver already does for us.

Fixes: 1c1008c7 ("net: bcmgenet: add main driver file")
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Our internal GPHY might be powered off before we attempt scanning the
MDIO bus and bind a driver to it. The way we are currently determining
whether a PHY is internal or not is done *after* we have successfully
matched its driver. If the PHY is powered down, it will not respond to
the MDIO bus, so we will not be able to bind a driver to it.

Our Device Tree for GENET interfaces specifies a "phy-mode" value:
"internal" which tells if this internal uses an internal PHY or not.

If of_get_phy_mode() fails to parse the 'phy-mode' property, do an
additional manual lookup, and if we find "internal" set the
corresponding internal variable accordingly.

Replace all uses of phy_is_internal() with a check against
priv->internal_phy to avoid having to rely on whether or not
priv->phydev is set correctly.

Fixes: 1c1008c7 ("net: bcmgenet: add main driver file")
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We are currently disabling the GPHY interface during bcmgenet_close(),
and attempting to power it back on during bcmgenet_open(). This works
fine for the first time, because we called bcmgenet_mii_config() which
took care of enabling the interface, however, bcmgenet_power_up() really
needs to power on the GPHY for correctness.

This will be particularly important as we want to move
bcmgenet_mii_probe() down to bcmgenet_open() to avoid seeing the "PHY
already attached" message.

Fixes: a642c4f7 ("net: bcmgenet: power up and down integrated GPHY when unused")
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

bcmgenet_open()'s error path call free_irq() with a dev_id argument
different from the one we used to call request_irq() with, this will
make us trip over the warning in kernel/irq/manage.c:__free_irq()

Fixes: 1c1008c7 ("net: bcmgenet: add main driver file")
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We are currently issuing multiple PHY resets during a suspend/resume,
first during bcmgenet_power_up() which does a hardware reset, then a
software reset by calling bcmgenet_mii_reset(). This is both unnecessary
and can take as long as 10ms per MDIO transactions while we re-apply
workarounds because we do not yet have MDIO interrupts enabled.

phy_resume() takes care of re-apply our workarounds in case we need any,
and bcmgenet_power_up() does a PHY hardware reset, all of this is more
than enough to guarantee that the PHY operates correctly.

Fixes: 1c1008c7 ("net: bcmgenet: add main driver file")
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

bcmgenet driver needs to separate packet drops from packet errors.

When the driver has to drop a *good* packet, due to lack of buffers or
replacement skbs, increment only dev->stats.[rx|tx]_dropped.

When the driver encounters a bad Rx packet or Tx error, increment only
dev->stats.[rx|tx]_errors + relevant detailed error counter.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Dump useful ring statistics along with interrupt status, software
maintained pointers and hardware registers to help troubleshoot TX queue
stalls.

When a timeout occurs, disable TX NAPI for the rings, dump their states
while interrupts are disabled, re-enable interrupts, NAPI and queue flow
control to help with the recovery.
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Currently, MoCA fixed PHYs are always in link-up state, regardless of
whether the link is actually up or not.

Add code to properly detect MoCA link state changes and to reflect the
new state in MoCA fixed PHY. Only GENET V3 and V4 MACs are capable of
detecting MoCA link state changes.

The code works as follows:
1. GENET MAC detects MoCA link state change and issues UMAC_IRQ_LINK_UP
   or UMAC_IRQ_LINK_DOWN interrupt.
2. Link up/down interrupt is processed in bcmgenet_irq_task(), which
   calls phy_mac_interrupt().
3. phy_mac_interrupt() updates the fixed PHY phydev->link and kicks
   the PHY state machine.
4. PHY state machine proceeds to read the fixed PHY link status
   register.
5. When the fixed PHY link status register is being read, the new
   function bcmgenet_fixed_phy_link_update() gets called. It copies
   the fixed PHY phydev->link value to the fixed PHY status->link.
6. PHY state machine receives the new link state of the fixed PHY.
7. MoCA fixed PHY link state now correctly reflects the real MoCA
   hardware link state.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

On phy_mac_interrupt() call, the new_link parameter should be 0 or 1.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add #define UMAC_IRQ_LINK_EVENT to simplify link event handling code.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

If bcmgenet_init_dma() fails, it cleans up after itself. Rx and Tx
DMAs are off, and NAPI instances haven't been netif_napi_add()'ed.
Therefore, we need to skip calling bcmgenet_fini_dma() on the error
handling path. bcmgenet_resume() already does this correctly.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

bcmgenet_fini_dma() is the only caller of __bcmgenet_fini_dma().
Move __bcmgenet_fini_dma() code inside bcmgenet_fini_dma().
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add #define for UMAC_IRQ_RXDMA_DONE and UMAC_IRQ_TXDMA_DONE in order
to simplify the code that handles Rx and Tx default queue interrupts.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add support for multiple Rx queues:
1. Add NAPI context per Rx queue
2. Modify Rx interrupt and Rx NAPI code to handle multiple Rx queues
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Introduce new bcmgenet functions to handle the NAPI calls to:
netif_napi_add()
napi_enable()
napi_disable()
netif_napi_del()
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Introduce new bcmgenet functions to handle the NAPI calls to:
netif_napi_add()
napi_enable()
napi_disable()
netif_napi_del()
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use more meaningful variable names int0_enable and int1_enable when
enabling bcmgenet interrupts.

For Rx default queue interrupts, use:
UMAC_IRQ_RXDMA_BDONE | UMAC_IRQ_RXDMA_PDONE

For Tx default queue interrupts, use:
UMAC_IRQ_TXDMA_BDONE | UMAC_IRQ_TXDMA_PDONE
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Do the two kcalloc() calls first, before proceeding into Rx/Tx DMA init.
Makes the error case handling much simpler.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Reviewed-by: NJaedon Shin <jaedon.shin@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Remove unnecessary function parameter priv. Use ring->priv instead.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Remove unused priv->int0_mask and priv->int1_mask.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In case the interface is not used, power down the integrated GPHY during
suspend. Similarly to bcmgenet_open(), bcmgenet_resume() powers on the GPHY
prior to any UniMAC activity.
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Power up the GPHY while we are bringing-up the network interface, and
conversely, upon bring down, power the GPHY down. In order to avoid
creating hardware hazards, make sure that the GPHY gets powered on
during bcmgenet_open() prior to the UniMAC being reset as the UniMAC may
start creating activity towards the GPHY if we reverse the steps.
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

If bcmgenet_power_down() fails, we would want to propagate a return
value from bcmgenet_wol_power_down_cfg() to know about this.
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add support for Hardware Filter Block (HFB) so that incoming Rx traffic
can be matched and directed to desired Rx queues.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Delay the update of the TDMA producer index unless this is the last SKB
in a batch, or the queue is already stopped. Move the check for whether
the queue should be stopped before the xmit_more check to avoid locking
the transmit queue in case there was a SKB submitted which has xmit_more
set.
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There is no need to have both bcmgenet_xmit_single() and
bcmgenet_xmit_frag() perform a free_bds decrement and a prod_index
increment by one. In case one of these functions fails to map a SKB or
fragment for transmit, we will return and exit bcmgenet_xmit() with an
error.

We can therefore safely use our local copy of nr_frags to know by how
much we should decrement the number of free buffers available, and by
how much the producer count must be incremented and do this in the tail
of bcmgenet_xmit().
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Acked-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Currently, bcmgenet_desc_rx() calls bcmgenet_rx_refill() at the end of
Rx packet processing loop, after the current Rx packet has already been
passed to napi_gro_receive(). However, bcmgenet_rx_refill() might fail
to allocate a new Rx skb, thus leaving a hole on the Rx queue where no
valid Rx buffer exists.

To eliminate this situation:
1. Rewrite bcmgenet_rx_refill() to retain the current Rx skb on the Rx
   queue if a new replacement Rx skb can't be allocated and DMA-mapped.
   In this case, the data on the current Rx skb is effectively dropped.
2. Modify bcmgenet_desc_rx() to call bcmgenet_rx_refill() at the top of
   Rx packet processing loop, so that the new replacement Rx skb is
   already in place before the current Rx skb is processed.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Tested-by: Jaedon Shin <jaedon.shin@gmail.com>--
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Tested-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Bits 31:16 of RDMA_PROD_INDEX contain Rx discarded packet count, which
are the Rx packets that had to be dropped by MAC hardware since there
was no room on the Rx queue. Add code to collect this information into
the netdev stats.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

1. Add struct bcmgenet_rx_ring to hold all necessary information
   for a single Rx queue.
2. Add bcmgenet_init_rx_queues() to initialize all Rx queues.
3. Modify bcmgenet_init_rx_ring() to initialize a single Rx queue.
4. Modify Rx interrupt path code to use per-queue data.
5. Modify bcmgenet_rx_refill() to use RxCB->bd_addr.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In preparation for supporting multiple Rx queues:
1. Move the initialization of priv->num_rx_bds, priv->rx_bds, and
   priv->rx_cbs from bcmgenet_init_rx_ring() to bcmgenet_init_dma()
   since they are not specific to a single Rx queue. Mimics the Tx
   init model where priv->num_tx_bds, priv->tx_bds, and priv->tx_cbs
   are initialized in bcmgenet_init_dma().
2. Program DMA_MBUF_DONE_THRESH = 1 so that future Rx queues Q0-Q15
   will get per-packet Rx interrupt.
3. Group DMA_START_ADDR, RDMA_READ_PTR, RDMA_WRITE_PTR, and DMA_END_ADDR
   initialization together. Mimics the Tx init model.
4. There is 1-to-1 mapping between RxCBs and RxBDs.
   Precalculate RxCB->bd_addr so that it can be used in the future.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In preparation for supporting multiple Rx queues, add GENET_Q16_RX_BD_CNT
and hw_params->rx_bds_per_q.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In preparation for supporting multiple Rx queues, adjust the call to
alloc_etherdev_mqs() to allow max GENET_MAX_MQ_CNT + 1 Rx queues.

The actual number of Rx queues in use is correctly adjusted with:
netif_set_real_num_rx_queues(priv->dev, priv->hw_params->rx_queues + 1);
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

bcmgenet driver doesn't yet support multiple Rx queues.
Set hw_params->rx_queues = 0 accordingly.
The default Rx queue (Q16) is still created and operational.
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

1. Use c_index and ring->c_index to determine how many TxCBs/TxBDs are
   ready for cleanup
   - c_index = the current value of TDMA_CONS_INDEX
   - TDMA_CONS_INDEX is HW-incremented and auto-wraparound (0x0-0xFFFF)
   - ring->c_index = __bcmgenet_tx_reclaim() cleaned up to this point on
     the previous invocation

2. Add bcmgenet_tx_ring->clean_ptr
   - index of the next TxCB to be cleaned
   - incremented as TxCBs/TxBDs are processed
   - value always in range [ring->cb_ptr, ring->end_ptr]

3. Fix incrementing of dev->stats.tx_packets
   - should be incremented only when tx_cb_ptr->skb != NULL

These changes simplify __bcmgenet_tx_reclaim(). Furthermore, Tx ring size
can now be any value.

With the old code, Tx ring size had to be a power-of-2:
   num_tx_bds = ring->size;
   c_index &= (num_tx_bds - 1);
   last_c_index &= (num_tx_bds - 1);
Signed-off-by: NPetri Gynther <pgynther@google.com>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Commit 44c8bc3c ("net: bcmgenet: log RX buffer allocation and RX/TX dma
failures") added a few software maintained statistics using
BCMGENET_STAT_MIB_RX and BCMGENET_STAT_MIB_TX. These statistics are read from
the hardware MIB counters, such that bcmgenet_update_mib_counters() was trying
to read from a non-existing MIB offset for these counters.

Fix this by introducing a special type: BCMGENET_STAT_SOFT, similar to
BCMGENET_STAT_NETDEV, such that bcmgenet_get_ethtool_stats will read from the
software mib.

Fixes: 44c8bc3c ("net: bcmgenet: log RX buffer allocation and RX/TX dma failures")
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds bcmgenet_tx_poll for the tx_rings. This can reduce the
interrupt load and send xmit in network stack on time. This also
separated for the completion of tx_ring16 from bcmgenet_poll.

The bcmgenet_tx_reclaim of tx_ring[{0,1,2,3}] operative by an interrupt
is to be not more than a certain number TxBDs. It is caused by too
slowly reclaiming the transmitted skb. Therefore, performance
degradation of xmit after 605ad7f1 ("tcp: refine TSO autosizing").
Signed-off-by: NJaedon Shin <jaedon.shin@gmail.com>
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

bcmgenet_hw_params->bds_cnt and GENET_DEFAULT_BD_CNT are used only in Tx init.
Rename them accordingly:
- bcmgenet_hw_params->bds_cnt => bcmgenet_hw_params->tx_bds_per_q
- GENET_DEFAULT_BD_CNT => GENET_Q16_TX_BD_CNT
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There is 1-to-1 mapping between TxCBs and TxBDs. Precalculate TxCB->bd_addr
once in bcmgenet_init_dma() instead of doing it over and over needlessly in
bcmgenet_get_txcb().
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

1. Rename bcmgenet_init_multiq() to bcmgenet_init_tx_queues()
2. Fix bcmgenet_init_tx_queues() function description
3. Move Tx default queue init inside bcmgenet_init_tx_queues()
4. Modify bcmgenet_init_dma() to call bcmgenet_init_tx_queues()
Signed-off-by: NPetri Gynther <pgynther@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>