These ops allow to control number of channels driver is allowed to
work with at cpdma level. The maximum number of channels is 8 for
rx and 8 for tx. In dual_emac mode the h/w channels are shared
between two interfaces and changing number on one interface changes
number of channels on another.

How many channels are supported and enabled:
$ ethtool -l ethX

Change number of channels (up to 8)
$ ethtool -L ethX rx 6 tx 6

Per-channel statistic:
$ ethtool -S ethX
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Keep the driver internals in C file. Currently it's not required for
drivers to know rx or tx a channel is, except create function.
So correct "channel create" function, and use all channel struct
macroses only for internal use.
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The cpsw h/w supports up to 8 tx and 8 rx channels. This patch adds
multi-queue support to the driver only, shaper configuration will
be added with separate patch series. Default shaper mode, as
before, priority mode, but with corrected priority order, 0 - is
highest priority, 7 - lowest.

The poll function handles all unprocessed channels, till all of
them are free, beginning from hi priority channel.

In dual_emac mode the channels are shared between two network devices,
as it's with single-queue default mode.

The statistic for every channel can be read with:
$ ethtool -S ethX
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The interrupts shouldn't be disabled while receiving skb, but while
ctrl_stop, the channels are stopped and all remaining packets are
handled with netif_receive_skb(), it can cause WARN_ONCE when ctrl
is stopping while not all packets were handled with NAPIs:

lock_irq_save
    cpdma_ctlr_stop
       cpdma_chan_top
           __cpdma_chan_free
               cpsw_rx_handler
                   netif_receive_skb

So, split locking while ctrl stop thus interrupts are still
enabled while skbs handling. It can cause WARN_ONCE in rare
cases when ctrl is stopping while not all packets were handled
with NAPIs.
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Tx channels share same pool of descriptors. Thus one channel can
block another if pool is emptied by one. But, the shaper should
decide which channel is allowed to send packets. To avoid such
impact of one channel on another, let every channel to have its
own piece of pool.
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The ale, cpts, version, rx_packet_max, bus_freq, interrupt pacing
parameters are common per net device that uses the same h/w. So,
move them to common driver structure.
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The napi structs are common for both net devices in dual_emac
mode, In order to not hold duplicate links to them, move to
cpsw_common.
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

These data are common for net devs in dual_emac mode. No need to hold
it for every priv instance, so move them under cpsw_common.
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The irq data are common for net devs in dual_emac mode. So no need to
hold these data in every priv struct, move them under cpsw_common.
Also delete irq_num var, as after optimization it's not needed.
Correct number of irqs to 2, as anyway, driver is using only 2,
at least for now.
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Every net device private struct holds links to shared cpdma resources.
No need to save and every time synchronize these resources per net dev.
So, move it to common driver struct.
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The pointers on h/w registers are common for every cpsw_private
instance, so no need to hold them for every ndev.
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

No need to hold pdev link when only dev is needed.
This allows to simplify a bunch of cpsw->pdev->dev now and farther.
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch simply create holder for common data and as a start moves
pdev var to it.
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

No need to check const slave num in runtime for every packet,
and ndev for slaves w/o ndev is anyway NULL. So remove redundant
check and macro.
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There is no need to hold link to clk, it's used only once
while probe.
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Reviewed-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There is no need in priv here.
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Reviewed-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

At poll handler no possibility to figure out which network device is
handling packets, as cpdma channels are common for both network
devices in dual_emac mode. Currently, the messages are printed only
for one device, in fact, there is two. This print msg is incorrect
and seems is not very useful, so drop it from poll handler.
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

As second net dev is created only in case of dual_emac mode, port
number can be figured out in simpler way. Also no need to pass
redundant ndev struct.
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Reviewed-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This is a preparatory patch for converting qdisc linked list into a
hashtable. As we'll need to include hashtable.h in netdevice.h, we first
have to make sure that this will not introduce symbol conflicts for any of
the netdevice.h users.
Reviewed-by: NCong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Kmemleak reports following false positive memory leaks for each sk
buffers allocated by CPSW (__netdev_alloc_skb_ip_align()) in
cpsw_ndo_open() and cpsw_rx_handler():

unreferenced object 0xea915000 (size 2048):
  comm "systemd-network", pid 713, jiffies 4294938323 (age 102.180s)
  hex dump (first 32 bytes):
    00 58 91 ea ff ff ff ff ff ff ff ff ff ff ff ff  .X..............
    ff ff ff ff ff ff fd 0f 00 00 00 00 00 00 00 00  ................
  backtrace:
    [<c0108680>] __kmalloc_track_caller+0x1a4/0x230
    [<c0529eb4>] __alloc_skb+0x68/0x16c
    [<c052c884>] __netdev_alloc_skb+0x40/0x104
    [<bf1ad29c>] cpsw_ndo_open+0x374/0x670 [ti_cpsw]
    [<c053c3d4>] __dev_open+0xb0/0x114
    [<c053c690>] __dev_change_flags+0x9c/0x14c
    [<c053c760>] dev_change_flags+0x20/0x50
    [<c054bdcc>] do_setlink+0x2cc/0x78c
    [<c054c358>] rtnl_setlink+0xcc/0x100
    [<c054b34c>] rtnetlink_rcv_msg+0x184/0x224
    [<c056467c>] netlink_rcv_skb+0xa8/0xc4
    [<c054b1c0>] rtnetlink_rcv+0x2c/0x34
    [<c0564018>] netlink_unicast+0x16c/0x1f8
    [<c0564498>] netlink_sendmsg+0x334/0x348
    [<c052015c>] sock_sendmsg+0x1c/0x2c
    [<c05213e0>] SyS_sendto+0xc0/0xe8

unreferenced object 0xec861780 (size 192):
  comm "softirq", pid 0, jiffies 4294938759 (age 109.540s)
  hex dump (first 32 bytes):
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00 00 00 00 00 b0 5a ed 00 00 00 00 00 00 00 00  ......Z.........
  backtrace:
    [<c0107830>] kmem_cache_alloc+0x190/0x208
    [<c052c768>] __build_skb+0x30/0x98
    [<c052c8fc>] __netdev_alloc_skb+0xb8/0x104
    [<bf1abc54>] cpsw_rx_handler+0x68/0x1e4 [ti_cpsw]
    [<bf11aa30>] __cpdma_chan_free+0xa8/0xc4 [davinci_cpdma]
    [<bf11ab98>] __cpdma_chan_process+0x14c/0x16c [davinci_cpdma]
    [<bf11abfc>] cpdma_chan_process+0x44/0x5c [davinci_cpdma]
    [<bf1adc78>] cpsw_rx_poll+0x1c/0x9c [ti_cpsw]
    [<c0539180>] net_rx_action+0x1f0/0x2ec
    [<c003881c>] __do_softirq+0x134/0x258
    [<c0038a00>] do_softirq+0x68/0x70
    [<c0038adc>] __local_bh_enable_ip+0xd4/0xe8
    [<c0640994>] _raw_spin_unlock_bh+0x30/0x34
    [<c05f4e9c>] igmp6_group_added+0x4c/0x1bc
    [<c05f6600>] ipv6_dev_mc_inc+0x398/0x434
    [<c05dba74>] addrconf_dad_work+0x224/0x39c

This happens because CPSW allocates SK buffers and then passes
pointers on them in CPDMA where they stored in internal CPPI RAM
(SRAM) which belongs to DEV MMIO space. Kmemleak does not scan IO
memory and so reports memory leaks.

Hence, mark allocated sk buffers as false positive explicitly.

Cc: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The struct cpdma_desc_pool->used_desc field can be safely removed from
CPDMA driver (and hot patch) because used_descs counter is used just
for pool consistency check at CPDMA deinitialization and now this
check can be re-implemnted using gen_pool_size(pool->gen_pool) !=
gen_pool_avail(pool->gen_pool).
More over, this will allow to get rid of warnings in
cpdma_desc_pool_destro()-> WARN_ON(pool->used_desc) which may happen
because the used_descs is used unprotected, since CPDMA has been
switched to use genalloc, and may get wrong values on SMP.

Hence, remove used_desc from struct cpdma_desc_pool.
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Reviewed-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

of_node_put needs to be called when the device node which is got
from of_parse_phandle has finished using.
Signed-off-by: NPeter Chen <peter.chen@nxp.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use of_platform_depopulate() in cpsw_remove() instead of
of_device_unregister(), because CSPW child devices will not be
recreated otherwise on next insmod. of_platform_depopulate() is
correct way now as it will ensure that all steps done in
of_platform_populate() are reverted, including cleaning up of
OF_POPULATED flag.
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The L3 error will be generated and system will crash during unloading
of CPSW driver if CPSW is used as module and ethX devices are down.
This happens because CPSW can be power off by PM runtime now when ethX
devices are down.

Hence, ensure that CPSW powered up by PM runtime before performing any
deinitialization actions which require CPSW registers access. In case
of PM runtime error just leave cpsw_remove() as we can't do anything
anymore.
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Fix deadlock in cpdma_ctlr_destroy() which is triggered now on
cpsw module removal:
 cpsw_remove()
 - cpdma_ctlr_destroy()
   - spin_lock_irqsave(&ctlr->lock, flags)
   - cpdma_ctlr_stop()
     - spin_lock_irqsave(&ctlr->lock, flags);
   - cpdma_chan_destroy()
     - spin_lock_irqsave(&ctlr->lock, flags);

The issue has not been observed before because CPDMA channels have
been destroyed manually by CPSW until commit d941ebe8 ("net:
ethernet: ti: cpsw: use destroy ctlr to destroy channels") was merged.
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Reviewed-by: NMugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Such a big dump of register values is hardly useful on a production
system.

Another downside of the now removed functions is that calling
emac_dump_regs resulted in at least 87 calls to dev_info while holding a
spinlock and having irqs off which is a big source of latency.
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add the missing free_netdev() before return from function
cpmac_probe() in the error handling case.
This patch revert commit 0465be8f ("net: cpmac: fix in
releasing resources"), which changed to only free_netdev
while register_netdev failed.
Signed-off-by: NWei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In 'cpmac_open', 'dma_alloc_coherent' has been used to allocate some
resources, so we need to free them using 'dma_free_coherent' instead
of 'kfree'.

Also, we don't need to free these resources if the allocation has failed.
So I have slighly modified the goto label in this case.
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

TI_CPSW_PHY_SEL depended on TI_CPSW and was selected by the latter. So
there is no reason to have this symbol visible.

A further optimisation would be to put the code for both symbols into a
single module which would allow to not export at least cpsw_phy_sel()
and simplify the module load process.
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There are two generics functions phy_ethtool_{get|set}_link_ksettings,
so we can use them instead of defining the same code in the driver.

There was a check on CAP_NET_ADMIN in cpmac_set_settings, but this
check is already done in dev_ethtool, so no need to repeat it before
calling the generic function.
Signed-off-by: NPhilippe Reynes <tremyfr@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The private structure contain a pointer to phydev, but the structure
net_device already contain such pointer. So we can remove the pointer
phy in the private structure, and update the driver to use the
one contained in struct net_device.
Signed-off-by: NPhilippe Reynes <tremyfr@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There are two generics functions phy_ethtool_{get|set}_link_ksettings,
so we can use them instead of defining the same code in the driver.
Signed-off-by: NPhilippe Reynes <tremyfr@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The private structure contain a pointer to phydev, but the structure
net_device already contain such pointer. So we can remove the pointer
phy in the private structure, and update the driver to use the
one contained in struct net_device.
Signed-off-by: NPhilippe Reynes <tremyfr@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

TI CPDMA currently uses a bitmap for tracking descriptors alloactions
allocations, but The genalloc already handles the same and can be used
as with special memory (SRAM) as with DMA cherent memory chank
(dma_alloc_coherent()). Hence, switch to using genalloc and add
desc_num property for each channel for limitation of max number of
allowed descriptors for each CPDMA channel. This patch do not affect
on net throuput.
Acked-by: NMugunthan V N <mugunthanvnm@ti.com>
Tested-by: NIvan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use "ti,cpsw-mdio" to enable PM runtime auto-suspend on supported
platforms, where MDIO is implemented as part of TI CPSW.
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Davinci MDIO is always used as slave device which services
read/write requests from MDIO/PHY core. It doesn't use IRQ also.

As result, It's possible to relax PM runtime constraints for Davinci
MDIO and enable it on demand, instead of powering it during probe
and powering off during removal.

Hence, implement PM runtime autosuspend for Davinci MDIO, but keep it
disabled by default, because Davinci MDIO is integrated in big set of
TI devices and not all of them expected to work corectly with RPM
 autosuspend enabled:
- expected to work on SoCs where MDIO is part of TI CPSW
(cpsw.c DRA7/am57x, am437x, am335x, dm814x)
- not verified on Keystone 2 and other SoCs where MDIO is used with TI EMAC IP
(davinci_emac.c:  dm6467-emac, am3517-emac, dm816-emac).

Davinci MDIO RPM autosuspend can be enabled through sysfs:
 echo 100 > /sys/devices/../48484000.ethernet/48485000.mdio/power/autosuspend_delay_ms
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add PM runtime .runtime_suspend()/.runtime_resume() callbacks and
perform Davinci MDIO enabling/disabling from these callbacks. This
allows to reuse pm_runtime_force_suspend/resume() APIs during System
suspend and required for further implementation of PM runtime
autosuspend.
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The Davinci MDIO MDIO_CONTROL.CLKDIV can be calculated only once
during probe, hence split __davinci_mdio_reset() on
davinci_mdio_init_clk() and davinci_mdio_enable(). Initialize and
save CLKDIV in .probe(). Then just use saved value.
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

It's not expected Davinci MDIO to be accessible after its suspend
callbacks have been called:
 - all consumers of Davinci MDIO will stop/disconnect phys at Device
suspend stage;
 - all phys are expected to be suspned already by PHY/MDIO core;
 - MDIO locking is done by MDIO Bus code.

Hence, it's safe to drop "suspended" and "lock" fields from mdio_data.
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

PM runtime is disabled when Davinci MDIO .suspend_late() and
.resume_early() callbacks are called. As result, any PM runtime calls here will
be just a nop and can be removed.
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>