Add SYSFW defined rchan_oes_offset number for J721S2 SoC in soc data.
Signed-off-by: NAswath Govindraju <a-govindraju@ti.com>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20211119132315.15901-2-a-govindraju@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

udma_get_*() checks if rchan/tchan/rflow is already allocated by checking
if it has a NON NULL value. For the error cases, rchan/tchan/rflow will
have error value and udma_get_*() considers this as already allocated
(PASS) since the error values are NON NULL. This results in NULL pointer
dereference error while de-referencing rchan/tchan/rflow.

Reset the value of rchan/tchan/rflow to NULL if a channel request fails.

CC: stable@vger.kernel.org
Acked-by: NPeter Ujfalusi <peter.ujfalusi@gmail.com>
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Link: https://lore.kernel.org/r/20211031032411.27235-3-kishon@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

bcdma_get_*() checks if bchan is already allocated by checking if it
has a NON NULL value. For the error cases, bchan will have error value
and bcdma_get_*() considers this as already allocated (PASS) since the
error values are NON NULL. This results in NULL pointer dereference
error while de-referencing bchan.

Reset the value of bchan to NULL if a channel request fails.

CC: stable@vger.kernel.org
Acked-by: NPeter Ujfalusi <peter.ujfalusi@gmail.com>
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Link: https://lore.kernel.org/r/20211031032411.27235-2-kishon@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Fix the following fallthrough warning:

drivers/dma/ti/k3-udma.c:4951:2: warning: unannotated fall-through between switch labels [-Wimplicit-fallthrough]
Reported-by: NNathan Chancellor <nathan@kernel.org>
Signed-off-by: NGustavo A. R. Silva <gustavoars@kernel.org>

In 'dma_pool_create()', we return -ENOMEM, but don't release the resources
already allocated, as in all the other error handling paths.

Go to 'err_res_free' instead of returning directly.

Fixes: 01779473 ("dmaengine: ti: k3-udma: Initial support for K3 BCDMA")
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20210124070923.724479-1-christophe.jaillet@wanadoo.frSigned-off-by: NVinod Koul <vkoul@kernel.org>

The ret does not need to be initialized to 0 in the tisci channel config
functions.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20210113114923.9231-4-peter.ujfalusi@gmail.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

The UDMA and BCDMA can provide higher throughput if the burst_size of the
channel is changed from it's default (which is 64 bytes) for Ultra-high
and high capacity channels.

This performance benefit is even more visible when the buffers are aligned
with the burst_size configuration.

The am654 does not have a way to change the burst size, but it is using
64 bytes burst, so increasing the copy_align from 8 bytes to 64 (and
clients taking that into account) can increase the throughput as well.

Numbers gathered on j721e:
echo 8000000 > /sys/module/dmatest/parameters/test_buf_size
echo 2000 > /sys/module/dmatest/parameters/timeout
echo 50 > /sys/module/dmatest/parameters/iterations
echo 1 > /sys/module/dmatest/parameters/max_channels

Prior this patch:       ~1.3 GB/s
After this patch:       ~1.8 GB/s
 with 1 byte alignment: ~1.7 GB/s
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@gmail.com>
Tested-by: NKishon Vijay Abraham I <kishon@ti.com>
Link: https://lore.kernel.org/r/20210113114923.9231-3-peter.ujfalusi@gmail.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

BCDMA RX channels have one flow per channel, therefore set the rflow_cnt
to rchan_cnt.

Without this patch, request for BCDMA RX channel allocation fails as
rflow_cnt is 0 thus fails to reserve a rflow for the channel.

Fixes: 88448980 ("dmaengine: ti: k3-udma: Add support for BCDMA channel TPL handling")
Signed-off-by: NVignesh Raghavendra <vigneshr@ti.com>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20210112141403.30286-1-vigneshr@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Instead of initializing the rchan_tpl the initial commit re-initialized
the tchan_tpl.

Fixes: d2abc982 ("dmaengine: ti: k3-udma: Initial support for K3 PKTDMA")
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201216154833.20821-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

One of the DMAs introduced with AM64 is the Packet DMA (PKTDMA).
It serves similar purpose as K3 UDMAP channels in packet mode, but with
notable differences, like tflow support and channels being allocated to
service specific peripherals.
The rings for the PKTDMA is integrated within the DMA itself instead of
using rings from the general purpose ringacc.

PKTDMA can be used to service PSI-L peripherals, similarly to
K3 UDMA channels.

Most of the driver code can be reused for PKTDMA tchan/rchan support but
new setup and allocation functions are needed to handle the differences
between the DMAs.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-20-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Unlike UDMAP the BCDMA defines the channel TPL levels per channel type.
In UDMAP the number of high and ultra-high channels applies to both tchan
and rchan.

BCDMA defines the TPL per channel types: bchan, tchan and rchan can have
different number of high and ultra-high channels.

In order to support BCDMA channel TPL we need to move the tpl information
as per channel type property for the DMAs.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-19-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

One of the DMAs introduced with AM64 is the Block Copy DMA (BCDMA).
It serves similar purpose as K3 UDMAP channels in TR mode.

The rings for the BCDMA is integrated within the DMA itself instead of
using rings from the general purpose ringacc.

A BCDMA have two different type of channels:
- Block Copy Channels (bchan)
- Split Channels (tchan and rchan)

tchan and rchan can be used to service PSI-L peripherals, similarly to
K3 UDMA channels.

bchan can be only used for block copy operation (TR type15) like the
paired K3 UDMA tchan/rchan configured in block copy mode.
bchans can be also used to service peripherals directly if an external
trigger is selected for the channel.

Most of the driver code can be reused for BCDMA bchan/tchan/rchan support
but new setup and allocation functions are needed to handle the
differences between the DMAs.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-18-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Resource allocation via sysfw can use up to two ranges per resource subtype
to support more complex resource assignment, mainly for DMA channels.

Take the second range also into consideration when setting up the maps for
available resources.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-4-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Set the TDTYPE if it is supported on the platform (j721e) which will cause
UDMAP to wait for the remote peer to finish the teardown before returning
the teardown completed message.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Tested-by: NKeerthy <j-keerthy@ti.com>
Reviewed-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-3-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

According to different sections of the TRM, the hchan_cnt of CAP3 includes
the number of uchan in UDMA, thus the start offset of the normal channels
are hchan_cnt.

Fixes: daf4ad04 ("dmaengine: ti: k3-udma: Query throughput level information from hardware")
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-2-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Running in hardIRQ, disabling IRQ is redundant since hardIRQ has disabled
IRQ. This patch removes the irqsave and irqstore to save some instruction
cycles.
Signed-off-by: NBarry Song <song.bao.hua@hisilicon.com>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201027215252.25820-3-song.bao.hua@hisilicon.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Use the devm_platform_ioremap_resource_byname() helper instead of
calling platform_get_resource_byname() and devm_ioremap_resource()
separately.
Signed-off-by: NZhang Qilong <zhangqilong3@huawei.com>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200921093701.102208-1-zhangqilong3@huawei.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

In preparation for unconditionally passing the
struct tasklet_struct pointer to all tasklet
callbacks, switch to using the new tasklet_setup()
and from_tasklet() to pass the tasklet pointer explicitly.
Signed-off-by: NAllen Pais <allen.lkml@gmail.com>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200831103542.305571-36-allen.lkml@gmail.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Use separate data for SoC dependent parameters. These parameters depends
on the DMA integration (either in HW or in SYSFW), the DMA controller
itself remains compatible with either the am654 or j721e variant.

j7200 have the same DMA as j721e with different number of channels, which
can be queried from HW, but SYSFW defines different rchan_oes_offset
number for j7200 (0x80) compared to j721e (0x400).
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200910124329.21206-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

SYSFW ABI 3.0 has changed the rchan_oes_offset value for am654 to support
SR2.

Since the kernel now needs SYSFW API 3.0 to work because the merged irqchip
update, we need to also update the am654 rchan_oes_offset.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200831091019.25273-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

The direction has been already validated in the main callback and there is
no need to check it again in the TR mode handlers for slave_sg and cyclic.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200824120120.9270-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

The TR which needs to be initialized for the next sg entry is indexed by
tr_idx and not by the running i counter.

In case any sub element in the SG needs more than one TR, the code would
corrupt an already configured TR.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200824120108.9178-1-peter.ujfalusi@ti.com
Fixes: 6cf668a4 ("dmaengine: ti: k3-udma: Use the TR counter helper for slave_sg and cyclic")
Signed-off-by: NVinod Koul <vkoul@kernel.org>

The CAP3 register contains information about the number of
HCHAN (High Capacity) and UCHAN (Ultra High Capacity) channels in UDMAP.

Based on this information the start indexes of the levels can be calculated
without a need of a table in the match data.

On am654 the CAP3 does not contain information about the number different
channels. Set up the tpl information based on the available documentation.

This change will allow to use the same compatible for different SoCs where
the only difference is the number of channel types.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200717120903.8774-3-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Add defines for the CAP register fields to make the code self explaining.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200717120903.8774-2-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

We only request ring pairs via K3 DMA driver, switch to use the new
k3_ringacc_request_rings_pair() to simplify the code.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Acked-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NSantosh Shilimkar <santosh.shilimkar@oracle.com>

Make use of the struct_size() helper instead of an open-coded version
in order to avoid any potential type mistakes.

This code was detected with the help of Coccinelle and, audited and
fixed manually.
Signed-off-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Addresses-KSPP-ID: https://github.com/KSPP/linux/issues/83
Link: https://lore.kernel.org/r/20200619224334.GA7857@embeddedorSigned-off-by: NVinod Koul <vkoul@kernel.org>

Move the uc->tchan/rchan checks to the IO wrappers itself instead of
calling the functions with tchan/rchan directly.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20200707102352.28773-6-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

The register offsets and functions are the same among TCHAN and RCHAN.
Use generic, common names for them.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20200707102352.28773-4-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

The ring_get_occ is redundant as the k3_ringacc_ring_pop() is also
checking the occ of the ring.

With removing the ring_get_occ, the function can be simplified as well.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20200707102352.28773-3-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

The descriptors are allocated via wither dma_pool or dma_alloc_coherent.

There is no need for the dma_sync_singel_* calls.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NGrygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20200707102352.28773-2-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

INIT_DELAYED_WORK_ONSTACK() must be used with on-stack delayed work, which
is not the case here.
Use normal delayed_work for the channels instead.

Fixes: 25dcb5dd ("dmaengine: ti: New driver for K3 UDMA")
Reported-by: NTomi Valkeinen <tomi.valkeinen@ti.com>
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200618114004.6268-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

In the unlikely case when the channel is running (RT enabled) during
alloc_chan_resources then we should use udma_reset_chan() and not
udma_stop() as the later is trying to initiate a teardown on the channel,
which is not valid at this point.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200527070612.636-3-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Some of the earlier errors should be sent to the error cleanup path to
make sure that the uchan struct is reset, the dma_pool (if allocated) is
released and memcpy channel pairs are released in a correct way.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200527070612.636-2-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

The "ti,udma-atype" property is expected in the UDMA node and not in the
parent navss node.

Fixes: 0ebcf1a2 ("dmaengine: ti: k3-udma: Implement support for atype (for virtualization)")
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200527065357.30791-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

There is a regular need in the kernel to provide a way to declare having a
dynamically sized set of trailing elements in a structure. Kernel code should
always use “flexible array members”[1] for these cases. The older style of
one-element or zero-length arrays should no longer be used[2].

[1] https://en.wikipedia.org/wiki/Flexible_array_member
[2] https://github.com/KSPP/linux/issues/21Signed-off-by: NGustavo A. R. Silva <gustavoars@kernel.org>

In udma_alloc_chan_resources() if the channel is not willing to stop then
the function should return with error code.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200512134519.5642-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

The in_ring_cnt is not used for anything, it can be removed.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200512134611.6015-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

The TR mode rx flush descriptor did not had a dma_sync_single_for_device()
call to make sure that the DMA see the correct information.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200512134544.5839-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

cppi5_tr_csf_set() clears previously set Configuration Specific Flags.
Setting the EOP flag clears the SUPR_EVT flag for the last TR which is not
desirable as we do not want to have events from the TR.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200512134531.5742-1-peter.ujfalusi@ti.comSigned-off-by: NVinod Koul <vkoul@kernel.org>

Fixes coccicheck warnings:

drivers/dma/ti/k3-udma.c:1294:1-3: WARNING: PTR_ERR_OR_ZERO can be used
drivers/dma/ti/k3-udma.c:1311:1-3: WARNING: PTR_ERR_OR_ZERO can be used
drivers/dma/ti/k3-udma.c:1376:1-3: WARNING: PTR_ERR_OR_ZERO can be used
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NSamuel Zou <zou_wei@huawei.com>
Link: https://lore.kernel.org/r/1588757146-38858-1-git-send-email-zou_wei@huawei.comSigned-off-by: NVinod Koul <vkoul@kernel.org>