[ Upstream commit ab0fb17c7d46406e1aac2dda265874751946626d ]

A recent commit removed the incorrect use of phys_to_dma from this
driver, but failed to remove the dma-direct.h include, so do that
now.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>
Signed-off-by: NSasha Levin <sashal@kernel.org>

commit 81d9bdf59092e4755fc4307c93c4589ef0fe2e0f upstream.

This patch fixes a memory corruption that occurred in the
qcom-nandc driver since it was converted to nand_scan().

On boot, an affected device will panic from a NPE at a weird place:
| Unable to handle kernel NULL pointer dereference at virtual address 0
| pgd = (ptrval)
| [00000000] *pgd=00000000
| Internal error: Oops: 80000005 [#1] SMP ARM
| CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.19.9 #0
| Hardware name: Generic DT based system
| PC is at   (null)
| LR is at nand_block_isbad+0x90/0xa4
| pc : [<00000000>]    lr : [<c0592240>]    psr: 80000013
| sp : cf839d40  ip : 00000000  fp : cfae9e20
| r10: cf815810  r9 : 00000000  r8 : 00000000
| r7 : 00000000  r6 : 00000000  r5 : 00000001  r4 : cf815810
| r3 : 00000000  r2 : cfae9810  r1 : ffffffff  r0 : cf815810
| Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
| Control: 10c5387d  Table: 8020406a  DAC: 00000051
| Process swapper/0 (pid: 1, stack limit = 0x(ptrval))
| [<c0592240>] (nand_block_isbad) from [<c0580a94>]
| [<c0580a94>] (allocate_partition) from [<c05811e4>]
| [<c05811e4>] (add_mtd_partitions) from [<c0581164>]
| [<c0581164>] (parse_mtd_partitions) from [<c057def4>]
| [<c057def4>] (mtd_device_parse_register) from [<c059d274>]
| [<c059d274>] (qcom_nandc_probe) from [<c0567f00>]

The problem is that the nand_scan()'s qcom_nand_attach_chip callback
is updating the nandc->max_cwperpage from 1 to 4. This causes the
sg_init_table of clear_bam_transaction() in the driver's
qcom_nandc_block_bad() to memset much more than what was initially
allocated by alloc_bam_transaction().

This patch restores the old behavior by reallocating the shared bam
transaction alloc_bam_transaction() after the chip was identified,
but before mtd_device_parse_register() (which is an alias for
mtd_device_register() - see panic) gets called. This fixes the
corruption and the driver is working again.

Cc: stable@vger.kernel.org
Fixes: 6a3cec64 ("mtd: rawnand: qcom: convert driver to nand_scan()")
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Acked-by: NMiquel Raynal <miquel.raynal@bootlin.com>
Signed-off-by: NBoris Brezillon <bbrezillon@kernel.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

[ Upstream commit 33bf5519ae5dd356b182a94e3622f42860274a38 ]

PAGE_READ is used by RISC-V arch code included through mm headers,
and it makes sense to bring in a prefix on these in the driver.

drivers/mtd/nand/raw/qcom_nandc.c:153: warning: "PAGE_READ" redefined
 #define PAGE_READ   0x2
In file included from include/linux/memremap.h:7,
                 from include/linux/mm.h:27,
                 from include/linux/scatterlist.h:8,
                 from include/linux/dma-mapping.h:11,
                 from drivers/mtd/nand/raw/qcom_nandc.c:17:
arch/riscv/include/asm/pgtable.h:48: note: this is the location of the previous definition

Caught by riscv allmodconfig.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Reviewed-by: NMiquel Raynal <miquel.raynal@bootlin.com>
Signed-off-by: NBoris Brezillon <boris.brezillon@bootlin.com>
Signed-off-by: NSasha Levin <sashal@kernel.org>

Two helpers have been added to the core to do all kind of controller
side configuration/initialization between the detection phase and the
final NAND scan. Implement these hooks so that we can convert the driver
to just use nand_scan() instead of the nand_scan_ident() +
nand_scan_tail() pair.
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>
Reviewed-by: NBoris Brezillon <boris.brezillon@bootlin.com>

In the raw NAND core, a NAND chip is described by a nand_chip structure,
while a NAND controller is described with a nand_hw_control structure
which is not very meaningful.

Rename this structure nand_controller.

As the structure gets renamed, it is logical to also rename the core
function initializing it from nand_hw_control_init() to
nand_controller_init().

Lastly, the 'hwcontrol' entry of the nand_chip structure is not
meaningful neither while it has the role of fallback when no controller
structure is provided by the driver (the controller driver is dumb and
can only control a single chip). Thus, it is renamed dummy_controller.
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>
Acked-by: NBoris Brezillon <boris.brezillon@bootlin.com>

Compile-testing this driver on x86 caused a link error:

ERROR: "__phys_to_dma" [drivers/mtd/nand/raw/qcom_nandc.ko] undefined!

The problem here is that the driver attempts to convert the physical
address into the DMA controller as a dma_addr_t and calls phys_to_dma()
to do the conversion.

The correct way to do the conversion is using the dma mapping interfaces.

Fixes: c76b78d8 ("mtd: nand: Qualcomm NAND controller driver")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Reviewed-by: NBoris Brezillon <boris.brezillon@bootlin.com>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

NAND parts can have bitflips in an erased page due to the
process technology used. In this case, QCOM NAND controller
is not able to identify that page as an erased page.
Currently the driver calls nand_check_erased_ecc_chunk() for
identifying the erased pages but this won’t work always since the
checking is being with ECC engine returned data. In case of
bitflips, the ECC engine tries to correct the data and then it
generates the uncorrectable error. Now, this data is not equal to
original raw data. For erased CW identification, the raw data
should be read again from NAND device and this
nand_check_erased_ecc_chunk function() should be called for raw
data only.

Now following logic is being added to identify the erased
codeword bitflips.

1. In most of the cases, not all the codewords will have bitflips
   and only single CW will have bitflips. So, there is no need to
   read the complete raw page data. The NAND raw read can be
   scheduled for any CW in page. The NAND controller works on CW
   basis and it will update the status register after each CW read.
   Maintain the bitmask for the CW which generated the uncorrectable
   error.
2. Do raw read for all the CW's which generated the uncorrectable
   error.
3. Both DATA and OOB need to be checked for number of 0. The
   top-level API can be called with only data buf or OOB buf so use
   chip->databuf if data buf is null and chip->oob_poi if
   OOB buf is null for copying the raw bytes temporarily.
4. For each CW, check the number of 0 in cw_data and usable
   oob bytes, The bbm and spare (unused) bytes bit flip won’t
   affect the ECC so don’t check the number of bitflips in this area.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

Make separate function to perform raw read for one codeword and
call this function multiple times for each codeword in case of
raw page read. This separate function will help in subsequent
patches related with erased codeword bitflip detection.

It will decrease throughput for raw page read. Raw page read
is used for debug purpose so it won't affect normal flash
operations.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

Currently there is no error checking for raw read. For raw
reads, there won’t be any ECC failure but the operational
failures are possible, so schedule the NAND_FLASH_STATUS read
after each codeword.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

Fix value returned by ->read_page_raw() to be the
actual operation status, instead of always 0.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

QCOM NAND controller layout protects available OOB data bytes with
ECC also so when ecc->write_oob() is being called then it
can't update just OOB bytes. Currently, it first reads the last
codeword which includes old OOB bytes. Then it updates the old OOB
bytes with new ones and then again writes the codeword back.
The reading codeword is unnecessary since user is responsible to
have these bytes cleared to 0xFF.

This patch removes the read part and updates the OOB bytes with
data area padded with OxFF.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

read_page and read_oob both calls the read_page_ecc function.
The QCOM NAND controller protect the OOB available bytes with
ECC so read errors should be checked for read_oob also.
This patch moves the error checking code inside read_page_ecc
so caller does not have to check explicitly for read errors.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

parse_read_errors can be called with only oob_buf in which case
data_buf will be NULL.  If data_buf is NULL, then don’t
treat this page as completely erased in case of ECC uncorrectable
error for RS ECC. For BCH ECC, the controller itself tells
regarding erased page in status register.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

Following is the flow in the HW if controller tries to read erased
page:

1. First ECC uncorrectable error will be generated from ECC engine
   since ECC engine first calculates the ECC with all 0xff and match
   the calculated ECC with ECC code in OOB (which is again all 0xff).
2. After getting ECC error, erased CW detection logic will be
   applied which is different for BCH and RS ECC
    a. For BCH, HW checks if all the bytes in page are 0xff and then
       it updates the status in separate register
       NAND_ERASED_CW_DETECT_STATUS.
    b. For RS ECC, the HW reports the same error when reading an
       erased CW, but it notifies that it is an erased CW by
       placing special characters at certain offsets in the
       buffer.

So the erased CW detect status should be checked only if ECC engine
generated the uncorrectable error.

Currently for all other operational errors also (like TIMEOUT, MPU
errors, etc.), the erased CW detect logic is being applied so fix this
and return EIO for other operational errors.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

The BAM has 3 channels - tx, rx and command. command channel
is used for register read/writes, tx channel for data writes
and rx channel for data reads. Currently, the driver assumes the
transfer completion once it gets all the command descriptors
completed. Sometimes, there is race condition between data channel
(tx/rx) and command channel completion. In these cases,
the data present in buffer is not valid during small window
between command descriptor completion and data descriptor
completion.

This patch generates NAND transfer completion when both
(Data and Command) DMA channels have completed all its DMA
descriptors. It assigns completion callback in last
DMA descriptors of that channel and wait for completion.

Fixes: 8d6b6d7e ("mtd: nand: qcom: support for command descriptor formation")
Cc: stable@vger.kernel.org
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

Currently the driver uses the ECC strength specified in DT.
The QPIC/EBI2 NAND supports 4 or 8-bit ECC correction. The same
kind of board can have different NAND parts so use the ECC
strength from device parameters if it is not specified in DT.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

QCOM NAND controller supports only one step size (512) so
nand-ecc-step-size DT property is redundant. This property
can be removed and ecc step size can be assigned with 512 value.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>

The devm_kzalloc() function has a 2-factor argument form, devm_kcalloc().
This patch replaces cases of:

        devm_kzalloc(handle, a * b, gfp)

with:
        devm_kcalloc(handle, a * b, gfp)

as well as handling cases of:

        devm_kzalloc(handle, a * b * c, gfp)

with:

        devm_kzalloc(handle, array3_size(a, b, c), gfp)

as it's slightly less ugly than:

        devm_kcalloc(handle, array_size(a, b), c, gfp)

This does, however, attempt to ignore constant size factors like:

        devm_kzalloc(handle, 4 * 1024, gfp)

though any constants defined via macros get caught up in the conversion.

Any factors with a sizeof() of "unsigned char", "char", and "u8" were
dropped, since they're redundant.

Some manual whitespace fixes were needed in this patch, as Coccinelle
really liked to write "=devm_kcalloc..." instead of "= devm_kcalloc...".

The Coccinelle script used for this was:

// Fix redundant parens around sizeof().
@@
expression HANDLE;
type TYPE;
expression THING, E;
@@

(
  devm_kzalloc(HANDLE,
-	(sizeof(TYPE)) * E
+	sizeof(TYPE) * E
  , ...)
|
  devm_kzalloc(HANDLE,
-	(sizeof(THING)) * E
+	sizeof(THING) * E
  , ...)
)

// Drop single-byte sizes and redundant parens.
@@
expression HANDLE;
expression COUNT;
typedef u8;
typedef __u8;
@@

(
  devm_kzalloc(HANDLE,
-	sizeof(u8) * (COUNT)
+	COUNT
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(__u8) * (COUNT)
+	COUNT
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(char) * (COUNT)
+	COUNT
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(unsigned char) * (COUNT)
+	COUNT
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(u8) * COUNT
+	COUNT
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(__u8) * COUNT
+	COUNT
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(char) * COUNT
+	COUNT
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(unsigned char) * COUNT
+	COUNT
  , ...)
)

// 2-factor product with sizeof(type/expression) and identifier or constant.
@@
expression HANDLE;
type TYPE;
expression THING;
identifier COUNT_ID;
constant COUNT_CONST;
@@

(
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(TYPE) * (COUNT_ID)
+	COUNT_ID, sizeof(TYPE)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(TYPE) * COUNT_ID
+	COUNT_ID, sizeof(TYPE)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(TYPE) * (COUNT_CONST)
+	COUNT_CONST, sizeof(TYPE)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(TYPE) * COUNT_CONST
+	COUNT_CONST, sizeof(TYPE)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(THING) * (COUNT_ID)
+	COUNT_ID, sizeof(THING)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(THING) * COUNT_ID
+	COUNT_ID, sizeof(THING)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(THING) * (COUNT_CONST)
+	COUNT_CONST, sizeof(THING)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(THING) * COUNT_CONST
+	COUNT_CONST, sizeof(THING)
  , ...)
)

// 2-factor product, only identifiers.
@@
expression HANDLE;
identifier SIZE, COUNT;
@@

- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	SIZE * COUNT
+	COUNT, SIZE
  , ...)

// 3-factor product with 1 sizeof(type) or sizeof(expression), with
// redundant parens removed.
@@
expression HANDLE;
expression THING;
identifier STRIDE, COUNT;
type TYPE;
@@

(
  devm_kzalloc(HANDLE,
-	sizeof(TYPE) * (COUNT) * (STRIDE)
+	array3_size(COUNT, STRIDE, sizeof(TYPE))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(TYPE) * (COUNT) * STRIDE
+	array3_size(COUNT, STRIDE, sizeof(TYPE))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(TYPE) * COUNT * (STRIDE)
+	array3_size(COUNT, STRIDE, sizeof(TYPE))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(TYPE) * COUNT * STRIDE
+	array3_size(COUNT, STRIDE, sizeof(TYPE))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(THING) * (COUNT) * (STRIDE)
+	array3_size(COUNT, STRIDE, sizeof(THING))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(THING) * (COUNT) * STRIDE
+	array3_size(COUNT, STRIDE, sizeof(THING))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(THING) * COUNT * (STRIDE)
+	array3_size(COUNT, STRIDE, sizeof(THING))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(THING) * COUNT * STRIDE
+	array3_size(COUNT, STRIDE, sizeof(THING))
  , ...)
)

// 3-factor product with 2 sizeof(variable), with redundant parens removed.
@@
expression HANDLE;
expression THING1, THING2;
identifier COUNT;
type TYPE1, TYPE2;
@@

(
  devm_kzalloc(HANDLE,
-	sizeof(TYPE1) * sizeof(TYPE2) * COUNT
+	array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(TYPE1) * sizeof(THING2) * (COUNT)
+	array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(THING1) * sizeof(THING2) * COUNT
+	array3_size(COUNT, sizeof(THING1), sizeof(THING2))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(THING1) * sizeof(THING2) * (COUNT)
+	array3_size(COUNT, sizeof(THING1), sizeof(THING2))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(TYPE1) * sizeof(THING2) * COUNT
+	array3_size(COUNT, sizeof(TYPE1), sizeof(THING2))
  , ...)
|
  devm_kzalloc(HANDLE,
-	sizeof(TYPE1) * sizeof(THING2) * (COUNT)
+	array3_size(COUNT, sizeof(TYPE1), sizeof(THING2))
  , ...)
)

// 3-factor product, only identifiers, with redundant parens removed.
@@
expression HANDLE;
identifier STRIDE, SIZE, COUNT;
@@

(
  devm_kzalloc(HANDLE,
-	(COUNT) * STRIDE * SIZE
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  devm_kzalloc(HANDLE,
-	COUNT * (STRIDE) * SIZE
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  devm_kzalloc(HANDLE,
-	COUNT * STRIDE * (SIZE)
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  devm_kzalloc(HANDLE,
-	(COUNT) * (STRIDE) * SIZE
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  devm_kzalloc(HANDLE,
-	COUNT * (STRIDE) * (SIZE)
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  devm_kzalloc(HANDLE,
-	(COUNT) * STRIDE * (SIZE)
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  devm_kzalloc(HANDLE,
-	(COUNT) * (STRIDE) * (SIZE)
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  devm_kzalloc(HANDLE,
-	COUNT * STRIDE * SIZE
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
)

// Any remaining multi-factor products, first at least 3-factor products,
// when they're not all constants...
@@
expression HANDLE;
expression E1, E2, E3;
constant C1, C2, C3;
@@

(
  devm_kzalloc(HANDLE, C1 * C2 * C3, ...)
|
  devm_kzalloc(HANDLE,
-	(E1) * E2 * E3
+	array3_size(E1, E2, E3)
  , ...)
|
  devm_kzalloc(HANDLE,
-	(E1) * (E2) * E3
+	array3_size(E1, E2, E3)
  , ...)
|
  devm_kzalloc(HANDLE,
-	(E1) * (E2) * (E3)
+	array3_size(E1, E2, E3)
  , ...)
|
  devm_kzalloc(HANDLE,
-	E1 * E2 * E3
+	array3_size(E1, E2, E3)
  , ...)
)

// And then all remaining 2 factors products when they're not all constants,
// keeping sizeof() as the second factor argument.
@@
expression HANDLE;
expression THING, E1, E2;
type TYPE;
constant C1, C2, C3;
@@

(
  devm_kzalloc(HANDLE, sizeof(THING) * C2, ...)
|
  devm_kzalloc(HANDLE, sizeof(TYPE) * C2, ...)
|
  devm_kzalloc(HANDLE, C1 * C2 * C3, ...)
|
  devm_kzalloc(HANDLE, C1 * C2, ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(TYPE) * (E2)
+	E2, sizeof(TYPE)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(TYPE) * E2
+	E2, sizeof(TYPE)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(THING) * (E2)
+	E2, sizeof(THING)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	sizeof(THING) * E2
+	E2, sizeof(THING)
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	(E1) * E2
+	E1, E2
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	(E1) * (E2)
+	E1, E2
  , ...)
|
- devm_kzalloc
+ devm_kcalloc
  (HANDLE,
-	E1 * E2
+	E1, E2
  , ...)
)
Signed-off-by: NKees Cook <keescook@chromium.org>

SET/GET FEATURES are flagged ONFI-compliant because of their name. This
is not accurate as non-ONFI NAND chips support it and use it.

Rename the hooks and helpers to remove the "onfi" prefix.
Signed-off-by: NMiquel Raynal <miquel.raynal@bootlin.com>
Signed-off-by: NBoris Brezillon <boris.brezillon@bootlin.com>

As part of the process of sharing more code between different NAND
based devices, we need to move all raw NAND related code to the raw/
subdirectory.
Signed-off-by: NBoris Brezillon <boris.brezillon@bootlin.com>

phys_to_dma, dma_to_phys and dma_capable are helpers published by
architecture code for use of swiotlb and xen-swiotlb only.  Drivers are
not supposed to use these directly, but use the DMA API instead.

Move these to a new asm/dma-direct.h helper, included by a
linux/dma-direct.h wrapper that provides the default linear mapping
unless the architecture wants to override it.

In the MIPS case the existing dma-coherent.h is reused for now as
untangling it will take a bit of work.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NRobin Murphy <robin.murphy@arm.com>

devm_kasprintf() may fail, so we should better add a NULL check
and propagate an error on failure.
Signed-off-by: NFabio Estevam <fabio.estevam@nxp.com>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

The core currently send the READ0 and SEQIN+PAGEPROG commands in
nand_do_read/write_ops(). This is inconsistent with
->read/write_oob[_raw]() hooks behavior which are expected to send
these commands.

There's already a flag (NAND_ECC_CUSTOM_PAGE_ACCESS) to inform the core
that a specific controller wants to send the READ/SEQIN+PAGEPROG
commands on its own, but it's an opt-in flag, and existing drivers are
unlikely to be updated to pass it.

Moreover, some controllers cannot dissociate the READ/PAGEPROG commands
from the associated data transfer and ECC engine activation, and
developers have to hack things in their ->cmdfunc() implementation to
handle such complex cases, or have to accept the perf penalty of sending
twice the same command.
To address this problem we are planning on adding a new interface which
is passed all information about a NAND operation (including the amount
of data to transfer) and replacing all calls to ->cmdfunc() to calls to
this new ->exec_op() hook. But, in order to do that, we need to have all
->cmdfunc() calls placed near their associated ->read/write_buf/byte()
calls.

Modify the core and relevant drivers to make NAND_ECC_CUSTOM_PAGE_ACCESS
the default case, and remove this flag.
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>
[miquel.raynal@free-electrons.com: tested, fixed and rebased on nand/next]
Signed-off-by: NMiquel Raynal <miquel.raynal@free-electrons.com>
Acked-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

This is part of the process of removing direct calls to ->cmdfunc()
outside of the core in order to introduce a better interface to execute
NAND operations.

Here we provide several helpers and make use of them to remove all
direct calls to ->cmdfunc(). This way, we can easily modify those
helpers to make use of the new ->exec_op() interface when available.
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>
[miquel.raynal@free-electrons.com: rebased and fixed some conflicts]
Signed-off-by: NMiquel Raynal <miquel.raynal@free-electrons.com>
Acked-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

1. Add the function for command descriptor preparation which will
   be used only by BAM DMA and it will form the DMA descriptors
   containing command elements
2. DMA_PREP_CMD flag should be used for forming command DMA
   descriptors
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

All the QPIC register read/write through BAM DMA requires
command descriptor which contains the array of command elements.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

Add the compatible string for IPQ8074 QPIC NAND controller
version 1.5.0 which uses BAM DMA and its FLASH_DEV_CMD registers
starting offset is 0x7000.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

Add the compatible string for IPQ4019 QPIC NAND controller
version 1.4.0 which uses BAM DMA.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

The FLASH_DEV_CMD registers starting offset is not same in
different QPIC NAND controller versions. This patch adds
the starting offset in NAND controller properties and uses
the same for calculating the actual offset of these registers.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

1. Add the data descriptor preparation function which will be used
   only by BAM DMA for forming the data SGL’s
2. Add clear BAM transaction and call it before every new request
3. Check DMA mode for ADM or BAM and call the appropriate
   descriptor formation function.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

1. DM_EN is only required for EBI2 NAND controller which uses ADM
2. BAM mode will be disabled after power on reset which needs to
   be enabled before starting any BAM transfers.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

The NAND controller returns ECC failure during read of completely
erased codeword. The NAND controller has hardware functionality
to detect erased codeword in case of BCH ECC algorithm. The
NAND_ERASED_CW_DETECT_CFG register controls the erased
codeword/page detection controller. This register should be reset
before every page read by setting and clearing bit 0 of
NAND_ERASED_CW_DETECT_CFG.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

In EBI2, all codeword data will be read in FLASH_BUF_ACC buffer
and ADM will copy the data from source (FLASH_BUF_ACC) to
destination (memory for data read).

In QPIC, there is no FLASH_BUF_ACC and all the codeword data will
held in QPIC BAM FIFO buffers. It provides multiple READ_LOCATION
registers which will be used for copying the data from FIFO to
memory. The READ_LOCATION register will be used to read a
specific amount of data from a specific offset within the flash
buffer. It supports sequential offset requests. Each request is
composed of the following fields:

a. Offset within the flash buffer from which data should be
   read
b. Amount of data to be read
c. Flag bit specifying the last read request from the flash
   buffer. Following the last read request the NANDc refers to the
   buffer as empty.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

The QPIC NAND BAM has multiple flags to control the transfer. This
patch adds flags parameter in register and data transfer DMA helper
functions and modifies all these functions call with appropriate
flags using following rule

1. Read and write can’t go in single command descriptor so
   separate SGL should be used.
2. For some of the requests, NWD flag should be set in BAM
   DMA descriptor.
3. For Data write, the BAM has internal buffer for each codeword.
   All write request will modify the data in internal buffer and
   this buffer will be flushed to NAND device once EOT flag is set.
   So for all the write requests in single codeword, the EOT should
   be cleared for all tx data descriptors except the last one.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

1. prepare_bam_async_desc is the function which will call
   all the DMA API’s. It will fetch the outstanding scatter gather
   list for passed channel and will do the DMA descriptor formation.
   The DMA flag is dependent upon the type of channel.

2. For ADM DMA, the descriptor is being formed for every DMA
   request so its sgl count will be always 1 while in BAM DMA, the
   clubbing of descriptor is being done to increase throughput.

3. ADM DMA uses only one channel while in BAM DMA, data descriptors
   will be submitted to tx channel (for write) or rx channel
   (for read) and all the registers read/write descriptors in
   command channel.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

- The BAM transaction is the core data structure which will be used
  for all the data transfers in QPIC NAND. Since the core framework
  in nand_base.c is serializing all the NAND requests so allocating
  BAM transaction before every transfer will be overhead. The memory
  for it be allocated during probe time and before every transfer,
  it will be cleared.

- The BAM transaction contains the array of
  command and data scatter gather list and indexes. For
  every transfer, all the resource will be taken from BAM
  transaction.

- The size of the buffer used for BAM transactions
  is calculated based on the NAND device with the maximum page size,
  among all the devices connected to the
  controller.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

The EBI2 NAND controller directly remaps register read buffer with
dma_map_sg and DMA address of this buffer will be passed to DMA
API’s. While, on QPIC NAND controller, which uses BAM DMA, we read
the controller registers by preparing a BAM command descriptor. This
command descriptor requires the

  - controller register address
  - the DMA address in which we want to store the value read
    back from the controller register.

This command descriptor will be remapped with dma_map_sg
and its DMA address will be passed to DMA API’s. Therefore,
it's required that we also map our register read buffer for
DMA (using dma_map_single). We use the returned DMA address
for preparing entries in our command descriptor.

This patch adds the DMA mapping support for register read
buffer. This buffer will be DMA mapped during allocation
time. Before starting of any operation, this buffer will
be synced for device operation and after operation
completion, it will be synced again for CPU.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

1. QPIC NAND controller uses 3 BAM channels: command, data tx
   and data rx while EBI2 NAND controller uses only single ADM
   channel.
2. CRCI is only required for ADM DMA and it's not required for
   BAM DMA.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

The current driver only supports EBI2 NAND controller which uses
ADM DMA. The latest QCOM SoC uses QPIC NAND controller with BAM
DMA. NAND registers and programming sequence are same for EBI2
and QPIC NAND so the same driver can support QPIC NAND also by
adding the BAM DMA support. This patch adds the is_bam in NAND
property which will be checked for determining the DMA engine type.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>

Currently driver data is being assigned directly with ECC modes.
Now, the plan is to add more NAND controller versions which will
have different properties. This patch reorganizes the current driver
data assignment by creating NAND controller properties structure
which will contain all properties specific to NAND controller.
Reviewed-by: NArchit Taneja <architt@codeaurora.org>
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>