This wraps single ioctl() commands into block requests using
the custom block layer request types REQ_OP_DRV_IN and
REQ_OP_DRV_OUT.

By doing this we are loosening the grip on the big host lock,
since two calls to mmc_get_card()/mmc_put_card() are removed.

We are storing the ioctl() in/out argument as a pointer in
the per-request struct mmc_blk_request container. Since we
now let the block layer allocate this data, blk_get_request()
will allocate it for us and we can immediately dereference
it and use it to pass the argument into the block layer.

We refactor the if/else/if/else ladder in mmc_blk_issue_rq()
as part of the job, keeping some extra attention to the
case when a NULL req is passed into this function and
making that pipeline flush more explicit.

Tested on the ux500 with the userspace:
mmc extcsd read /dev/mmcblk3
resulting in a successful EXTCSD info dump back to the
console.

This commit fixes a starvation issue in the MMC/SD stack
that can be easily provoked in the following way by
issueing the following commands in sequence:

> dd if=/dev/mmcblk3 of=/dev/null bs=1M &
> mmc extcs read /dev/mmcblk3

Before this patch, the extcsd read command would hang
(starve) while waiting for the dd command to finish since
the block layer was holding the card/host lock.

After this patch, the extcsd ioctl() command is nicely
interpersed with the rest of the block commands and we
can issue a bunch of ioctl()s from userspace while there
is some busy block IO going on without any problems.

Conversely userspace ioctl()s can no longer starve
the block layer by holding the card/host lock.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NAvri Altman <Avri.Altman@sandisk.com>

The variable is_rpmb is clearly a bool and even assigned true
and false, yet declared as an int.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

The mmc_queue_req is a per-request state container the MMC core uses
to carry bounce buffers, pointers to asynchronous requests and so on.
Currently allocated as a static array of objects, then as a request
comes in, a mmc_queue_req is assigned to it, and used during the
lifetime of the request.

This is backwards compared to how other block layer drivers work:
they usally let the block core provide a per-request struct that get
allocated right beind the struct request, and which can be obtained
using the blk_mq_rq_to_pdu() helper. (The _mq_ infix in this function
name is misleading: it is used by both the old and the MQ block
layer.)

The per-request struct gets allocated to the size stored in the queue
variable .cmd_size initialized using the .init_rq_fn() and
cleaned up using .exit_rq_fn().

The block layer code makes the MMC core rely on this mechanism to
allocate the per-request mmc_queue_req state container.

Doing this make a lot of complicated queue handling go away. We only
need to keep the .qnct that keeps count of how many request are
currently being processed by the MMC layer. The MQ block layer will
replace also this once we transition to it.

Doing this refactoring is necessary to move the ioctl() operations
into custom block layer requests tagged with REQ_OP_DRV_[IN|OUT]
instead of the custom code using the BigMMCHostLock that we have
today: those require that per-request data be obtainable easily from
a request after creating a custom request with e.g.:

struct request *rq = blk_get_request(q, REQ_OP_DRV_IN, __GFP_RECLAIM);
struct mmc_queue_req *mq_rq = req_to_mq_rq(rq);

And this is not possible with the current construction, as the request
is not immediately assigned the per-request state container, but
instead it gets assigned when the request finally enters the MMC
queue, which is way too late for custom requests.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
[Ulf: Folded in the fix to drop a call to blk_cleanup_queue()]
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NHeiner Kallweit <hkallweit1@gmail.com>

This option is activated by all multiplatform configs and what
not so we almost always have it turned on, and the memory it
saves is negligible, even more so moving forward. The actual
bounce buffer only gets allocated only when used, the only
thing the ifdefs are saving is a little bit of code.

It is highly improper to have this as a Kconfig option that
get turned on by Kconfig, make this a pure runtime-thing and
let the host decide whether we use bounce buffers. We add a
new property "disable_bounce" to the host struct.

Notice that mmc_queue_calc_bouncesz() already disables the
bounce buffers if host->max_segs != 1, so any arch that has a
maximum number of segments higher than 1 will have bounce
buffers disabled.

The option CONFIG_MMC_BLOCK_BOUNCE is default y so the
majority of platforms in the kernel already have it on, and
it then gets turned off at runtime since most of these have
a host->max_segs > 1. The few exceptions that have
host->max_segs == 1 and still turn off the bounce buffering
are those that disable it in their defconfig.

Those are the following:

arch/arm/configs/colibri_pxa300_defconfig
arch/arm/configs/zeus_defconfig
- Uses MMC_PXA, drivers/mmc/host/pxamci.c
- Sets host->max_segs = NR_SG, which is 1
- This needs its bounce buffer deactivated so we set
  host->disable_bounce to true in the host driver

arch/arm/configs/davinci_all_defconfig
- Uses MMC_DAVINCI, drivers/mmc/host/davinci_mmc.c
- This driver sets host->max_segs to MAX_NR_SG, which is 16
- That means this driver anyways disabled bounce buffers
- No special action needed for this platform

arch/arm/configs/lpc32xx_defconfig
arch/arm/configs/nhk8815_defconfig
arch/arm/configs/u300_defconfig
- Uses MMC_ARMMMCI, drivers/mmc/host/mmci.[c|h]
- This driver by default sets host->max_segs to NR_SG,
  which is 128, unless a DMA engine is used, and in that case
  the number of segments are also > 1
- That means this driver already disables bounce buffers
- No special action needed for these platforms

arch/arm/configs/sama5_defconfig
- Uses MMC_SDHCI, MMC_SDHCI_PLTFM, MMC_SDHCI_OF_AT91, MMC_ATMELMCI
- Uses drivers/mmc/host/sdhci.c
- Normally sets host->max_segs to SDHCI_MAX_SEGS which is 128 and
  thus disables bounce buffers
- Sets host->max_segs to 1 if SDHCI_USE_SDMA is set
- SDHCI_USE_SDMA is only set by SDHCI on PCI adapers
- That means that for this platform bounce buffers are already
  disabled at runtime
- No special action needed for this platform

arch/blackfin/configs/CM-BF533_defconfig
arch/blackfin/configs/CM-BF537E_defconfig
- Uses MMC_SPI (a simple MMC card connected on SPI pins)
- Uses drivers/mmc/host/mmc_spi.c
- Sets host->max_segs to MMC_SPI_BLOCKSATONCE which is 128
- That means this platform already disables bounce buffers at
  runtime
- No special action needed for these platforms

arch/mips/configs/cavium_octeon_defconfig
- Uses MMC_CAVIUM_OCTEON, drivers/mmc/host/cavium.c
- Sets host->max_segs to 16 or 1
- Setting host->disable_bounce to be sure for the 1 case

arch/mips/configs/qi_lb60_defconfig
- Uses MMC_JZ4740, drivers/mmc/host/jz4740_mmc.c
- This sets host->max_segs to 128 so bounce buffers are
  already runtime disabled
- No action needed for this platform

It would be interesting to come up with a list of the platforms
that actually end up using bounce buffers. I have not been
able to infer such a list, but it occurs when
host->max_segs == 1 and the bounce buffering is not explicitly
disabled.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

For hosts not supporting MMC_CAP2_SDIO_IRQ_NOTHREAD but MMC_CAP_SDIO_IRQ,
the SDIO IRQs are processed from a dedicated kernel thread. For these
cases, the host calls mmc_signal_sdio_irq() from its ISR to signal a new
SDIO IRQ.

Signaling an SDIO IRQ makes the host's ->enable_sdio_irq() callback to be
invoked to temporary disable the IRQs, before the kernel thread is woken up
to process it. When processing of the IRQs are completed, they are
re-enabled by the kernel thread, again via invoking the host's
->enable_sdio_irq().

The observation from this, is that the execution path is being unnecessary
complex, as the host driver already knows that it needs to temporary
disable the IRQs before signaling a new one. Moreover, replacing the kernel
thread with a work/workqueue would not only greatly simplify the code, but
also make it more robust.

To address the above problems, let's continue to build upon the support for
MMC_CAP2_SDIO_IRQ_NOTHREAD, as it already implements SDIO IRQs to be
processed without using the clumsy kernel thread and without the ping-pong
calls of the host's ->enable_sdio_irq() callback for each processed IRQ.

Therefore, let's add new API sdio_signal_irq(), which enables hosts to
signal/process SDIO IRQs by using a work/workqueue, rather than using the
kernel thread.

Add also a new host callback ->ack_sdio_irq(), which the work invokes when
the SDIO IRQs have been processed. This informs the host about when it
shall re-enable the SDIO IRQs. Potentially, we could re-use the existing
->enable_sdio_irq() callback instead of adding a new one, however it has
turned out that it's more convenient for hosts to get this information via
a separate callback.

Hosts that wants to use this new method to signal/process SDIO IRQs, must
enable MMC_CAP2_SDIO_IRQ_NOTHREAD and implement the ->ack_sdio_irq()
callback.
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NDouglas Anderson <dianders@chromium.org>
Reviewed-by: NDouglas Anderson <dianders@chromium.org>

In cases when MMC_CAP2_SDIO_IRQ_NOTHREAD is set, there is a minor window
for when the mmc host could call sdio_run_irqs(), while in fact an SDIO
func driver could have decided to released the SDIO IRQ via a call to
sdio_release_irq(). In this scenario, processing of the SDIO IRQs are done
even if there is none IRQ claimed, which is not what we want.

To prevent this from happen, close the window by validating that at least
one SDIO IRQs is claimed, before deciding to process them.
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NDouglas Anderson <dianders@chromium.org>
Reviewed-by: NDouglas Anderson <dianders@chromium.org>

In case if a pwrseq-emmc has been bound to the host, a call to
mmc_power_up() triggers an eMMC HW reset via the pwrseq_emmc's
->post_power_on() callback. This isn't really what we want, as
mmc_power_up() is called each time when resuming the card.

As a matter of fact, the current approach may also violate the eMMC spec,
as the involved delays managed in pwrseq_emmc assumes both VCC and VCCQ has
been turned on, which isn't the case for VCCQ, unless the regulator is
always on.

Fix this behaviour by aligning to the same procedure used when the mmc host
implements the ->hw_reset() callback and has the MMC_CAP_HW_RESET flag set.
In this way the eMMC HW reset is issued at card detection scan, to cope
with bogus bootloaders and in the error recovery path via the mmc specific
bus_ops->reset() callback.
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NMarek Szyprowski <m.szyprowski@samsung.com>

The ->reset() callback is needed to implement a better support for eMMC HW
reset. The following changes will take advantage of the new callback.
Suggested-by: NHeiner Kallweit <hkallweit1@gmail.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NMarek Szyprowski <m.szyprowski@samsung.com>

If the optional power-off-delay-us property is found, insert the
corresponding delay after asserting the GPIO during power off. This enables
a graceful shutdown sequence for some devices.

Cc: linux-mmc@vger.kernel.org
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>

Some hosts controllers, like Cavium, needs to know whether the card
operates in byte- or block-address mode. Therefore export a new API,
mmc_card_is_blockaddr(), which provides this information.
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Signed-off-by: NSteven J. Hill <Steven.Hill@cavium.com>
Acked-by: NDavid Daney <david.daney@cavium.com>

The host may issue an I/O abort by writing to the CCCR at any time
during I/O read operation via CMD52. And host may need suspend
transcation during write busy stage in SDIO suspend/resume scenario.
>From other side, a card may accept CMD52 during data transfer phase.

Previous implement would block issuing above command in busy stage.
It cause function driver can't implement as proper way and has no
opportunity to do some coverage in error case via I/O abort etc.

We need bypass some necessary operation during busy check stage.
Signed-off-by: NJiajie Hao <jiajie.hao@mediatek.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Annotate big endian values correctly and make sparse happy.
In mmc_app_send_scr remove scr function parameter as it was
updating card->raw_scr anyway.
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

We have this construction:

if (a && b && !c)
   finalize;
else
   block;
   finalize;

Which is equivalent by boolean logic to:

if (!a || !b || c)
   block;
finalize;

Which is simpler code.
Reviewed-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

mmc_wait_for_data_req_done() is called in exactly one place,
and having it spread out is making things hard to oversee.
Factor this function into mmc_finalize_areq().
Reviewed-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

"previous" is a better name for the variable storing the previous
asynchronous request, better than the opaque name "data" atleast.
We see that we assign the return status to the returned variable
on all code paths, so we might as well just do that immediately
after calling mmc_finalize_areq().
Reviewed-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

In preparation to reuse the code for CQE support.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

In preparation to reuse the code for CQE support.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Factor out data preparation into a separate function mmc_blk_data_prep()
which can be re-used for command queuing.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

mmc_apply_rel_rw() will be used by Software Command Queuing also. In that
case the command argument is not the block address so change
mmc_apply_rel_rw() to get block address from the request.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

RPMB does not allow Command Queue commands. Disable and re-enable the
Command Queue when switching.

Note that the driver only switches partitions when the queue is empty.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NHarjani Ritesh <riteshh@codeaurora.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Normal read and write commands may not be used while the command queue is
enabled. Disable the Command Queue when mmc_test is probed and re-enable it
when it is removed.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NHarjani Ritesh <riteshh@codeaurora.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Add helper functions to enable or disable the Command Queue.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

eMMC can have multiple internal partitions that are represented as separate
disks / queues. However switching between partitions is only done when the
queue is empty. Consequently the array of mmc requests that are queued can
be shared between partitions saving memory.

Keep a pointer to the mmc request queue on the card, and use that instead
of allocating a new one for each partition.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Change from viewing the requests in progress as 'current' and 'previous',
to viewing them as a queue. The current request is allocated to the first
free slot. The presence of incomplete requests is determined from the
count (mq->qcnt) of entries in the queue. Non-read-write requests (i.e.
discards and flushes) are not added to the queue at all and require no
special handling. Also no special handling is needed for the
MMC_BLK_NEW_REQUEST case.

As well as allowing an arbitrarily sized queue, the queue thread function
is significantly simpler.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

A subsequent patch will remove 'mq->mqrq_cur'. Prepare for that by
assigning it to a local variable.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Improve mmc_io_rw_extended a little:
- using DIV_ROUND_UP achieves the same but is better readable
- simplify code by using sg_set_buf
- simplify one statement by using -= operator
Signed-off-by: NHeiner Kallweit <hkallweit1@gmail.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

File contains multiple functions doing variations on the same thing,
sdio_readb(), sdio_writeb()f, sdio_readw(), sdio_writew()
etc. Although the functions have very similar logic the code is laid
out in a variety of ways. This makes it overly complicated to
read. There is a already a nice clean chunk of code, if we use this
format for all instances then we will have cleaned up the code,
reduced the line count and lessened the cognitive load required while
reading. Less lines equals less bugs.

Pick the most simple and clear code flow and change all functions to
be the same.
Signed-off-by: NTobin C. Harding <me@tobin.cc>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Various functions take as parameter an optional pointer. Pointer
should be guarded with non-NULL check before dereferencing.

Add non-NULL check before dereference of pointer.
Signed-off-by: NTobin C. Harding <me@tobin.cc>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Certain 64-bit systems (e.g. Amlogic Meson GX) require buffers to be
used for DMA to be 8-byte-aligned. struct sdio_func has an embedded
small DMA buffer not meeting this requirement.
When testing switching to descriptor chain mode in meson-gx driver
SDIO is broken therefore. Fix this by allocating the small DMA buffer
separately as kmalloc ensures that the returned memory area is
properly aligned for every basic data type.
Signed-off-by: NHeiner Kallweit <hkallweit1@gmail.com>
Tested-by: NHelmut Klein <hgkr.klein@gmail.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

mmc only supports discarding on large alignments, so the zeroing code
would always fall back to explicit writings of zeroes.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NHannes Reinecke <hare@suse.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

Fix compilation warning:

drivers/mmc/core/block.c:1563:24: warning: variable ‘mq_rq’ set but not
used [-Wunused-but-set-variable]  struct mmc_queue_req *mq_rq;
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Commit 4e1f7800 ("mmc: block: break out mmc_blk_rw_cmd_abort()")
assumed the request had not completed, but in one case it had. Fix that.

Fixes: 4e1f7800 ("mmc: block: break out mmc_blk_rw_cmd_abort()")
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Commit 15520111 ("mmc: core: Further fix thread wake-up") allowed a
queue to release the host with is_waiting_last_req set to true. A queue
waiting to claim the host will not reset it, which can result in the
queue getting stuck in a loop.

Fixes: 15520111 ("mmc: core: Further fix thread wake-up")
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Cc: stable@vger.kernel.org # v4.10+
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

HS400-ES devices fail to initialize with the following error messages.

mmc1: power class selection to bus width 8 ddr 0 failed
mmc1: error -110 whilst initialising MMC card

This was seen on Samsung Chromebook Plus. Code analysis points to
commit 3d4ef329 ("mmc: core: fix multi-bit bus width without
high-speed mode"), which attempts to set the bus width for all but
HS200 devices unconditionally. However, for HS400-ES, the bus width
is already selected.

Cc: Anssi Hannula <anssi.hannula@bitwise.fi>
Cc: Douglas Anderson <dianders@chromium.org>
Cc: Brian Norris <briannorris@chromium.org>
Fixes: 3d4ef329 ("mmc: core: fix multi-bit bus width ...")
Signed-off-by: NGuenter Roeck <linux@roeck-us.net>
Reviewed-by: NDouglas Anderson <dianders@chromium.org>
Reviewed-by: NShawn Lin <shawn.lin@rock-chip.com>
Tested-by: NHeiko Stuebner <heiko@sntech.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

We are going to move scheduler ABI details to <uapi/linux/sched/types.h>,
which will be used from a number of .c files.

Create empty placeholder header that maps to <linux/types.h>.

Include the new header in the files that are going to need it.
Acked-by: NLinus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: NIngo Molnar <mingo@kernel.org>

That makes all the quirks table look more consistent.
Signed-off-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

It's not appreciated to place quirks everywhere, let's
put them together just like what we do for USB, PCI etc.
Signed-off-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Rename mmc_fixup_methods to sdio_fixup_methods to better
reflect that it's for sdio devices. So we could also pass
on it from sdio card's probe sequence just like what we do
for eMMC and block there.
Signed-off-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Consolidate all the sdio devices' IDs into sdio_ids.
Signed-off-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Rename quirks.c to quirks.h, and include it for
individual C files which need it.
Signed-off-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>