In function mmc_exit_request, the request_queue structure(*q) is not used.
I remove the unnecessary code related to the request_queue structure.
Signed-off-by: NChanWoo Lee <cw9316.lee@samsung.com>
Acked-by: NAdrian Hunter <adrian.hunter@intel.com>
Link: https://lore.kernel.org/r/20210825074601.8881-1-cw9316.lee@samsung.comSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Each function has a different data type for max_segments,
Modify to match unsigned short(host->max_segs).
* unsigned short          max_segs;       /* see blk_queue_max_segments */

1) Return type : unsigned int
static unsigned int mmc_get_max_segments(struct mmc_host *host)
{
       return host->can_dma_map_merge ? MMC_DMA_MAP_MERGE_SEGMENTS :
                                         host->max_segs;
}

2) Parameter type : int
 mmc_alloc_sg(mmc_get_max_segments(host), gfp);
   -> static struct scatterlist *mmc_alloc_sg(int sg_len, gfp_t gfp)

3) Parameter type : unsigned short
 blk_queue_max_segments(mq->queue, mmc_get_max_segments(host));
   -> void blk_queue_max_segments(struct request_queue *q,
                                        unsigned short max_segments)
Signed-off-by: NChanWoo Lee <cw9316.lee@samsung.com>
Acked-by: NColy Li <colyli@suse.de>
Link: https://lore.kernel.org/r/20210824073934.19727-1-cw9316.lee@samsung.comSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Use the blk_mq_alloc_disk to allocate the request_queue and gendisk
together.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NUlf Hansson <ulf.hansson@linaro.org>
Link: https://lore.kernel.org/r/20210616053934.880951-3-hch@lst.deSigned-off-by: NJens Axboe <axboe@kernel.dk>

The host->cqe_enabled is already containing the needed information about
whether the CQE is enabled or not, hence there is no need to keep another
copy of it around.
Signed-off-by: NLuca Porzio <lporzio@micron.com>
Signed-off-by: NZhan Liu <zliua@micron.com>
Link: https://lore.kernel.org/r/20210215003217.GA12240@lupo-laptopSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

MMC_CQE_QUEUE_FULL is not set and is only cleared.
Therefore, define is unnecessary.
Signed-off-by: NChanWoo Lee <cw9316.lee@samsung.com>
Acked-by: NAdrian Hunter <adrian.hunter@intel.com>
Link: https://lore.kernel.org/r/20210203072014.30272-1-cw9316.lee@samsung.comSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

From the 4.19 kernel, thread related code has been removed in queue.c.
So we can exclude unnecessary header file.
Signed-off-by: NChanWoo Lee <cw9316.lee@samsung.com>
Acked-by: NColy Li <colyli@suse.de>
Link: https://lore.kernel.org/r/20210125064355.28545-1-cw9316.lee@samsung.comSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

In preparation for adding CQHCI crypto engine (inline encryption)
support, add the code required to make mmc_core and mmc_block aware of
inline encryption.  Specifically:

- Add a capability flag MMC_CAP2_CRYPTO to struct mmc_host.  Drivers
  will set this if the host and driver support inline encryption.

- Embed a blk_keyslot_manager in struct mmc_host.  Drivers will
  initialize this (as a device-managed resource) if the host and driver
  support inline encryption.  mmc_block registers this keyslot manager
  with the request_queue of any MMC card attached to the host.

- Make mmc_block copy the crypto keyslot and crypto data unit number
  from struct request to struct mmc_request, so that drivers will have
  access to them.

- If the MMC host is reset, reprogram all the keyslots to ensure that
  the software state stays in sync with the hardware state.
Co-developed-by: NSatya Tangirala <satyat@google.com>
Signed-off-by: NSatya Tangirala <satyat@google.com>
Acked-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NSatya Tangirala <satyat@google.com>
Reviewed-and-tested-by: NPeng Zhou <peng.zhou@mediatek.com>
Signed-off-by: NEric Biggers <ebiggers@google.com>
Link: https://lore.kernel.org/r/20210126001456.382989-2-ebiggers@kernel.orgSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

If extended CSD was not available, the eMMC driver would incorrectly
set the block size to 0, as the data_sector_size field of ext_csd
was never initialized. This issue was exposed by commit 817046ec
("block: Align max_hw_sectors to logical blocksize") which caused
max_sectors and max_hw_sectors to be set to 0 after setting the block
size to 0, resulting in a kernel panic in bio_split when attempting
to read from the device. Fix it by only reading the block size from
ext_csd if it is available.

Fixes: a5075eb9 ("mmc: block: Allow disabling 512B sector size emulation")
Signed-off-by: NPeter Collingbourne <pcc@google.com>
Reviewed-by: NDamien Le Moal <damien.lemoal@wdc.com>
Link: https://linux-review.googlesource.com/id/If244d178da4d86b52034459438fec295b02d6e60Acked-by: NAdrian Hunter <adrian.hunter@intel.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20210114201405.2934886-1-pcc@google.comSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

In mmc_queue_setup_discard() the mmc driver queue's discard_granularity
might be set as 0 (when card->pref_erase > max_discard) while the mmc
device still declares to support discard operation. This is buggy and
triggered the following kernel warning message,

WARNING: CPU: 0 PID: 135 at __blkdev_issue_discard+0x200/0x294
CPU: 0 PID: 135 Comm: f2fs_discard-17 Not tainted 5.9.0-rc6 #1
Hardware name: Google Kevin (DT)
pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--)
pc : __blkdev_issue_discard+0x200/0x294
lr : __blkdev_issue_discard+0x54/0x294
sp : ffff800011dd3b10
x29: ffff800011dd3b10 x28: 0000000000000000 x27: ffff800011dd3cc4 x26: ffff800011dd3e18 x25: 000000000004e69b x24: 0000000000000c40 x23: ffff0000f1deaaf0 x22: ffff0000f2849200 x21: 00000000002734d8 x20: 0000000000000008 x19: 0000000000000000 x18: 0000000000000000 x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 x14: 0000000000000394 x13: 0000000000000000 x12: 0000000000000000 x11: 0000000000000000 x10: 00000000000008b0 x9 : ffff800011dd3cb0 x8 : 000000000004e69b x7 : 0000000000000000 x6 : ffff0000f1926400 x5 : ffff0000f1940800 x4 : 0000000000000000 x3 : 0000000000000c40 x2 : 0000000000000008 x1 : 00000000002734d8 x0 : 0000000000000000 Call trace:
__blkdev_issue_discard+0x200/0x294
__submit_discard_cmd+0x128/0x374
__issue_discard_cmd_orderly+0x188/0x244
__issue_discard_cmd+0x2e8/0x33c
issue_discard_thread+0xe8/0x2f0
kthread+0x11c/0x120
ret_from_fork+0x10/0x1c
---[ end trace e4c8023d33dfe77a ]---

This patch fixes the issue by setting discard_granularity as SECTOR_SIZE
instead of 0 when (card->pref_erase > max_discard) is true. Now no more
complain from __blkdev_issue_discard() for the improper value of discard
granularity.

This issue is exposed after commit b35fd742 ("block: check queue's
limits.discard_granularity in __blkdev_issue_discard()"), a "Fixes:" tag
is also added for the commit to make sure people won't miss this patch
after applying the change of __blkdev_issue_discard().

Fixes: e056a1b5 ("mmc: queue: let host controllers specify maximum discard timeout")
Fixes: b35fd742 ("block: check queue's limits.discard_granularity in __blkdev_issue_discard()").
Reported-and-tested-by: NVicente Bergas <vicencb@gmail.com>
Signed-off-by: NColy Li <colyli@suse.de>
Acked-by: NAdrian Hunter <adrian.hunter@intel.com>
Cc: Ulf Hansson <ulf.hansson@linaro.org>
Link: https://lore.kernel.org/r/20201002013852.51968-1-colyli@suse.deSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

The BDI_CAP_STABLE_WRITES is one of the few bits of information in the
backing_dev_info shared between the block drivers and the writeback code.
To help untangling the dependency replace it with a queue flag and a
superblock flag derived from it.  This also helps with the case of e.g.
a file system requiring stable writes due to its own checksumming, but
not forcing it on other users of the block device like the swap code.

One downside is that we an't support the stable_pages_required bdi
attribute in sysfs anymore.  It is replaced with a queue attribute which
also is writable for easier testing.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJan Kara <jack@suse.cz>
Reviewed-by: NJohannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Correcting this misspelling squashes the following W=1 build warning(s):

 mmc/core/queue.c:212: warning: Function parameter or member 'mq' not described in '__mmc_init_request'
 mmc/core/queue.c:212: warning: Excess function parameter 'q' description in '__mmc_init_request'
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Link: https://lore.kernel.org/r/20200701124702.908713-8-lee.jones@linaro.orgSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

First, it should be noted that the CQE timeout (60 seconds) is substantial
so a CQE request that times out is really stuck, and the race between
timeout and completion is extremely unlikely. Nevertheless this patch
fixes an issue with it.

Commit ad73d6fe ("mmc: complete requests from ->timeout")
preserved the existing functionality, to complete the request.
However that had only been necessary because the block layer
timeout handler had been marking the request to prevent it from being
completed normally. That restriction was removed at the same time, the
result being that a request that has gone will have been completed anyway.
That is, the completion was unnecessary.

At the time, the unnecessary completion was harmless because the block
layer would ignore it, although that changed in kernel v5.0.

Note for stable, this patch will not apply cleanly without patch "mmc:
core: Fix recursive locking issue in CQE recovery path"
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Fixes: ad73d6fe ("mmc: complete requests from ->timeout")
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20200508062227.23144-1-adrian.hunter@intel.comSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Consider the following stack trace

-001|raw_spin_lock_irqsave
-002|mmc_blk_cqe_complete_rq
-003|__blk_mq_complete_request(inline)
-003|blk_mq_complete_request(rq)
-004|mmc_cqe_timed_out(inline)
-004|mmc_mq_timed_out

mmc_mq_timed_out acquires the queue_lock for the first
time. The mmc_blk_cqe_complete_rq function also tries to acquire
the same queue lock resulting in recursive locking where the task
is spinning for the same lock which it has already acquired leading
to watchdog bark.

Fix this issue with the lock only for the required critical section.

Cc: <stable@vger.kernel.org>
Fixes: 1e8e55b6 ("mmc: block: Add CQE support")
Suggested-by: NSahitya Tummala <stummala@codeaurora.org>
Signed-off-by: NSarthak Garg <sartgarg@codeaurora.org>
Acked-by: NAdrian Hunter <adrian.hunter@intel.com>
Link: https://lore.kernel.org/r/1588868135-31783-1-git-send-email-vbadigan@codeaurora.orgSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Now the MMC read/write stack will always wait for previous request is
completed by mmc_blk_rw_wait(), before sending a new request to hardware,
or queue a work to complete request, that will bring context switching
overhead and spend some extra time to poll the card for busy completion
for I/O writes via sending CMD13, especially for high I/O per second
rates, to affect the IO performance.

Thus this patch introduces MMC software queue interface based on the
hardware command queue engine's interfaces, which is similar with the
hardware command queue engine's idea, that can remove the context
switching. Moreover we set the default queue depth as 64 for software
queue, which allows more requests to be prepared, merged and inserted
into IO scheduler to improve performance, but we only allow 2 requests
in flight, that is enough to let the irq handler always trigger the
next request without a context switch, as well as avoiding a long latency.

Moreover the host controller should support HW busy detection for I/O
operations when enabling the host software queue. That means, the host
controller must not complete a data transfer request, until after the
card stops signals busy.

From the fio testing data in cover letter, we can see the software
queue can improve some performance with 4K block size, increasing
about 16% for random read, increasing about 90% for random write,
though no obvious improvement for sequential read and write.

Moreover we can expand the software queue interface to support MMC
packed request or packed command in future.
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBaolin Wang <baolin.wang@linaro.org>
Signed-off-by: NBaolin Wang <baolin.wang7@gmail.com>
Link: https://lore.kernel.org/r/4409c1586a9b3ed20d57ad2faf6c262fc3ccb6e2.1581478568.git.baolin.wang7@gmail.comSigned-off-by: NUlf Hansson <ulf.hansson@linaro.org>

The commit 38c38cb7 ("mmc: queue: use bigger segments if DMA MAP
layer can merge the segments") always enables the bugger segments
if DMA MAP layer can merge the segments, but some controllers (SDHCI)
have strictly limitation about the segments size, and then the commit
breaks on the controllers.

To fix the issue, this patch adds a new flag MMC_CAP2_MERGE_CAPABLE
into the struct mmc_host and the bigger segments usage is disabled
as default.
Reported-by: NThierry Reding <treding@nvidia.com>
Fixes: 38c38cb7 ("mmc: queue: use bigger segments if DMA MAP layer can merge the segments")
Signed-off-by: NYoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Acked-by: NUlf Hansson <ulf.hansson@linaro.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

When the max_segs of a mmc host is smaller than 512, the mmc
subsystem tries to use 512 segments if DMA MAP layer can merge
the segments, and then the mmc subsystem exposes such information
to the block layer by using blk_queue_can_use_dma_map_merging().
Signed-off-by: NYoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: NUlf Hansson <ulf.hansson@linaro.org>
Reviewed-by: NSimon Horman <horms+renesas@verge.net.au>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

While using the mmc_spi driver occasionally errors like this popped up:

mmcblk0: error -84 transferring data end_request: I/O error, dev mmcblk0, sector 581756

I looked on the Internet for occurrences of the same problem and came
across a helpful post [1]. It includes source code to reproduce the bug.
There is also an analysis about the cause. During transmission data in the
supplied buffer is being modified. Thus the previously calculated checksum
is not correct anymore.

After some digging I found out that device drivers are supposed to report
they need stable writes. To fix this I set the appropriate flag at queue
initialization if CRC checksumming is enabled for that SPI host.

[1]
https://groups.google.com/forum/#!msg/sim1/gLlzWeXGFr8/KevXinUXfc8JSigned-off-by: NAndreas Koop <andreas.koop@zf.com>
[shihpo: Rebase on top of v5.3-rc1]
Signed-off-by: NShihPo Hung <shihpo.hung@sifive.com>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
CC: stable@vger.kernel.org
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Just like we do for all other block drivers. Especially as the limit
imposed at the moment might be way to pessimistic for iommus.

This also means we are not going to set a bounce limit for the queue, in
case we have a dma mask. On most architectures it was never needed, the
major hold out was x86-32 with PAE, but that has been fixed by now.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Based on 2 normalized pattern(s):

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license version 2 as
  published by the free software foundation

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license version 2 as
  published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

  GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Reviewed-by: NEnrico Weigelt <info@metux.net>
Reviewed-by: NKate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: NAllison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.deSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

If we only set the max_segment_size on the queue an IOMMU merge might
create bigger segments again, so limit the IOMMU merges as well.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

The tag set is allocated in mmc_init_queue but never freed. This results
in a memory leak. This change makes sure we free the tag set when the
queue is also freed.
Signed-off-by: NRaul E Rangel <rrangel@chromium.org>
Reviewed-by: NJens Axboe <axboe@kernel.dk>
Acked-by: NAdrian Hunter <adrian.hunter@intel.com>
Fixes: 81196976 ("mmc: block: Add blk-mq support")
Cc: stable@vger.kernel.org
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

Logical block size is the lowest possible block size that the storage
device can address. Max segment size is often related with controller's
DMA capability. And it is reasonable to align max segment size with
logical block size.

SDHCI sets un-aligned max segment size, and causes ADMA error, so
fix it by aligning max segment size with logical block size.
Reported-by: NNaresh Kamboju <naresh.kamboju@linaro.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Naresh Kamboju <naresh.kamboju@linaro.org>
Cc: Faiz Abbas <faiz_abbas@ti.com>
Cc: linux-block@vger.kernel.org
Signed-off-by: NMing Lei <ming.lei@redhat.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too.
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NOmar Sandoval <osandov@fb.com>
Signed-off-by: NMing Lei <ming.lei@redhat.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Replace the lock in mmc_blk_data that is only used through a pointer
in struct mmc_queue and to protect fields in that structure with
an actual lock in struct mmc_queue.
Suggested-by: NUlf Hansson <ulf.hansson@linaro.org>
Reviewed-by: NUlf Hansson <ulf.hansson@linaro.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

mmc uses the block layer struct request pointer to indirect their own
lock to the mmc_queue structure, given that the original lock isn't
reachable outside of block.c.  Add a lock pointer to struct mmc_queue
instead and stop overriding the block layer lock which protects fields
entirely separate from the mmc use.
Reviewed-by: NHannes Reinecke <hare@suse.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Merge three functions initializing the queue into a single one, and drop
an unused argument for it.
Reviewed-by: NUlf Hansson <ulf.hansson@linaro.org>
Reviewed-by: NHannes Reinecke <hare@suse.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

The mmc block driver does not support parallel dispatch of requests. In
normal circumstances, all requests are anyway funneled through a single
work item, so parallel dispatch never happens. However it can happen if
there is no elevator.

Fix that by detecting if a dispatch is in progress and returning busy
(BLK_STS_RESOURCE) in that case

Fixes: 81196976 ("mmc: block: Add blk-mq support")
Cc: stable@vger.kernel.org # v4.16+
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

By completing the request entirely in the driver we can remove the
BLK_EH_HANDLED return value and thus the split responsibility between the
driver and the block layer that has been causing trouble.

[While this keeps existing behavior it seems to mismatch the comment,
 maintainers please chime in!]
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NHannes Reinecke <hare@suse.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

This patch has been generated as follows:

for verb in set_unlocked clear_unlocked set clear; do
  replace-in-files queue_flag_${verb} blk_queue_flag_${verb%_unlocked} \
    $(git grep -lw queue_flag_${verb} drivers block/bsg*)
done

Except for protecting all queue flag changes with the queue lock
this patch does not change any functionality.

Cc: Mike Snitzer <snitzer@redhat.com>
Cc: Shaohua Li <shli@fb.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Ming Lei <ming.lei@redhat.com>
Signed-off-by: NBart Van Assche <bart.vanassche@wdc.com>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Acked-by: NMartin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Remove code no longer needed after the switch to blk-mq.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

The function mmc_cqe_issue_type is local to the source and does
not need to be in global scope, so make it static.

Cleans up sparse warning:
drivers/mmc/core/queue.c:62:21: warning: symbol 'mmc_cqe_issue_type'
was not declared. Should it be static?
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Acked-by: NAdrian Hunter <adrian.hunter@intel.com>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

For blk-mq, add support for completing requests directly in the ->done
callback. That means that error handling and urgent background operations
must be handled by recovery_work in that case.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>

Add CQE support to the block driver, including:
    - optionally using DCMD for flush requests
    - "manually" issuing discard requests
    - issuing read / write requests to the CQE
    - supporting block-layer timeouts
    - handling recovery
    - supporting re-tuning

CQE offers 25% - 50% better random multi-threaded I/O.  There is a slight
(e.g. 2%) drop in sequential read speed but no observable change to sequential
write.

CQE automatically sends the commands to complete requests.  However it only
supports reads / writes and so-called "direct commands" (DCMD).  Furthermore
DCMD is limited to one command at a time, but discards require 3 commands.
That makes issuing discards through CQE very awkward, but some CQE's don't
support DCMD anyway.  So for discards, the existing non-CQE approach is
taken, where the mmc core code issues the 3 commands one at a time i.e.
mmc_erase(). Where DCMD is used, is for issuing flushes.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>

Define and use a blk-mq queue. Discards and flushes are processed
synchronously, but reads and writes asynchronously. In order to support
slow DMA unmapping, DMA unmapping is not done until after the next request
is started. That means the request is not completed until then. If there is
no next request then the completion is done by queued work.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>

Use blk_cleanup_queue() to shutdown the queue when the driver is removed,
and instead get an extra reference to the queue to prevent the queue being
freed before the final mmc_blk_put().
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>

mmc_cleanup_queue() is not used by a different module. Do not export it.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>

Factor out some common code that will also be used with blk-mq.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

This function is used by the block layer queue to bail out of
requests if the current request is towards an RPMB
"block device".

This was done to avoid boot time scanning of this "block
device" which was never really a block device, thus duct-taping
over the fact that it was badly engineered.

This problem is now gone as we removed the offending RPMB block
device in another patch and replaced it with a character
device.

Cc: Tomas Winkler <tomas.winkler@intel.com>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

In may, Steven sent a patch deleting the bounce buffer handling
and the CONFIG_MMC_BLOCK_BOUNCE option.

I chose the less invasive path of making it a runtime config
option, and we merged that successfully for kernel v4.12.

The code is however just standing in the way and taking up
space for seemingly no gain on any systems in wide use today.

Pierre says the code was there to improve speed on TI SDHCI
controllers on certain HP laptops and possibly some Ricoh
controllers as well. Early SDHCI controllers lacked the
scatter-gather feature, which made software bounce buffers
a significant speed boost.

We are clearly talking about the list of SDHCI PCI-based
MMC/SD card readers found in the pci_ids[] list in
drivers/mmc/host/sdhci-pci-core.c.

The TI SDHCI derivative is not supported by the upstream
kernel. This leaves the Ricoh.

What we can however notice is that the x86 defconfigs in the
kernel did not enable CONFIG_MMC_BLOCK_BOUNCE option, which
means that any such laptop would have to have a custom
configured kernel to actually take advantage of this
bounce buffer speed-up. It simply seems like there was
a speed optimization for the Ricoh controllers that noone
was using. (I have not checked the distro defconfigs but
I am pretty sure the situation is the same there.)

Bounce buffers increased performance on the OMAP HSMMC
at one point, and was part of the original submission in
commit a45c6cb8 ("[ARM] 5369/1: omap mmc: Add new
   omap hsmmc controller for 2430 and 34xx, v3")

This optimization was removed in
commit 0ccd76d4 ("omap_hsmmc: Implement scatter-gather
   emulation")
which found that scatter-gather emulation provided even
better performance.

The same was introduced for SDHCI in
commit 2134a922 ("sdhci: scatter-gather (ADMA) support")

I am pretty positively convinced that software
scatter-gather emulation will do for any host controller what
the bounce buffers were doing. Essentially, the bounce buffer
was a reimplementation of software scatter-gather-emulation in
the MMC subsystem, and it should be done away with.

Cc: Pierre Ossman <pierre@ossman.eu>
Cc: Juha Yrjola <juha.yrjola@solidboot.com>
Cc: Steven J. Hill <Steven.Hill@cavium.com>
Cc: Shawn Lin <shawn.lin@rock-chips.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Suggested-by: NSteven J. Hill <Steven.Hill@cavium.com>
Suggested-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>

mmc_init_request() depends on card->bouncesz so it must be calculated
before blk_init_allocated_queue() starts allocating requests.
Reported-by: NSeraphime Kirkovski <kirkseraph@gmail.com>
Fixes: 304419d8 ("mmc: core: Allocate per-request data using the..")
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Tested-by: NSeraphime Kirkovski <kirkseraph@gmail.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NUlf Hansson <ulf.hansson@linaro.org>
Tested-by: NPavel Machek <pavel@ucw.cz>