iommu: Fix a boundary issue to avoid performance drop

stable inclusion
from stable-5.10.37
commit 620aa5821aaa2636d1675c72ab89ee1f26ee4fa1
bugzilla: 51868
CVE: NA

--------------------------------

[ Upstream commit 3431c3f6 ]

After the change of patch ("iommu: Switch gather->end to the
inclusive end"), the performace drops from 1600+K IOPS to 1200K in our
kunpeng ARM64 platform.
We find that the range [start1, end1) actually is joint from the range
[end1, end2), but it is considered as disjoint after the change,
so it needs more times of TLB sync, and spends more time on it.
So fix the boundary issue to avoid performance drop.

Fixes: 862c3715 ("iommu: Switch gather->end to the inclusive end")
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Acked-by: NWill Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/1616643504-120688-1-git-send-email-chenxiang66@hisilicon.comSigned-off-by: NJoerg Roedel <jroedel@suse.de>
Signed-off-by: NSasha Levin <sashal@kernel.org>
Signed-off-by: NChen Jun <chenjun102@huawei.com>
Acked-by: NWeilong Chen <chenweilong@huawei.com>
Signed-off-by: NZheng Zengkai <zhengzengkai@huawei.com>

include/linux/iommu.h

@@ -561,7 +561,7 @@ static inline void iommu_iotlb_gather_add_page(struct iommu_domain *domain,
 	 * structure can be rewritten.
 	 */
 	if (gather->pgsize != size ||
-	    end < gather->start || start > gather->end) {
+	    end + 1 < gather->start || start > gather->end + 1) {
 		if (gather->pgsize)
 			iommu_iotlb_sync(domain, gather);
 		gather->pgsize = size;