Based on include/xen/mm.h [1], Linux is mistakenly using MFN when GFN
is meant, I suspect this is because the first support for Xen was for
PV. This resulted in some misimplementation of helpers on ARM and
confused developers about the expected behavior.

For instance, with pfn_to_mfn, we expect to get an MFN based on the name.
Although, if we look at the implementation on x86, it's returning a GFN.

For clarity and avoid new confusion, replace any reference to mfn with
gfn in any helpers used by PV drivers. The x86 code will still keep some
reference of pfn_to_mfn which may be used by all kind of guests
No changes as been made in the hypercall field, even
though they may be invalid, in order to keep the same as the defintion
in xen repo.

Note that page_to_mfn has been renamed to xen_page_to_gfn to avoid a
name to close to the KVM function gfn_to_page.

Take also the opportunity to simplify simple construction such
as pfn_to_mfn(page_to_pfn(page)) into xen_page_to_gfn. More complex clean up
will come in follow-up patches.

[1] http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=e758ed14f390342513405dd766e874934573e6cbSigned-off-by: NJulien Grall <julien.grall@citrix.com>
Reviewed-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: NDmitry Torokhov <dmitry.torokhov@gmail.com>
Acked-by: NWei Liu <wei.liu2@citrix.com>
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>

After the commit introducing convertion between DMA and guest addresses,
all the callers of pfn_to_mfn are expecting to get a GFN (Guest Frame
Number). On ARM, all the guests are auto-translated so the GFN is equal
to the Linux PFN (Pseudo-physical Frame Number).

The current implementation may return an MFN if the caller is passing a
PFN associated to a mapped foreign grant. In pratice, I haven't seen
the problem on running guest but we should fix it for the sake of
correctness.

Correct the implementation by always returning the pfn passed in parameter.

A follow-up patch will take care to rename pfn_to_mfn to a suitable
name.
Signed-off-by: NJulien Grall <julien.grall@citrix.com>
Reviewed-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>

The swiotlb is required when programming a DMA address on ARM when a
device is not protected by an IOMMU.

In this case, the DMA address should always be equal to the machine address.
For DOM0 memory, Xen ensure it by have an identity mapping between the
guest address and host address. However, when mapping a foreign grant
reference, the 1:1 model doesn't work.

For ARM guest, most of the callers of pfn_to_mfn expects to get a GFN
(Guest Frame Number), i.e a PFN (Page Frame Number) from the Linux point
of view given that all ARM guest are auto-translated.

Even though the name pfn_to_mfn is misleading, we need to ensure that
those caller get a GFN and not by mistake a MFN. In pratical, I haven't
seen error related to this but we should fix it for the sake of
correctness.

In order to fix the implementation of pfn_to_mfn on ARM in a follow-up
patch, we have to introduce new helpers to return the DMA from a PFN and
the invert.

On x86, the new helpers will be an alias of pfn_to_mfn and mfn_to_pfn.

The helpers will be used in swiotlb and xen_biovec_phys_mergeable.

This is necessary in the latter because we have to ensure that the
biovec code will not try to merge a biovec using foreign page and
another using Linux memory.

Lastly, the helper mfn_to_local_pfn has been renamed to bfn_to_local_pfn
given that the only usage was in swiotlb.
Signed-off-by: NJulien Grall <julien.grall@citrix.com>
Reviewed-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>

ARM guests are always HVM. The current implementation is assuming a 1:1
mapping which is only true for DOM0 and may not be at all in the future.

Furthermore, all the helpers but arbitrary_virt_to_machine are used in
x86 specific code (or only compiled for).

The helper arbitrary_virt_to_machine is only used in PV specific code.
Therefore we should never call the function.

Add a BUG() in this helper and drop all the others.
Signed-off-by: NJulien Grall <julien.grall@citrix.com>
Acked-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>

Signed-off-by: NJulien Grall <julien.grall@citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>

Make sure that xen_swiotlb_init allocates buffers that are DMA capable
when at least one memblock is available below 4G. Otherwise we assume
that all devices on the SoC can cope with >4G addresses. We do this on
ARM and ARM64, where dom0 is mapped 1:1, so pfn == mfn in this case.

No functional changes on x86.

From: Chen Baozi <baozich@gmail.com>
Signed-off-by: NChen Baozi <baozich@gmail.com>
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Tested-by: NChen Baozi <baozich@gmail.com>
Acked-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>

When unmapping grants, instead of converting the kernel map ops to
unmap ops on the fly, pre-populate the set of unmap ops.

This allows the grant unmap for the kernel mappings to be trivially
batched in the future.
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>
Reviewed-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>

Introduce an arch specific function to find out whether a particular dma
mapping operation needs to bounce on the swiotlb buffer.

On ARM and ARM64, if the page involved is a foreign page and the device
is not coherent, we need to bounce because at unmap time we cannot
execute any required cache maintenance operations (we don't know how to
find the pfn from the mfn).

No change of behaviour for x86.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: NDavid Vrabel <david.vrabel@citrix.com>
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NIan Campbell <ian.campbell@citrix.com>
Acked-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Remove the rbtree used to keep track of machine to physical mappings:
the frontend can grant the same page multiple times, leading to errors
inserting or removing entries from the mach_to_phys tree.

Linux only needed to know the physical address corresponding to a given
machine address in swiotlb-xen. Now that swiotlb-xen can call the
xen_dma_* functions passing the machine address directly, we can remove
it.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Tested-by: NDenis Schneider <v1ne2go@gmail.com>

virt_to_pfn has been defined in asm/memory.h by the commit e26a9e00 "ARM: Better
virt_to_page() handling"

This will result of a compilation warning when CONFIG_XEN is enabled.

arch/arm/include/asm/xen/page.h:80:0: warning: "virt_to_pfn" redefined [enabled by default]
 #define virt_to_pfn(v)          (PFN_DOWN(__pa(v)))
 ^
In file included from arch/arm/include/asm/page.h:163:0,
                 from arch/arm/include/asm/xen/page.h:4,
                 from include/xen/page.h:4,
                 from arch/arm/xen/grant-table.c:33:

The definition in memory.h is nearly the same (it directly expand PFN_DOWN),
so we can safely drop virt_to_pfn in xen include.
Signed-off-by: NJulien Grall <julien.grall@linaro.org>
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>

virt_to_pfn has been defined in asm/memory.h by the commit e26a9e00 "ARM: Better
virt_to_page() handling"

This will result of a compilation warning when CONFIG_XEN is enabled.

arch/arm/include/asm/xen/page.h:80:0: warning: "virt_to_pfn" redefined [enabled by default]
 #define virt_to_pfn(v)          (PFN_DOWN(__pa(v)))
 ^
In file included from arch/arm/include/asm/page.h:163:0,
                 from arch/arm/include/asm/xen/page.h:4,
                 from include/xen/page.h:4,
                 from arch/arm/xen/grant-table.c:33:

The definition in memory.h is nearly the same (it directly expand PFN_DOWN),
so we can safely drop virt_to_pfn in xen include.
Signed-off-by: NJulien Grall <julien.grall@linaro.org>
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>

The grant mapping API does m2p_override unnecessarily: only gntdev needs it,
for blkback and future netback patches it just cause a lock contention, as
those pages never go to userspace. Therefore this series does the following:
- the bulk of the original function (everything after the mapping hypercall)
  is moved to arch-dependent set/clear_foreign_p2m_mapping
- the "if (xen_feature(XENFEAT_auto_translated_physmap))" branch goes to ARM
- therefore the ARM function could be much smaller, the m2p_override stubs
  could be also removed
- on x86 the set_phys_to_machine calls were moved up to this new funcion
  from m2p_override functions
- and m2p_override functions are only called when there is a kmap_ops param

It also removes a stray space from arch/x86/include/asm/xen/page.h.
Signed-off-by: NZoltan Kiss <zoltan.kiss@citrix.com>
Suggested-by: NAnthony Liguori <aliguori@amazon.com>
Suggested-by: NDavid Vrabel <david.vrabel@citrix.com>
Suggested-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NDavid Vrabel <david.vrabel@citrix.com>
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>

The 'xen_hvm_resume_frames' used to be an 'unsigned long'
and contain the virtual address of the grants. That was OK
for most architectures (PVHVM, ARM) were the grants are contiguous
in memory. That however is not the case for PVH - in which case
we will have to do a lookup for each virtual address for the PFN.

Instead of doing that, lets make it a structure which will contain
the array of PFNs, the virtual address and the count of said PFNs.

Also provide a generic functions: gnttab_setup_auto_xlat_frames and
gnttab_free_auto_xlat_frames to populate said structure with
appropriate values for PVHVM and ARM.

To round it off, change the name from 'xen_hvm_resume_frames' to
a more descriptive one - 'xen_auto_xlat_grant_frames'.

For PVH, in patch "xen/pvh: Piggyback on PVHVM for grant driver"
we will populate the 'xen_auto_xlat_grant_frames' by ourselves.

v2 moves the xen_remap in the gnttab_setup_auto_xlat_frames
and also introduces xen_unmap for gnttab_free_auto_xlat_frames.
Suggested-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
[v3: Based on top of 'asm/xen/page.h: remove redundant semicolon']
Acked-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>

Signed-off-by: NWei Liu <liuw@liuw.name>
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>

ioremap_cache is more aligned with other architectures.
There are only 2 users of this in the kernel: pxa2xx-flash and Xen.

This fixes Xen build failures on arm64:

drivers/tty/hvc/hvc_xen.c:233:2: error: implicit declaration of function 'ioremap_cached' [-Werror=implicit-function-declaration]
drivers/xen/grant-table.c:1174:3: error: implicit declaration of function 'ioremap_cached' [-Werror=implicit-function-declaration]
drivers/xen/xenbus/xenbus_probe.c:778:4: error: implicit declaration of function 'ioremap_cached' [-Werror=implicit-function-declaration]
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ioremap_cache is more aligned with other architectures. There are only
2 users of this in the kernel: pxa2xx-flash and Xen.

This fixes Xen build failures on arm64 caused by commit c04e8e2f (arm64:
allow ioremap_cache() to use existing RAM mappings)

drivers/tty/hvc/hvc_xen.c:233:2: error: implicit declaration of function 'ioremap_cached' [-Werror=implicit-function-declaration]
drivers/xen/grant-table.c:1174:3: error: implicit declaration of function 'ioremap_cached' [-Werror=implicit-function-declaration]
drivers/xen/xenbus/xenbus_probe.c:778:4: error: implicit declaration of function 'ioremap_cached' [-Werror=implicit-function-declaration]
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Some common Xen drivers, like balloon.c, call pfn_to_mfn and mfn_to_pfn
even for autotranslate guests, expecting the argument back.
The following commit broke these drivers by changing the behavior of
pfn_to_mfn and mfn_to_pfn:

commit 4a19138c
Author: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Date:   Thu Oct 17 16:22:27 2013 +0000

    arm/xen,arm64/xen: introduce p2m

They now return INVALID_P2M_ENTRY if Linux doesn't actually know what is
the mfn backing a pfn or what is the pfn corresponding to an mfn.
Fix the regression by switching to the old behavior.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: NIan Campbell <ian.campbell@citrix.com>
Tested-by: NIan Campbell <ian.campbell@citrix.com>

Xen on arm and arm64 needs SWIOTLB_XEN: when running on Xen we need to
program the hardware with mfns rather than pfns for dma addresses.
Remove SWIOTLB_XEN dependency on X86 and PCI and make XEN select
SWIOTLB_XEN on arm and arm64.

At the moment always rely on swiotlb-xen, but when Xen starts supporting
hardware IOMMUs we'll be able to avoid it conditionally on the presence
of an IOMMU on the platform.

Implement xen_create_contiguous_region on arm and arm64: for the moment
we assume that dom0 has been mapped 1:1 (physical addresses == machine
addresses) therefore we don't need to call XENMEM_exchange. Simply
return the physical address as dma address.

Initialize the xen-swiotlb from xen_early_init (before the native
dma_ops are initialized), set xen_dma_ops to &xen_swiotlb_dma_ops.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>


Changes in v8:
- assume dom0 is mapped 1:1, no need to call XENMEM_exchange.

Changes in v7:
- call __set_phys_to_machine_multi from xen_create_contiguous_region and
xen_destroy_contiguous_region to update the P2M;
- don't call XENMEM_unpin, it has been removed;
- call XENMEM_exchange instead of XENMEM_exchange_and_pin;
- set nr_exchanged to 0 before calling the hypercall.

Changes in v6:
- introduce and export xen_dma_ops;
- call xen_mm_init from as arch_initcall.

Changes in v4:
- remove redefinition of DMA_ERROR_CODE;
- update the code to use XENMEM_exchange_and_pin and XENMEM_unpin;
- add a note about hardware IOMMU in the commit message.

Changes in v3:
- code style changes;
- warn on XENMEM_put_dma_buf failures.

Introduce physical to machine and machine to physical tracking
mechanisms based on rbtrees for arm/xen and arm64/xen.

We need it because any guests on ARM are an autotranslate guests,
therefore a physical address is potentially different from a machine
address. When programming a device to do DMA, we need to be
extra-careful to use machine addresses rather than physical addresses to
program the device. Therefore we need to know the physical to machine
mappings.

For the moment we assume that dom0 starts with a 1:1 physical to machine
mapping, in other words physical addresses correspond to machine
addresses. However when mapping a foreign grant reference, obviously the
1:1 model doesn't work anymore. So at the very least we need to be able
to track grant mappings.

We need locking to protect accesses to the two trees.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>

Changes in v8:
- move pfn_to_mfn and mfn_to_pfn to page.h as static inline functions;
- no need to walk the tree if phys_to_mach.rb_node is NULL;
- correctly handle multipage p2m entries;
- substitute the spin_lock with a rwlock.

Define xen_remap as ioremap_cache (MT_MEMORY and MT_DEVICE_CACHED end up
having the same AttrIndx encoding).

Remove include asm/mach/map.h, not unneeded.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>

ioremap can't be used to map ring pages on ARM because it uses device
memory caching attributes (MT_DEVICE*).

Introduce a Xen specific abstraction to map ring pages, called
xen_remap, that is defined as ioremap on x86 (no behavioral changes).
On ARM it explicitly calls __arm_ioremap with the right caching
attributes: MT_MEMORY.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

This makes common code less ifdef-y and is consistent with PVHVM on
x86.

Also note that phys_to_machine_mapping_valid should take a pfn
argument and make it do so.

Add __set_phys_to_machine, make set_phys_to_machine a simple wrapper
(on systems with non-nop implementations the outer one can allocate
new p2m pages).

Make __set_phys_to_machine check for identity mapping or invalid only.
Acked-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NIan Campbell <ian.campbell@citrix.com>
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

ARM Xen guests always use paging in hardware, like PV on HVM guests in
the X86 world.

Changes in v3:

- improve comments.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>