Move the few remaining bits of swiotlb glue towards their callers,
and remove the pointless on ia64 swiotlb variable.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NChristian König <christian.koenig@amd.com>

These are identical to the ia64 ops, and would also support CMA
if enabled on ia64.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NChristian König <christian.koenig@amd.com>

ia64 uses ZONE_DMA for allocations below 32-bits.  These days we
name the zone for that ZONE_DMA32, which will allow to use the
dma-direct and generic swiotlb code as-is, so rename it.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NChristian König <christian.koenig@amd.com>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

We'll need that name for a generic implementation soon.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

We'll need that name for a generic implementation soon.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

We'll need that name for a generic implementation soon.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

We'll need that name for a generic implementation soon.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>

Replace the bare-bones h8300 direct dma mapping implementation with
the fully featured generic dma-direct one.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

cris currently has an incomplete direct mapping dma_map_ops implementation
if PCI support is enabled.  Replace it with the fully feature generic
dma-direct implementation.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJesper Nilsson <jesper.nilsson@axis.com>

So that they don't need to indirect through the operation vector.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NVladimir Murzin <vladimir.murzin@arm.com>

The trivial direct mapping implementation already does a virtual to
physical translation which isn't strictly a noop, and will soon learn
to do non-direct but linear physical to dma translations through the
device offset and a few small tricks.  Rename it to a better fitting
name.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NVladimir Murzin <vladimir.murzin@arm.com>

For architectures that just use the generic dma_noop_ops we can provide
a generic version of dma-mapping.h.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

To implement the x86 forbid_dac and iommu_sac_force we want an arch hook
so that it can apply the global options across all dma_map_ops
implementations.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Lift the code from x86 so that we behave consistently.  In the future we
should probably warn if any of these is set.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJesper Nilsson <jesper.nilsson@axis.com>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> [m68k]

This is not needed in drivers, so move it to a private header.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Signed-off-by: NChristoph Hellwig <hch@lst.de>

Always returning 1 is the same behavior as not supplying a method at all.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

This frees the dma_direct_* namespace for a generic implementation.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

We want to use the dma_direct_ namespace for a generic implementation,
so rename powerpc to the second best choice: dma_nommu_.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NRichard Kuo <rkuo@codeaurora.org>

And unlike the other helpers we don't require a <asm/dma-direct.h> as
this helper is a special case for ia64 only, and this keeps it as
simple as possible.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

phys_to_dma, dma_to_phys and dma_capable are helpers published by
architecture code for use of swiotlb and xen-swiotlb only.  Drivers are
not supposed to use these directly, but use the DMA API instead.

Move these to a new asm/dma-direct.h helper, included by a
linux/dma-direct.h wrapper that provides the default linear mapping
unless the architecture wants to override it.

In the MIPS case the existing dma-coherent.h is reused for now as
untangling it will take a bit of work.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NRobin Murphy <robin.murphy@arm.com>

This makes it match the generic version.
Reported-by: NVladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

The generic version now takes dma_pfn_offset into account, so there is no
more need for an architecture override.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>

Signed-off-by: NChristoph Hellwig <hch@lst.de>

Signed-off-by: NChristoph Hellwig <hch@lst.de>

Signed-off-by: NChristoph Hellwig <hch@lst.de>

We always use the stub definitions, so remove the unused other code.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NVineet Gupta <vgupta@synopsys.com>

Signed-off-by: NChristoph Hellwig <hch@lst.de>

Signed-off-by: NChristoph Hellwig <hch@lst.de>

Signed-off-by: NChristoph Hellwig <hch@lst.de>

CONFIG_ALPHA_JENSEN has failed to compile since commit 6aca0503
("alpha/dma: use common noop dma ops"), so mark it as broken.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Add qemu idle sleep support when running under qemu with SeaBIOS PDC
firmware.

Like the power architecture we use the "or" assembler instructions,
which translate to nops on real hardware, to indicate that qemu shall
idle sleep.
Signed-off-by: NHelge Deller <deller@gmx.de>
Cc: Richard Henderson <rth@twiddle.net>
CC: stable@vger.kernel.org # v4.9+

Guest GPR values are live in the hardware GPRs at VM-exit.  Do not
leave any guest values in hardware GPRs after the guest GPR values are
saved to the vcpu_vmx structure.

This is a partial mitigation for CVE 2017-5715 and CVE 2017-5753.
Specifically, it defeats the Project Zero PoC for CVE 2017-5715.
Suggested-by: NEric Northup <digitaleric@google.com>
Signed-off-by: NJim Mattson <jmattson@google.com>
Reviewed-by: NEric Northup <digitaleric@google.com>
Reviewed-by: NBenjamin Serebrin <serebrin@google.com>
Reviewed-by: NAndrew Honig <ahonig@google.com>
[Paolo: Add AMD bits, Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>]
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Use the name associated with the particular attack which needs page table
isolation for mitigation.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NDavid Woodhouse <dwmw@amazon.co.uk>
Cc: Alan Cox <gnomes@lxorguk.ukuu.org.uk>
Cc: Jiri Koshina <jikos@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Andi Lutomirski  <luto@amacapital.net>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Turner <pjt@google.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Greg KH <gregkh@linux-foundation.org>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Kees Cook <keescook@google.com>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/alpine.DEB.2.20.1801051525300.1724@nanos

Where an ALTERNATIVE is used in the middle of an inline asm block, this
would otherwise lead to the following instruction being appended directly
to the trailing ".popsection", and a failed compile.

Fixes: 9cebed42 ("x86, alternative: Use .pushsection/.popsection")
Signed-off-by: NDavid Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Cc: gnomes@lxorguk.ukuu.org.uk
Cc: Rik van Riel <riel@redhat.com>
Cc: ak@linux.intel.com
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Turner <pjt@google.com>
Cc: Jiri Kosina <jikos@kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Kees Cook <keescook@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linux-foundation.org>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20180104143710.8961-8-dwmw@amazon.co.uk

gcc -fisolate-erroneous-paths-dereference can generate calls to abort()
from modular code too.

[arnd@arndb.de: drop duplicate exports of abort()]
  Link: http://lkml.kernel.org/r/20180102103311.706364-1-arnd@arndb.deReported-by: NVineet Gupta <Vineet.Gupta1@synopsys.com>
Cc: Sudip Mukherjee <sudipm.mukherjee@gmail.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Alexey Brodkin <Alexey.Brodkin@synopsys.com>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
Cc: Jose Abreu <Jose.Abreu@synopsys.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The recent changes for PTI touch cpu_tlbstate from various tlb_flush
inlines. cpu_tlbstate is exported as GPL symbol, so this causes a
regression when building out of tree drivers for certain graphics cards.

Aside of that the export was wrong since it was introduced as it should
have been EXPORT_PER_CPU_SYMBOL_GPL().

Use the correct PER_CPU export and drop the _GPL to restore the previous
state which allows users to utilize the cards they payed for.

As always I'm really thrilled to make this kind of change to support the
#friends (or however the hot hashtag of today is spelled) from that closet
sauce graphics corp.

Fixes: 1e02ce4c ("x86: Store a per-cpu shadow copy of CR4")
Fixes: 6fd166aa ("x86/mm: Use/Fix PCID to optimize user/kernel switches")
Reported-by: NKees Cook <keescook@google.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: stable@vger.kernel.org

Thomas reported the following warning:

 BUG: using smp_processor_id() in preemptible [00000000] code: ovsdb-server/4498
 caller is native_flush_tlb_single+0x57/0xc0
 native_flush_tlb_single+0x57/0xc0
 __set_pte_vaddr+0x2d/0x40
 set_pte_vaddr+0x2f/0x40
 cea_set_pte+0x30/0x40
 ds_update_cea.constprop.4+0x4d/0x70
 reserve_ds_buffers+0x159/0x410
 x86_reserve_hardware+0x150/0x160
 x86_pmu_event_init+0x3e/0x1f0
 perf_try_init_event+0x69/0x80
 perf_event_alloc+0x652/0x740
 SyS_perf_event_open+0x3f6/0xd60
 do_syscall_64+0x5c/0x190

set_pte_vaddr is used to map the ds buffers into the cpu entry area, but
there are two problems with that:

 1) The resulting flush is not supposed to be called in preemptible context

 2) The cpu entry area is supposed to be per CPU, but the debug store
    buffers are mapped for all CPUs so these mappings need to be flushed
    globally.

Add the necessary preemption protection across the mapping code and flush
TLBs globally.

Fixes: c1961a46 ("x86/events/intel/ds: Map debug buffers in cpu_entry_area")
Reported-by: NThomas Zeitlhofer <thomas.zeitlhofer+lkml@ze-it.at>
Signed-off-by: NPeter Zijlstra <peterz@infradead.org>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Tested-by: NThomas Zeitlhofer <thomas.zeitlhofer+lkml@ze-it.at>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20180104170712.GB3040@hirez.programming.kicks-ass.net