The debug trapping code is pretty heavy on the "inline" attribute,
but most functions are actually referenced in the sysreg tables,
making the inlining imposible.

Removing the useless inline qualifier seems the right thing to do,
having verified that the output code is similar.

Cc: Alex Bennée <alex.bennee@linaro.org>
Acked-by: NChristoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

David Binderman reported that the exception injection code had a
couple of unused variables lingering around.

Upon examination, it looked like this code could do with an
anticipated spring cleaning, which amounts to deduplicating
the CPSR/SPSR update, and making it look a bit more like
the architecture spec.

The spurious variables are removed in the process.
Reported-by: NDavid Binderman <dcb314@hotmail.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

As we've now switched to the new world switch implementation,
remove the weak attributes, as nobody is supposed to override
it anymore.
Acked-by: NChristoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

As we've now rewritten most of our code-base in C, most of the
KVM-specific code in asm-offset.c is useless. Delete-time again!
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NChristoffer Dall <christoffer.dall@linaro.org>

Having the system register numbers as #defines has been a pain
since day one, as the ordering is pretty fragile, and moving
things around leads to renumbering and epic conflict resolutions.

Now that we're mostly acessing the sysreg file in C, an enum is
a much better type to use, and we can clean things up a bit.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NChristoffer Dall <christoffer.dall@linaro.org>

This is it. We remove all of the code that has now been rewritten.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NChristoffer Dall <christoffer.dall@linaro.org>

In order to run C code in HYP, we must make sure that the kernel's
RO section is mapped into HYP (otherwise things break badly).
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NChristoffer Dall <christoffer.dall@linaro.org>

So far, we've implemented the new world switch with a completely
different namespace, so that we could have both implementation
compiled in.

Let's take things one step further by adding weak aliases that
have the same names as the original implementation. The weak
attributes allows the new implementation to be overriden by the
old one, and everything still work.

At a later point, we'll be able to simply drop the old code, and
everything will hopefully keep working, thanks to the aliases we
have just added. This also saves us repainting all the callers.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NChristoffer Dall <christoffer.dall@linaro.org>

Add the panic handler, together with the small bits of assembly
code to call the kernel's panic implementation.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>

Add the entry points for HYP mode (both for hypercalls and
exception handling).
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>

Implement the TLB handling as a direct translation of the assembly
code version.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>

Implement the fpsimd save restore, keeping the lazy part in
assembler (as returning to C would be overkill).
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>

Implement the core of the world switch in C. Not everything is there
yet, and there is nothing to re-enter the world switch either.

But this already outlines the code structure well enough.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>

KVM so far relies on code patching, and is likely to use it more
in the future. The main issue is that our alternative system works
at the instruction level, while we'd like to have alternatives at
the function level.

In order to cope with this, add the "hyp_alternate_select" macro that
outputs a brief sequence of code that in turn can be patched, allowing
an alternative function to be selected.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Contrary to the previous patch, the guest entry is fairly different
from its assembly counterpart, mostly because it is only concerned
with saving/restoring the GP registers, and nothing else.
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Implement the debug save restore as a direct translation of
the assembly code version.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Tested-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>

Implement the 32bit system register save/restore as a direct
translation of the assembly code version.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>

Implement the system register save/restore as a direct translation of
the assembly code version.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>

Implement the timer save restore as a direct translation of
the assembly code version.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Implement the vgic-v3 save restore as a direct translation of
the assembly code version.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

We store GICv3 LRs in reverse order so that the CPU can save/restore
them in rever order as well (don't ask why, the design is crazy),
and yet generate memory traffic that doesn't completely suck.

We need this macro to be available to the C version of save/restore.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Implement the vgic-v2 save restore (mostly) as a direct translation
of the assembly code version.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

In order to expose the various EL2 services that are private to
the hypervisor, add a new hyp.h file.

So far, it only contains mundane things such as section annotation
and VA manipulation.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Rather than crafting custom macros for reading/writing each system
register provide generics accessors, read_sysreg and write_sysreg, for
this purpose.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Cc: Suzuki Poulose <suzuki.poulose@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

It would add guest exit statistics to debugfs, this can be helpful
while measuring KVM performance.

  [ Renamed some of the field names - Christoffer ]
Signed-off-by: NAmit Singh Tomar <amittomer25@gmail.com>
Signed-off-by: NChristoffer Dall <christoffer.dall@linaro.org>

vgic_io_ops is only referenced within vgic.c, so it can be declared
static.
Signed-off-by: NJisheng Zhang <jszhang@marvell.com>
Signed-off-by: NChristoffer Dall <christoffer.dall@linaro.org>

Jan Stancek reported that I wrecked things for him by fixing things for
Vladimir :/

His report was due to an UNINTERRUPTIBLE wait getting -EINTR, which
should not be possible, however my previous patch made this possible by
unconditionally checking signal_pending().

We cannot use current->state as was done previously, because the
instruction after the store to that variable it can be changed.  We must
instead pass the initial state along and use that.

Fixes: 68985633 ("sched/wait: Fix signal handling in bit wait helpers")
Reported-by: NJan Stancek <jstancek@redhat.com>
Reported-by: NChris Mason <clm@fb.com>
Tested-by: NJan Stancek <jstancek@redhat.com>
Tested-by: NVladimir Murzin <vladimir.murzin@arm.com>
Tested-by: NChris Mason <clm@fb.com>
Reviewed-by: NPaul Turner <pjt@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: tglx@linutronix.de
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: hpa@zytor.com
Signed-off-by: NPeter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Pull NFS client bugfix from Trond Myklebust:
 "SUNRPC: Fix a NFSv4.1 callback channel regression"

* tag 'nfs-for-4.4-3' of git://git.linux-nfs.org/projects/trondmy/linux-nfs:
  SUNRPC: Fix callback channel

Pull timer fixlets from Thomas Gleixner:
 "Two trivial fixes which add missing header fileas and forward
  declarations so the code will compile even when the magic include
  chains are different"

* 'irq-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  irqchip/gic-v3: Add missing include for barrier.h
  irqchip/gic-v3: Add missing struct device_node declaration

Pull timer fix from Thomas Gleixner:
 "A single fix to unbreak a clocksource driver which has more than 32bit
  counter width"

* 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  clocksource: Mmio: remove artificial 32bit limitation

Pull fpga driver fixes from Greg KH:
 "Only two small fpga driver fixes here, both have been in linux-next
  for a while, and resolve some reported issues"

* tag 'char-misc-4.4-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc:
  fpga manager: Fix firmware resource leak on error
  fpga manager: remove label

Pull staging driver fixes from Greg KH:
 "Here are a few staging and IIO driver fixes for 4.4-rc5.

  All of them resolve reported problems and have been in linux-next for
  a while.  Nothing major here, just small fixes where needed"

* tag 'staging-4.4-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging:
  staging: lustre: echo_copy.._lsm() dereferences userland pointers directly
  iio: adc: spmi-vadc: add missing of_node_put
  iio: fix some warning messages
  iio: light: apds9960: correct ->last_busy count
  iio: lidar: return -EINVAL on invalid signal
  staging: iio: dummy: complete IIO events delivery to userspace

Pull USB driver fixes from Greg KH:
 "Here are a number of small USB fixes for 4.4-rc5.  All of them have
  been in linux-next.  The majority are gadget and phy issues, with a
  few new quirks and device ids added as well"

* tag 'usb-4.4-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb: (32 commits)
  USB: add quirk for devices with broken LPM
  xhci: fix usb2 resume timing and races.
  usb: musb: fail with error when no DMA controller set
  usb: gadget: uvc: fix permissions of configfs attributes
  usb: musb: core: Fix pm runtime for deferred probe
  usb: phy: msm: fix a possible NULL dereference
  USB: host: ohci-at91: fix a crash in ohci_hcd_at91_overcurrent_irq
  usb: Quiet down false peer failure messages
  usb: xhci: fix config fail of FS hub behind a HS hub with MTT
  xhci: Fix memory leak in xhci_pme_acpi_rtd3_enable()
  usb: Use the USB_SS_MULT() macro to decode burst multiplier for log message
  USB: whci-hcd: add check for dma mapping error
  usb: core : hub: Fix BOS 'NULL pointer' kernel panic
  USB: quirks: Apply ALWAYS_POLL to all ELAN devices
  usb-storage: Fix scsi-sd failure "Invalid field in cdb" for USB adapter JMicron
  USB: quirks: Fix another ELAN touchscreen
  usb: dwc3: gadget: don't prestart interrupt endpoints
  USB: serial: Another Infineon flash loader USB ID
  USB: cdc_acm: Ignore Infineon Flash Loader utility
  USB: cp210x: Remove CP2110 ID from compatibility list
  ...

Pull ARM SoC fixes from Arnd Bergmann:
 "Here are a bunch of small bug fixes for various ARM platforms, nothing
  really sticks out this week, most of either fixes bugs in code that
  was just added in 4.4, or that has been broken for many years without
  anyone noticing.

  at91/sama5d2:
   - fix sama5de hardware setup of sd/mmc interface
   - proper selection of pinctrl drivers.  PIO4 is necessary for sama5d2

  berlin:
   - fix incorrect clock input for SDIO

  exynos:
   - Fix potential NULL pointer dereference in Exynos PMU driver.

  imx:
   - Fix vf610 SAI clock configuration bug which is discovered by the
     newly added master mode support in SAI audio driver.
   - Fix buggy L2 cache latency values in vf610 device trees, which may
     cause system hang when cpu runs at a higher frequency.

  ixp4xx:
   - fix prototypes for readl/writel functions

  ls2080a:
   - use little-endian register access for GPIO and SDHCI

  omap:
   - Fix clock source for ARM TWD and global timers on am437x
   - Always select REGULATOR_FIXED_VOLTAGE for omap2+ instead of when
     MACH_OMAP3_PANDORA is selected
   - Fix SPI DMA handles for dm816x as only some were mapped
   - Fix up mbox cells for dm816x to make mailbox usable

  pxa:
   - use PWM lookup table for all ezx machines

  s3c24xx:
   - Remove incorrect __init annotation from s3c24xx cpufreq driver
     structures.

  versatile:
   - fix PCI IRQ mapping on Versatile PB"

* tag 'fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc:
  ls2080a/dts: Add little endian property for GPIO IP block
  dt-bindings: define little-endian property for QorIQ GPIO
  ARM64: dts: ls2080a: fix eSDHC endianness
  ARM: dts: vf610: use reset values for L2 cache latencies
  ARM: pxa: use PWM lookup table for all machines
  ARM: dts: berlin: add 2nd clock for BG2Q sdhci0 and sdhci1
  ARM: dts: berlin: correct BG2Q's sdhci2 2nd clock
  ARM: dts: am4372: fix clock source for arm twd and global timers
  ARM: at91: fix pinctrl driver selection
  ARM: at91/dt: add always-on to 1.8V regulator
  ARM: dts: vf610: fix clock definition for SAI2
  ARM: imx: clk-vf610: fix SAI clock tree
  ARM: ixp4xx: fix read{b,w,l} return types
  irqchip/versatile-fpga: Fix PCI IRQ mapping on Versatile PB
  ARM: OMAP2+: enable REGULATOR_FIXED_VOLTAGE
  ARM: dts: add dm816x missing spi DT dma handles
  ARM: dts: add dm816x missing #mbox-cells
  cpufreq: s3c24xx: Do not mark s3c2410_plls_add as __init
  ARM: EXYNOS: Fix potential NULL pointer access in exynos_sys_powerdown_conf

Pull powerpc fixes from Michael Ellerman:
 - opal-irqchip: Fix double endian conversion from Alistair Popple
 - cxl: Set endianess of kernel contexts from Frederic Barrat
 - sbc8641: drop bogus PHY IRQ entries from DTS file from Paul Gortmaker
 - Revert "powerpc/eeh: Don't unfreeze PHB PE after reset" from Andrew
   Donnellan

* tag 'powerpc-4.4-4' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux:
  Revert "powerpc/eeh: Don't unfreeze PHB PE after reset"
  powerpc/sbc8641: drop bogus PHY IRQ entries from DTS file
  cxl: Set endianess of kernel contexts
  powerpc/opal-irqchip: Fix double endian conversion

Merge misc fixes from Andrew Morton:
 "17 fixes"

* emailed patches from Andrew Morton <akpm@linux-foundation.org>:
  MIPS: fix DMA contiguous allocation
  sh64: fix __NR_fgetxattr
  ocfs2: fix SGID not inherited issue
  mm/oom_kill.c: avoid attempting to kill init sharing same memory
  drivers/base/memory.c: prohibit offlining of memory blocks with missing sections
  tmpfs: fix shmem_evict_inode() warnings on i_blocks
  mm/hugetlb.c: fix resv map memory leak for placeholder entries
  mm: hugetlb: call huge_pte_alloc() only if ptep is null
  kernel: remove stop_machine() Kconfig dependency
  mm: kmemleak: mark kmemleak_init prototype as __init
  mm: fix kerneldoc on mem_cgroup_replace_page
  osd fs: __r4w_get_page rely on PageUptodate for uptodate
  MAINTAINERS: make Vladimir co-maintainer of the memory controller
  mm, vmstat: allow WQ concurrency to discover memory reclaim doesn't make any progress
  mm: fix swapped Movable and Reclaimable in /proc/pagetypeinfo
  memcg: fix memory.high target
  mm: hugetlb: fix hugepage memory leak caused by wrong reserve count

Pull parisc fixes from Helge Deller:
 "Fix the boot crash on Mako machines with Huge Pages, prevent a panic
  with SATA controllers (and others) by correctly calculating the IOMMU
  space, hook up the mlock2 syscall and drop unneeded code in the parisc
  pci code"

* 'parisc-4.4-3' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux:
  parisc: Disable huge pages on Mako machines
  parisc: Wire up mlock2 syscall
  parisc: Remove unused pcibios_init_bus()
  parisc iommu: fix panic due to trying to allocate too large region

Pull block layer fixes from Jens Axboe:
 "A set of fixes for the current series.  This contains:

   - A bunch of fixes for lightnvm, should be the last round for this
     series.  From Matias and Wenwei.

   - A writeback detach inode fix from Ilya, also marked for stable.

   - A block (though it says SCSI) fix for an OOPS in SCSI runtime power
     management.

   - Module init error path fixes for null_blk from Minfei"

* 'for-linus' of git://git.kernel.dk/linux-block:
  null_blk: Fix error path in module initialization
  lightnvm: do not compile in debugging by default
  lightnvm: prevent gennvm module unload on use
  lightnvm: fix media mgr registration
  lightnvm: replace req queue with nvmdev for lld
  lightnvm: comments on constants
  lightnvm: check mm before use
  lightnvm: refactor spin_unlock in gennvm_get_blk
  lightnvm: put blks when luns configure failed
  lightnvm: use flags in rrpc_get_blk
  block: detach bdev inode from its wb in __blkdev_put()
  SCSI: Fix NULL pointer dereference in runtime PM

Pull arm64 fixes from Catalin Marinas:

 - Update the linker script to use L1_CACHE_BYTES instead of hard-coded
   64.  We recently changed L1_CACHE_BYTES to 128

 - Improve race condition reporting on set_pte_at() and change the BUG
   to WARN_ONCE.  With hardware update of the accessed/dirty state, we
   need to ensure that set_pte_at() does not inadvertently override
   hardware updated state.  The patch also makes the checks ignore
   !pte_valid() new entries

* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
  arm64: Improve error reporting on set_pte_at() checks
  arm64: update linker script to increased L1_CACHE_BYTES value