The address space enums that must be used when modifying the address
space part of a psw with the psw_bits() macro can easily be confused
with the psw defines that are used to mask and compare directly the
mask part of a psw.
We have e.g. PSW_AS_PRIMARY vs PSW_ASC_PRIMARY.

To avoid confusion rename the PSW_AS_* enums to PSW_BITS_AS_*.

In addition also rename the PSW_AMODE_* enums, so they also follow the
same naming scheme: PSW_BITS_AMODE_*.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Right now the kernel uses the primary address space until finally the
switch to the correct home address space will be done when the idle
PSW will be loaded within psw_idle().

Correct this and simply use the home address space when DAT is enabled
for the first time.

This doesn't really fix a bug, but fixes odd behavior.
Reviewed-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

This reverts the two commits

7afbeb6d ("s390/ipl: always use load normal for CCW-type re-IPL")
0f7451ff ("s390/ipl: use load normal for LPAR re-ipl")

The two commits did not take into account that behavior of standby
memory changes fundamentally if the re-IPL method is changed from
Load Clear to Load Normal.

In case of the old re-IPL clear method all memory that was initially
in standby state will be put into standby state again within the
re-IPL process. Or in other words: memory that was brought online
before a re-IPL will be offline again after a reboot.

Given that we use different re-IPL methods depending on the hypervisor
and CCW-type vs SCSI re-IPL it is not easy to tell in advance when and
why memory will stay online or will be offline after a re-IPL.
This does also have other side effects, since memory that is online
from the beginning will be in ZONE_NORMAL by default vs ZONE_MOVABLE
for memory that is offline.

Therefore, before the change, a user could online and offline memory
easily since standby memory was always in ZONE_NORMAL.  After the
change, and a re-IPL, this depended on which memory parts were online
before the re-IPL.

From a usability point of view the current behavior is more than
suboptimal. Therefore revert these changes until we have a better
solution and get back to a consistent behavior. The bad thing about
this is that the time required for a re-IPL will be significantly
increased for configurations with several 100GB or 1TB of memory.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Remove raw stack dumps that are printed before call traces in case of
a warning, or the 'l' sysrq trigger (show a stack backtrace for all
active CPUs).

Besides that a raw stack dump should not be shown for the 'l' sysrq
trigger the value of the dump is close to zero. That's also why we
don't print it in case of a panic since ages anymore. That this is
still printed on warnings is just a leftover. So get rid of this
completely.

The following won't be printed anymore with this change:

Stack:
       00000000bbc4fbc8 00000000bbc4fc58 0000000000000003 0000000000000000
       00000000bbc4fcf8 00000000bbc4fc70 00000000bbc4fc70 0000000000000020
       000000007fe00098 00000000bfe8be00 00000000bbc4fe94 000000000000000a
       000000000000000c 00000000bbc4fcc0 0000000000000000 0000000000000000
       000000000095b930 0000000000113366 00000000bbc4fc58 00000000bbc4fca0
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Move the CONFIG_PCI device so that ioremap and iounmap are always
available. This looks safe as there's nothing PCI specific in the
implementation of these functions.

I have designs to use these functions in scatterlist.c where they'd likely
never be called without CONFIG_PCI set, but this is needed to compile
such changes.
Signed-off-by: NLogan Gunthorpe <logang@deltatee.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Sebastian Ott <sebott@linux.vnet.ibm.com>
Cc: Al Viro <viro@ZenIV.linux.org.uk>
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Command 'perf list pmu' displays events which contain
an invalid string "(null)=xxx", where xxx is the pmu event
name, for example:
   cpum_cf/AES_BLOCKED_CYCLES,(null)=AES_BLOCKED_CYCLES/
This is not correct, the invalid string should not be
displayed at all.

It is caused by an obsolete term in the
sysfs attribute file for each s390 CPUMF counter event.
Reading from the sysfs file also displays the event
name.

Fix this by omitting the event name.  This patch makes
s390 CPUMF sysfs files consistent with other plattforms.

This is an interface change between user and kernel
but does not break anything. Reading from a counter event
sysfs file should only list terms mentioned in the
/sys/bus/event_source/devices/<cpumf>/format directory.
Name is not listed.
Reported-by: NZvonko Kosic <zvonko.kosic@de.ibm.com>
Signed-off-by: NThomas Richter <tmricht@linux.vnet.ibm.com>
Reviewed-by: NHendrik Brueckner <brueckner@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Introduce and use p?d_folded() functions to clarify the page table
code a bit more.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

_REGION3_ENTRY_ORIGIN defines a wrong mask which can be used to
extract a segment table origin from a region 3 table entry. It removes
only the lower 11 instead of 12 bits from a region 3 table entry.
Luckily this bit is currently always zero, so nothing bad happened yet.

In order to avoid future bugs just remove the region 3 specific mask
and use the correct generic _REGION_ENTRY_ORIGIN mask.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The regset functions for guarded storage are supposed to work on
the current task as well. For task == current add the required
load and store instructions for the guarded storage control block.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

All call sites of "stfle" check if the instruction is available before
executing it. Therefore there is no reason to have the corresponding
facility bit set within the architecture level set.
This removes the last more or less odd bit from the list.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The code in arch/s390/crypto checks for the availability of the
'message security assist' facility on its own, either by using
module_cpu_feature_match(MSA, ...) or by checking the facility
bit during cpacf_query(). Thus setting the MSA facility bit in
gen_facilities.c as hard requirement is not necessary. We can
remove it here, so that the kernel can also run on systems that
do not provide the MSA facility yet (like the emulated environment
of QEMU, for example).
Signed-off-by: NThomas Huth <thuth@redhat.com>
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

When masking an ASCE to get its origin use the corresponding define
instead of the unrelated PAGE_MASK.
This doesn't fix a bug since both masks are identical.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Use proper define instead of open-coding the condition code value.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

I get number of CPUs - 1 kmemleak hits like

unreferenced object 0x37ec6f000 (size 1024):
  comm "swapper/0", pid 1, jiffies 4294937330 (age 889.690s)
  hex dump (first 32 bytes):
    6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
    6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
  backtrace:
    [<000000000034a848>] kmem_cache_alloc+0x2b8/0x3d0
    [<00000000001164de>] __cpu_up+0x456/0x488
    [<000000000016f60c>] bringup_cpu+0x4c/0xd0
    [<000000000016d5d2>] cpuhp_invoke_callback+0xe2/0x9e8
    [<000000000016f3c6>] cpuhp_up_callbacks+0x5e/0x110
    [<000000000016f988>] _cpu_up+0xe0/0x158
    [<000000000016faf0>] do_cpu_up+0xf0/0x110
    [<0000000000dae1ee>] smp_init+0x126/0x130
    [<0000000000d9bd04>] kernel_init_freeable+0x174/0x2e0
    [<000000000089fc62>] kernel_init+0x2a/0x148
    [<00000000008adce2>] kernel_thread_starter+0x6/0xc
    [<00000000008adcdc>] kernel_thread_starter+0x0/0xc
    [<ffffffffffffffff>] 0xffffffffffffffff

The pointer of this data structure is stored in the prefix page of that
CPU together with some extra bits ORed into the the low bits.
Mark the data structure as non-leak.
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The s390_paes and the s390_aes kernel module used just one
config symbol CONFIG_CRYPTO_AES. As paes has a dependency
to PKEY and this requires ZCRYPT the aes module also had
a dependency to the zcrypt device driver which is not true.
Fixed by introducing a new config symbol CONFIG_CRYPTO_PAES
which has dependencies to PKEY and ZCRYPT. Removed the
dependency for the aes module to ZCRYPT.
Signed-off-by: NHarald Freudenberger <freude@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Switch to the improved _install_special_mapping function to install
the vdso mapping. This has two advantages, the arch_vma_name function
is not needed anymore and the vdso vma still has its name after its
memory location has been changed with mremap.
Tested-by: NMichael Holzheu <holzheu@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The account_system_index_scaled gets two cputime values, a raw value
derived from CPU timer deltas and a scaled value. The scaled value
is always calculated from the raw value, the code can be simplified
by moving the scale_vtime call into account_system_index_scaled.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

I missed at least these two header files where we can make use of the
generic ones.  vga.h is another one, however that is already addressed
by a patch from Jiri Slaby.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Add __rcu annotations so sparse correctly warns only if "slot" gets
derefenced without using rcu_dereference(). Right now we get warnings
because of the missing annotation:

arch/s390/mm/gmap.c:135:17: warning: incorrect type in assignment (different address spaces)
arch/s390/mm/gmap.c:135:17:    expected void **slot
arch/s390/mm/gmap.c:135:17:    got void [noderef] <asn:4>**
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Add missing include statements to make sure that prototypes match
implementation. As reported by sparse:

arch/s390/crypto/arch_random.c:18:1:
  warning: symbol 's390_arch_random_available' was not declared. Should it be static?
arch/s390/kernel/traps.c:279:13: warning:
  symbol 'trap_init' was not declared. Should it be static?
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Add the logic to upgrade the page table for a 64-bit process to
five levels. This increases the TASK_SIZE from 8PB to 16EB-4K.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

For most cases a protection exception in the host (e.g. copy
on write or dirty tracking) on the sie instruction will indicate
an instruction length of 4. Turns out that there are some corner
cases (e.g. runtime instrumentation) where this is not necessarily
true and the ILC is unpredictable.

Let's replace our 4 byte rewind_pad with 3 byte nops to prepare for
all possible ILCs.
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Cc: stable@vger.kernel.org
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Cc: Tigran Aivazian <aivazian.tigran@gmail.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Shubham was recently asking on netdev why in arm64 JIT we don't multiply
the index for accessing the tail call map by 8. That led me into testing
out arm64 JIT wrt tail calls and it turned out I got a NULL pointer
dereference on the tail call.

The buggy access is at:

  prog = array->ptrs[index];
  if (prog == NULL)
      goto out;

  [...]
  00000060:  d2800e0a  mov x10, #0x70 // #112
  00000064:  f86a682a  ldr x10, [x1,x10]
  00000068:  f862694b  ldr x11, [x10,x2]
  0000006c:  b40000ab  cbz x11, 0x00000080
  [...]

The code triggering the crash is f862694b. x1 at the time contains the
address of the bpf array, x10 offsetof(struct bpf_array, ptrs). Meaning,
above we load the pointer to the program at map slot 0 into x10. x10
can then be NULL if the slot is not occupied, which we later on try to
access with a user given offset in x2 that is the map index.

Fix this by emitting the following instead:

  [...]
  00000060:  d2800e0a  mov x10, #0x70 // #112
  00000064:  8b0a002a  add x10, x1, x10
  00000068:  d37df04b  lsl x11, x2, #3
  0000006c:  f86b694b  ldr x11, [x10,x11]
  00000070:  b40000ab  cbz x11, 0x00000084
  [...]

This basically adds the offset to ptrs to the base address of the bpf
array we got and we later on access the map with an index * 8 offset
relative to that. The tail call map itself is basically one large area
with meta data at the head followed by the array of prog pointers.
This makes tail calls working again, tested on Cavium ThunderX ARMv8.

Fixes: ddb55992 ("arm64: bpf: implement bpf_tail_call() helper")
Reported-by: NShubham Bansal <illusionist.neo@gmail.com>
Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

refcount_t type and corresponding API should be
used instead of atomic_t when the variable is used as
a reference counter. This allows to avoid accidental
refcounter overflows that might lead to use-after-free
situations.
Signed-off-by: NElena Reshetova <elena.reshetova@intel.com>
Signed-off-by: NHans Liljestrand <ishkamiel@gmail.com>
Signed-off-by: NKees Cook <keescook@chromium.org>
Signed-off-by: NDavid Windsor <dwindsor@gmail.com>
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

When booted as pv-guest the p2m list presented by the Xen is already
mapped to virtual addresses. In dom0 case the hypervisor might make use
of 2M- or 1G-pages for this mapping. Unfortunately while being properly
aligned in virtual and machine address space, those pages might not be
aligned properly in guest physical address space.

So when trying to obtain the guest physical address of such a page
pud_pfn() and pmd_pfn() must be avoided as those will mask away guest
physical address bits not being zero in this special case.
Signed-off-by: NJuergen Gross <jgross@suse.com>
Reviewed-by: NJan Beulich <jbeulich@suse.com>
Signed-off-by: NJuergen Gross <jgross@suse.com>

When running as Xen pv guest X86_BUG_SYSRET_SS_ATTRS must not be set
on AMD cpus.

This bug/feature bit is kind of special as it will be used very early
when switching threads. Setting the bit and clearing it a little bit
later leaves a critical window where things can go wrong. This time
window has enlarged a little bit by using setup_clear_cpu_cap() instead
of the hypervisor's set_cpu_features callback. It seems this larger
window now makes it rather easy to hit the problem.

The proper solution is to never set the bit in case of Xen.
Signed-off-by: NJuergen Gross <jgross@suse.com>
Reviewed-by: NBoris Ostrovsky <boris.ostrovsky@oracle.com>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJuergen Gross <jgross@suse.com>

When CONFIG_ARM64_MODULE_PLTS is enabled, the first allocation using the
module space fails, because the module is too big, and then the module
allocation is attempted from vmalloc space. Silence the first allocation
failure in that case by setting __GFP_NOWARN.
Reviewed-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

When CONFIG_ARM_MODULE_PLTS is enabled, the first allocation using the
module space fails, because the module is too big, and then the module
allocation is attempted from vmalloc space. Silence the first allocation
failure in that case by setting __GFP_NOWARN.
Acked-by: NRussell King <rmk+kernel@armlinux.org.uk>
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

As of commits d8f347ba35cf ("nios2: enable earlycon support"),
0dcc0542 ("serial: altera_jtaguart: add earlycon support") and
4d9d7d89 ("serial: altera_uart: add earlycon support"), the nios2
architecture and the altera_uart/altera_jtaguart drivers support
earlycon. Thus, the custom early console implementation for nios2 is no
longer necessary to get early boot messages. Remove it and rely fully on
earlycon support.
Signed-off-by: NTobias Klauser <tklauser@distanz.ch>

Signed-off-by: NNicolas Dichtel <nicolas.dichtel@6wind.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

This patch removes the need of subdir-y. Now all files/directories under
arch/<arch>/include/uapi/ are exported.

The only change for userland is the layout of the command 'make
headers_install_all': directories asm-<arch> are replaced by arch-<arch>/.
Those new directories contains all files/directories of the specified arch.

Note that only cris and tile have more directories than only asm:
 - arch-v[10|32] for cris;
 - arch for tile.
Signed-off-by: NNicolas Dichtel <nicolas.dichtel@6wind.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Regularly, when a new header is created in include/uapi/, the developer
forgets to add it in the corresponding Kbuild file. This error is usually
detected after the release is out.

In fact, all headers under uapi directories should be exported, thus it's
useless to have an exhaustive list.

After this patch, the following files, which were not exported, are now
exported (with make headers_install_all):
asm-arc/kvm_para.h
asm-arc/ucontext.h
asm-blackfin/shmparam.h
asm-blackfin/ucontext.h
asm-c6x/shmparam.h
asm-c6x/ucontext.h
asm-cris/kvm_para.h
asm-h8300/shmparam.h
asm-h8300/ucontext.h
asm-hexagon/shmparam.h
asm-m32r/kvm_para.h
asm-m68k/kvm_para.h
asm-m68k/shmparam.h
asm-metag/kvm_para.h
asm-metag/shmparam.h
asm-metag/ucontext.h
asm-mips/hwcap.h
asm-mips/reg.h
asm-mips/ucontext.h
asm-nios2/kvm_para.h
asm-nios2/ucontext.h
asm-openrisc/shmparam.h
asm-parisc/kvm_para.h
asm-powerpc/perf_regs.h
asm-sh/kvm_para.h
asm-sh/ucontext.h
asm-tile/shmparam.h
asm-unicore32/shmparam.h
asm-unicore32/ucontext.h
asm-x86/hwcap2.h
asm-xtensa/kvm_para.h
drm/armada_drm.h
drm/etnaviv_drm.h
drm/vgem_drm.h
linux/aspeed-lpc-ctrl.h
linux/auto_dev-ioctl.h
linux/bcache.h
linux/btrfs_tree.h
linux/can/vxcan.h
linux/cifs/cifs_mount.h
linux/coresight-stm.h
linux/cryptouser.h
linux/fsmap.h
linux/genwqe/genwqe_card.h
linux/hash_info.h
linux/kcm.h
linux/kcov.h
linux/kfd_ioctl.h
linux/lightnvm.h
linux/module.h
linux/nbd-netlink.h
linux/nilfs2_api.h
linux/nilfs2_ondisk.h
linux/nsfs.h
linux/pr.h
linux/qrtr.h
linux/rpmsg.h
linux/sched/types.h
linux/sed-opal.h
linux/smc.h
linux/smc_diag.h
linux/stm.h
linux/switchtec_ioctl.h
linux/vfio_ccw.h
linux/wil6210_uapi.h
rdma/bnxt_re-abi.h

Note that I have removed from this list the files which are generated in every
exported directories (like .install or .install.cmd).

Thanks to Julien Floret <julien.floret@6wind.com> for the tip to get all
subdirs with a pure makefile command.

For the record, note that exported files for asm directories are a mix of
files listed by:
 - include/uapi/asm-generic/Kbuild.asm;
 - arch/<arch>/include/uapi/asm/Kbuild;
 - arch/<arch>/include/asm/Kbuild.
Signed-off-by: NNicolas Dichtel <nicolas.dichtel@6wind.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: NRussell King <rmk+kernel@armlinux.org.uk>
Acked-by: NMark Salter <msalter@redhat.com>
Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Even if this file was not in an uapi directory, it was exported because
it was listed in the Kbuild file.

Fixes: b72e7464 ("x86/uapi: Do not export <asm/msr-index.h> as part of the user API headers")
Suggested-by: NBorislav Petkov <bp@alien8.de>
Signed-off-by: NNicolas Dichtel <nicolas.dichtel@6wind.com>
Acked-by: NIngo Molnar <mingo@kernel.org>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

This header file is exported, but from a userland pov, it's just a wrapper
to asm-generic/setup.h.
Signed-off-by: NNicolas Dichtel <nicolas.dichtel@6wind.com>
Reviewed-by: NTobias Klauser <tklauser@distanz.ch>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

This header file is exported, thus move it to uapi.
Signed-off-by: NNicolas Dichtel <nicolas.dichtel@6wind.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

When any of the functions contained in NGbzero.S and GENbzero.S
vector through *bzero_from_clear_user, we may end up taking a
fault when executing one of the store alternate address space
instructions. If this happens, the exception handler does not
restore the %asi register.

This commit fixes the issue by introducing a new exception
handler that ensures the %asi register is restored when
a fault is handled.

Orabug: 25577560
Signed-off-by: NDave Aldridge <david.j.aldridge@oracle.com>
Reviewed-by: NRob Gardner <rob.gardner@oracle.com>
Reviewed-by: NBabu Moger <babu.moger@oracle.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use memdup_user_nul() helper instead of open-coding to simplify the code.
Signed-off-by: NGeliang Tang <geliangtang@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Commit 11e63f6d added cache flushing for unaligned writes from an
iovec, covering the first and last cache line of a >= 8 byte write and
the first cache line of a < 8 byte write.  But an unaligned write of
2-7 bytes can still cover two cache lines, so make sure we flush both
in that case.

Cc: <stable@vger.kernel.org>
Fixes: 11e63f6d ("x86, pmem: fix broken __copy_user_nocache ...")
Signed-off-by: NBen Hutchings <ben.hutchings@codethink.co.uk>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>