Skip to content

K
Kernel

openeuler / Kernel
接近 2 年 前同步成功

通知 8
Star 0
Fork 0
K
Kernel
切换分支/标签
  1. 05 10月, 2016 2 次提交
  3. 04 10月, 2016 2 次提交
    • J
      MIPS: c-r4k: Split user/kernel flush_icache_range() · 01882b4d
      由 James Hogan 提交于 
      flush_icache_range() is used for both user addresses (i.e.
cacheflush(2)), and kernel addresses (as the API documentation
describes).

This isn't really suitable however for Enhanced Virtual Addressing (EVA)
where cache operations on usermode addresses must use a different
instruction, and the protected cache ops assume user addresses, making
flush_icache_range() ineffective on kernel addresses.

Split out a new __flush_icache_user_range() and
__local_flush_icache_user_range() for users which actually want to flush
usermode addresses (note that flush_icache_user_range() already exists
on various architectures but with different arguments).

The implementation of flush_icache_range() will be changed in an
upcoming commit to use unprotected normal cache ops so as to always work
on the kernel mode address space.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/14152/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      01882b4d
    • J
      MIPS: c-r4k: Drop bc_wback_inv() from icache flush · d260d97e
      由 James Hogan 提交于 
      The EVA conditional bc_wback_inv() at the end of flush_icache_range() to
flush the modified code all the way back to RAM was apparently there for
debug purposes and to accommodate the Malta EVA configuration which
makes use of a physical alias, and didn't use the CP0_EBase.WG (Write
Gate) bit to put the exception vector in the same physical alias where
the exception vector code is written and is being flushed.

Now that CP0_EBase.WG is used, lets drop this flush.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/14151/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      d260d97e
  5. 13 9月, 2016 1 次提交
  7. 29 7月, 2016 10 次提交
    • J
      MIPS: c-r4k: Use SMP calls for CM indexed cache ops · 11f76903
      由 James Hogan 提交于 
      The MIPS Coherence Manager (CM) can propagate address-based ("hit")
cache operations to other cores in the coherent system, alleviating
software of the need to use SMP calls, however indexed cache operations
are not propagated by hardware since doing so makes no sense for
separate caches.

Update r4k_op_needs_ipi() to report that only hit cache operations are
globalized by the CM, requiring indexed cache operations to be
globalized by software via an SMP call.

r4k_on_each_cpu() previously had a special case for CONFIG_MIPS_MT_SMP,
intended to avoid the SMP calls when the only other CPUs in the system
were other VPEs in the same core, and hence sharing the same caches.
This was changed by commit cccf34e9 ("MIPS: c-r4k: Fix cache
flushing for MT cores") to apparently handle multi-core multi-VPE
systems, but it focussed mainly on hit cache ops, so the SMP calls were
still disabled entirely for CM systems.

This doesn't normally cause problems, but tests can be written to hit
these corner cases by using multiple threads, or changing task
affinities to force the process to migrate cores. For example the
failure of mprotect RW->RX to globally sync icaches (via
flush_cache_range) can be detected by modifying and mprotecting a code
page on one core, and migrating to a different core to execute from it.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13807/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      11f76903
    • J
      MIPS: c-r4k: Avoid small flush_icache_range SMP calls · f70ddc07
      由 James Hogan 提交于 
      Avoid SMP calls for flushing small icache ranges. On non-CM platforms,
and CM platforms too after we make r4k_on_each_cpu() take the cache op
type into account, it will be called on multiple CPUs due to the
possibility that local_r4k_flush_icache_range_ipi() could do
non-globalized indexed cache ops. This rougly copies the range size
check out into r4k_flush_icache_range(), which can disallow indexed
cache ops and allow r4k_on_each_cpu() to skip the SMP call.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13805/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      f70ddc07
    • J
      MIPS: c-r4k: Local flush_icache_range cache op override · 27b93d9c
      由 James Hogan 提交于 
      Allow the permitted cache op types used by
local_r4k_flush_icache_range_ipi() to be overridden by the SMP caller.
This will allow SMP calls to be avoided under certain circumstances,
falling back to a single CPU performing globalized hit cache ops only.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13803/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      27b93d9c
    • J
      MIPS: c-r4k: Split r4k_flush_kernel_vmap_range() · a9341ae2
      由 James Hogan 提交于 
      Split the operation of r4k_flush_kernel_vmap_range() into separate
SMP callbacks for the indexed cache flush and hit cache flush cases,
since the logic to determine which to use can be determined by the
initiating CPU prior to doing any SMP calls.

This will help when we change r4k_on_each_cpu() to distinguish indexed
and hit cache ops in a later patch, preventing globalized hit cache ops
being performed redundantly on multiple CPUs.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13806/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      a9341ae2
    • J
      MIPS: c-r4k: Exclude sibling CPUs in SMP calls · 640511ae
      由 James Hogan 提交于 
      When performing SMP calls to foreign cores, exclude sibling CPUs from
the provided map, as we already handle the local core on the current
CPU. This prevents an SMP call from for example core 0, VPE 1 to VPE 0
on the same core.

In the process the cpu_foreign_map cpumask is turned into an array of
cpumasks, so that each CPU has its own version of it which excludes
sibling CPUs. r4k_op_needs_ipi() is also updated to reflect that cache
management SMP calls are not needed when all CPUs are siblings (i.e.
there are no foreign CPUs according to the new cpu_foreign_map[]
semantics which exclude siblings).
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: Felix Fietkau <nbd@nbd.name>
Cc: Jayachandran C. <jchandra@broadcom.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13801/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      640511ae
    • J
      MIPS: c-r4k: Fix valid ASID optimisation · 6d758bfc
      由 James Hogan 提交于 
      Several cache operations are optimised to return early from the SMP call
handler if the memory map in question has no valid ASID on the current
CPU, or any online CPU in the case of MIPS_MT_SMP. The idea is that if a
memory map has never been used on a CPU it shouldn't have cache lines in
need of flushing.

However this doesn't cover all cases when ASIDs for other CPUs need to
be checked:
- Offline VPEs may have recently been online and brought lines into the
  (shared) cache, so they should also be checked, rather than only
  online CPUs.
- SMP systems with a Coherence Manager (CM), but with MT disabled still
  have globalized hit cache ops, but don't use SMP calls, so all present
  CPUs should be taken into account.
- R6 systems have a different multithreading implementation, so
  MIPS_MT_SMP won't be set, but as above may still have a CM which
  globalizes hit cache ops.

Additionally for non-globalized cache operations where an SMP call to a
single VPE in each foreign core is used, it is not necessary to check
every CPU in the system, only sibling CPUs sharing the same first level
cache.

Fix this by making has_valid_asid() take a cache op type argument like
r4k_on_each_cpu(), so it can determine whether r4k_on_each_cpu() will
have done SMP calls to other cores. It can then determine which set of
CPUs to check the ASIDs of based on that, excluding foreign CPUs if an
SMP call will have been performed.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13804/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      6d758bfc
    • J
      MIPS: c-r4k: Add r4k_on_each_cpu cache op type arg · d374d937
      由 James Hogan 提交于 
      The r4k_on_each_cpu() function calls the specified cache flush helper on
other CPUs if deemed necessary due to the cache ops not being
globalized by hardware. However this really depends on the cache op
addressing type, as the MIPS Coherence Manager (CM) if present will
globalize "hit" cache ops (addressed by virtual address), but not
"index" cache ops (addressed by cache index). This results in index
cache ops only being performed on a single CPU when CM is present.

Most (but not all) of the functions called by r4k_on_each_cpu() perform
cache operations exclusively with a single cache op type, so add a type
argument and modify the callers to pass in some combination of R4K_HIT
(global kernel virtual addressing or user virtual addressing
conditional upon matching active_mm) and R4K_INDEX (index into cache).

This will allow r4k_on_each_cpu() to later distinguish these cases and
decide whether to perform an SMP call based on it.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13798/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      d374d937
    • J
      MIPS: c-r4k: Avoid dcache flush for sigtramps · 8bd646e9
      由 James Hogan 提交于 
      Avoid the dcache and scache flush in local_r4k_flush_cache_sigtramp() if
the icache fills straight from the dcache.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13802/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      8bd646e9
    • J
      MIPS: c-r4k: Fix sigtramp SMP call to use kmap · e523f289
      由 James Hogan 提交于 
      Fix r4k_flush_cache_sigtramp() and local_r4k_flush_cache_sigtramp() to
flush the delay slot emulation trampoline cacheline through a kmap
rather than directly when the active_mm doesn't match that of the task
initiating the flush, a bit like local_r4k_flush_cache_page() does.

This would fix a corner case on SMP systems without hardware globalized
hit cache ops, where a migration to another CPU after the flush, where
that CPU did not have the same mm active at the time of the flush, could
result in stale icache content being executed instead of the trampoline,
e.g. from a previous delay slot emulation with a similar stack pointer.

This case was artificially triggered by replacing the icache flush with
a full indexed flush (not globalized on CM systems) and forcing the SMP
call to take place, with a test program that alternated two FPU delay
slots with a parent process repeatedly changing scheduler affinity.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13797/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      e523f289
    • J
      MIPS: SMP: Clear ASID without confusing has_valid_asid() · a05c3920
      由 James Hogan 提交于 
      The SMP flush_tlb_*() functions may clear the memory map's ASIDs for
other CPUs if the mm has only a single user (the current CPU) in order
to avoid SMP calls. However this makes it appear to has_valid_asid(),
which is used by various cache flush functions, as if the CPUs have
never run in the mm, and therefore can't have cached any of its memory.

For flush_tlb_mm() this doesn't sound unreasonable.

flush_tlb_range() corresponds to flush_cache_range() which does do full
indexed cache flushes, but only on the icache if the specified mapping
is executable, otherwise it doesn't guarantee that there are no cache
contents left for the mm.

flush_tlb_page() corresponds to flush_cache_page(), which will perform
address based cache ops on the specified page only, and also only
touches the icache if the page is executable. It does not guarantee that
there are no cache contents left for the mm.

For example, this affects flush_cache_range() which uses the
has_valid_asid() optimisation. It is required to flush the icache when
mappings are made executable (e.g. using mprotect) so they are
immediately usable. If some code is changed to non executable in order
to be modified then it will not be flushed from the icache during that
time, but the ASID on other CPUs may still be cleared for TLB flushing.
When the code is changed back to executable, flush_cache_range() will
assume the code hasn't run on those other CPUs due to the zero ASID, and
won't invalidate the icache on them.

This is fixed by clearing the other CPUs ASIDs to 1 instead of 0 for the
above two flush_tlb_*() functions when the corresponding cache flushes
are likely to be incomplete (non executable range flush, or any page
flush). This ASID appears valid to has_valid_asid(), but still triggers
ASID regeneration due to the upper ASID version bits being 0, which is
less than the minimum ASID version of 1 and so always treated as stale.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13795/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      a05c3920
  9. 06 7月, 2016 1 次提交
    • R
      MIPS: Remove cpu_has_safe_index_cacheops · c00ab489
      由 Ralf Baechle 提交于 
      Very early versions of the 1004K had an hardware issue that made index
cache ops unsafe so they had to be avoided and hit ops be used instead.
This may significantly slow down cache maintenance operations.  Only
very early FPGA versions of the 1004K were affected so let's get rid
of the workaround which was only implemented for the DMA cache
maintenance operations anyway.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      c00ab489
  11. 16 6月, 2016 1 次提交
  13. 13 5月, 2016 9 次提交
  15. 09 5月, 2016 2 次提交
    • J
      MIPS: I6400: Icache fills from dcache · 47f2ac50
      由 James Hogan 提交于 
      Coherence Manager 3 (CM3) as present in I6400 can fill icache lines
effectively from dirty dcaches, so there is no need to flush dirty lines
from dcaches through to L2 prior to icache invalidation.

Set the MIPS_CACHE_IC_F_DC flag such that cpu_has_ic_fills_f_dc
evaluates to true, which avoids those dcache flushes.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: Manuel Lauss <manuel.lauss@gmail.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/12180/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      47f2ac50
    • J
      MIPS: c-r4k: Sync icache when it fills from dcache · b2a3c5be
      由 James Hogan 提交于 
      It is still necessary to handle icache coherency in flush_cache_range()
and copy_to_user_page() when the icache fills from the dcache, even
though the dcache does not need to be written back. However when this
handling was added in commit 2eaa7ec2 ("[MIPS] Handle I-cache
coherency in flush_cache_range()"), it did not do any icache flushing
when it fills from dcache.

Therefore fix r4k_flush_cache_range() to run
local_r4k_flush_cache_range() without taking into account whether icache
fills from dcache, so that the icache coherency gets handled. Checks are
also added in local_r4k_flush_cache_range() so that the dcache blast
doesn't take place when icache fills from dcache.

A test to mmap a page PROT_READ|PROT_WRITE, modify code in it, and
mprotect it to VM_READ|VM_EXEC (similar to case described in above
commit) can hit this case quite easily to verify the fix.

A similar check was added in commit f8829cae ("[MIPS] Fix aliasing
bug in copy_to_user_page / copy_from_user_page"), so also fix
copy_to_user_page() similarly, to call flush_cache_page() without taking
into account whether icache fills from dcache, since flush_cache_page()
already takes that into account to avoid performing a dcache flush.
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: Manuel Lauss <manuel.lauss@gmail.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/12179/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      b2a3c5be
  17. 16 1月, 2016 1 次提交
  19. 26 8月, 2015 1 次提交
  21. 10 7月, 2015 2 次提交
  23. 22 6月, 2015 1 次提交
  25. 17 6月, 2015 1 次提交
  27. 06 6月, 2015 1 次提交
  29. 02 4月, 2015 1 次提交
    • M
      MIPS: c-r4k.c: Fix the 74K D-cache alias erratum workaround · e2e7f29a
      由 Maciej W. Rozycki 提交于 
      Fix the 74K D-cache alias erratum workaround so that it actually works.
Our current code sets MIPS_CACHE_VTAG for the D-cache, but that flag
only has any effect for the I-cache.  Additionally MIPS_CACHE_PINDEX is
set for the D-cache if CP0.Config7.AR is also set for an affected
processor, leading to confusing information in the bootstrap log (the
flag isn't used beyond that).

So delete the setting of MIPS_CACHE_VTAG and rely on MIPS_CACHE_ALIASES,
set in a common place, removing I-cache coherency issues seen in GDB
testing with software breakpoints, gdbserver and ptrace(2), on affected
systems.

While at it add a little piece of explanation of what CP0.Config6.SYND
is so that people do not have to chase documentation.
Signed-off-by: NMaciej W. Rozycki <macro@codesourcery.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/8507/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
      e2e7f29a
  31. 01 4月, 2015 1 次提交
  33. 17 2月, 2015 2 次提交
  35. 16 2月, 2015 1 次提交