1. 13 2月, 2015 1 次提交
  2. 11 2月, 2015 1 次提交
  3. 10 10月, 2014 2 次提交
    • S
      arm: mm: enable RCU fast_gup · b8cd51af
      Steve Capper 提交于
      Activate the RCU fast_gup for ARM.  We also need to force THP splits to
      broadcast an IPI s.t.  we block in the fast_gup page walker.  As THP
      splits are comparatively rare, this should not lead to a noticeable
      performance degradation.
      
      Some pre-requisite functions pud_write and pud_page are also added.
      Signed-off-by: NSteve Capper <steve.capper@linaro.org>
      Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
      Cc: Dann Frazier <dann.frazier@canonical.com>
      Cc: Hugh Dickins <hughd@google.com>
      Cc: Russell King <rmk@arm.linux.org.uk>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Mel Gorman <mel@csn.ul.ie>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Christoffer Dall <christoffer.dall@linaro.org>
      Cc: Andrea Arcangeli <aarcange@redhat.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      b8cd51af
    • S
      arm: mm: introduce special ptes for LPAE · bd951303
      Steve Capper 提交于
      We need a mechanism to tag ptes as being special, this indicates that no
      attempt should be made to access the underlying struct page * associated
      with the pte.  This is used by the fast_gup when operating on ptes as it
      has no means to access VMAs (that also contain this information)
      locklessly.
      
      The L_PTE_SPECIAL bit is already allocated for LPAE, this patch modifies
      pte_special and pte_mkspecial to make use of it, and defines
      __HAVE_ARCH_PTE_SPECIAL.
      
      This patch also excludes special ptes from the icache/dcache sync logic.
      Signed-off-by: NSteve Capper <steve.capper@linaro.org>
      Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
      Cc: Dann Frazier <dann.frazier@canonical.com>
      Cc: Hugh Dickins <hughd@google.com>
      Cc: Russell King <rmk@arm.linux.org.uk>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Mel Gorman <mel@csn.ul.ie>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Christoffer Dall <christoffer.dall@linaro.org>
      Cc: Andrea Arcangeli <aarcange@redhat.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      bd951303
  4. 24 7月, 2014 2 次提交
    • S
      ARM: 8109/1: mm: Modify pte_write and pmd_write logic for LPAE · ded94779
      Steven Capper 提交于
      For LPAE, we have the following means for encoding writable or dirty
      ptes:
                                    L_PTE_DIRTY       L_PTE_RDONLY
          !pte_dirty && !pte_write        0               1
          !pte_dirty && pte_write         0               1
          pte_dirty && !pte_write         1               1
          pte_dirty && pte_write          1               0
      
      So we can't distinguish between writeable clean ptes and read only
      ptes. This can cause problems with ptes being incorrectly flagged as
      read only when they are writeable but not dirty.
      
      This patch renumbers L_PTE_RDONLY from AP[2] to a software bit #58,
      and adds additional logic to set AP[2] whenever the pte is read only
      or not dirty. That way we can distinguish between clean writeable ptes
      and read only ptes.
      
      HugeTLB pages will use this new logic automatically.
      
      We need to add some logic to Transparent HugePages to ensure that they
      correctly interpret the revised pgprot permissions (L_PTE_RDONLY has
      moved and no longer matches PMD_SECT_AP2). In the process of revising
      THP, the names of the PMD software bits have been prefixed with L_ to
      make them easier to distinguish from their hardware bit counterparts.
      Signed-off-by: NSteve Capper <steve.capper@linaro.org>
      Reviewed-by: NWill Deacon <will.deacon@arm.com>
      Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
      ded94779
    • S
      ARM: 8108/1: mm: Introduce {pte,pmd}_isset and {pte,pmd}_isclear · f2950706
      Steven Capper 提交于
      Long descriptors on ARM are 64 bits, and some pte functions such as
      pte_dirty return a bitwise-and of a flag with the pte value. If the
      flag to be tested resides in the upper 32 bits of the pte, then we run
      into the danger of the result being dropped if downcast.
      
      For example:
      	gather_stats(page, md, pte_dirty(*pte), 1);
      where pte_dirty(*pte) is downcast to an int.
      
      This patch introduces a new macro pte_isset which performs the bitwise
      and, then performs a double logical invert (where needed) to ensure
      predictable downcasting. The logical inverse pte_isclear is also
      introduced.
      
      Equivalent pmd functions for Transparent HugePages have also been
      added.
      Signed-off-by: NSteve Capper <steve.capper@linaro.org>
      Reviewed-by: NWill Deacon <will.deacon@arm.com>
      Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
      f2950706
  5. 10 2月, 2014 1 次提交
  6. 11 12月, 2013 1 次提交
  7. 29 10月, 2013 1 次提交
    • S
      ARM: 7858/1: mm: make UACCESS_WITH_MEMCPY huge page aware · a3a9ea65
      Steven Capper 提交于
      The memory pinning code in uaccess_with_memcpy.c does not check
      for HugeTLB or THP pmds, and will enter an infinite loop should
      a __copy_to_user or __clear_user occur against a huge page.
      
      This patch adds detection code for huge pages to pin_page_for_write.
      As this code can be executed in a fast path it refers to the actual
      pmds rather than the vma. If a HugeTLB or THP is found (they have
      the same pmd representation on ARM), the page table spinlock is
      taken to prevent modification whilst the page is pinned.
      
      On ARM, huge pages are only represented as pmds, thus no huge pud
      checks are performed. (For huge puds one would lock the page table
      in a similar manner as in the pmd case).
      
      Two helper functions are introduced; pmd_thp_or_huge will check
      whether or not a page is huge or transparent huge (which have the
      same pmd layout on ARM), and pmd_hugewillfault will detect whether
      or not a page fault will occur on write to the page.
      
      Running the following test (with the chunking from read_zero
      removed):
       $ dd if=/dev/zero of=/dev/null bs=10M count=1024
      Gave:  2.3 GB/s backed by normal pages,
             2.9 GB/s backed by huge pages,
             5.1 GB/s backed by huge pages, with page mask=HPAGE_MASK.
      
      After some discussion, it was decided not to adopt the HPAGE_MASK,
      as this would have a significant detrimental effect on the overall
      system latency due to page_table_lock being held for too long.
      This could be revisited if split huge page locks are adopted.
      Signed-off-by: NSteve Capper <steve.capper@linaro.org>
      Reviewed-by: NNicolas Pitre <nico@linaro.org>
      Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
      a3a9ea65
  8. 18 10月, 2013 1 次提交
  9. 04 6月, 2013 3 次提交
  10. 30 5月, 2013 2 次提交
  11. 17 4月, 2013 1 次提交
  12. 24 1月, 2013 1 次提交
  13. 09 11月, 2012 2 次提交
    • W
      ARM: mm: introduce present, faulting entries for PAGE_NONE · 26ffd0d4
      Will Deacon 提交于
      PROT_NONE mappings apply the page protection attributes defined by _P000
      which translate to PAGE_NONE for ARM. These attributes specify an XN,
      RDONLY pte that is inaccessible to userspace. However, on kernels
      configured without support for domains, such a pte *is* accessible to
      the kernel and can be read via get_user, allowing tasks to read
      PROT_NONE pages via syscalls such as read/write over a pipe.
      
      This patch introduces a new software pte flag, L_PTE_NONE, that is set
      to identify faulting, present entries.
      Signed-off-by: NWill Deacon <will.deacon@arm.com>
      26ffd0d4
    • W
      ARM: mm: introduce L_PTE_VALID for page table entries · dbf62d50
      Will Deacon 提交于
      For long-descriptor translation table formats, the ARMv7 architecture
      defines the last two bits of the second- and third-level descriptors to
      be:
      
      	x0b	- Invalid
      	01b	- Block (second-level), Reserved (third-level)
      	11b	- Table (second-level), Page (third-level)
      
      This allows us to define L_PTE_PRESENT as (3 << 0) and use this value to
      create ptes directly. However, when determining whether a given pte
      value is present in the low-level page table accessors, we only need to
      check the least significant bit of the descriptor, allowing us to write
      faulting, present entries which are required for PROT_NONE mappings.
      
      This patch introduces L_PTE_VALID, which can be used to test whether a
      pte should fault, and updates the low-level page table accessors
      accordingly.
      Signed-off-by: NWill Deacon <will.deacon@arm.com>
      dbf62d50
  14. 12 5月, 2012 1 次提交
  15. 08 12月, 2011 2 次提交
    • C
      ARM: LPAE: Page table maintenance for the 3-level format · da028779
      Catalin Marinas 提交于
      This patch modifies the pgd/pmd/pte manipulation functions to support
      the 3-level page table format. Since there is no need for an 'ext'
      argument to cpu_set_pte_ext(), this patch conditionally defines a
      different prototype for this function when CONFIG_ARM_LPAE.
      
      The patch also introduces the L_PGD_SWAPPER flag to mark pgd entries
      pointing to pmd tables pre-allocated in the swapper_pg_dir and avoid
      trying to free them at run-time. This flag is 0 with the classic page
      table format.
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      da028779
    • C
      ARM: LPAE: Introduce the 3-level page table format definitions · dcfdae04
      Catalin Marinas 提交于
      This patch introduces the pgtable-3level*.h files with definitions
      specific to the LPAE page table format (3 levels of page tables).
      
      Each table is 4KB and has 512 64-bit entries. An entry can point to a
      40-bit physical address. The young, write and exec software bits share
      the corresponding hardware bits (negated). Other software bits use spare
      bits in the PTE.
      
      The patch also changes some variable types from unsigned long or int to
      pteval_t or pgprot_t.
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      dcfdae04