1. 15 11月, 2006 1 次提交
    • H
      [PATCH] hugetlb: prepare_hugepage_range check offset too · 68589bc3
      Hugh Dickins 提交于
      (David:)
      
      If hugetlbfs_file_mmap() returns a failure to do_mmap_pgoff() - for example,
      because the given file offset is not hugepage aligned - then do_mmap_pgoff
      will go to the unmap_and_free_vma backout path.
      
      But at this stage the vma hasn't been marked as hugepage, and the backout path
      will call unmap_region() on it.  That will eventually call down to the
      non-hugepage version of unmap_page_range().  On ppc64, at least, that will
      cause serious problems if there are any existing hugepage pagetable entries in
      the vicinity - for example if there are any other hugepage mappings under the
      same PUD.  unmap_page_range() will trigger a bad_pud() on the hugepage pud
      entries.  I suspect this will also cause bad problems on ia64, though I don't
      have a machine to test it on.
      
      (Hugh:)
      
      prepare_hugepage_range() should check file offset alignment when it checks
      virtual address and length, to stop MAP_FIXED with a bad huge offset from
      unmapping before it fails further down.  PowerPC should apply the same
      prepare_hugepage_range alignment checks as ia64 and all the others do.
      
      Then none of the alignment checks in hugetlbfs_file_mmap are required (nor
      is the check for too small a mapping); but even so, move up setting of
      VM_HUGETLB and add a comment to warn of what David Gibson discovered - if
      hugetlbfs_file_mmap fails before setting it, do_mmap_pgoff's unmap_region
      when unwinding from error will go the non-huge way, which may cause bad
      behaviour on architectures (powerpc and ia64) which segregate their huge
      mappings into a separate region of the address space.
      Signed-off-by: NHugh Dickins <hugh@veritas.com>
      Cc: "Luck, Tony" <tony.luck@intel.com>
      Cc: "David S. Miller" <davem@davemloft.net>
      Acked-by: NAdam Litke <agl@us.ibm.com>
      Acked-by: NDavid Gibson <david@gibson.dropbear.id.au>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      68589bc3
  2. 01 11月, 2006 1 次提交
  3. 24 8月, 2006 1 次提交
    • A
      [POWERPC] hugepage BUG fix · c9169f87
      Adam Litke 提交于
      On Tue, 2006-08-15 at 08:22 -0700, Dave Hansen wrote:
      > kernel BUG in cache_free_debugcheck at mm/slab.c:2748!
      
      Alright, this one is only triggered when slab debugging is enabled.  The
      slabs are assumed to be aligned on a HUGEPTE_TABLE_SIZE boundary.  The free
      path makes use of this assumption and uses the lowest nibble to pass around
      an index into an array of kmem_cache pointers.  With slab debugging turned
      on, the slab is still aligned, but the "working" object pointer is not.
      This would break the assumption above that a full nibble is available for
      the PGF_CACHENUM_MASK.
      
      The following patch reduces PGF_CACHENUM_MASK to cover only the two least
      significant bits, which is enough to cover the current number of 4 pgtable
      cache types.  Then use this constant to mask out the appropriate part of
      the huge pte pointer.
      Signed-off-by: NAdam Litke <agl@us.ibm.com>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      c9169f87
  4. 28 4月, 2006 1 次提交
    • D
      [PATCH] powerpc: Fix pagetable bloat for hugepages · f10a04c0
      David Gibson 提交于
      At present, ARCH=powerpc kernels can waste considerable space in
      pagetables when making large hugepage mappings.  Hugepage PTEs go in
      PMD pages, but each PMD page maps 256M and so contains only 16
      hugepage PTEs (128 bytes of data), but takes up a 1024 byte
      allocation.  With CONFIG_PPC_64K_PAGES enabled (64k base page size),
      the situation is worse.  Now hugepage PTEs are at the PTE page level
      (also mapping 256M), so we store 16 hugepage PTEs in a 64k allocation.
      
      The PowerPC MMU already means that any 256M region is either all
      hugepage, or all normal pages.  Thus, with some care, we can use a
      different allocation for the hugepage PTE tables and only allocate the
      128 bytes necessary.
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      f10a04c0
  5. 22 3月, 2006 1 次提交
    • D
      [PATCH] hugepage: is_aligned_hugepage_range() cleanup · 42b88bef
      David Gibson 提交于
      Quite a long time back, prepare_hugepage_range() replaced
      is_aligned_hugepage_range() as the callback from mm/mmap.c to arch code to
      verify if an address range is suitable for a hugepage mapping.
      is_aligned_hugepage_range() stuck around, but only to implement
      prepare_hugepage_range() on archs which didn't implement their own.
      
      Most archs (everything except ia64 and powerpc) used the same
      implementation of is_aligned_hugepage_range().  On powerpc, which
      implements its own prepare_hugepage_range(), the custom version was never
      used.
      
      In addition, "is_aligned_hugepage_range()" was a bad name, because it
      suggests it returns true iff the given range is a good hugepage range,
      whereas in fact it returns 0-or-error (so the sense is reversed).
      
      This patch cleans up by abolishing is_aligned_hugepage_range().  Instead
      prepare_hugepage_range() is defined directly.  Most archs use the default
      version, which simply checks the given region is aligned to the size of a
      hugepage.  ia64 and powerpc define custom versions.  The ia64 one simply
      checks that the range is in the correct address space region in addition to
      being suitably aligned.  The powerpc version (just as previously) checks
      for suitable addresses, and if necessary performs low-level MMU frobbing to
      set up new areas for use by hugepages.
      
      No libhugetlbfs testsuite regressions on ppc64 (POWER5 LPAR).
      Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
      Signed-off-by: NZhang Yanmin <yanmin.zhang@intel.com>
      Cc: "David S. Miller" <davem@davemloft.net>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: William Lee Irwin III <wli@holomorphy.com>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      42b88bef
  6. 09 1月, 2006 2 次提交
  7. 09 12月, 2005 2 次提交
    • D
      [PATCH] powerpc: Fix SLB flushing path in hugepage · 23ed6cb9
      David Gibson 提交于
      On ppc64, when opening a new hugepage region, we need to make sure any
      old normal-page SLBs for the area are flushed on all CPUs.  There was
      a bug in this logic - after putting the new hugepage area masks into
      the thread structure, we copied it into the paca (read by the SLB miss
      handler) only on one CPU, not on all.  This could cause incorrect SLB
      entries to be loaded when a multithreaded program was running
      simultaneously on several CPUs.  This patch corrects the error,
      copying the context information into the PACA on all CPUs using the mm
      in question before flushing any existing SLB entries.
      Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      23ed6cb9
    • D
      [PATCH] powerpc: Add missing icache flushes for hugepages · cbf52afd
      David Gibson 提交于
      On most powerpc CPUs, the dcache and icache are not coherent so
      between writing and executing a page, the caches must be flushed.
      Userspace programs assume pages given to them by the kernel are icache
      clean, so we must do this flush between the kernel clearing a page and
      it being mapped into userspace for execute.  We were not doing this
      for hugepages, this patch corrects the situation.
      
      We use the same lazy mechanism as we use for normal pages, delaying
      the flush until userspace actually attempts to execute from the page
      in question.
      
      Tested on G5.
      Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      cbf52afd
  8. 08 12月, 2005 1 次提交
  9. 25 11月, 2005 1 次提交
    • D
      [PATCH] powerpc: More hugepage boundary case fixes · 9a94c579
      David Gibson 提交于
      Blah.  The patch [0] I recently sent fixing errors with
      in_hugepage_area() and prepare_hugepage_range() for powerpc itself has
      an off-by-one bug.  Furthermore, the related functions
      touches_hugepage_*_range() and within_hugepage_*_range() are also
      buggy.  Some of the bugs, like those addressed in [0] originated with
      commit 7d24f0b8 where we tweaked the
      semantics of where hugepages are allowed.  Other bugs have been there
      essentially forever, and are due to the undefined behaviour of '<<'
      with shift counts greater than the type width (LOW_ESID_MASK could
      return non-zero for high ranges with the right congruences).
      
      The good news is that I now have a testsuite which should pick up
      things like this if they creep in again.
      
      [0] "powerpc-fix-for-hugepage-areas-straddling-4gb-boundary"
      Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      9a94c579
  10. 24 11月, 2005 2 次提交
  11. 07 11月, 2005 2 次提交
    • D
      [PATCH] ppc64: Fix bug in SLB miss handler for hugepages · 7d24f0b8
      David Gibson 提交于
      This patch, however, should be applied on top of the 64k-page-size patch to
      fix some problems with hugepage (some pre-existing, another introduced by
      this patch).
      
      The patch fixes a bug in the SLB miss handler for hugepages on ppc64
      introduced by the dynamic hugepage patch (commit id
      c594adad) due to a misunderstanding of the
      srd instruction's behaviour (mea culpa).  The problem arises when a 64-bit
      process maps some hugepages in the low 4GB of the address space (unusual).
      In this case, as well as the 256M segment in question being marked for
      hugepages, other segments at 32G intervals will be incorrectly marked for
      hugepages.
      
      In the process, this patch tweaks the semantics of the hugepage bitmaps to
      be more sensible.  Previously, an address below 4G was marked for hugepages
      if the appropriate segment bit in the "low areas" bitmask was set *or* if
      the low bit in the "high areas" bitmap was set (which would mark all
      addresses below 1TB for hugepage).  With this patch, any given address is
      governed by a single bitmap.  Addresses below 4GB are marked for hugepage
      if and only if their bit is set in the "low areas" bitmap (256M
      granularity).  Addresses between 4GB and 1TB are marked for hugepage iff
      the low bit in the "high areas" bitmap is set.  Higher addresses are marked
      for hugepage iff their bit in the "high areas" bitmap is set (1TB
      granularity).
      
      To avoid conflicts, this patch must be applied on top of BenH's pending
      patch for 64k base page size [0].  As such, this patch also addresses a
      hugepage problem introduced by that patch.  That patch allows hugepages of
      1MB in size on hardware which supports it, however, that won't work when
      using 4k pages (4 level pagetable), because in that case hugepage PTEs are
      stored at the PMD level, and each PMD entry maps 2MB.  This patch simply
      disallows hugepages in that case (we can do something cleverer to re-enable
      them some other day).
      
      Built, booted, and a handful of hugepage related tests passed on POWER5
      LPAR (both ARCH=powerpc and ARCH=ppc64).
      
      [0] http://gate.crashing.org/~benh/ppc64-64k-pages.diffSigned-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Paul Mackerras <paulus@samba.org>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      7d24f0b8
    • B
      [PATCH] ppc64: support 64k pages · 3c726f8d
      Benjamin Herrenschmidt 提交于
      Adds a new CONFIG_PPC_64K_PAGES which, when enabled, changes the kernel
      base page size to 64K.  The resulting kernel still boots on any
      hardware.  On current machines with 4K pages support only, the kernel
      will maintain 16 "subpages" for each 64K page transparently.
      
      Note that while real 64K capable HW has been tested, the current patch
      will not enable it yet as such hardware is not released yet, and I'm
      still verifying with the firmware architects the proper to get the
      information from the newer hypervisors.
      Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      3c726f8d
  12. 10 10月, 2005 1 次提交
  13. 24 9月, 2005 1 次提交
    • B
      [PATCH] ppc64: Fix huge pages MMU mapping bug · 67b10813
      Benjamin Herrenschmidt 提交于
      Current kernel has a couple of sneaky bugs in the ppc64 hugetlb code that
      cause huge pages to be potentially left stale in the hash table and TLBs
      (improperly invalidated), with all the nasty consequences that can have.
      
      One is that we forgot to set the "secondary" bit in the hash PTEs when
      hashing a huge page in the secondary bucket (fortunately very rare).
      
      The other one is on non-LPAR machines (like Apple G5s), flush_hash_range()
      which is used to flush a batch of PTEs simply did not work for huge pages.
      Historically, our huge page code didn't batch, but this was changed without
      fixing this routine.  This patch fixes both.
      Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      67b10813
  14. 06 9月, 2005 1 次提交
    • D
      [PATCH] Invert sense of SLB class bit · 14b34661
      David Gibson 提交于
      Currently, we set the class bit in kernel SLB entries, and clear it on
      user SLB entries.  On POWER5, ERAT entries created in real mode have
      the class bit clear.  So to avoid flushing kernel ERAT entries on each
      context switch, this patch inverts our usage of the class bit, setting
      it on user SLB entries and clearing it on kernel SLB entries.
      
      Booted on POWER5 and G5.
      Signed-off-by: NDavid Gibson <dwg@au1.ibm.com>
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      14b34661
  15. 29 8月, 2005 2 次提交
    • D
      [PATCH] Dynamic hugepage addresses for ppc64 · c594adad
      David Gibson 提交于
      Paulus, I think this is now a reasonable candidate for the post-2.6.13
      queue.
      
      Relax address restrictions for hugepages on ppc64
      
      Presently, 64-bit applications on ppc64 may only use hugepages in the
      address region from 1-1.5T.  Furthermore, if hugepages are enabled in
      the kernel config, they may only use hugepages and never normal pages
      in this area.  This patch relaxes this restriction, allowing any
      address to be used with hugepages, but with a 1TB granularity.  That
      is if you map a hugepage anywhere in the region 1TB-2TB, that entire
      area will be reserved exclusively for hugepages for the remainder of
      the process's lifetime.  This works analagously to hugepages in 32-bit
      applications, where hugepages can be mapped anywhere, but with 256MB
      (mmu segment) granularity.
      
      This patch applies on top of the four level pagetable patch
      (http://patchwork.ozlabs.org/linuxppc64/patch?id=1936).
      Signed-off-by: NDavid Gibson <dwg@au1.ibm.com>
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      c594adad
    • D
      [PATCH] Four level pagetables for ppc64 · e28f7faf
      David Gibson 提交于
      Implement 4-level pagetables for ppc64
      
      This patch implements full four-level page tables for ppc64, thereby
      extending the usable user address range to 44 bits (16T).
      
      The patch uses a full page for the tables at the bottom and top level,
      and a quarter page for the intermediate levels.  It uses full 64-bit
      pointers at every level, thus also increasing the addressable range of
      physical memory.  This patch also tweaks the VSID allocation to allow
      matching range for user addresses (this halves the number of available
      contexts) and adds some #if and BUILD_BUG sanity checks.
      Signed-off-by: NDavid Gibson <dwg@au1.ibm.com>
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      e28f7faf
  16. 14 7月, 2005 1 次提交
  17. 22 6月, 2005 2 次提交
    • W
      [PATCH] Avoiding mmap fragmentation · 1363c3cd
      Wolfgang Wander 提交于
      Ingo recently introduced a great speedup for allocating new mmaps using the
      free_area_cache pointer which boosts the specweb SSL benchmark by 4-5% and
      causes huge performance increases in thread creation.
      
      The downside of this patch is that it does lead to fragmentation in the
      mmap-ed areas (visible via /proc/self/maps), such that some applications
      that work fine under 2.4 kernels quickly run out of memory on any 2.6
      kernel.
      
      The problem is twofold:
      
        1) the free_area_cache is used to continue a search for memory where
           the last search ended.  Before the change new areas were always
           searched from the base address on.
      
           So now new small areas are cluttering holes of all sizes
           throughout the whole mmap-able region whereas before small holes
           tended to close holes near the base leaving holes far from the base
           large and available for larger requests.
      
        2) the free_area_cache also is set to the location of the last
           munmap-ed area so in scenarios where we allocate e.g.  five regions of
           1K each, then free regions 4 2 3 in this order the next request for 1K
           will be placed in the position of the old region 3, whereas before we
           appended it to the still active region 1, placing it at the location
           of the old region 2.  Before we had 1 free region of 2K, now we only
           get two free regions of 1K -> fragmentation.
      
      The patch addresses thes issues by introducing yet another cache descriptor
      cached_hole_size that contains the largest known hole size below the
      current free_area_cache.  If a new request comes in the size is compared
      against the cached_hole_size and if the request can be filled with a hole
      below free_area_cache the search is started from the base instead.
      
      The results look promising: Whereas 2.6.12-rc4 fragments quickly and my
      (earlier posted) leakme.c test program terminates after 50000+ iterations
      with 96 distinct and fragmented maps in /proc/self/maps it performs nicely
      (as expected) with thread creation, Ingo's test_str02 with 20000 threads
      requires 0.7s system time.
      
      Taking out Ingo's patch (un-patch available per request) by basically
      deleting all mentions of free_area_cache from the kernel and starting the
      search for new memory always at the respective bases we observe: leakme
      terminates successfully with 11 distinctive hardly fragmented areas in
      /proc/self/maps but thread creating is gringdingly slow: 30+s(!) system
      time for Ingo's test_str02 with 20000 threads.
      
      Now - drumroll ;-) the appended patch works fine with leakme: it ends with
      only 7 distinct areas in /proc/self/maps and also thread creation seems
      sufficiently fast with 0.71s for 20000 threads.
      Signed-off-by: NWolfgang Wander <wwc@rentec.com>
      Credit-to: "Richard Purdie" <rpurdie@rpsys.net>
      Signed-off-by: NKen Chen <kenneth.w.chen@intel.com>
      Acked-by: Ingo Molnar <mingo@elte.hu> (partly)
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      1363c3cd
    • D
      [PATCH] Hugepage consolidation · 63551ae0
      David Gibson 提交于
      A lot of the code in arch/*/mm/hugetlbpage.c is quite similar.  This patch
      attempts to consolidate a lot of the code across the arch's, putting the
      combined version in mm/hugetlb.c.  There are a couple of uglyish hacks in
      order to covert all the hugepage archs, but the result is a very large
      reduction in the total amount of code.  It also means things like hugepage
      lazy allocation could be implemented in one place, instead of six.
      
      Tested, at least a little, on ppc64, i386 and x86_64.
      
      Notes:
      	- this patch changes the meaning of set_huge_pte() to be more
      	  analagous to set_pte()
      	- does SH4 need s special huge_ptep_get_and_clear()??
      Acked-by: NWilliam Lee Irwin <wli@holomorphy.com>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      63551ae0
  18. 01 5月, 2005 1 次提交
  19. 20 4月, 2005 2 次提交
    • H
      [PATCH] freepgt: hugetlb area is clean · 021740dc
      Hugh Dickins 提交于
      Once we're strict about clearing away page tables, hugetlb_prefault can assume
      there are no page tables left within its range.  Since the other arches
      continue if !pte_none here, let i386 do the same.
      Signed-off-by: NHugh Dickins <hugh@veritas.com>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      021740dc
    • H
      [PATCH] freepgt: hugetlb_free_pgd_range · 3bf5ee95
      Hugh Dickins 提交于
      ia64 and ppc64 had hugetlb_free_pgtables functions which were no longer being
      called, and it wasn't obvious what to do about them.
      
      The ppc64 case turns out to be easy: the associated tables are noted elsewhere
      and freed later, safe to either skip its hugetlb areas or go through the
      motions of freeing nothing.  Since ia64 does need a special case, restore to
      ppc64 the special case of skipping them.
      
      The ia64 hugetlb case has been broken since pgd_addr_end went in, though it
      probably appeared to work okay if you just had one such area; in fact it's
      been broken much longer if you consider a long munmap spanning from another
      region into the hugetlb region.
      
      In the ia64 hugetlb region, more virtual address bits are available than in
      the other regions, yet the page tables are structured the same way: the page
      at the bottom is larger.  Here we need to scale down each addr before passing
      it to the standard free_pgd_range.  Was about to write a hugely_scaled_down
      macro, but found htlbpage_to_page already exists for just this purpose.  Fixed
      off-by-one in ia64 is_hugepage_only_range.
      
      Uninline free_pgd_range to make it available to ia64.  Make sure the
      vma-gathering loop in free_pgtables cannot join a hugepage_only_range to any
      other (safe to join huges?  probably but don't bother).
      Signed-off-by: NHugh Dickins <hugh@veritas.com>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      3bf5ee95
  20. 17 4月, 2005 1 次提交
    • L
      Linux-2.6.12-rc2 · 1da177e4
      Linus Torvalds 提交于
      Initial git repository build. I'm not bothering with the full history,
      even though we have it. We can create a separate "historical" git
      archive of that later if we want to, and in the meantime it's about
      3.2GB when imported into git - space that would just make the early
      git days unnecessarily complicated, when we don't have a lot of good
      infrastructure for it.
      
      Let it rip!
      1da177e4