1. 24 6月, 2009 10 次提交
    • T
      s390: switch to dynamic percpu allocator · 9a0ef292
      Tejun Heo 提交于
      64bit s390 shares the same problem with alpha regarding percpu symbol
      addressing from modules.  It needs assembly magic to force GOTENT
      reference when building module as the percpu address will be outside
      the usual 4G range from the module text.  This can be solved by using
      weak percpu variable definitions.
      
      This patch makes s390 use weak definitions and switch to dynamic
      percpu allocator.  Please note that weak attribute is not added if
      !SMP as percpu variables behave exactly the same as normal variables
      on UP.
      
      Compile tested.  Generation of GOTENT reference verified.
      
      This patch is based on Ivan Kokshaysky's alpha percpu patch.
      
      [ Impact: use dynamic percpu allocator ]
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
      Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
      9a0ef292
    • T
      alpha: switch to dynamic percpu allocator · 9b7dbc7d
      Tejun Heo 提交于
      Alpha implements custom SHIFT_PERCPU_PTR for modules because percpu
      area can be located far away from the 4G area where the module text is
      located.  The custom SHIFT_PERCPU_PTR forces GOT usage using ldq
      instruction with literal relocation; however, the relocation can't be
      used with dynamically allocated percpu variables.  Fortunately,
      similar result can be achieved by using weak percpu variable
      definitions.
      
      This patch makes alpha use weak definitions and switch to dynamic
      percpu allocator.
      
      asm/tlbflush.h was getting linux/sched.h via asm/percpu.h which no
      longer needs it.  Include linux/sched.h directly in asm/tlbflush.h.
      
      Compile tested.  Generation of litereal relocation verified.
      
      This patch is based on Ivan Kokshaysky's alpha percpu patch.
      
      [ Impact: use dynamic percpu allocator ]
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Acked-by: NIvan Kokshaysky <ink@jurassic.park.msu.ru>
      Cc: Richard Henderson <rth@twiddle.net>
      9b7dbc7d
    • T
      alpha: kill unnecessary __used attribute in PER_CPU_ATTRIBUTES · 6088464c
      Tejun Heo 提交于
      With the previous percpu variable definition change, all percpu
      variables are global and there's no need to specify __used, which only
      triggers on recent compilers anyway.  Kill it.
      
      [ Impact: remove unnecessary percpu attribute ]
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
      Cc: Richard Henderson <rth@twiddle.net>
      6088464c
    • T
      percpu: implement optional weak percpu definitions · 7c756e6e
      Tejun Heo 提交于
      Some archs (alpha and s390) need to use weak definitions for percpu
      variables in modules so that the compiler generates external
      references for them.
      
      This patch implements weak percpu definitions which arch can enable by
      defining ARCH_NEEDS_WEAK_PER_CPU in arch percpu header file.  This
      weak definition adds the following two restrictions on percpu variable
      definitions.
      
        1. percpu symbols must be unique whether static or not
        2. percpu variables can't be defined inside a function
      
      To ensure that these restrictions are observed in generic code, config
      option DEBUG_FORCE_WEAK_PER_CPU enables weak percpu definitions for
      all cases.
      
      This patch is inspired by Ivan Kokshaysky's alpha percpu patch.
      
      [ Impact: stricter rules for percpu variables, one more debug config option ]
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
      7c756e6e
    • T
      percpu: clean up percpu variable definitions · 245b2e70
      Tejun Heo 提交于
      Percpu variable definition is about to be updated such that all percpu
      symbols including the static ones must be unique.  Update percpu
      variable definitions accordingly.
      
      * as,cfq: rename ioc_count uniquely
      
      * cpufreq: rename cpu_dbs_info uniquely
      
      * xen: move nesting_count out of xen_evtchn_do_upcall() and rename it
      
      * mm: move ratelimits out of balance_dirty_pages_ratelimited_nr() and
        rename it
      
      * ipv4,6: rename cookie_scratch uniquely
      
      * x86 perf_counter: rename prev_left to pmc_prev_left, irq_entry to
        pmc_irq_entry and nmi_entry to pmc_nmi_entry
      
      * perf_counter: rename disable_count to perf_disable_count
      
      * ftrace: rename test_event_disable to ftrace_test_event_disable
      
      * kmemleak: rename test_pointer to kmemleak_test_pointer
      
      * mce: rename next_interval to mce_next_interval
      
      [ Impact: percpu usage cleanups, no duplicate static percpu var names ]
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Reviewed-by: NChristoph Lameter <cl@linux-foundation.org>
      Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
      Cc: Jens Axboe <jens.axboe@oracle.com>
      Cc: Dave Jones <davej@redhat.com>
      Cc: Jeremy Fitzhardinge <jeremy@xensource.com>
      Cc: linux-mm <linux-mm@kvack.org>
      Cc: David S. Miller <davem@davemloft.net>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Steven Rostedt <srostedt@redhat.com>
      Cc: Li Zefan <lizf@cn.fujitsu.com>
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Andi Kleen <andi@firstfloor.org>
      245b2e70
    • T
      percpu: use DEFINE_PER_CPU_SHARED_ALIGNED() · b9bf3121
      Tejun Heo 提交于
      There are a few places where ___cacheline_aligned* is used with
      DEFINE_PER_CPU().  Use DEFINE_PER_CPU_SHARED_ALIGNED() instead.
      
      DEFINE_PER_CPU_SHARED_ALIGNED() applies alignment only on SMPs.  While
      all other converted places used _in_smp variant or only get compiled
      for SMP, net/rds used unconditional ____cacheline_aligned.  I don't
      see any reason these data structures should be aligned on UP and thus
      converted together.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Cc: Mike Frysinger <vapier@gentoo.org>
      Cc: Tony Luck <tony.luck@intel.com>
      Cc: Andy Grover <andy.grover@oracle.com>
      b9bf3121
    • T
      percpu: cleanup percpu array definitions · 204fba4a
      Tejun Heo 提交于
      Currently, the following three different ways to define percpu arrays
      are in use.
      
      1. DEFINE_PER_CPU(elem_type[array_len], array_name);
      2. DEFINE_PER_CPU(elem_type, array_name[array_len]);
      3. DEFINE_PER_CPU(elem_type, array_name)[array_len];
      
      Unify to #1 which correctly separates the roles of the two parameters
      and thus allows more flexibility in the way percpu variables are
      defined.
      
      [ Impact: cleanup ]
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Reviewed-by: NChristoph Lameter <cl@linux-foundation.org>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: Tony Luck <tony.luck@intel.com>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Jeremy Fitzhardinge <jeremy@xensource.com>
      Cc: linux-mm@kvack.org
      Cc: Christoph Lameter <cl@linux-foundation.org>
      Cc: David S. Miller <davem@davemloft.net>
      204fba4a
    • J
      CRIS: Change DEFINE_PER_CPU of current_pgd to be non volatile. · fe87f94f
      Jesper Nilsson 提交于
      The DEFINE_PER_CPU of current_pgd was on CRIS defined using volatile,
      which is not needed. Remove volatile.
      
      Tested on an ARTPEC-3 (CRISv32) board.
      
      tj: extern DEFINE_PER_CPU() replaced with DECLARE_PER_CPU()
      
      [ Impact: code cleanup ]
      Signed-off-by: NJesper Nilsson <jesper.nilsson@axis.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      fe87f94f
    • T
      linker script: throw away .discard section · 405d967d
      Tejun Heo 提交于
      x86 throws away .discard section but no other archs do.  Also,
      .discard is not thrown away while linking modules.  Make every arch
      and module linking throw it away.  This will be used to define dummy
      variables for percpu declarations and definitions.
      
      This patch is based on Ivan Kokshaysky's alpha percpu patch.
      
      [ Impact: always throw away everything in .discard ]
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
      Cc: Richard Henderson <rth@twiddle.net>
      Cc: Russell King <linux@arm.linux.org.uk>
      Cc: Haavard Skinnemoen <hskinnemoen@atmel.com>
      Cc: Bryan Wu <cooloney@kernel.org>
      Cc: Mikael Starvik <starvik@axis.com>
      Cc: Jesper Nilsson <jesper.nilsson@axis.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
      Cc: Tony Luck <tony.luck@intel.com>
      Cc: Hirokazu Takata <takata@linux-m32r.org>
      Cc: Geert Uytterhoeven <geert@linux-m68k.org>
      Cc: Michal Simek <monstr@monstr.eu>
      Cc: Ralf Baechle <ralf@linux-mips.org>
      Cc: Kyle McMartin <kyle@mcmartin.ca>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
      Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
      Cc: Paul Mundt <lethal@linux-sh.org>
      Cc: David S. Miller <davem@davemloft.net>
      Cc: Jeff Dike <jdike@addtoit.com>
      Cc: Chris Zankel <chris@zankel.net>
      Cc: Rusty Russell <rusty@rustcorp.com.au>
      Cc: Ingo Molnar <mingo@elte.hu>
      405d967d
    • T
      percpu: use dynamic percpu allocator as the default percpu allocator · e74e3962
      Tejun Heo 提交于
      This patch makes most !CONFIG_HAVE_SETUP_PER_CPU_AREA archs use
      dynamic percpu allocator.  The first chunk is allocated using
      embedding helper and 8k is reserved for modules.  This ensures that
      the new allocator behaves almost identically to the original allocator
      as long as static percpu variables are concerned, so it shouldn't
      introduce much breakage.
      
      s390 and alpha use custom SHIFT_PERCPU_PTR() to work around addressing
      range limit the addressing model imposes.  Unfortunately, this breaks
      if the address is specified using a variable, so for now, the two
      archs aren't converted.
      
      The following architectures are affected by this change.
      
      * sh
      * arm
      * cris
      * mips
      * sparc(32)
      * blackfin
      * avr32
      * parisc (broken, under investigation)
      * m32r
      * powerpc(32)
      
      As this change makes the dynamic allocator the default one,
      CONFIG_HAVE_DYNAMIC_PER_CPU_AREA is replaced with its invert -
      CONFIG_HAVE_LEGACY_PER_CPU_AREA, which is added to yet-to-be converted
      archs.  These archs implement their own setup_per_cpu_areas() and the
      conversion is not trivial.
      
      * powerpc(64)
      * sparc(64)
      * ia64
      * alpha
      * s390
      
      Boot and batch alloc/free tests on x86_32 with debug code (x86_32
      doesn't use default first chunk initialization).  Compile tested on
      sparc(32), powerpc(32), arm and alpha.
      
      Kyle McMartin reported that this change breaks parisc.  The problem is
      still under investigation and he is okay with pushing this patch
      forward and fixing parisc later.
      
      [ Impact: use dynamic allocator for most archs w/o custom percpu setup ]
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Acked-by: NRusty Russell <rusty@rustcorp.com.au>
      Acked-by: NDavid S. Miller <davem@davemloft.net>
      Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
      Acked-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>
      Reviewed-by: NChristoph Lameter <cl@linux.com>
      Cc: Paul Mundt <lethal@linux-sh.org>
      Cc: Russell King <rmk@arm.linux.org.uk>
      Cc: Mikael Starvik <starvik@axis.com>
      Cc: Ralf Baechle <ralf@linux-mips.org>
      Cc: Bryan Wu <cooloney@kernel.org>
      Cc: Kyle McMartin <kyle@mcmartin.ca>
      Cc: Matthew Wilcox <matthew@wil.cx>
      Cc: Grant Grundler <grundler@parisc-linux.org>
      Cc: Hirokazu Takata <takata@linux-m32r.org>
      Cc: Richard Henderson <rth@twiddle.net>
      Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
      Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      e74e3962
  2. 22 6月, 2009 24 次提交
  3. 21 6月, 2009 6 次提交