1. 04 8月, 2009 3 次提交
    • T
      x86, percpu: Collect hot percpu variables into one cacheline · bdf977b3
      Tejun Heo 提交于
      On x86_64, percpu variables current_task and kernel_stack are used for
      get_current() and current_thread_info() respectively and thus are
      often used close to each other.  Move definition of current_task to
      kernel/cpu/common.c right above kernel_stack definition and align it
      to cacheline so that they always fall into the same cacheline.  Two
      percpu variables defined there together - irq_stack_ptr and irq_count
      - are also pretty hot and will benefit from sharing the cacheline.
      
      For consistency, current_task definition for x86_32 is also moved to
      kernel/cpu/common.c.
      
      Putting current_task and kernel_stack into the same cacheline was
      suggested by Linus Torvalds.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Ingo Molnar <mingo@elte.hu>
      Signed-off-by: NH. Peter Anvin <hpa@zytor.com>
      bdf977b3
    • T
      x86, percpu: Fix DECLARE/DEFINE_PER_CPU_PAGE_ALIGNED() · 3e352aa8
      Tejun Heo 提交于
      DECLARE/DEFINE_PER_CPU_PAGE_ALIGNED() put percpu variables in
      .page_aligned section without adding any alignment restrictions.
      Currently, this doesn't cause any problem because all users of the
      macros have explicit page alignment and page-sized but it's much safer
      to enforce page alignment from the macros.  After all, it's what they
      claim to do.
      
      Add __aligned(PAGE_SIZE) to DECLARE/DEFINE_PER_CPU_PAGE_ALIGNED() and
      drop explicit alignment from it users.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@elte.hu>
      Signed-off-by: NH. Peter Anvin <hpa@zytor.com>
      3e352aa8
    • L
      x86, percpu: Add 'percpu_read_stable()' interface for cacheable accesses · ed8d9adf
      Linus Torvalds 提交于
      This is very useful for some common things like 'get_current()' and
      'get_thread_info()', which can be used multiple times in a function, and
      where the result is cacheable.
      
      tj: Added the magical undocumented "P" modifier to UP __percpu_arg()
          to force gcc to dereference the pointer value passed in via the
          "p" input constraint.  Without this, percpu_read_stable() returns
          the address of the percpu variable.  Also added comment explaining
          the difference between percpu_read() and percpu_read_stable().
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Signed-off-by: NH. Peter Anvin <hpa@zytor.com>
      ed8d9adf
  2. 03 8月, 2009 21 次提交
  3. 02 8月, 2009 10 次提交
  4. 01 8月, 2009 6 次提交