1. 21 1月, 2009 9 次提交
    • T
      x86: make x86_32 use tlb_64.c · 02cf94c3
      Tejun Heo 提交于
      Impact: less contention when issuing invalidate IPI, cleanup
      
      Make x86_32 use the same tlb code as 64bit.  The 64bit code uses
      multiple IPI vectors for tlb shootdown to reduce contention.  This
      patch makes x86_32 allocate the same 8 IPIs as x86_64 and share the
      code paths.
      
      Note that the usage of asmlinkage is inconsistent for x86_32 and 64
      and calls for further cleanup.  This has been noted with a FIXME
      comment in tlb_64.c.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      02cf94c3
    • T
      x86: prepare for tlb merge · 6dd01bed
      Tejun Heo 提交于
      Impact: clean up, ipi vector number reordering for x86_32
      
      Make the following changes to prepare for tlb merge.
      
      * reorder x86_32 ip vectors
      
      * adjust tlb_32.c and tlb_64.c such that their logics coincide exactly
      	- on spurious invalidate ipi, tlb_32 acks the irq
      	- tlb_64 now has proper memory barriers around clearing
                flush_cpumask (no change in generated code)
      
      * unexport flush_tlb_page from tlb_32.c, there's no user
      
      * use unsigned int for cpu id
      
      * drop unnecessary includes from tlb_64.c
      Signed-off-by: NTejun Heo <tj@kernel.org>
      6dd01bed
    • T
      x86: uv cleanup · bdbcdd48
      Tejun Heo 提交于
      Impact: cleanup
      
      Make the following uv related cleanups.
      
      * collect visible uv related definitions and interfaces into uv/uv.h
        and use it.  this cleans up the messy situation where on 64bit, uv
        is defined properly, on 32bit generic it's dummy and on the rest
        undefined.  after this clean up, uv is defined on 64 and dummy on
        32.
      
      * update uv_flush_tlb_others() such that it takes cpumask of
        to-be-flushed cpus as argument, instead of that minus self, and
        returns yet-to-be-flushed cpumask, instead of modifying the passed
        in parameter.  this interface change will ease dummy implementation
        of uv_flush_tlb_others() and makes uv tlb flush related stuff
        defined in tlb_uv proper.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      bdbcdd48
    • B
      x86: merge irq_regs.h · d650a514
      Brian Gerst 提交于
      Impact: cleanup, better irq_regs code generation for x86_64
      
      Make 64-bit use the same optimizations as 32-bit.
      Signed-off-by: NBrian Gerst <brgerst@gmail.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      d650a514
    • B
      x86: merge mmu_context.h · 6826c8ff
      Brian Gerst 提交于
      Impact: cleanup
      
      tj: * changed cpu to unsigned as was done on mmu_context_64.h as cpu
            id is officially unsigned int
          * added missing ';' to 32bit version of deactivate_mm()
      Signed-off-by: NBrian Gerst <brgerst@gmail.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      6826c8ff
    • B
      x86: set %fs to __KERNEL_PERCPU unconditionally for x86_32 · 0dd76d73
      Brian Gerst 提交于
      Impact: cleanup
      
      %fs is currently set to __KERNEL_DS at boot, and conditionally
      switched to __KERNEL_PERCPU for secondary cpus.  Instead, initialize
      GDT_ENTRY_PERCPU to the same attributes as GDT_ENTRY_KERNEL_DS and
      set %fs to __KERNEL_PERCPU unconditionally.
      Signed-off-by: NBrian Gerst <brgerst@gmail.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      0dd76d73
    • B
      x86: fix percpu_write with 64-bit constants · 299e2699
      Brian Gerst 提交于
      Impact: slightly better code generation for percpu_to_op()
      
      The processor will sign-extend 32-bit immediate values in 64-bit
      operations.  Use the 'e' constraint ("32-bit signed integer constant,
      or a symbolic reference known to fit that range") for 64-bit constants.
      Signed-off-by: NBrian Gerst <brgerst@gmail.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      299e2699
    • B
      x86: clean up gdt_page definition · 06deef89
      Brian Gerst 提交于
      Impact: cleanup && more compact percpu area layout with future changes
      
      Move 64-bit GDT to page-aligned section and clean up comment
      formatting.
      Signed-off-by: NBrian Gerst <brgerst@gmail.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      06deef89
    • T
      x86: update canary handling during switch · 67e68bde
      Tejun Heo 提交于
      Impact: cleanup
      
      In switch_to(), instead of taking offset to irq_stack_union.stack,
      make it a proper percpu access using __percpu_arg() and per_cpu_var().
      Signed-off-by: NTejun Heo <tj@kernel.org>
      67e68bde
  2. 20 1月, 2009 10 次提交
  3. 19 1月, 2009 3 次提交
  4. 18 1月, 2009 12 次提交
  5. 17 1月, 2009 2 次提交
  6. 16 1月, 2009 4 次提交
    • T
      x86_64: initialize this_cpu_off to __per_cpu_load · cd3adf52
      Tejun Heo 提交于
      On x86_64, if get_per_cpu_var() is used before per cpu area is setup
      (if lockdep is turned on, it happens), it needs this_cpu_off to point
      to __per_cpu_load.  Initialize accordingly.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      cd3adf52
    • T
      x86: fix build bug introduced during merge · a338af2c
      Tejun Heo 提交于
      EXPORT_PER_CPU_SYMBOL() got misplaced during merge leading to build
      failure.  Fix it.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      a338af2c
    • I
      percpu: add optimized generic percpu accessors · 6dbde353
      Ingo Molnar 提交于
      It is an optimization and a cleanup, and adds the following new
      generic percpu methods:
      
        percpu_read()
        percpu_write()
        percpu_add()
        percpu_sub()
        percpu_and()
        percpu_or()
        percpu_xor()
      
      and implements support for them on x86. (other architectures will fall
      back to a default implementation)
      
      The advantage is that for example to read a local percpu variable,
      instead of this sequence:
      
       return __get_cpu_var(var);
      
       ffffffff8102ca2b:	48 8b 14 fd 80 09 74 	mov    -0x7e8bf680(,%rdi,8),%rdx
       ffffffff8102ca32:	81
       ffffffff8102ca33:	48 c7 c0 d8 59 00 00 	mov    $0x59d8,%rax
       ffffffff8102ca3a:	48 8b 04 10          	mov    (%rax,%rdx,1),%rax
      
      We can get a single instruction by using the optimized variants:
      
       return percpu_read(var);
      
       ffffffff8102ca3f:	65 48 8b 05 91 8f fd 	mov    %gs:0x7efd8f91(%rip),%rax
      
      I also cleaned up the x86-specific APIs and made the x86 code use
      these new generic percpu primitives.
      
      tj: * fixed generic percpu_sub() definition as Roel Kluin pointed out
          * added percpu_and() for completeness's sake
          * made generic percpu ops atomic against preemption
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      6dbde353
    • T
      x86: misc clean up after the percpu update · 004aa322
      Tejun Heo 提交于
      Do the following cleanups:
      
      * kill x86_64_init_pda() which now is equivalent to pda_init()
      
      * use per_cpu_offset() instead of cpu_pda() when initializing
        initial_gs
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      004aa322