Skip to content

cloud-kernel
cloud-kernel

openanolis / cloud-kernel
1 年多 前同步成功

通知 160
Star 36
Fork 7
cloud-kernel
cloud-kernel
切换分支/标签
  1. 08 1月, 2011 28 次提交
  3. 07 1月, 2011 12 次提交
    • N
      fs: scale mntget/mntput · b3e19d92
      由 Nick Piggin 提交于 
      The problem that this patch aims to fix is vfsmount refcounting scalability.
We need to take a reference on the vfsmount for every successful path lookup,
which often go to the same mount point.

The fundamental difficulty is that a "simple" reference count can never be made
scalable, because any time a reference is dropped, we must check whether that
was the last reference. To do that requires communication with all other CPUs
that may have taken a reference count.

We can make refcounts more scalable in a couple of ways, involving keeping
distributed counters, and checking for the global-zero condition less
frequently.

- check the global sum once every interval (this will delay zero detection
  for some interval, so it's probably a showstopper for vfsmounts).

- keep a local count and only taking the global sum when local reaches 0 (this
  is difficult for vfsmounts, because we can't hold preempt off for the life of
  a reference, so a counter would need to be per-thread or tied strongly to a
  particular CPU which requires more locking).

- keep a local difference of increments and decrements, which allows us to sum
  the total difference and hence find the refcount when summing all CPUs. Then,
  keep a single integer "long" refcount for slow and long lasting references,
  and only take the global sum of local counters when the long refcount is 0.

This last scheme is what I implemented here. Attached mounts and process root
and working directory references are "long" references, and everything else is
a short reference.

This allows scalable vfsmount references during path walking over mounted
subtrees and unattached (lazy umounted) mounts with processes still running
in them.

This results in one fewer atomic op in the fastpath: mntget is now just a
per-CPU inc, rather than an atomic inc; and mntput just requires a spinlock
and non-atomic decrement in the common case. However code is otherwise bigger
and heavier, so single threaded performance is basically a wash.
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      b3e19d92
    • N
      fs: rename vfsmount counter helpers · c6653a83
      由 Nick Piggin 提交于 
      Suggested by Andreas, mnt_ prefix is clearer namespace, follows kernel
conventions better, and is easier for tab complete. I introduced these
names so I'll admit they were not good choices.
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      c6653a83
    • N
      fs: implement faster dentry memcmp · 9d55c369
      由 Nick Piggin 提交于 
      The standard memcmp function on a Westmere system shows up hot in
profiles in the `git diff` workload (both parallel and single threaded),
and it is likely due to the costs associated with trapping into
microcode, and little opportunity to improve memory access (dentry
name is not likely to take up more than a cacheline).

So replace it with an open-coded byte comparison. This increases code
size by 8 bytes in the critical __d_lookup_rcu function, but the
speedup is huge, averaging 10 runs of each:

git diff st   user   sys   elapsed  CPU
before        1.15   2.57  3.82      97.1
after         1.14   2.35  3.61      96.8

git diff mt   user   sys   elapsed  CPU
before        1.27   3.85  1.46     349
after         1.26   3.54  1.43     333

Elapsed time for single threaded git diff at 95.0% confidence:
        -0.21  +/- 0.01
        -5.45% +/- 0.24%

It's -0.66% +/- 0.06% elapsed time on my Opteron, so rep cmp costs on the
fam10h seem to be relatively smaller, but there is still a win.
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      9d55c369
    • N
      fs: prefetch inode data in dcache lookup · e1bb5782
      由 Nick Piggin 提交于 
      This makes single threaded git diff -1.25% +/- 0.05% elapsed time on my
2s12c24t Westmere system, and -0.86% +/- 0.05% on my 2s8c Barcelona, by
prefetching the important first cacheline of the inode in while we do the
actual name compare and other operations on the dentry.

There was no measurable slowdown in the single file stat case, or the creat
case (where negative dentries would be common).
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      e1bb5782
    • N
      fs: improve scalability of pseudo filesystems · 4b936885
      由 Nick Piggin 提交于 
      Regardless of how much we possibly try to scale dcache, there is likely
always going to be some fundamental contention when adding or removing children
under the same parent. Pseudo filesystems do not seem need to have connected
dentries because by definition they are disconnected.
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      4b936885
    • N
      fs: dcache per-inode inode alias locking · 873feea0
      由 Nick Piggin 提交于 
      dcache_inode_lock can be replaced with per-inode locking. Use existing
inode->i_lock for this. This is slightly non-trivial because we sometimes
need to find the inode from the dentry, which requires d_inode to be
stabilised (either with refcount or d_lock).
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      873feea0
    • N
      fs: dcache per-bucket dcache hash locking · ceb5bdc2
      由 Nick Piggin 提交于 
      We can turn the dcache hash locking from a global dcache_hash_lock into
per-bucket locking.
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      ceb5bdc2
    • N
      bit_spinlock: add required includes · 626d6074
      由 Nick Piggin 提交于 
      Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      626d6074
    • N
      kernel: add bl_list · 4e35e607
      由 Nick Piggin 提交于 
      Introduce a type of hlist that can support the use of the lowest bit in the
hlist_head. This will be subsequently used to implement per-bucket bit spinlock
for inode and dentry hashes, and may be useful in other cases such as network
hashes.
Reviewed-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      4e35e607
    • N
      xfs: provide simple rcu-walk ACL implementation · 880566e1
      由 Nick Piggin 提交于 
      This simple implementation just checks for no ACLs on the inode, and
if so, then the rcu-walk may proceed, otherwise fail it.
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      880566e1
    • N
      btrfs: provide simple rcu-walk ACL implementation · 258a5aa8
      由 Nick Piggin 提交于 
      This simple implementation just checks for no ACLs on the inode, and
if so, then the rcu-walk may proceed, otherwise fail it.
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      258a5aa8
    • N
      ext2,3,4: provide simple rcu-walk ACL implementation · 73598611
      由 Nick Piggin 提交于 
      This simple implementation just checks for no ACLs on the inode, and
if so, then the rcu-walk may proceed, otherwise fail it.
Signed-off-by: NNick Piggin <npiggin@kernel.dk>
      73598611