• M
    mm: non-atomically mark page accessed during page cache allocation where possible · 2457aec6
    Mel Gorman 提交于
    aops->write_begin may allocate a new page and make it visible only to have
    mark_page_accessed called almost immediately after.  Once the page is
    visible the atomic operations are necessary which is noticable overhead
    when writing to an in-memory filesystem like tmpfs but should also be
    noticable with fast storage.  The objective of the patch is to initialse
    the accessed information with non-atomic operations before the page is
    visible.
    
    The bulk of filesystems directly or indirectly use
    grab_cache_page_write_begin or find_or_create_page for the initial
    allocation of a page cache page.  This patch adds an init_page_accessed()
    helper which behaves like the first call to mark_page_accessed() but may
    called before the page is visible and can be done non-atomically.
    
    The primary APIs of concern in this care are the following and are used
    by most filesystems.
    
    	find_get_page
    	find_lock_page
    	find_or_create_page
    	grab_cache_page_nowait
    	grab_cache_page_write_begin
    
    All of them are very similar in detail to the patch creates a core helper
    pagecache_get_page() which takes a flags parameter that affects its
    behavior such as whether the page should be marked accessed or not.  Then
    old API is preserved but is basically a thin wrapper around this core
    function.
    
    Each of the filesystems are then updated to avoid calling
    mark_page_accessed when it is known that the VM interfaces have already
    done the job.  There is a slight snag in that the timing of the
    mark_page_accessed() has now changed so in rare cases it's possible a page
    gets to the end of the LRU as PageReferenced where as previously it might
    have been repromoted.  This is expected to be rare but it's worth the
    filesystem people thinking about it in case they see a problem with the
    timing change.  It is also the case that some filesystems may be marking
    pages accessed that previously did not but it makes sense that filesystems
    have consistent behaviour in this regard.
    
    The test case used to evaulate this is a simple dd of a large file done
    multiple times with the file deleted on each iterations.  The size of the
    file is 1/10th physical memory to avoid dirty page balancing.  In the
    async case it will be possible that the workload completes without even
    hitting the disk and will have variable results but highlight the impact
    of mark_page_accessed for async IO.  The sync results are expected to be
    more stable.  The exception is tmpfs where the normal case is for the "IO"
    to not hit the disk.
    
    The test machine was single socket and UMA to avoid any scheduling or NUMA
    artifacts.  Throughput and wall times are presented for sync IO, only wall
    times are shown for async as the granularity reported by dd and the
    variability is unsuitable for comparison.  As async results were variable
    do to writback timings, I'm only reporting the maximum figures.  The sync
    results were stable enough to make the mean and stddev uninteresting.
    
    The performance results are reported based on a run with no profiling.
    Profile data is based on a separate run with oprofile running.
    
    async dd
                                        3.15.0-rc3            3.15.0-rc3
                                           vanilla           accessed-v2
    ext3    Max      elapsed     13.9900 (  0.00%)     11.5900 ( 17.16%)
    tmpfs	Max      elapsed      0.5100 (  0.00%)      0.4900 (  3.92%)
    btrfs   Max      elapsed     12.8100 (  0.00%)     12.7800 (  0.23%)
    ext4	Max      elapsed     18.6000 (  0.00%)     13.3400 ( 28.28%)
    xfs	Max      elapsed     12.5600 (  0.00%)      2.0900 ( 83.36%)
    
    The XFS figure is a bit strange as it managed to avoid a worst case by
    sheer luck but the average figures looked reasonable.
    
            samples percentage
    ext3       86107    0.9783  vmlinux-3.15.0-rc4-vanilla        mark_page_accessed
    ext3       23833    0.2710  vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed
    ext3        5036    0.0573  vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed
    ext4       64566    0.8961  vmlinux-3.15.0-rc4-vanilla        mark_page_accessed
    ext4        5322    0.0713  vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed
    ext4        2869    0.0384  vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed
    xfs        62126    1.7675  vmlinux-3.15.0-rc4-vanilla        mark_page_accessed
    xfs         1904    0.0554  vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed
    xfs          103    0.0030  vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed
    btrfs      10655    0.1338  vmlinux-3.15.0-rc4-vanilla        mark_page_accessed
    btrfs       2020    0.0273  vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed
    btrfs        587    0.0079  vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed
    tmpfs      59562    3.2628  vmlinux-3.15.0-rc4-vanilla        mark_page_accessed
    tmpfs       1210    0.0696  vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed
    tmpfs         94    0.0054  vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed
    
    [akpm@linux-foundation.org: don't run init_page_accessed() against an uninitialised pointer]
    Signed-off-by: NMel Gorman <mgorman@suse.de>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Jan Kara <jack@suse.cz>
    Cc: Michal Hocko <mhocko@suse.cz>
    Cc: Hugh Dickins <hughd@google.com>
    Cc: Dave Hansen <dave.hansen@intel.com>
    Cc: Theodore Ts'o <tytso@mit.edu>
    Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
    Cc: Oleg Nesterov <oleg@redhat.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Tested-by: NPrabhakar Lad <prabhakar.csengg@gmail.com>
    Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
    2457aec6
swap.h 18.0 KB