1. 30 8月, 2017 1 次提交
  2. 22 8月, 2017 1 次提交
  3. 16 8月, 2017 2 次提交
  4. 10 8月, 2017 1 次提交
  5. 01 8月, 2017 2 次提交
    • C
      f2fs: enhance on-disk inode structure scalability · 7a2af766
      Chao Yu 提交于
      This patch add new flag F2FS_EXTRA_ATTR storing in inode.i_inline
      to indicate that on-disk structure of current inode is extended.
      
      In order to extend, we changed the inode structure a bit:
      
      Original one:
      
      struct f2fs_inode {
      	...
      	struct f2fs_extent i_ext;
      	__le32 i_addr[DEF_ADDRS_PER_INODE];
      	__le32 i_nid[DEF_NIDS_PER_INODE];
      }
      
      Extended one:
      
      struct f2fs_inode {
              ...
              struct f2fs_extent i_ext;
      	union {
      		struct {
      			__le16 i_extra_isize;
      			__le16 i_padding;
      			__le32 i_extra_end[0];
      		};
      		__le32 i_addr[DEF_ADDRS_PER_INODE];
      	};
              __le32 i_nid[DEF_NIDS_PER_INODE];
      }
      
      Once F2FS_EXTRA_ATTR is set, we will steal four bytes in the head of
      i_addr field for storing i_extra_isize and i_padding. with i_extra_isize,
      we can calculate actual size of reserved space in i_addr, available
      attribute fields included in total extra attribute fields for current
      inode can be described as below:
      
        +--------------------+
        | .i_mode            |
        | ...                |
        | .i_ext             |
        +--------------------+
        | .i_extra_isize     |-----+
        | .i_padding         |     |
        | .i_prjid           |     |
        | .i_atime_extra     |     |
        | .i_ctime_extra     |     |
        | .i_mtime_extra     |<----+
        | .i_inode_cs        |<----- store blkaddr/inline from here
        | .i_xattr_cs        |
        | ...                |
        +--------------------+
        |                    |
        |    block address   |
        |                    |
        +--------------------+
        | .i_nid             |
        +--------------------+
        |   node_footer      |
        | (nid, ino, offset) |
        +--------------------+
      
      Hence, with this patch, we would enhance scalability of f2fs inode for
      storing more newly added attribute.
      Signed-off-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      7a2af766
    • C
      f2fs: make background threads of f2fs being aware of freezing · dc6febb6
      Chao Yu 提交于
      When ->freeze_fs is called from lvm for doing snapshot, it needs to
      make sure there will be no more changes in filesystem's data, however,
      previously, background threads like GC thread wasn't aware of freezing,
      so in environment with active background threads, data of snapshot
      becomes unstable.
      
      This patch fixes this issue by adding sb_{start,end}_intwrite in
      below background threads:
      - GC thread
      - flush thread
      - discard thread
      
      Note that, don't use sb_start_intwrite() in gc_thread_func() due to:
      
      generic/241 reports below bug:
      
       ======================================================
       WARNING: possible circular locking dependency detected
       4.13.0-rc1+ #32 Tainted: G           O
       ------------------------------------------------------
       f2fs_gc-250:0/22186 is trying to acquire lock:
        (&sbi->gc_mutex){+.+...}, at: [<f8fa7f0b>] f2fs_sync_fs+0x7b/0x1b0 [f2fs]
      
       but task is already holding lock:
        (sb_internal#2){++++.-}, at: [<f8fb5609>] gc_thread_func+0x159/0x4a0 [f2fs]
      
       which lock already depends on the new lock.
      
       the existing dependency chain (in reverse order) is:
      
       -> #2 (sb_internal#2){++++.-}:
              __lock_acquire+0x405/0x7b0
              lock_acquire+0xae/0x220
              __sb_start_write+0x11d/0x1f0
              f2fs_evict_inode+0x2d6/0x4e0 [f2fs]
              evict+0xa8/0x170
              iput+0x1fb/0x2c0
              f2fs_sync_inode_meta+0x3f/0xf0 [f2fs]
              write_checkpoint+0x1b1/0x750 [f2fs]
              f2fs_sync_fs+0x85/0x1b0 [f2fs]
              f2fs_do_sync_file.isra.24+0x137/0xa30 [f2fs]
              f2fs_sync_file+0x34/0x40 [f2fs]
              vfs_fsync_range+0x4a/0xa0
              do_fsync+0x3c/0x60
              SyS_fdatasync+0x15/0x20
              do_fast_syscall_32+0xa1/0x1b0
              entry_SYSENTER_32+0x4c/0x7b
      
       -> #1 (&sbi->cp_mutex){+.+...}:
              __lock_acquire+0x405/0x7b0
              lock_acquire+0xae/0x220
              __mutex_lock+0x4f/0x830
              mutex_lock_nested+0x25/0x30
              write_checkpoint+0x2f/0x750 [f2fs]
              f2fs_sync_fs+0x85/0x1b0 [f2fs]
              sync_filesystem+0x67/0x80
              generic_shutdown_super+0x27/0x100
              kill_block_super+0x22/0x50
              kill_f2fs_super+0x3a/0x40 [f2fs]
              deactivate_locked_super+0x3d/0x70
              deactivate_super+0x40/0x60
              cleanup_mnt+0x39/0x70
              __cleanup_mnt+0x10/0x20
              task_work_run+0x69/0x80
              exit_to_usermode_loop+0x57/0x92
              do_fast_syscall_32+0x18c/0x1b0
              entry_SYSENTER_32+0x4c/0x7b
      
       -> #0 (&sbi->gc_mutex){+.+...}:
              validate_chain.isra.36+0xc50/0xdb0
              __lock_acquire+0x405/0x7b0
              lock_acquire+0xae/0x220
              __mutex_lock+0x4f/0x830
              mutex_lock_nested+0x25/0x30
              f2fs_sync_fs+0x7b/0x1b0 [f2fs]
              f2fs_balance_fs_bg+0xb9/0x200 [f2fs]
              gc_thread_func+0x302/0x4a0 [f2fs]
              kthread+0xe9/0x120
              ret_from_fork+0x19/0x24
      
       other info that might help us debug this:
      
       Chain exists of:
         &sbi->gc_mutex --> &sbi->cp_mutex --> sb_internal#2
      
        Possible unsafe locking scenario:
      
              CPU0                    CPU1
              ----                    ----
         lock(sb_internal#2);
                                      lock(&sbi->cp_mutex);
                                      lock(sb_internal#2);
         lock(&sbi->gc_mutex);
      
        *** DEADLOCK ***
      
       1 lock held by f2fs_gc-250:0/22186:
        #0:  (sb_internal#2){++++.-}, at: [<f8fb5609>] gc_thread_func+0x159/0x4a0 [f2fs]
      
       stack backtrace:
       CPU: 2 PID: 22186 Comm: f2fs_gc-250:0 Tainted: G           O    4.13.0-rc1+ #32
       Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
       Call Trace:
        dump_stack+0x5f/0x92
        print_circular_bug+0x1b3/0x1bd
        validate_chain.isra.36+0xc50/0xdb0
        ? __this_cpu_preempt_check+0xf/0x20
        __lock_acquire+0x405/0x7b0
        lock_acquire+0xae/0x220
        ? f2fs_sync_fs+0x7b/0x1b0 [f2fs]
        __mutex_lock+0x4f/0x830
        ? f2fs_sync_fs+0x7b/0x1b0 [f2fs]
        mutex_lock_nested+0x25/0x30
        ? f2fs_sync_fs+0x7b/0x1b0 [f2fs]
        f2fs_sync_fs+0x7b/0x1b0 [f2fs]
        f2fs_balance_fs_bg+0xb9/0x200 [f2fs]
        gc_thread_func+0x302/0x4a0 [f2fs]
        ? preempt_schedule_common+0x2f/0x4d
        ? f2fs_gc+0x540/0x540 [f2fs]
        kthread+0xe9/0x120
        ? f2fs_gc+0x540/0x540 [f2fs]
        ? kthread_create_on_node+0x30/0x30
        ret_from_fork+0x19/0x24
      
      The deadlock occurs in below condition:
      GC Thread			Thread B
      - sb_start_intwrite
      				- f2fs_sync_file
      				 - f2fs_sync_fs
      				  - mutex_lock(&sbi->gc_mutex)
      				   - write_checkpoint
      				    - block_operations
      				     - f2fs_sync_inode_meta
      				      - iput
      				       - sb_start_intwrite
       - mutex_lock(&sbi->gc_mutex)
      
      Fix this by altering sb_start_intwrite to sb_start_write_trylock.
      Signed-off-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      dc6febb6
  6. 24 5月, 2017 7 次提交
  7. 04 5月, 2017 1 次提交
  8. 03 5月, 2017 1 次提交
  9. 25 4月, 2017 2 次提交
  10. 12 4月, 2017 1 次提交
  11. 11 4月, 2017 2 次提交
  12. 30 3月, 2017 2 次提交
    • J
      f2fs: relax node version check for victim data in gc · c13ff37e
      Jaegeuk Kim 提交于
      - has_not_enough_free_secs
      node_secs: 0  dent_secs: 0  freed:0  free_segments:103  reserved:104
      
                - f2fs_gc
                   - get_victim_by_default
      alloc_mode 0, gc_mode 1, max_search 2672, offset 4654, ofs_unit 1
      
                      - do_garbage_collect
      start_segno 3976, end_segno 3977   type 0
      
                        - is_alive
      nid 22797, blkaddr 2131882, ofs_in_node 0, version 0x8/0x0
      
                         - gc_data_segment 766, segno 3976, block 512/426 not alive
      
      So, this patch fixes subtle corrupted case where node version does not match
      to summary version which results in infinite loop by gc.
      Reported-by: NYunlei He <heyunlei@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      c13ff37e
    • J
      f2fs: allocate node and hot data in the beginning of partition · 7a20b8a6
      Jaegeuk Kim 提交于
      In order to give more spatial locality, this patch changes the block allocation
      policy which assigns beginning of partition for small and hot data/node blocks.
      In order to do this, we set noheap allocation by default and introduce another
      mount option, heap, to reset it back.
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      7a20b8a6
  13. 29 3月, 2017 1 次提交
  14. 25 3月, 2017 1 次提交
  15. 28 2月, 2017 4 次提交
  16. 25 2月, 2017 1 次提交
  17. 24 2月, 2017 2 次提交
    • H
      f2fs: add ovp valid_blocks check for bg gc victim to fg_gc · e93b9865
      Hou Pengyang 提交于
      For foreground gc, greedy algorithm should be adapted, which makes
      this formula work well:
      
      	(2 * (100 / config.overprovision + 1) + 6)
      
      But currently, we fg_gc have a prior to select bg_gc victim segments to gc
      first, these victims are selected by cost-benefit algorithm, we can't guarantee
      such segments have the small valid blocks, which may destroy the f2fs rule, on
      the worstest case, would consume all the free segments.
      
      This patch fix this by add a filter in check_bg_victims, if segment's has # of
      valid blocks over overprovision ratio, skip such segments.
      
      Cc: <stable@vger.kernel.org>
      Signed-off-by: NHou Pengyang <houpengyang@huawei.com>
      Reviewed-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      e93b9865
    • Y
      f2fs: replace __get_victim by dirty_segments in FG_GC · 05eeb118
      Yunlei He 提交于
      In FG_GC process, it will search victim section twice. This will
      cause some dirty section with less valid blocks skip garbage
      collection.
      
      section # 26425 : valid blocks # 3
      142.037567: get_victim_by_default: victim 26425 : valid blocks # 3
      142.037585: f2fs_get_victim: dev = (259,30), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 26425 ofs_unit = 1, pre_victim_secno = 26425, prefree = 0, free = 244
      142.039494: f2fs_get_victim: dev = (259,30), type = Hot DATA, policy = (Background GC, SSR-mode, Greedy), victim = 19022 ofs_unit = 1, pre_victim_secno = 26425, prefree = 0, free = 24
      142.070247: new_curseg: Debug: alloc new segment 26746
      142.244341: f2fs_get_victim: dev = (259,30), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 26054 ofs_unit = 1, pre_victim_secno = 26054, prefree = 0, free = 243
      142.254475: do_garbage_collect: Debug: FG_GC, seg_freed = 1
      142.293131: f2fs_get_victim: dev = (259,30), type = Warm DATA, policy = (Background GC, SSR-mode, Greedy), victim = 23466 ofs_unit = 1, pre_victim_secno = -1, prefree = 0, free = 244
      142.319001: f2fs_get_victim: dev = (259,30), type = Warm DATA, policy = (Background GC, SSR-mode, Greedy), victim = 23467 ofs_unit = 1, pre_victim_secno = -1, prefree = 0, free = 244
      142.368879: get_victim_by_default: victim 26425 : valid blocks # 3
      142.368894: f2fs_get_victim: dev = (259,30), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 26425 ofs_unit = 1, pre_victim_secno = 26425, prefree = 0, free = 244
      142.378127: f2fs_get_victim: dev = (259,30), type = Hot DATA, policy = (Background GC, SSR-mode, Greedy), victim = 19612 ofs_unit = 1, pre_victim_secno = 26425, prefree = 0, free = 24
      142.416917: new_curseg: Debug: alloc new segment 26054
      142.656794: f2fs_get_victim: dev = (259,30), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 25404 ofs_unit = 1, pre_victim_secno = 25404, prefree = 0, free = 243
      142.662139: do_garbage_collect: Debug: FG_GC, seg_freed = 1
      142.684159: new_curseg: Debug: alloc new segment 25197
      142.685059: get_victim_by_default: victim 26425 : valid blocks # 3
      142.685079: f2fs_get_victim: dev = (259,30), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 26425 ofs_unit = 1, pre_victim_secno = 26425, prefree = 0, free = 243
      142.701427: f2fs_get_victim: dev = (259,30), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 26238 ofs_unit = 1, pre_victim_secno = 26238, prefree = 0, free = 243
      142.707105: do_garbage_collect: Debug: FG_GC, seg_freed = 1
      142.802444: f2fs_get_victim: dev = (259,30), type = Warm DATA, policy = (Background GC, SSR-mode, Greedy), victim = 23473 ofs_unit = 1, pre_victim_secno = -1, prefree = 0, free = 244
      142.804422: get_victim_by_default: victim 26425 : valid blocks # 3
      142.804443: f2fs_get_victim: dev = (259,30), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 26425 ofs_unit = 1, pre_victim_secno = 26425, prefree = 0, free = 244
      142.851567: f2fs_get_victim: dev = (259,30), type = Hot DATA, policy = (Background GC, SSR-mode, Greedy), victim = 19092 ofs_unit = 1, pre_victim_secno = 26425, prefree = 0, free = 24
      142.865014: new_curseg: Debug: alloc new segment 26238
      143.082245: f2fs_get_victim: dev = (259,30), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 26307 ofs_unit = 1, pre_victim_secno = 26307, prefree = 0, free = 244
      143.088252: do_garbage_collect: Debug: FG_GC, seg_freed = 1
      143.128307: new_curseg: Debug: alloc new segment 25404
      143.181846: get_victim_by_default: victim 26425 : valid blocks # 3
      143.181872: f2fs_get_victim: dev = (259,30), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 26425 ofs_unit = 1, pre_victim_secno = 26425, prefree = 0, free = 244
      Signed-off-by: NYunlei He <heyunlei@huawei.com>
      Reviewed-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      05eeb118
  18. 23 2月, 2017 1 次提交
    • C
      f2fs: introduce FI_ATOMIC_COMMIT · 5fe45743
      Chao Yu 提交于
      This patch introduces a new flag to indicate inode status of doing atomic
      write committing, so that, we can keep atomic write status for inode
      during atomic committing, then we can skip GCing pages of atomic write inode,
      that avoids random GCed datas being mixed with current transaction, so
      isolation of transaction can be kept.
      Signed-off-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      5fe45743
  19. 26 11月, 2016 2 次提交
    • C
      f2fs: don't wait writeback for datas during checkpoint · 36951b38
      Chao Yu 提交于
      Normally, while committing checkpoint, we will wait on all pages to be
      writebacked no matter the page is data or metadata, so in scenario where
      there are lots of data IO being submitted with metadata, we may suffer
      long latency for waiting writeback during checkpoint.
      
      Indeed, we only care about persistence for pages with metadata, but not
      pages with data, as file system consistent are only related to metadate,
      so in order to avoid encountering long latency in above scenario, let's
      recognize and reference metadata in submitted IOs, wait writeback only
      for metadatas.
      Signed-off-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      36951b38
    • J
      f2fs: avoid BG_GC in f2fs_balance_fs · 7702bdbe
      Jaegeuk Kim 提交于
      If many threads hit has_not_enough_free_secs() in f2fs_balance_fs() at the same
      time, all the threads would do FG_GC or BG_GC.
      In this critical path, we totally don't need to do BG_GC at all.
      Let's avoid that.
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      7702bdbe
  20. 24 11月, 2016 2 次提交
  21. 01 11月, 2016 1 次提交
  22. 13 10月, 2016 1 次提交
  23. 01 10月, 2016 1 次提交