1. 11 1月, 2011 4 次提交
    • J
      ext4: flush the i_completed_io_list during ext4_truncate · 3889fd57
      Jiaying Zhang 提交于
      Ted first found the bug when running 2.6.36 kernel with dioread_nolock
      mount option that xfstests #13 complained about wrong file size during fsck.
      However, the bug exists in the older kernels as well although it is
      somehow harder to trigger.
      
      The problem is that ext4_end_io_work() can happen after we have truncated an
      inode to a smaller size. Then when ext4_end_io_work() calls 
      ext4_convert_unwritten_extents(), we may reallocate some blocks that have 
      been truncated, so the inode size becomes inconsistent with the allocated
      blocks. 
      
      The following patch flushes the i_completed_io_list during truncate to reduce 
      the risk that some pending end_io requests are executed later and convert 
      already truncated blocks to initialized. 
      
      Note that although the fix helps reduce the problem a lot there may still 
      be a race window between vmtruncate() and ext4_end_io_work(). The fundamental
      problem is that if vmtruncate() is called without either i_mutex or i_alloc_sem
      held, it can race with an ongoing write request so that the io_end request is
      processed later when the corresponding blocks have been truncated.
      
      Ted and I have discussed the problem offline and we saw a few ways to fix
      the race completely:
      
      a) We guarantee that i_mutex lock and i_alloc_sem write lock are both hold 
      whenever vmtruncate() is called. The i_mutex lock prevents any new write
      requests from entering writeback and the i_alloc_sem prevents the race
      from ext4_page_mkwrite(). Currently we hold both locks if vmtruncate()
      is called from do_truncate(), which is probably the most common case.
      However, there are places where we may call vmtruncate() without holding
      either i_mutex or i_alloc_sem. I would like to ask for other people's
      opinions on what locks are expected to be held before calling vmtruncate().
      There seems a disagreement among the callers of that function.
      
      b) We change the ext4 write path so that we change the extent tree to contain 
      the newly allocated blocks and update i_size both at the same time --- when 
      the write of the data blocks is completed.
      
      c) We add some additional locking to synchronize vmtruncate() and 
      ext4_end_io_work(). This approach may have performance implications so we
      need to be careful.
      
      All of the above proposals may require more substantial changes, so
      we may consider to take the following patch as a bandaid.
      Signed-off-by: NJiaying Zhang <jiayingz@google.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      3889fd57
    • T
      ext4: drop ec_type from the ext4_ext_cache structure · b05e6ae5
      Theodore Ts'o 提交于
      We can encode the ec_type information by using ee_len == 0 to denote
      EXT4_EXT_CACHE_NO, ee_start == 0 to denote EXT4_EXT_CACHE_GAP, and if
      neither is true, then the cache type must be EXT4_EXT_CACHE_EXTENT.
      This allows us to reduce the size of ext4_ext_inode by another 8
      bytes.  (ec_type is 4 bytes, plus another 4 bytes of padding)
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      b05e6ae5
    • T
      ext4: use ext4_lblk_t instead of sector_t for logical blocks · 01f49d0b
      Theodore Ts'o 提交于
      This fixes a number of places where we used sector_t instead of
      ext4_lblk_t for logical blocks, which for ext4 are still 32-bit data
      types.  No point wasting space in the ext4_inode_info structure, and
      requiring 64-bit arithmetic on 32-bit systems, when it isn't
      necessary.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      01f49d0b
    • K
      ext4: fix 32bit overflow in ext4_ext_find_goal() · ad4fb9ca
      Kazuya Mio 提交于
      ext4_ext_find_goal() returns an ideal physical block number that the block
      allocator tries to allocate first. However, if a required file offset is
      smaller than the existing extent's one, ext4_ext_find_goal() returns
      a wrong block number because it may overflow at
      "block - le32_to_cpu(ex->ee_block)". This patch fixes the problem.
      
      ext4_ext_find_goal() will also return a wrong block number in case
      a file offset of the existing extent is too big. In this case,
      the ideal physical block number is fixed in ext4_mb_initialize_context(),
      so it's no problem.
      
      reproduce:
      # dd if=/dev/zero of=/mnt/mp1/tmp bs=127M count=1 oflag=sync
      # dd if=/dev/zero of=/mnt/mp1/file bs=512K count=1 seek=1 oflag=sync
      # filefrag -v /mnt/mp1/file
      Filesystem type is: ef53
      File size of /mnt/mp1/file is 1048576 (256 blocks, blocksize 4096)
       ext logical physical expected length flags
         0     128    67456             128 eof
      /mnt/mp1/file: 2 extents found
      # rm -rf /mnt/mp1/tmp
      # echo $((512*4096)) > /sys/fs/ext4/loop0/mb_stream_req
      # dd if=/dev/zero of=/mnt/mp1/file bs=512K count=1 oflag=sync conv=notrunc
      
      result (linux-2.6.37-rc2 + ext4 patch queue):
      # filefrag -v /mnt/mp1/file
      Filesystem type is: ef53
      File size of /mnt/mp1/file is 1048576 (256 blocks, blocksize 4096)
       ext logical physical expected length flags
         0       0    33280             128 
         1     128    67456    33407    128 eof
      /mnt/mp1/file: 2 extents found
      
      result(apply this patch):
      # filefrag -v /mnt/mp1/file
      Filesystem type is: ef53
      File size of /mnt/mp1/file is 1048576 (256 blocks, blocksize 4096)
       ext logical physical expected length flags
         0       0    66560             128 
         1     128    67456    66687    128 eof
      /mnt/mp1/file: 2 extents found
      Signed-off-by: NKazuya Mio <k-mio@sx.jp.nec.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      ad4fb9ca
  2. 28 10月, 2010 6 次提交
  3. 27 7月, 2010 1 次提交
  4. 17 6月, 2010 1 次提交
  5. 15 6月, 2010 1 次提交
  6. 17 5月, 2010 9 次提交
  7. 16 5月, 2010 1 次提交
  8. 12 5月, 2010 1 次提交
  9. 04 4月, 2010 1 次提交
  10. 04 3月, 2010 2 次提交
  11. 03 3月, 2010 1 次提交
  12. 05 3月, 2010 1 次提交
    • J
      ext4: use ext4_get_block_write in buffer write · 744692dc
      Jiaying Zhang 提交于
      Allocate uninitialized extent before ext4 buffer write and
      convert the extent to initialized after io completes.
      The purpose is to make sure an extent can only be marked
      initialized after it has been written with new data so
      we can safely drop the i_mutex lock in ext4 DIO read without
      exposing stale data. This helps to improve multi-thread DIO
      read performance on high-speed disks.
      
      Skip the nobh and data=journal mount cases to make things simple for now.
      Signed-off-by: NJiaying Zhang <jiayingz@google.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      744692dc
  13. 03 3月, 2010 1 次提交
  14. 24 2月, 2010 1 次提交
  15. 05 3月, 2010 1 次提交
  16. 16 2月, 2010 1 次提交
  17. 25 1月, 2010 1 次提交
    • T
      ext4: Use bitops to read/modify EXT4_I(inode)->i_state · 19f5fb7a
      Theodore Ts'o 提交于
      At several places we modify EXT4_I(inode)->i_state without holding
      i_mutex (ext4_release_file, ext4_bmap, ext4_journalled_writepage,
      ext4_do_update_inode, ...). These modifications are racy and we can
      lose updates to i_state. So convert handling of i_state to use bitops
      which are atomic.
      
      Cc: Jan Kara <jack@suse.cz>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      19f5fb7a
  18. 15 1月, 2010 1 次提交
  19. 25 1月, 2010 1 次提交
  20. 01 1月, 2010 1 次提交
    • T
      ext4: Calculate metadata requirements more accurately · 9d0be502
      Theodore Ts'o 提交于
      In the past, ext4_calc_metadata_amount(), and its sub-functions
      ext4_ext_calc_metadata_amount() and ext4_indirect_calc_metadata_amount()
      badly over-estimated the number of metadata blocks that might be
      required for delayed allocation blocks.  This didn't matter as much
      when functions which managed the reserved metadata blocks were more
      aggressive about dropping reserved metadata blocks as delayed
      allocation blocks were written, but unfortunately they were too
      aggressive.  This was fixed in commit 0637c6f4, but as a result the
      over-estimation by ext4_calc_metadata_amount() would lead to reserving
      2-3 times the number of pending delayed allocation blocks as
      potentially required metadata blocks.  So if there are 1 megabytes of
      blocks which have been not yet been allocation, up to 3 megabytes of
      space would get reserved out of the user's quota and from the file
      system free space pool until all of the inode's data blocks have been
      allocated.
      
      This commit addresses this problem by much more accurately estimating
      the number of metadata blocks that will be required.  It will still
      somewhat over-estimate the number of blocks needed, since it must make
      a worst case estimate not knowing which physical blocks will be
      needed, but it is much more accurate than before.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      9d0be502
  21. 30 12月, 2009 1 次提交
  22. 14 12月, 2009 1 次提交
  23. 10 12月, 2009 1 次提交
    • T
      ext4: Fix potential fiemap deadlock (mmap_sem vs. i_data_sem) · fab3a549
      Theodore Ts'o 提交于
      Fix the following potential circular locking dependency between
      mm->mmap_sem and ei->i_data_sem:
      
          =======================================================
          [ INFO: possible circular locking dependency detected ]
          2.6.32-04115-gec044c5 #37
          -------------------------------------------------------
          ureadahead/1855 is trying to acquire lock:
           (&mm->mmap_sem){++++++}, at: [<ffffffff81107224>] might_fault+0x5c/0xac
      
          but task is already holding lock:
           (&ei->i_data_sem){++++..}, at: [<ffffffff811be1fd>] ext4_fiemap+0x11b/0x159
      
          which lock already depends on the new lock.
      
          the existing dependency chain (in reverse order) is:
      
          -> #1 (&ei->i_data_sem){++++..}:
                 [<ffffffff81099bfa>] __lock_acquire+0xb67/0xd0f
                 [<ffffffff81099e7e>] lock_acquire+0xdc/0x102
                 [<ffffffff81516633>] down_read+0x51/0x84
                 [<ffffffff811a2414>] ext4_get_blocks+0x50/0x2a5
                 [<ffffffff811a3453>] ext4_get_block+0xab/0xef
                 [<ffffffff81154f39>] do_mpage_readpage+0x198/0x48d
                 [<ffffffff81155360>] mpage_readpages+0xd0/0x114
                 [<ffffffff811a104b>] ext4_readpages+0x1d/0x1f
                 [<ffffffff810f8644>] __do_page_cache_readahead+0x12f/0x1bc
                 [<ffffffff810f86f2>] ra_submit+0x21/0x25
                 [<ffffffff810f0cfd>] filemap_fault+0x19f/0x32c
                 [<ffffffff81107b97>] __do_fault+0x55/0x3a2
                 [<ffffffff81109db0>] handle_mm_fault+0x327/0x734
                 [<ffffffff8151aaa9>] do_page_fault+0x292/0x2aa
                 [<ffffffff81518205>] page_fault+0x25/0x30
                 [<ffffffff812a34d8>] clear_user+0x38/0x3c
                 [<ffffffff81167e16>] padzero+0x20/0x31
                 [<ffffffff81168b47>] load_elf_binary+0x8bc/0x17ed
                 [<ffffffff81130e95>] search_binary_handler+0xc2/0x259
                 [<ffffffff81166d64>] load_script+0x1b8/0x1cc
                 [<ffffffff81130e95>] search_binary_handler+0xc2/0x259
                 [<ffffffff8113255f>] do_execve+0x1ce/0x2cf
                 [<ffffffff81027494>] sys_execve+0x43/0x5a
                 [<ffffffff8102918a>] stub_execve+0x6a/0xc0
      
          -> #0 (&mm->mmap_sem){++++++}:
                 [<ffffffff81099aa4>] __lock_acquire+0xa11/0xd0f
                 [<ffffffff81099e7e>] lock_acquire+0xdc/0x102
                 [<ffffffff81107251>] might_fault+0x89/0xac
                 [<ffffffff81139382>] fiemap_fill_next_extent+0x95/0xda
                 [<ffffffff811bcb43>] ext4_ext_fiemap_cb+0x138/0x157
                 [<ffffffff811be069>] ext4_ext_walk_space+0x178/0x1f1
                 [<ffffffff811be21e>] ext4_fiemap+0x13c/0x159
                 [<ffffffff811390e6>] do_vfs_ioctl+0x348/0x4d6
                 [<ffffffff811392ca>] sys_ioctl+0x56/0x79
                 [<ffffffff81028cb2>] system_call_fastpath+0x16/0x1b
      
          other info that might help us debug this:
      
          1 lock held by ureadahead/1855:
           #0:  (&ei->i_data_sem){++++..}, at: [<ffffffff811be1fd>] ext4_fiemap+0x11b/0x159
      
          stack backtrace:
          Pid: 1855, comm: ureadahead Not tainted 2.6.32-04115-gec044c5 #37
          Call Trace:
           [<ffffffff81098c70>] print_circular_bug+0xa8/0xb7
           [<ffffffff81099aa4>] __lock_acquire+0xa11/0xd0f
           [<ffffffff8102f229>] ? sched_clock+0x9/0xd
           [<ffffffff81099e7e>] lock_acquire+0xdc/0x102
           [<ffffffff81107224>] ? might_fault+0x5c/0xac
           [<ffffffff81107251>] might_fault+0x89/0xac
           [<ffffffff81107224>] ? might_fault+0x5c/0xac
           [<ffffffff81124b44>] ? __kmalloc+0x13b/0x18c
           [<ffffffff81139382>] fiemap_fill_next_extent+0x95/0xda
           [<ffffffff811bcb43>] ext4_ext_fiemap_cb+0x138/0x157
           [<ffffffff811bca0b>] ? ext4_ext_fiemap_cb+0x0/0x157
           [<ffffffff811be069>] ext4_ext_walk_space+0x178/0x1f1
           [<ffffffff811be21e>] ext4_fiemap+0x13c/0x159
           [<ffffffff81107224>] ? might_fault+0x5c/0xac
           [<ffffffff811390e6>] do_vfs_ioctl+0x348/0x4d6
           [<ffffffff8129f6d0>] ? __up_read+0x8d/0x95
           [<ffffffff81517fb5>] ? retint_swapgs+0x13/0x1b
           [<ffffffff811392ca>] sys_ioctl+0x56/0x79
           [<ffffffff81028cb2>] system_call_fastpath+0x16/0x1b
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      fab3a549