1. 10 9月, 2011 6 次提交
  2. 04 9月, 2011 1 次提交
    • T
      ext4: improve handling of conflicting mount options · 56889787
      Theodore Ts'o 提交于
      If the user explicitly specifies conflicting mount options for
      delalloc or dioread_nolock and data=journal, fail the mount, instead
      of printing a warning and continuing (since many user's won't look at
      dmesg and notice the warning).
      
      Also, print a single warning that data=journal implies that delayed
      allocation is not on by default (since it's not supported), and
      furthermore that O_DIRECT is not supported.  Improve the text in
      Documentation/filesystems/ext4.txt so this is clear there as well.
      
      Similarly, if the dioread_nolock mount option is specified when the
      file system block size != PAGE_SIZE, fail the mount instead of
      printing a warning message and ignoring the mount option.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      56889787
  3. 03 9月, 2011 1 次提交
    • A
      ext4: Add new ext4_discard_partial_page_buffers routines · 4e96b2db
      Allison Henderson 提交于
      This patch adds two new routines: ext4_discard_partial_page_buffers
      and ext4_discard_partial_page_buffers_no_lock.
      
      The ext4_discard_partial_page_buffers routine is a wrapper
      function to ext4_discard_partial_page_buffers_no_lock.
      The wrapper function locks the page and passes it to
      ext4_discard_partial_page_buffers_no_lock.
      Calling functions that already have the page locked can call
      ext4_discard_partial_page_buffers_no_lock directly.
      
      The ext4_discard_partial_page_buffers_no_lock function
      zeros a specified range in a page, and unmaps the
      corresponding buffer heads.  Only block aligned regions of the
      page will have their buffer heads unmapped.  Unblock aligned regions
      will be mapped if needed so that they can be updated with the
      partial zero out.  This function is meant to
      be used to update a page and its buffer heads to be zeroed
      and unmapped when the corresponding blocks have been released
      or will be released.
      
      This routine is used in the following scenarios:
      * A hole is punched and the non page aligned regions
        of the head and tail of the hole need to be discarded
      
      * The file is truncated and the partial page beyond EOF needs
        to be discarded
      
      * The end of a hole is in the same page as EOF.  After the
        page is flushed, the partial page beyond EOF needs to be
        discarded.
      
      * A write operation begins or ends inside a hole and the partial
        page appearing before or after the write needs to be discarded
      
      * A write operation extends EOF and the partial page beyond EOF
        needs to be discarded
      
      This function takes a flag EXT4_DISCARD_PARTIAL_PG_ZERO_UNMAPPED
      which is used when a write operation begins or ends in a hole.
      When the EXT4_DISCARD_PARTIAL_PG_ZERO_UNMAPPED flag is used, only
      buffer heads that are already unmapped will have the corresponding
      regions of the page zeroed.
      Signed-off-by: NAllison Henderson <achender@linux.vnet.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      4e96b2db
  4. 31 8月, 2011 2 次提交
    • T
      ext2,ext3,ext4: don't inherit APPEND_FL or IMMUTABLE_FL for new inodes · 1cd9f097
      Theodore Ts'o 提交于
      This doesn't make much sense, and it exposes a bug in the kernel where
      attempts to create a new file in an append-only directory using
      O_CREAT will fail (but still leave a zero-length file).  This was
      discovered when xfstests #79 was generalized so it could run on all
      file systems.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      Cc:stable@kernel.org
      1cd9f097
    • J
      ext4: remove i_mutex lock in ext4_evict_inode to fix lockdep complaining · 8c0bec21
      Jiaying Zhang 提交于
      The i_mutex lock and flush_completed_IO() added by commit 2581fdc8
      in ext4_evict_inode() causes lockdep complaining about potential
      deadlock in several places.  In most/all of these LOCKDEP complaints
      it looks like it's a false positive, since many of the potential
      circular locking cases can't take place by the time the
      ext4_evict_inode() is called; but since at the very least it may mask
      real problems, we need to address this.
      
      This change removes the flush_completed_IO() and i_mutex lock in
      ext4_evict_inode().  Instead, we take a different approach to resolve
      the software lockup that commit 2581fdc8 intends to fix.  Rather
      than having ext4-dio-unwritten thread wait for grabing the i_mutex
      lock of an inode, we use mutex_trylock() instead, and simply requeue
      the work item if we fail to grab the inode's i_mutex lock.
      
      This should speed up work queue processing in general and also
      prevents the following deadlock scenario: During page fault,
      shrink_icache_memory is called that in turn evicts another inode B.
      Inode B has some pending io_end work so it calls ext4_ioend_wait()
      that waits for inode B's i_ioend_count to become zero.  However, inode
      B's ioend work was queued behind some of inode A's ioend work on the
      same cpu's ext4-dio-unwritten workqueue.  As the ext4-dio-unwritten
      thread on that cpu is processing inode A's ioend work, it tries to
      grab inode A's i_mutex lock.  Since the i_mutex lock of inode A is
      still hold before the page fault happened, we enter a deadlock.
      Signed-off-by: NJiaying Zhang <jiayingz@google.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      8c0bec21
  5. 01 8月, 2011 1 次提交
  6. 27 7月, 2011 4 次提交
  7. 21 7月, 2011 1 次提交
    • J
      fs: push i_mutex and filemap_write_and_wait down into ->fsync() handlers · 02c24a82
      Josef Bacik 提交于
      Btrfs needs to be able to control how filemap_write_and_wait_range() is called
      in fsync to make it less of a painful operation, so push down taking i_mutex and
      the calling of filemap_write_and_wait() down into the ->fsync() handlers.  Some
      file systems can drop taking the i_mutex altogether it seems, like ext3 and
      ocfs2.  For correctness sake I just pushed everything down in all cases to make
      sure that we keep the current behavior the same for everybody, and then each
      individual fs maintainer can make up their mind about what to do from there.
      Thanks,
      Acked-by: NJan Kara <jack@suse.cz>
      Signed-off-by: NJosef Bacik <josef@redhat.com>
      Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
      02c24a82
  8. 11 7月, 2011 2 次提交
    • T
      ext4: Speed up FITRIM by recording flags in ext4_group_info · 3d56b8d2
      Tao Ma 提交于
      In ext4, when FITRIM is called every time, we iterate all the
      groups and do trim one by one. It is a bit time wasting if the
      group has been trimmed and there is no change since the last
      trim.
      
      So this patch adds a new flag in ext4_group_info->bb_state to
      indicate that the group has been trimmed, and it will be cleared
      if some blocks is freed(in release_blocks_on_commit). Another
      trim_minlen is added in ext4_sb_info to record the last minlen
      we use to trim the volume, so that if the caller provide a small
      one, we will go on the trim regardless of the bb_state.
      
      A simple test with my intel x25m ssd:
      df -h shows:
      /dev/sdb1              40G   21G   17G  56% /mnt/ext4
      Block size:               4096
      
      run the FITRIM with the following parameter:
      range.start = 0;
      range.len = UINT64_MAX;
      range.minlen = 1048576;
      
      without the patch:
      [root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
      real	0m5.505s
      user	0m0.000s
      sys	0m1.224s
      [root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
      real	0m5.359s
      user	0m0.000s
      sys	0m1.178s
      [root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
      real	0m5.228s
      user	0m0.000s
      sys	0m1.151s
      
      with the patch:
      [root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
      real	0m5.625s
      user	0m0.000s
      sys	0m1.269s
      [root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
      real	0m0.002s
      user	0m0.000s
      sys	0m0.001s
      [root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
      real	0m0.002s
      user	0m0.000s
      sys	0m0.001s
      
      A big improvement for the 2nd and 3rd run.
      
      Even after I delete some big image files, it is still much
      faster than iterating the whole disk.
      
      [root@boyu-tm test]# time ./ftrim /mnt/ext4/a
      real	0m1.217s
      user	0m0.000s
      sys	0m0.196s
      
      Cc: Lukas Czerner <lczerner@redhat.com>
      Reviewed-by: NAndreas Dilger <adilger.kernel@dilger.ca>
      Signed-off-by: NTao Ma <boyu.mt@taobao.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      3d56b8d2
    • M
      ext4: fix i_blocks/quota accounting when extent insertion fails · 7132de74
      Maxim Patlasov 提交于
      The current implementation of ext4_free_blocks() always calls
      dquot_free_block This looks quite sensible in the most cases: blocks
      to be freed are associated with inode and were accounted in quota and
      i_blocks some time ago.
      
      However, there is a case when blocks to free were not accounted by the
      time calling ext4_free_blocks() yet:
      
      1. delalloc is on, write_begin pre-allocated some space in quota
      2. write-back happens, ext4 allocates some blocks in ext4_ext_map_blocks()
      3. then ext4_ext_map_blocks() gets an error (e.g.  ENOSPC) from
         ext4_ext_insert_extent() and calls ext4_free_blocks().
      
      In this scenario, ext4_free_blocks() calls dquot_free_block() who, in
      turn, decrements i_blocks for blocks which were not accounted yet (due
      to delalloc) After clean umount, e2fsck reports something like:
      
      > Inode 21, i_blocks is 5080, should be 5128.  Fix<y>?
      because i_blocks was erroneously decremented as explained above.
      
      The patch fixes the problem by passing the new flag
      EXT4_FREE_BLOCKS_NO_QUOT_UPDATE to ext4_free_blocks(), to request
      that the dquot_free_block() call be skipped.
      Signed-off-by: NMaxim Patlasov <maxim.patlasov@gmail.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      Cc: stable@kernel.org
      7132de74
  9. 28 6月, 2011 4 次提交
  10. 27 5月, 2011 1 次提交
    • C
      fs: pass exact type of data dirties to ->dirty_inode · aa385729
      Christoph Hellwig 提交于
      Tell the filesystem if we just updated timestamp (I_DIRTY_SYNC) or
      anything else, so that the filesystem can track internally if it
      needs to push out a transaction for fdatasync or not.
      
      This is just the prototype change with no user for it yet.  I plan
      to push large XFS changes for the next merge window, and getting
      this trivial infrastructure in this window would help a lot to avoid
      tree interdependencies.
      
      Also remove incorrect comments that ->dirty_inode can't block.  That
      has been changed a long time ago, and many implementations rely on it.
      Signed-off-by: NChristoph Hellwig <hch@lst.de>
      Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
      aa385729
  11. 25 5月, 2011 6 次提交
    • V
      ext4: do not normalize block requests from fallocate() · 556b27ab
      Vivek Haldar 提交于
      Currently, an fallocate request of size slightly larger than a power of
      2 is turned into two block requests, each a power of 2, with the extra
      blocks pre-allocated for future use. When an application calls
      fallocate, it already has an idea about how large the file may grow so
      there is usually little benefit to reserve extra blocks on the
      preallocation list. This reduces disk fragmentation.
      
      Tested: fsstress. Also verified manually that fallocat'ed files are
      contiguously laid out with this change (whereas without it they begin at
      power-of-2 boundaries, leaving blocks in between). CPU usage of
      fallocate is not appreciably higher.  In a tight fallocate loop, CPU
      usage hovers between 5%-8% with this change, and 5%-7% without it.
      
      Using a simulated file system aging program which the file system to
      70%, the percentage of free extents larger than 8MB (as measured by
      e2freefrag) increased from 38.8% without this change, to 69.4% with
      this change.
      Signed-off-by: NVivek Haldar <haldar@google.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      556b27ab
    • A
      ext4: enable "punch hole" functionality · a4bb6b64
      Allison Henderson 提交于
      This patch adds new routines: "ext4_punch_hole" "ext4_ext_punch_hole"
      and "ext4_ext_check_cache"
      
      fallocate has been modified to call ext4_punch_hole when the punch hole
      flag is passed.  At the moment, we only support punching holes in
      extents, so this routine is pretty much a wrapper for the ext4_ext_punch_hole
      routine.
      
      The ext4_ext_punch_hole routine first completes all outstanding writes
      with the associated pages, and then releases them.  The unblock
      aligned data is zeroed, and all blocks in between are punched out.
      
      The ext4_ext_check_cache routine is very similar to ext4_ext_in_cache
      except it accepts a ext4_ext_cache parameter instead of a ext4_extent
      parameter.  This routine is used by ext4_ext_punch_hole to check and
      see if a block in a hole that has been cached.  The ext4_ext_cache
      parameter is necessary because the members ext4_extent structure are
      not large enough to hold a 32 bit value.  The existing
      ext4_ext_in_cache routine has become a wrapper to this new function.
      
      [ext4 punch hole patch series 5/5 v7] 
      Signed-off-by: NAllison Henderson <achender@us.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      Reviewed-by: NMingming Cao <cmm@us.ibm.com>
      a4bb6b64
    • A
      ext4: add new function ext4_block_zero_page_range() · 30848851
      Allison Henderson 提交于
      This patch modifies the existing ext4_block_truncate_page() function
      which was used by the truncate code path, and which zeroes out block
      unaligned data, by adding a new length parameter, and renames it to
      ext4_block_zero_page_rage().  This function can now be used to zero out the
      head of a block, the tail of a block, or the middle
      of a block.
      
      The ext4_block_truncate_page() function is now a wrapper to
      ext4_block_zero_page_range().
      
      [ext4 punch hole patch series 2/5 v7] 
      Signed-off-by: NAllison Henderson <achender@us.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      Reviewed-by: NMingming Cao <cmm@us.ibm.com>
      30848851
    • A
      ext4: add flag to ext4_has_free_blocks · 55f020db
      Allison Henderson 提交于
      This patch adds an allocation request flag to the ext4_has_free_blocks
      function which enables the use of reserved blocks.  This will allow a
      punch hole to proceed even if the disk is full.  Punching a hole may
      require additional blocks to first split the extents.
      
      Because ext4_has_free_blocks is a low level function, the flag needs
      to be passed down through several functions listed below:
      
      ext4_ext_insert_extent
      ext4_ext_create_new_leaf
      ext4_ext_grow_indepth
      ext4_ext_split
      ext4_ext_new_meta_block
      ext4_mb_new_blocks
      ext4_claim_free_blocks
      ext4_has_free_blocks
      
      [ext4 punch hole patch series 1/5 v7]
      Signed-off-by: NAllison Henderson <achender@us.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      Reviewed-by: NMingming Cao <cmm@us.ibm.com>
      55f020db
    • A
      ext4: reserve inodes and feature code for 'quota' feature · ae812306
      Aditya Kali 提交于
      I am working on patch to add quota as a built-in feature for ext4
      filesystem. The implementation is based on the design given at
      https://ext4.wiki.kernel.org/index.php/Design_For_1st_Class_Quota_in_Ext4.
      This patch reserves the inode numbers 3 and 4 for quota purposes and
      also reserves EXT4_FEATURE_RO_COMPAT_QUOTA feature code.
      Signed-off-by: NAditya Kali <adityakali@google.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      ae812306
    • J
      ext4: add support for multiple mount protection · c5e06d10
      Johann Lombardi 提交于
      Prevent an ext4 filesystem from being mounted multiple times.
      A sequence number is stored on disk and is periodically updated (every 5
      seconds by default) by a mounted filesystem.
      At mount time, we now wait for s_mmp_update_interval seconds to make sure
      that the MMP sequence does not change.
      In case of failure, the nodename, bdevname and the time at which the MMP
      block was last updated is displayed.
      Signed-off-by: NAndreas Dilger <adilger@whamcloud.com>
      Signed-off-by: NJohann Lombardi <johann@whamcloud.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      c5e06d10
  12. 23 5月, 2011 1 次提交
  13. 21 5月, 2011 2 次提交
  14. 09 5月, 2011 1 次提交
  15. 19 4月, 2011 1 次提交
    • T
      ext4: check for ext[23] file system features when mounting as ext[23] · 2035e776
      Theodore Ts'o 提交于
      Provide better emulation for ext[23] mode by enforcing that the file
      system does not have any unsupported file system features as defined
      by ext[23] when emulating the ext[23] file system driver when
      CONFIG_EXT4_USE_FOR_EXT23 is defined.
      
      This causes the file system type information in /proc/mounts to be
      correct for the automatically mounted root file system.  This also
      means that "mount -t ext2 /dev/sda /mnt" will fail if /dev/sda
      contains an ext3 or ext4 file system, just as one would expect if the
      original ext2 file system driver were in use.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      2035e776
  16. 24 3月, 2011 1 次提交
  17. 12 2月, 2011 1 次提交
    • E
      ext4: serialize unaligned asynchronous DIO · e9e3bcec
      Eric Sandeen 提交于
      ext4 has a data corruption case when doing non-block-aligned
      asynchronous direct IO into a sparse file, as demonstrated
      by xfstest 240.
      
      The root cause is that while ext4 preallocates space in the
      hole, mappings of that space still look "new" and 
      dio_zero_block() will zero out the unwritten portions.  When
      more than one AIO thread is going, they both find this "new"
      block and race to zero out their portion; this is uncoordinated
      and causes data corruption.
      
      Dave Chinner fixed this for xfs by simply serializing all
      unaligned asynchronous direct IO.  I've done the same here.
      The difference is that we only wait on conversions, not all IO.
      This is a very big hammer, and I'm not very pleased with
      stuffing this into ext4_file_write().  But since ext4 is
      DIO_LOCKING, we need to serialize it at this high level.
      
      I tried to move this into ext4_ext_direct_IO, but by then
      we have the i_mutex already, and we will wait on the
      work queue to do conversions - which must also take the
      i_mutex.  So that won't work.
      
      This was originally exposed by qemu-kvm installing to
      a raw disk image with a normal sector-63 alignment.  I've
      tested a backport of this patch with qemu, and it does
      avoid the corruption.  It is also quite a lot slower
      (14 min for package installs, vs. 8 min for well-aligned)
      but I'll take slow correctness over fast corruption any day.
      
      Mingming suggested that we can track outstanding
      conversions, and wait on those so that non-sparse
      files won't be affected, and I've implemented that here;
      unaligned AIO to nonsparse files won't take a perf hit.
      
      [tytso@mit.edu: Keep the mutex as a hashed array instead
       of bloating the ext4 inode]
      
      [tytso@mit.edu: Fix up namespace issues so that global
       variables are protected with an "ext4_" prefix.]
      Signed-off-by: NEric Sandeen <sandeen@redhat.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      e9e3bcec
  18. 17 1月, 2011 1 次提交
    • C
      fallocate should be a file operation · 2fe17c10
      Christoph Hellwig 提交于
      Currently all filesystems except XFS implement fallocate asynchronously,
      while XFS forced a commit.  Both of these are suboptimal - in case of O_SYNC
      I/O we really want our allocation on disk, especially for the !KEEP_SIZE
      case where we actually grow the file with user-visible zeroes.  On the
      other hand always commiting the transaction is a bad idea for fast-path
      uses of fallocate like for example in recent Samba versions.   Given
      that block allocation is a data plane operation anyway change it from
      an inode operation to a file operation so that we have the file structure
      available that lets us check for O_SYNC.
      
      This also includes moving the code around for a few of the filesystems,
      and remove the already unnedded S_ISDIR checks given that we only wire
      up fallocate for regular files.
      Signed-off-by: NChristoph Hellwig <hch@lst.de>
      Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
      2fe17c10
  19. 11 1月, 2011 3 次提交
    • J
      ext4: flush the i_completed_io_list during ext4_truncate · 3889fd57
      Jiaying Zhang 提交于
      Ted first found the bug when running 2.6.36 kernel with dioread_nolock
      mount option that xfstests #13 complained about wrong file size during fsck.
      However, the bug exists in the older kernels as well although it is
      somehow harder to trigger.
      
      The problem is that ext4_end_io_work() can happen after we have truncated an
      inode to a smaller size. Then when ext4_end_io_work() calls 
      ext4_convert_unwritten_extents(), we may reallocate some blocks that have 
      been truncated, so the inode size becomes inconsistent with the allocated
      blocks. 
      
      The following patch flushes the i_completed_io_list during truncate to reduce 
      the risk that some pending end_io requests are executed later and convert 
      already truncated blocks to initialized. 
      
      Note that although the fix helps reduce the problem a lot there may still 
      be a race window between vmtruncate() and ext4_end_io_work(). The fundamental
      problem is that if vmtruncate() is called without either i_mutex or i_alloc_sem
      held, it can race with an ongoing write request so that the io_end request is
      processed later when the corresponding blocks have been truncated.
      
      Ted and I have discussed the problem offline and we saw a few ways to fix
      the race completely:
      
      a) We guarantee that i_mutex lock and i_alloc_sem write lock are both hold 
      whenever vmtruncate() is called. The i_mutex lock prevents any new write
      requests from entering writeback and the i_alloc_sem prevents the race
      from ext4_page_mkwrite(). Currently we hold both locks if vmtruncate()
      is called from do_truncate(), which is probably the most common case.
      However, there are places where we may call vmtruncate() without holding
      either i_mutex or i_alloc_sem. I would like to ask for other people's
      opinions on what locks are expected to be held before calling vmtruncate().
      There seems a disagreement among the callers of that function.
      
      b) We change the ext4 write path so that we change the extent tree to contain 
      the newly allocated blocks and update i_size both at the same time --- when 
      the write of the data blocks is completed.
      
      c) We add some additional locking to synchronize vmtruncate() and 
      ext4_end_io_work(). This approach may have performance implications so we
      need to be careful.
      
      All of the above proposals may require more substantial changes, so
      we may consider to take the following patch as a bandaid.
      Signed-off-by: NJiaying Zhang <jiayingz@google.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      3889fd57
    • T
      ext4: dynamically allocate the jbd2_inode in ext4_inode_info as necessary · 8aefcd55
      Theodore Ts'o 提交于
      Replace the jbd2_inode structure (which is 48 bytes) with a pointer
      and only allocate the jbd2_inode when it is needed --- that is, when
      the file system has a journal present and the inode has been opened
      for writing.  This allows us to further slim down the ext4_inode_info
      structure.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      8aefcd55
    • T
      ext4: drop i_state_flags on architectures with 64-bit longs · 353eb83c
      Theodore Ts'o 提交于
      We can store the dynamic inode state flags in the high bits of
      EXT4_I(inode)->i_flags, and eliminate i_state_flags.  This saves 8
      bytes from the size of ext4_inode_info structure, which when
      multiplied by the number of the number of in the inode cache, can save
      a lot of memory.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      353eb83c