1. 29 9月, 2009 3 次提交
    • M
      ext4: async direct IO for holes and fallocate support · 8d5d02e6
      Mingming Cao 提交于
      For async direct IO that covers holes or fallocate, the end_io
      callback function now queued the convertion work on workqueue but
      don't flush the work rightaway as it might take too long to afford.
      
      But when fsync is called after all the data is completed, user expects
      the metadata also being updated before fsync returns.
      
      Thus we need to flush the conversion work when fsync() is called.
      This patch keep track of a listed of completed async direct io that
      has a work queued on workqueue.  When fsync() is called, it will go
      through the list and do the conversion.
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      8d5d02e6
    • M
      ext4: Use end_io callback to avoid direct I/O fallback to buffered I/O · 4c0425ff
      Mingming Cao 提交于
      Currently the DIO VFS code passes create = 0 when writing to the
      middle of file.  It does this to avoid block allocation for holes, so
      as not to expose stale data out when there is a parallel buffered read
      (which does not hold the i_mutex lock).  Direct I/O writes into holes
      falls back to buffered IO for this reason.
      
      Since preallocated extents are treated as holes when doing a
      get_block() look up (buffer is not mapped), direct IO over fallocate
      also falls back to buffered IO.  Thus ext4 actually silently falls
      back to buffered IO in above two cases, which is undesirable.
      
      To fix this, this patch creates unitialized extents when a direct I/O
      write into holes in sparse files, and registering an end_io callback which
      converts the uninitialized extent to an initialized extent after the
      I/O is completed.
      Singed-Off-By: NMingming Cao <cmm@us.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      4c0425ff
    • M
      ext4: Split uninitialized extents for direct I/O · 0031462b
      Mingming Cao 提交于
      When writing into an unitialized extent via direct I/O, and the direct
      I/O doesn't exactly cover the unitialized extent, split the extent
      into uninitialized and initialized extents before submitting the I/O.
      This avoids needing to deal with an ENOSPC error in the end_io
      callback that gets used for direct I/O.
      
      When the IO is complete, the written extent will be marked as initialized.
      
      Singed-Off-By: Mingming Cao <cmm@us.ibm.com> 
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      0031462b
  2. 30 9月, 2009 1 次提交
    • T
      ext4: Adjust ext4_da_writepages() to write out larger contiguous chunks · 55138e0b
      Theodore Ts'o 提交于
      Work around problems in the writeback code to force out writebacks in
      larger chunks than just 4mb, which is just too small.  This also works
      around limitations in the ext4 block allocator, which can't allocate
      more than 2048 blocks at a time.  So we need to defeat the round-robin
      characteristics of the writeback code and try to write out as many
      blocks in one inode before allowing the writeback code to move on to
      another inode.  We add a a new per-filesystem tunable,
      max_writeback_mb_bump, which caps this to a default of 128mb per
      inode.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      55138e0b
  3. 17 9月, 2009 3 次提交
    • E
      ext4: replace MAX_DEFRAG_SIZE with EXT_MAX_BLOCK · 0a80e986
      Eric Sandeen 提交于
      There's no reason to redefine the maximum allowable offset
      in an extent-based file just for defrag; 
      EXT_MAX_BLOCK already does this.
      Signed-off-by: NEric Sandeen <sandeen@redhat.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      0a80e986
    • T
      ext4: store EXT4_EXT_MIGRATE in i_state instead of i_flags · 1b9c12f4
      Theodore Ts'o 提交于
      EXT4_EXT_MIGRATE is only intended to be used for an in-memory flag,
      and the hex value assigned to it collides with FS_DIRECTIO_FL (which
      is also stored in i_flags).  There's no reason for the
      EXT4_EXT_MIGRATE bit to be stored in i_flags, so we switch it to use
      i_state instead.
      
      Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      1b9c12f4
    • E
      ext4: limit block allocations for indirect-block files to < 2^32 · fb0a387d
      Eric Sandeen 提交于
      Today, the ext4 allocator will happily allocate blocks past
      2^32 for indirect-block files, which results in the block
      numbers getting truncated, and corruption ensues.
      
      This patch limits such allocations to < 2^32, and adds
      BUG_ONs if we do get blocks larger than that.
      
      This should address RH Bug 519471, ext4 bitmap allocator 
      must limit blocks to < 2^32
      
      * ext4_find_goal() is modified to choose a goal < UINT_MAX,
        so that our starting point is in an acceptable range.
      
      * ext4_xattr_block_set() is modified such that the goal block
        is < UINT_MAX, as above.
      
      * ext4_mb_regular_allocator() is modified so that the group
        search does not continue into groups which are too high
      
      * ext4_mb_use_preallocated() has a check that we don't use
        preallocated space which is too far out
      
      * ext4_alloc_blocks() and ext4_xattr_block_set() add some BUG_ONs
      
      No attempt has been made to limit inode locations to < 2^32,
      so we may wind up with blocks far from their inodes.  Doing
      this much already will lead to some odd ENOSPC issues when the
      "lower 32" gets full, and further restricting inodes could
      make that even weirder.
      
      For high inodes, choosing a goal of the original, % UINT_MAX,
      may be a bit odd, but then we're in an odd situation anyway,
      and I don't know of a better heuristic.
      Signed-off-by: NEric Sandeen <sandeen@redhat.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      fb0a387d
  4. 06 9月, 2009 1 次提交
  5. 01 9月, 2009 1 次提交
  6. 26 8月, 2009 1 次提交
    • E
      ext4: use ext4_grpblk_t more extensively · a36b4498
      Eric Sandeen 提交于
      unsigned  short is potentially too small to track blocks within
      a group; today it is safe due to restrictions in e2fsprogs but
      we have _lo / _hi bits for group blocks with the intent to go
      up to 32 bits, so clean this up now.
      
      There are many more places where we use unsigned/int/unsigned int
      to contain a group block but this should at least fix all the
      short types.
      
      I added a few comments to the struct ext4_group_info definition
      as well.
      Signed-off-by: NEric Sandeen <sandeen@redhat.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      a36b4498
  7. 18 8月, 2009 2 次提交
    • E
      ext4: open-code ext4_mb_update_group_info · 0373130d
      Eric Sandeen 提交于
      ext4_mb_update_group_info is only called in one place, and it's
      extremely simple.  There's no reason to have it in a separate function
      in a separate file as far as I can tell, it just obfuscates what's
      really going on.
      
      Perhaps it was intended to keep the grp->bb_* manipulation local to
      mballoc.c but we're already accessing other grp-> fields in balloc.c
      directly so this seems ok.
      Signed-off-by: NEric Sandeen <sandeen@redhat.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      0373130d
    • J
      ext4: Fix possible deadlock between ext4_truncate() and ext4_get_blocks() · 487caeef
      Jan Kara 提交于
      During truncate we are sometimes forced to start a new transaction as
      the amount of blocks to be journaled is both quite large and hard to
      predict. So far we restarted a transaction while holding i_data_sem
      and that violates lock ordering because i_data_sem ranks below a
      transaction start (and it can lead to a real deadlock with
      ext4_get_blocks() mapping blocks in some page while having a
      transaction open).
      
      We fix the problem by dropping the i_data_sem before restarting the
      transaction and acquire it afterwards. It's slightly subtle that this
      works:
      
      1) By the time ext4_truncate() is called, all the page cache for the
      truncated part of the file is dropped so get_block() should not be
      called on it (we only have to invalidate extent cache after we
      reacquire i_data_sem because some extent from not-truncated part could
      extend also into the part we are going to truncate).
      
      2) Writes, migrate or defrag hold i_mutex so they are stopped for all
      the time of the truncate.
      
      This bug has been found and analyzed by Theodore Tso <tytso@mit.edu>.
      Signed-off-by: NJan Kara <jack@suse.cz>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      487caeef
  8. 19 9月, 2009 1 次提交
    • T
      ext4: Avoid group preallocation for closed files · 50797481
      Theodore Ts'o 提交于
      Currently the group preallocation code tries to find a large (512)
      free block from which to do per-cpu group allocation for small files.
      The problem with this scheme is that it leaves the filesystem horribly
      fragmented.  In the worst case, if the filesystem is unmounted and
      remounted (after a system shutdown, for example) we forget the fact
      that wee were using a particular (now-partially filled) 512 block
      extent.  So the next time we try to allocate space for a small file,
      we will find *another* completely free 512 block chunk to allocate
      small files.  Given that there are 32,768 blocks in a block group,
      after 64 iterations of "mount, write one 4k file in a directory,
      unmount", the block group will have 64 files, each separated by 511
      blocks, and the block group will no longer have any free 512
      completely free chunks of blocks for group preallocation space.
      
      So if we try to allocate blocks for a file that has been closed, such
      that we know the final size of the file, and the filesystem is not
      busy, avoid using group preallocation.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      50797481
  9. 10 8月, 2009 2 次提交
    • T
      ext4: Fix bugs in mballoc's stream allocation mode · 4ba74d00
      Theodore Ts'o 提交于
      The logic around sbi->s_mb_last_group and sbi->s_mb_last_start was all
      screwed up.  These fields were getting unconditionally all the time,
      set even when stream allocation had not taken place, and if they were
      being used when the file was smaller than s_mb_stream_request, which
      is when the allocation should _not_ be doing stream allocation.
      
      Fix this by determining whether or not we stream allocation should
      take place once, in ext4_mb_group_or_file(), and setting a flag which
      gets used in ext4_mb_regular_allocator() and ext4_mb_use_best_found().
      This simplifies the code and assures that we are consistently using
      (or not using) the stream allocation logic.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      4ba74d00
    • T
      ext4: Display the mballoc flags in mb_history in hex instead of decimal · 0ef90db9
      Theodore Ts'o 提交于
      Displaying the flags in base 16 makes it easier to see which flags
      have been set.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      0ef90db9
  10. 24 6月, 2009 1 次提交
  11. 15 6月, 2009 1 次提交
  12. 13 6月, 2009 5 次提交
  13. 18 6月, 2009 1 次提交
  14. 05 6月, 2009 1 次提交
  15. 09 6月, 2009 1 次提交
    • J
      ext4: Get rid of EXTEND_DISKSIZE flag of ext4_get_blocks_handle() · 03f5d8bc
      Jan Kara 提交于
      Get rid of EXTEND_DISKSIZE flag of ext4_get_blocks_handle(). This
      seems to be a relict from some old days and setting disksize in this
      function does not make much sense.  Currently it was set only by
      ext4_getblk().  Since the parameter has some effect only if create ==
      1, it is easy to check by grepping through the sources that the three
      callers which end up calling ext4_getblk() with create == 1
      (ext4_append, ext4_quota_write, ext4_mkdir) do the right thing and set
      disksize themselves.
      Signed-off-by: NJan Kara <jack@suse.cz>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      03f5d8bc
  16. 13 7月, 2009 1 次提交
    • E
      ext4: naturally align struct ext4_allocation_request · 726447d8
      Eric Sandeen 提交于
      As Ted noted, the ext4_allocation_request isn't well aligned.  Looking
      at it with pahole we're wasting space on 64-bit arches:
      
      struct ext4_allocation_request {
              struct inode *             inode;              /*     0     8 */
              ext4_lblk_t                logical;            /*     8     4 */
      
              /* XXX 4 bytes hole, try to pack */
      
              ext4_fsblk_t               goal;               /*    16     8 */
              ext4_lblk_t                lleft;              /*    24     4 */
      
              /* XXX 4 bytes hole, try to pack */
      
              ext4_fsblk_t               pleft;              /*    32     8 */
              ext4_lblk_t                lright;             /*    40     4 */
      
              /* XXX 4 bytes hole, try to pack */
      
              ext4_fsblk_t               pright;             /*    48     8 */
              unsigned int               len;                /*    56     4 */
              unsigned int               flags;              /*    60     4 */
              /* --- cacheline 1 boundary (64 bytes) --- */
      
              /* size: 64, cachelines: 1, members: 9 */
              /* sum members: 52, holes: 3, sum holes: 12 */
      };
      
      Grouping 32-bit members together closes these holes and shrinks the
      structure by 12 bytes. which is important since ext4 can get on the
      hairy edge of stack overruns.
      Signed-off-by: NEric Sandeen <sandeen@redhat.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      726447d8
  17. 18 5月, 2009 1 次提交
    • T
      ext4: Add a comprehensive block validity check to ext4_get_blocks() · 6fd058f7
      Theodore Ts'o 提交于
      To catch filesystem bugs or corruption which could lead to the
      filesystem getting severly damaged, this patch adds a facility for
      tracking all of the filesystem metadata blocks by contiguous regions
      in a red-black tree.  This allows quick searching of the tree to
      locate extents which might overlap with filesystem metadata blocks.
      
      This facility is also used by the multi-block allocator to assure that
      it is not allocating blocks out of the system zone, as well as by the
      routines used when reading indirect blocks and extents information
      from disk to make sure their contents are valid.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      6fd058f7
  18. 15 5月, 2009 1 次提交
    • T
      ext4: Clean up ext4_get_blocks() so it does not depend on bh_result->b_state · 2ac3b6e0
      Theodore Ts'o 提交于
      The ext4_get_blocks() function was depending on the value of
      bh_result->b_state as an input parameter to decide whether or not
      update the delalloc accounting statistics by calling
      ext4_da_update_reserve_space().  We now use a separate flag,
      EXT4_GET_BLOCKS_UPDATE_RESERVE_SPACE, to requests this update, so that
      all callers of ext4_get_blocks() can clear map_bh.b_state before
      calling ext4_get_blocks() without worrying about any consistency
      issues.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      2ac3b6e0
  19. 14 5月, 2009 2 次提交
  20. 15 5月, 2009 1 次提交
  21. 03 5月, 2009 1 次提交
  22. 02 5月, 2009 2 次提交
  23. 04 5月, 2009 1 次提交
  24. 02 5月, 2009 1 次提交
  25. 01 5月, 2009 1 次提交
    • T
      ext4: Avoid races caused by on-line resizing and SMP memory reordering · 8df9675f
      Theodore Ts'o 提交于
      Ext4's on-line resizing adds a new block group and then, only at the
      last step adjusts s_groups_count.  However, it's possible on SMP
      systems that another CPU could see the updated the s_group_count and
      not see the newly initialized data structures for the just-added block
      group.  For this reason, it's important to insert a SMP read barrier
      after reading s_groups_count and before reading any (for example) the
      new block group descriptors allowed by the increased value of
      s_groups_count.
      
      Unfortunately, we rather blatently violate this locking protocol
      documented in fs/ext4/resize.c.  Fortunately, (1) on-line resizes
      happen relatively rarely, and (2) it seems rare that the filesystem
      code will immediately try to use just-added block group before any
      memory ordering issues resolve themselves.  So apparently problems
      here are relatively hard to hit, since ext3 has been vulnerable to the
      same issue for years with no one apparently complaining.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      8df9675f
  26. 01 4月, 2009 1 次提交
    • N
      mm: page_mkwrite change prototype to match fault · c2ec175c
      Nick Piggin 提交于
      Change the page_mkwrite prototype to take a struct vm_fault, and return
      VM_FAULT_xxx flags.  There should be no functional change.
      
      This makes it possible to return much more detailed error information to
      the VM (and also can provide more information eg.  virtual_address to the
      driver, which might be important in some special cases).
      
      This is required for a subsequent fix.  And will also make it easier to
      merge page_mkwrite() with fault() in future.
      Signed-off-by: NNick Piggin <npiggin@suse.de>
      Cc: Chris Mason <chris.mason@oracle.com>
      Cc: Trond Myklebust <trond.myklebust@fys.uio.no>
      Cc: Miklos Szeredi <miklos@szeredi.hu>
      Cc: Steven Whitehouse <swhiteho@redhat.com>
      Cc: Mark Fasheh <mfasheh@suse.com>
      Cc: Joel Becker <joel.becker@oracle.com>
      Cc: Artem Bityutskiy <dedekind@infradead.org>
      Cc: Felix Blyakher <felixb@sgi.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      c2ec175c
  27. 26 3月, 2009 1 次提交
  28. 17 3月, 2009 1 次提交
    • T
      ext4: Add auto_da_alloc mount option · afd4672d
      Theodore Ts'o 提交于
      Add a mount option which allows the user to disable automatic
      allocation of blocks whose allocation by delayed allocation when the
      file was originally truncated or when the file is renamed over an
      existing file.  This feature is intended to save users from the
      effects of naive application writers, but it reduces the effectiveness
      of the delayed allocation code.  This mount option disables this
      safety feature, which may be desirable for prodcutions systems where
      the risk of unclean shutdowns or unexpected system crashes is low.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      afd4672d