1. 10 6月, 2009 12 次提交
    • C
      Btrfs: avoid allocation clusters that are too spread out · c6044801
      Chris Mason 提交于
      In SSD mode for data, and all the time for metadata the allocator
      will try to find a cluster of nearby blocks for allocations.  This
      commit adds extra checks to make sure that each free block in the
      cluster is close to the last one.
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      c6044801
    • C
      Btrfs: Add mount -o nossd · 3b30c22f
      Chris Mason 提交于
      This allows you to turn off the ssd mode via remount.
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      3b30c22f
    • C
      Btrfs: avoid IO stalls behind congested devices in a multi-device FS · d644d8a1
      Chris Mason 提交于
      The btrfs IO submission threads try to service a bunch of devices with a small
      number of threads.  They do a congestion check to try and avoid waiting
      on requests for a busy device.
      
      The checks make sure we've sent a few requests down to a given device just so
      that we aren't bouncing between busy devices without actually sending down
      any IO.  The counter used to decide if we can switch to the next device
      is somewhat overloaded.  It is also being used to decide if we've done
      a good batch of requests between the WRITE_SYNC or regular priority lists.
      It may get reset to zero often, leaving us hammering on a busy device
      instead of moving on to another disk.
      
      This commit adds a new counter for the number of bios sent while
      servicing a device.  It doesn't get reset or fiddled with.  On
      multi-device filesystems, this fixes IO stalls in streaming
      write workloads.
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      d644d8a1
    • C
      Btrfs: don't allow WRITE_SYNC bios to starve out regular writes · d84275c9
      Chris Mason 提交于
      Btrfs uses dedicated threads to submit bios when checksumming is on,
      which allows us to make sure the threads dedicated to checksumming don't get
      stuck waiting for requests.  For each btrfs device, there are
      two lists of bios.  One list is for WRITE_SYNC bios and the other
      is for regular priority bios.
      
      The IO submission threads used to process all of the WRITE_SYNC bios first and
      then switch to the regular bios.  This commit makes sure we don't completely
      starve the regular bios by rotating between the two lists.
      
      WRITE_SYNC bios are still favored 2:1 over the regular bios, and this tries
      to run in batches to avoid seeking.  Benchmarking shows this eliminates
      stalls during streaming buffered writes on both multi-device and
      single device filesystems.
      
      If the regular bios starve, the system can end up with a large amount of ram
      pinned down in writeback pages.  If we are a little more fair between the two
      classes, we're able to keep throughput up and make progress on the bulk of
      our dirty ram.
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      d84275c9
    • C
      Btrfs: fix metadata dirty throttling limits · 585ad2c3
      Chris Mason 提交于
      Once a metadata block has been written, it must be recowed, so the
      btrfs dirty balancing call has a check to make sure a fair amount of metadata
      was actually dirty before it started writing it back to disk.
      
      A previous commit had changed the dirty tracking for metadata without
      updating the btrfs dirty balancing checks.  This commit switches it
      to use the correct counter.
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      585ad2c3
    • C
      Btrfs: reduce mount -o ssd CPU usage · 2c943de6
      Chris Mason 提交于
      The block allocator in SSD mode will try to find groups of free blocks
      that are close together.  This commit makes it loop less on a given
      group size before bumping it.
      
      The end result is that we are less likely to fill small holes in the
      available free space, but we don't waste as much CPU building the
      large cluster used by ssd mode.
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      2c943de6
    • C
      Btrfs: balance btree more often · cfbb9308
      Chris Mason 提交于
      With the new back reference code, the cost of a balance has gone down
      in terms of the number of back reference updates done.  This commit
      makes us more aggressively balance leaves and nodes as they become
      less full.
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      cfbb9308
    • C
      Btrfs: stop avoiding balancing at the end of the transaction. · b3612421
      Chris Mason 提交于
      When the delayed reference code was added, some checks were added
      to avoid extra balancing while the delayed references were being flushed.
      This made for less efficient btrees, but it reduced the chances of
      loops where no forward progress was made because the balances made
      more delayed ref updates.
      
      With the new dead root removal code and the mixed back references,
      the extent allocation tree is no longer using precise back refs, and
      the delayed reference updates don't carry the risk of looping forever
      anymore.  So, the balance avoidance is no longer required.
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      b3612421
    • Y
      Btrfs: Mixed back reference (FORWARD ROLLING FORMAT CHANGE) · 5d4f98a2
      Yan Zheng 提交于
      This commit introduces a new kind of back reference for btrfs metadata.
      Once a filesystem has been mounted with this commit, IT WILL NO LONGER
      BE MOUNTABLE BY OLDER KERNELS.
      
      When a tree block in subvolume tree is cow'd, the reference counts of all
      extents it points to are increased by one.  At transaction commit time,
      the old root of the subvolume is recorded in a "dead root" data structure,
      and the btree it points to is later walked, dropping reference counts
      and freeing any blocks where the reference count goes to 0.
      
      The increments done during cow and decrements done after commit cancel out,
      and the walk is a very expensive way to go about freeing the blocks that
      are no longer referenced by the new btree root.  This commit reduces the
      transaction overhead by avoiding the need for dead root records.
      
      When a non-shared tree block is cow'd, we free the old block at once, and the
      new block inherits old block's references. When a tree block with reference
      count > 1 is cow'd, we increase the reference counts of all extents
      the new block points to by one, and decrease the old block's reference count by
      one.
      
      This dead tree avoidance code removes the need to modify the reference
      counts of lower level extents when a non-shared tree block is cow'd.
      But we still need to update back ref for all pointers in the block.
      This is because the location of the block is recorded in the back ref
      item.
      
      We can solve this by introducing a new type of back ref. The new
      back ref provides information about pointer's key, level and in which
      tree the pointer lives. This information allow us to find the pointer
      by searching the tree. The shortcoming of the new back ref is that it
      only works for pointers in tree blocks referenced by their owner trees.
      
      This is mostly a problem for snapshots, where resolving one of these
      fuzzy back references would be O(number_of_snapshots) and quite slow.
      The solution used here is to use the fuzzy back references in the common
      case where a given tree block is only referenced by one root,
      and use the full back references when multiple roots have a reference
      on a given block.
      
      This commit adds per subvolume red-black tree to keep trace of cached
      inodes. The red-black tree helps the balancing code to find cached
      inodes whose inode numbers within a given range.
      
      This commit improves the balancing code by introducing several data
      structures to keep the state of balancing. The most important one
      is the back ref cache. It caches how the upper level tree blocks are
      referenced. This greatly reduce the overhead of checking back ref.
      
      The improved balancing code scales significantly better with a large
      number of snapshots.
      
      This is a very large commit and was written in a number of
      pieces.  But, they depend heavily on the disk format change and were
      squashed together to make sure git bisect didn't end up in a
      bad state wrt space balancing or the format change.
      Signed-off-by: NYan Zheng <zheng.yan@oracle.com>
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      5d4f98a2
    • Y
      btrfs: Fix set/clear_extent_bit for 'end == (u64)-1' · 5c939df5
      Yan Zheng 提交于
      There are some 'start = state->end + 1;' like code in set_extent_bit
      and clear_extent_bit. They overflow when end == (u64)-1.
      Signed-off-by: NYan Zheng <zheng.yan@oracle.com>
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      5c939df5
    • J
      jbd: fix race in buffer processing in commit code · a61d90d7
      Jan Kara 提交于
      In commit code, we scan buffers attached to a transaction.  During this
      scan, we sometimes have to drop j_list_lock and then we recheck whether
      the journal buffer head didn't get freed by journal_try_to_free_buffers().
       But checking for buffer_jbd(bh) isn't enough because a new journal head
      could get attached to our buffer head.  So add a check whether the journal
      head remained the same and whether it's still at the same transaction and
      list.
      
      This is a nasty bug and can cause problems like memory corruption (use after
      free) or trigger various assertions in JBD code (observed).
      Signed-off-by: NJan Kara <jack@suse.cz>
      Cc: <stable@kernel.org>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      a61d90d7
    • I
      autofs4: remove hashed check in validate_wait() · 463aea1a
      Ian Kent 提交于
      The recent ->lookup() deadlock correction required the directory inode
      mutex to be dropped while waiting for expire completion.  We were
      concerned about side effects from this change and one has been identified.
      
      I saw several error messages.
      
      They cause autofs to become quite confused and don't really point to the
      actual problem.
      
      Things like:
      
      handle_packet_missing_direct:1376: can't find map entry for (43,1827932)
      
      which is usually totally fatal (although in this case it wouldn't be
      except that I treat is as such because it normally is).
      
      do_mount_direct: direct trigger not valid or already mounted
      /test/nested/g3c/s1/ss1
      
      which is recoverable, however if this problem is at play it can cause
      autofs to become quite confused as to the dependencies in the mount tree
      because mount triggers end up mounted multiple times.  It's hard to
      accurately check for this over mounting case and automount shouldn't need
      to if the kernel module is doing its job.
      
      There was one other message, similar in consequence of this last one but I
      can't locate a log example just now.
      
      When checking if a mount has already completed prior to adding a new mount
      request to the wait queue we check if the dentry is hashed and, if so, if
      it is a mount point.  But, if a mount successfully completed while we
      slept on the wait queue mutex the dentry must exist for the mount to have
      completed so the test is not really needed.
      
      Mounts can also be done on top of a global root dentry, so for the above
      case, where a mount request completes and the wait queue entry has already
      been removed, the hashed test returning false can cause an incorrect
      callback to the daemon.  Also, d_mountpoint() is not sufficient to check
      if a mount has completed for the multi-mount case when we don't have a
      real mount at the base of the tree.
      Signed-off-by: NIan Kent <raven@themaw.net>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      463aea1a
  2. 07 6月, 2009 1 次提交
  3. 06 6月, 2009 2 次提交
    • A
      ext3/4 with synchronous writes gets wedged by Postfix · 72a43d63
      Al Viro 提交于
      OK, that's probably the easiest way to do that, as much as I don't like it...
      Since iget() et.al. will not accept I_FREEING (will wait to go away
      and restart), and since we'd better have serialization between new/free
      on fs data structures anyway, we can afford simply skipping I_FREEING
      et.al. in insert_inode_locked().
      
      We do that from new_inode, so it won't race with free_inode in any interesting
      ways and it won't race with iget (of any origin; nfsd or in case of fs
      corruption a lookup) since both still will wait for I_LOCK.
      Reviewed-by: N"Theodore Ts'o" <tytso@mit.edu>
      Acked-by: NJan Kara <jack@suse.cz>
      Tested-by: NDavid Watson <dbwatson@ukfsn.org>
      Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
      72a43d63
    • T
      Fix nobh_truncate_page() to not pass stack garbage to get_block() · 460bcf57
      Theodore Ts'o 提交于
      The nobh_truncate_page() function is used by ext2, exofs, and jfs.  Of
      these three, only ext2 and jfs's get_block() function pays attention
      to bh->b_size --- which is normally always the filesystem blocksize
      except when the get_block() function is called by either
      mpage_readpage(), mpage_readpages(), or the direct I/O routines in
      fs/direct_io.c.
      
      Unfortunately, nobh_truncate_page() does not initialize map_bh before
      calling the filesystem-supplied get_block() function.  So ext2 and jfs
      will try to calculate the number of blocks to map by taking stack
      garbage and shifting it left by inode->i_blkbits.  This should be
      *mostly* harmless (except the filesystem will do some unnneeded work)
      unless the stack garbage is less than filesystem's blocksize, in which
      case maxblocks will be zero, and the attempt to find out whether or
      not the filesystem has a hole at a given logical block will fail, and
      the page cache entry might not get zero'ed out.
      
      Also if the stack garbage in in map_bh->state happens to have the
      BH_Mapped bit set, there could be an attempt to call readpage() on a
      non-existent page, which could cause nobh_truncate_page() to return an
      error when it should not.
      
      Fix this by initializing map_bh->state and map_bh->size.
      
      Fortunately, it's probably fairly unlikely that ext2 and jfs users
      mount with nobh these days.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      Cc: Dave Kleikamp <shaggy@linux.vnet.ibm.com>
      Cc: linux-fsdevel@vger.kernel.org
      Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
      460bcf57
  4. 05 6月, 2009 1 次提交
    • C
      Btrfs: Fix oops and use after free during space balancing · 44fb5511
      Chris Mason 提交于
      The btrfs allocator uses list_for_each to walk the available block
      groups when searching for free blocks.  It starts off with a hint
      to help find the best block group for a given allocation.
      
      The hint is resolved into a block group, but we don't properly check
      to make sure the block group we find isn't in the middle of being
      freed due to filesystem shrinking or balancing.  If it is being
      freed, the list pointers in it are bogus and can't be trusted.  But,
      the code happily goes along and uses them in the list_for_each loop,
      leading to all kinds of fun.
      
      The fix used here is to check to make sure the block group we find really
      is on the list before we use it.  list_del_init is used when removing
      it from the list, so we can do a proper check.
      
      The allocation clustering code has a similar bug where it will trust
      the block group in the current free space cluster.  If our allocation
      flags have changed (going from single spindle dup to raid1 for example)
      because the drives in the FS have changed, we're not allowed to use
      the old block group any more.
      
      The fix used here is to check the current cluster against the
      current allocation flags.
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      44fb5511
  5. 04 6月, 2009 1 次提交
  6. 02 6月, 2009 3 次提交
  7. 30 5月, 2009 1 次提交
  8. 29 5月, 2009 4 次提交
  9. 28 5月, 2009 3 次提交
  10. 27 5月, 2009 2 次提交
  11. 24 5月, 2009 1 次提交
    • S
      [CIFS] Avoid open on possible directories since Samba now rejects them · 8db14ca1
      Steve French 提交于
      Small change (mostly formatting) to limit lookup based open calls to
      file create only.
      
      After discussion yesteday on samba-technical about the posix lookup
      regression,  and looking at a problem with cifs posix open to one
      particular Samba version, Jeff and JRA realized that Samba server's
      behavior changed in this area (posix open behavior on files vs.
      directories).   To make this behavior consistent, JRA just made a
      fix to Samba server to alter how it handles open of directories (now
      returning the equivalent of EISDIR instead of success). Since we don't
      know at lookup time whether the inode is a directory or file (and
      thus whether posix open will succeed with most current Samba server),
      this change avoids the posix open code on lookup open (just issues
      posix open on creates).    This gets the semantic benefits we want
      (atomicity, posix byte range locks, improved write semantics on newly
      created files) and file create still is fast, and we avoid the problem
      that Jeff noticed yesterday with "openat" (and some open directory
      calls) of non-cached directories to one version of Samba server, and
      will work with future Samba versions (which include the fix jra just
      pushed into Samba server).  I confirmed this approach with jra
      yesterday and with Shirish today.
      
      Posix open is only called (at lookup time) for file create now.
      For opens (rather than creates), because we do not know if it
      is a file or directory yet, and current Samba no longer allows
      us to do posix open on dirs, we could end up wasting an open call
      on what turns out to be a dir. For file opens, we wait to call posix
      open till cifs_open.  It could be added here (lookup) in the future
      but the performance tradeoff of the extra network request when EISDIR
      or EACCES is returned would have to be weighed against the 50%
      reduction in network traffic in the other paths.
      Reviewed-by: NShirish Pargaonkar <shirishp@us.ibm.com>
      Tested-by: NJeff Layton <jlayton@redhat.com>
      CC: Jeremy Allison <jra@samba.org>
      Signed-off-by: NSteve French <sfrench@us.ibm.com>
      8db14ca1
  12. 22 5月, 2009 2 次提交
  13. 19 5月, 2009 2 次提交
  14. 18 5月, 2009 3 次提交
  15. 15 5月, 2009 2 次提交