1. 30 9月, 2009 1 次提交
  2. 16 9月, 2009 1 次提交
  3. 11 9月, 2009 1 次提交
    • T
      ext4: Fix async commit mode to be safe by using a barrier · 0e3d2a63
      Theodore Ts'o 提交于
      Previously the journal_async_commit mount option was equivalent to
      using barrier=0 (and just as unsafe).  This patch fixes it so that we
      eliminate the barrier before the commit block (by not using ordered
      mode), and explicitly issuing an empty barrier bio after writing the
      commit block.  Because of the journal checksum, it is safe to do this;
      if the journal blocks are not all written before a power failure, the
      checksum in the commit block will prevent the last transaction from
      being replayed.
      
      Using the fs_mark benchmark, using journal_async_commit shows a 50%
      improvement:
      
      FSUse%        Count         Size    Files/sec     App Overhead
           8         1000        10240         30.5            28242
      
      vs.
      
      FSUse%        Count         Size    Files/sec     App Overhead
           8         1000        10240         45.8            28620
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      0e3d2a63
  4. 17 6月, 2009 1 次提交
  5. 14 4月, 2009 1 次提交
  6. 06 4月, 2009 1 次提交
  7. 26 3月, 2009 1 次提交
  8. 06 1月, 2009 1 次提交
    • J
      jbd2: Add buffer triggers · e06c8227
      Joel Becker 提交于
      Filesystems often to do compute intensive operation on some
      metadata.  If this operation is repeated many times, it can be very
      expensive.  It would be much nicer if the operation could be performed
      once before a buffer goes to disk.
      
      This adds triggers to jbd2 buffer heads.  Just before writing a metadata
      buffer to the journal, jbd2 will optionally call a commit trigger associated
      with the buffer.  If the journal is aborted, an abort trigger will be
      called on any dirty buffers as they are dropped from pending
      transactions.
      
      ocfs2 will use this feature.
      
      Initially I tried to come up with a more generic trigger that could be
      used for non-buffer-related events like transaction completion.  It
      doesn't tie nicely, because the information a buffer trigger needs
      (specific to a journal_head) isn't the same as what a transaction
      trigger needs (specific to a tranaction_t or perhaps journal_t).  So I
      implemented a buffer set, with the understanding that
      journal/transaction wide triggers should be implemented separately.
      
      There is only one trigger set allowed per buffer.  I can't think of any
      reason to attach more than one set.  Contrast this with a journal or
      transaction in which multiple places may want to watch the entire
      transaction separately.
      
      The trigger sets are considered static allocation from the jbd2
      perspective.  ocfs2 will just have one trigger set per block type,
      setting the same set on every bh of the same type.
      Signed-off-by: NJoel Becker <joel.becker@oracle.com>
      Cc: "Theodore Ts'o" <tytso@mit.edu>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NMark Fasheh <mfasheh@suse.com>
      e06c8227
  9. 05 1月, 2009 1 次提交
  10. 07 11月, 2008 1 次提交
  11. 26 11月, 2008 1 次提交
    • J
      jbd2: improve jbd2 fsync batching · e07f7183
      Josef Bacik 提交于
      This patch removes the static sleep time in favor of a more self
      optimizing approach where we measure the average amount of time it
      takes to commit a transaction to disk and the ammount of time a
      transaction has been running.  If somebody does a sync write or an
      fsync() traditionally we would sleep for 1 jiffies, which depending on
      the value of HZ could be a significant amount of time compared to how
      long it takes to commit a transaction to the underlying storage.  With
      this patch instead of sleeping for a jiffie, we check to see if the
      amount of time this transaction has been running is less than the
      average commit time, and if it is we sleep for the delta using
      schedule_hrtimeout to give us a higher precision sleep time.  This
      greatly benefits high end storage where you could end up sleeping for
      longer than it takes to commit the transaction and therefore sitting
      idle instead of allowing the transaction to be committed by keeping
      the sleep time to a minimum so you are sure to always be doing
      something.
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      e07f7183
  12. 06 1月, 2009 1 次提交
    • T
      jbd2: Add barrier not supported test to journal_wait_on_commit_record · fd98496f
      Theodore Ts'o 提交于
      Xen doesn't report that barriers are not supported until buffer I/O is
      reported as completed, instead of when the buffer I/O is submitted.
      Add a check and a fallback codepath to journal_wait_on_commit_record()
      to detect this case, so that attempts to mount ext4 filesystems on
      LVM/devicemapper devices on Xen guests don't blow up with an "Aborting
      journal on device XXX"; "Remounting filesystem read-only" error.
      
      Thanks to Andreas Sundstrom for reporting this issue.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      Cc: stable@kernel.org
      fd98496f
  13. 29 10月, 2008 1 次提交
  14. 17 10月, 2008 1 次提交
    • T
      ext4: Replace hackish ext4_mb_poll_new_transaction with commit callback · 3e624fc7
      Theodore Ts'o 提交于
      The multiblock allocator needs to be able to release blocks (and issue
      a blkdev discard request) when the transaction which freed those
      blocks is committed.  Previously this was done via a polling mechanism
      when blocks are allocated or freed.  A much better way of doing things
      is to create a jbd2 callback function and attaching the list of blocks
      to be freed directly to the transaction structure.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      3e624fc7
  15. 11 10月, 2008 2 次提交
    • H
      ext4: add an option to control error handling on file data · 5bf5683a
      Hidehiro Kawai 提交于
      If the journal doesn't abort when it gets an IO error in file data
      blocks, the file data corruption will spread silently.  Because
      most of applications and commands do buffered writes without fsync(),
      they don't notice the IO error.  It's scary for mission critical
      systems.  On the other hand, if the journal aborts whenever it gets
      an IO error in file data blocks, the system will easily become
      inoperable.  So this patch introduces a filesystem option to
      determine whether it aborts the journal or just call printk() when
      it gets an IO error in file data.
      
      If you mount an ext4 fs with data_err=abort option, it aborts on file
      data write error.  If you mount it with data_err=ignore, it doesn't
      abort, just call printk().  data_err=ignore is the default.
      
      Here is the corresponding patch of the ext3 version:
      http://kerneltrap.org/mailarchive/linux-kernel/2008/9/9/3239374Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      5bf5683a
    • H
      jbd2: don't dirty original metadata buffer on abort · 7ad7445f
      Hidehiro Kawai 提交于
      Currently, original metadata buffers are dirtied when they are
      unfiled whether the journal has aborted or not.  Eventually these
      buffers will be written-back to the filesystem by pdflush.  This
      means some metadata buffers are written to the filesystem without
      journaling if the journal aborts.  So if both journal abort and
      system crash happen at the same time, the filesystem would become
      inconsistent state.  Additionally, replaying journaled metadata
      can overwrite the latest metadata on the filesystem partly.
      Because, if the journal gets aborted, journaled metadata are
      preserved and replayed during the next mount not to lose
      uncheckpointed metadata.  This would also break the consistency
      of the filesystem.
      
      This patch prevents original metadata buffers from being dirtied
      on abort by clearing BH_JBDDirty flag from those buffers.  Thus,
      no metadata buffers are written to the filesystem without journaling.
      Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      7ad7445f
  16. 13 10月, 2008 1 次提交
    • H
      jbd2: abort when failed to log metadata buffers · 77e841de
      Hidehiro Kawai 提交于
      If we failed to write metadata buffers to the journal space and
      succeeded to write the commit record, stale data can be written
      back to the filesystem as metadata in the recovery phase.
      
      To avoid this, when we failed to write out metadata buffers,
      abort the journal before writing the commit record.
      
      We can also avoid this kind of corruption by using the journal
      checksum feature because it can detect invalid metadata blocks in the
      journal and avoid them from being replayed.  So we don't need to care
      about asynchronous commit record writeout with a checksum.
      Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      77e841de
  17. 07 10月, 2008 1 次提交
  18. 06 10月, 2008 1 次提交
  19. 17 9月, 2008 1 次提交
    • T
      jbd2: clean up how the journal device name is printed · 05496769
      Theodore Ts'o 提交于
      Calculate the journal device name once and stash it away in the
      journal_s structure.  This avoids needing to call bdevname()
      everywhere and reduces stack usage by not needing to allocate an
      on-stack buffer.  In addition, we eliminate the '/' that can appear in
      device names (e.g. "cciss/c0d0p9" --- see kernel bugzilla #11321) that
      can cause problems when creating proc directory names, and include the
      inode number to support ocfs2 which creates multiple journals with
      different inode numbers.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      05496769
  20. 05 8月, 2008 1 次提交
  21. 01 8月, 2008 1 次提交
    • H
      jbd2: don't abort if flushing file data failed · e9e34f4e
      Hidehiro Kawai 提交于
      In ordered mode, the current jbd2 aborts the journal if a file data buffer
      has an error.  But this behavior is unintended, and we found that it has
      been adopted accidentally.
      
      This patch undoes it and just calls printk() instead of aborting the
      journal.  Unlike a similar patch for ext3/jbd, file data buffers are
      written via generic_writepages().  But we also need to set AS_EIO
      into their mappings because wait_on_page_writeback_range() clears
      AS_EIO before a user process sees it.
      Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      e9e34f4e
  22. 12 7月, 2008 4 次提交
  23. 04 6月, 2008 1 次提交
  24. 16 5月, 2008 1 次提交
  25. 17 4月, 2008 2 次提交
    • J
      jbd2: fix possible journal overflow issues · 1dfc3220
      Josef Bacik 提交于
      There are several cases where the running transaction can get buffers
      added to its BJ_Metadata list which it never dirtied, which makes its
      t_nr_buffers counter end up larger than its t_outstanding_credits
      counter.
      
      This will cause issues when starting new transactions as while we are
      logging buffers we decrement t_outstanding_buffers, so when
      t_outstanding_buffers goes negative, we will report that we need less
      space in the journal than we actually need, so transactions will be
      started even though there may not be enough room for them.  In the worst
      case scenario (which admittedly is almost impossible to reproduce) this
      will result in the journal running out of space.
      
      The fix is to only refile buffers from the committing transaction to the
      running transactions BJ_Modified list when b_modified is set on that
      journal, which is the only way to be sure if the running transaction has
      modified that buffer.
      
      This patch also fixes an accounting error in journal_forget, it is
      possible that we can call journal_forget on a buffer without having
      modified it, only gotten write access to it, so instead of freeing a
      credit, we only do so if the buffer was modified.  The assert will help
      catch if this problem occurs.  Without these two patches I could hit
      this assert within minutes of running postmark, with them this issue no
      longer arises.
      
      Cc: <linux-ext4@vger.kernel.org>
      Cc: Jan Kara <jack@ucw.cz>
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      1dfc3220
    • J
      jbd2: fix the way the b_modified flag is cleared · 9fc7c63a
      Josef Bacik 提交于
      Currently at the start of a journal commit we loop through all of the buffers
      on the committing transaction and clear the b_modified flag (the flag that is
      set when a transaction modifies the buffer) under the j_list_lock.
      
      The problem is that everywhere else this flag is modified only under the jbd2
      lock buffer flag, so it will race with a running transaction who could
      potentially set it, and have it unset by the committing transaction.
      
      This is also a big waste, you can have several thousands of buffers that you
      are clearing the modified flag on when you may not need to.  This patch
      removes this code and instead clears the b_modified flag upon entering
      do_get_write_access/journal_get_create_access, so if that transaction does
      indeed use the buffer then it will be accounted for properly, and if it does
      not then we know we didn't use it.
      
      That will be important for the next patch in this series.  Tested thoroughly
      by myself using postmark/iozone/bonnie++.
      
      Cc: <linux-ext4@vger.kernel.org>
      Cc: Jan Kara <jack@ucw.cz>
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      9fc7c63a
  26. 10 2月, 2008 1 次提交
    • D
      JBD2: Clear buffer_ordered flag for barried IO request on success · c4e35e07
      Dave Kleikamp 提交于
      In JBD2 jbd2_journal_write_commit_record(), clear the buffer_ordered
      flag for the bh after barried IO has succeed. This prevents later, if
      the same buffer head were submitted to the underlying device, which has
      been reconfigured to not support barrier request, the JBD2 commit code
      could treat it as a normal IO (without barrier).
      
      This is a port from JBD/ext3 fix from Neil Brown.
      
      More details from Neil:
      
      Some devices - notably dm and md - can change their behaviour in
      response to BIO_RW_BARRIER requests.  They might start out accepting
      such requests but on reconfiguration, they find out that they cannot
      any more. JBD2 deal with this by always testing if BIO_RW_BARRIER
      requests fail with EOPNOTSUPP, and retrying the write
      requests without the barrier (probably after waiting for any pending
      writes to complete).
      
      However there is a bug in the handling this in JBD2 for ext4 .
      
      When ext4/JBD2 to submit a BIO_RW_BARRIER request,
      it sets the buffer_ordered flag on the buffer head.
      If the request completes successfully, the flag STAYS SET.
      
      Other code might then write the same buffer_head after the device has
      been reconfigured to not accept barriers.  This write will then fail,
      but the "other code" is not ready to handle EOPNOTSUPP errors and the
      error will be treated as fatal.
      
      Cc:  Neil Brown <neilb@suse.de>
      Signed-off-by: NDave Kleikamp <shaggy@linux.vnet.ibm.com>
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      c4e35e07
  27. 05 2月, 2008 3 次提交
  28. 30 1月, 2008 1 次提交
    • N
      spinlock: lockbreak cleanup · 95c354fe
      Nick Piggin 提交于
      The break_lock data structure and code for spinlocks is quite nasty.
      Not only does it double the size of a spinlock but it changes locking to
      a potentially less optimal trylock.
      
      Put all of that under CONFIG_GENERIC_LOCKBREAK, and introduce a
      __raw_spin_is_contended that uses the lock data itself to determine whether
      there are waiters on the lock, to be used if CONFIG_GENERIC_LOCKBREAK is
      not set.
      
      Rename need_lockbreak to spin_needbreak, make it use spin_is_contended to
      decouple it from the spinlock implementation, and make it typesafe (rwlocks
      do not have any need_lockbreak sites -- why do they even get bloated up
      with that break_lock then?).
      Signed-off-by: NNick Piggin <npiggin@suse.de>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
      95c354fe
  29. 29 1月, 2008 3 次提交
    • G
      ext4: Add the journal checksum feature · 818d276c
      Girish Shilamkar 提交于
      The journal checksum feature adds two new flags i.e
      JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT and JBD2_FEATURE_COMPAT_CHECKSUM.
      
      JBD2_FEATURE_CHECKSUM flag indicates that the commit block contains the
      checksum for the blocks described by the descriptor blocks.
      Due to checksums, writing of the commit record no longer needs to be
      synchronous. Now commit record can be sent to disk without waiting for
      descriptor blocks to be written to disk. This behavior is controlled
      using JBD2_FEATURE_ASYNC_COMMIT flag. Older kernels/e2fsck should not be
      able to recover the journal with _ASYNC_COMMIT hence it is made
      incompat.
      The commit header has been extended to hold the checksum along with the
      type of the checksum.
      
      For recovery in pass scan checksums are verified to ensure the sanity
      and completeness(in case of _ASYNC_COMMIT) of every transaction.
      Signed-off-by: NAndreas Dilger <adilger@clusterfs.com>
      Signed-off-by: NGirish Shilamkar <girish@clusterfs.com>
      Signed-off-by: NDave Kleikamp <shaggy@linux.vnet.ibm.com>
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      818d276c
    • J
      jbd2: jbd2 stats through procfs · 8e85fb3f
      Johann Lombardi 提交于
      The patch below updates the jbd stats patch to 2.6.20/jbd2.
      The initial patch was posted by Alex Tomas in December 2005
      (http://marc.info/?l=linux-ext4&m=113538565128617&w=2).
      It provides statistics via procfs such as transaction lifetime and size.
      
      Sometimes, investigating performance problems, i find useful to have
      stats from jbd about transaction's lifetime, size, etc. here is a
      patch for review and inclusion probably.
      
      for example, stats after creation of 3M files in htree directory:
      
      [root@bob ~]# cat /proc/fs/jbd/sda/history
      R/C  tid   wait  run   lock  flush log   hndls  block inlog ctime write drop  close
      R    261   8260  2720  0     0     750   9892   8170  8187
      C    259                                                    750   0     4885  1
      R    262   20    2200  10    0     770   9836   8170  8187
      R    263   30    2200  10    0     3070  9812   8170  8187
      R    264   0     5000  10    0     1340  0      0     0
      C    261                                                    8240  3212  4957  0
      R    265   8260  1470  0     0     4640  9854   8170  8187
      R    266   0     5000  10    0     1460  0      0     0
      C    262                                                    8210  2989  4868  0
      R    267   8230  1490  10    0     4440  9875   8171  8188
      R    268   0     5000  10    0     1260  0      0     0
      C    263                                                    7710  2937  4908  0
      R    269   7730  1470  10    0     3330  9841   8170  8187
      R    270   0     5000  10    0     830   0      0     0
      C    265                                                    8140  3234  4898  0
      C    267                                                    720   0     4849  1
      R    271   8630  2740  20    0     740   9819   8170  8187
      C    269                                                    800   0     4214  1
      R    272   40    2170  10    0     830   9716   8170  8187
      R    273   40    2280  0     0     3530  9799   8170  8187
      R    274   0     5000  10    0     990   0      0     0
      
      
      where,
      
      R     - line for transaction's life from T_RUNNING to T_FINISHED
      C     - line for transaction's checkpointing
      tid   - transaction's id
      wait  - for how long we were waiting for new transaction to start
               (the longest period journal_start() took in this transaction)
      run   - real transaction's lifetime (from T_RUNNING to T_LOCKED
      lock  - how long we were waiting for all handles to close
               (time the transaction was in T_LOCKED)
      flush - how long it took to flush all data (data=ordered)
      log   - how long it took to write the transaction to the log
      hndls - how many handles got to the transaction
      block - how many blocks got to the transaction
      inlog - how many blocks are written to the log (block + descriptors)
      ctime - how long it took to checkpoint the transaction
      write - how many blocks have been written during checkpointing
      drop  - how many blocks have been dropped during checkpointing
      close - how many running transactions have been closed to checkpoint this one
      
      all times are in msec.
      
      
      [root@bob ~]# cat /proc/fs/jbd/sda/info
      280 transaction, each upto 8192 blocks
      average:
        1633ms waiting for transaction
        3616ms running transaction
        5ms transaction was being locked
        1ms flushing data (in ordered mode)
        1799ms logging transaction
        11781 handles per transaction
        5629 blocks per transaction
        5641 logged blocks per transaction
      Signed-off-by: NJohann Lombardi <johann.lombardi@bull.net>
      Signed-off-by: NMariusz Kozlowski <m.kozlowski@tuxland.pl>
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      Signed-off-by: NEric Sandeen <sandeen@redhat.com>
      8e85fb3f
    • J
      jbd2: Fix assertion failure in fs/jbd2/checkpoint.c · f5a7a6b0
      Jan Kara 提交于
      Before we start committing a transaction, we call
      __journal_clean_checkpoint_list() to cleanup transaction's written-back
      buffers.
      
      If this call happens to remove all of them (and there were already some
      buffers), __journal_remove_checkpoint() will decide to free the transaction
      because it isn't (yet) a committing transaction and soon we fail some
      assertion - the transaction really isn't ready to be freed :).
      
      We change the check in __journal_remove_checkpoint() to free only a
      transaction in T_FINISHED state.  The locking there is subtle though (as
      everywhere in JBD ;().  We use j_list_lock to protect the check and a
      subsequent call to __journal_drop_transaction() and do the same in the end
      of journal_commit_transaction() which is the only place where a transaction
      can get to T_FINISHED state.
      
      Probably I'm too paranoid here and such locking is not really necessary -
      checkpoint lists are processed only from log_do_checkpoint() where a
      transaction must be already committed to be processed or from
      __journal_clean_checkpoint_list() where kjournald itself calls it and thus
      transaction cannot change state either.  Better be safe if something
      changes in future...
      Signed-off-by: NJan Kara <jack@suse.cz>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      f5a7a6b0
  30. 18 10月, 2007 2 次提交