1. 11 10月, 2008 2 次提交
    • H
      ext4: add an option to control error handling on file data · 5bf5683a
      Hidehiro Kawai 提交于
      If the journal doesn't abort when it gets an IO error in file data
      blocks, the file data corruption will spread silently.  Because
      most of applications and commands do buffered writes without fsync(),
      they don't notice the IO error.  It's scary for mission critical
      systems.  On the other hand, if the journal aborts whenever it gets
      an IO error in file data blocks, the system will easily become
      inoperable.  So this patch introduces a filesystem option to
      determine whether it aborts the journal or just call printk() when
      it gets an IO error in file data.
      
      If you mount an ext4 fs with data_err=abort option, it aborts on file
      data write error.  If you mount it with data_err=ignore, it doesn't
      abort, just call printk().  data_err=ignore is the default.
      
      Here is the corresponding patch of the ext3 version:
      http://kerneltrap.org/mailarchive/linux-kernel/2008/9/9/3239374Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      5bf5683a
    • H
      jbd2: don't dirty original metadata buffer on abort · 7ad7445f
      Hidehiro Kawai 提交于
      Currently, original metadata buffers are dirtied when they are
      unfiled whether the journal has aborted or not.  Eventually these
      buffers will be written-back to the filesystem by pdflush.  This
      means some metadata buffers are written to the filesystem without
      journaling if the journal aborts.  So if both journal abort and
      system crash happen at the same time, the filesystem would become
      inconsistent state.  Additionally, replaying journaled metadata
      can overwrite the latest metadata on the filesystem partly.
      Because, if the journal gets aborted, journaled metadata are
      preserved and replayed during the next mount not to lose
      uncheckpointed metadata.  This would also break the consistency
      of the filesystem.
      
      This patch prevents original metadata buffers from being dirtied
      on abort by clearing BH_JBDDirty flag from those buffers.  Thus,
      no metadata buffers are written to the filesystem without journaling.
      Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      7ad7445f
  2. 13 10月, 2008 1 次提交
    • H
      jbd2: abort when failed to log metadata buffers · 77e841de
      Hidehiro Kawai 提交于
      If we failed to write metadata buffers to the journal space and
      succeeded to write the commit record, stale data can be written
      back to the filesystem as metadata in the recovery phase.
      
      To avoid this, when we failed to write out metadata buffers,
      abort the journal before writing the commit record.
      
      We can also avoid this kind of corruption by using the journal
      checksum feature because it can detect invalid metadata blocks in the
      journal and avoid them from being replayed.  So we don't need to care
      about asynchronous commit record writeout with a checksum.
      Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      77e841de
  3. 07 10月, 2008 1 次提交
  4. 06 10月, 2008 1 次提交
  5. 17 9月, 2008 1 次提交
    • T
      jbd2: clean up how the journal device name is printed · 05496769
      Theodore Ts'o 提交于
      Calculate the journal device name once and stash it away in the
      journal_s structure.  This avoids needing to call bdevname()
      everywhere and reduces stack usage by not needing to allocate an
      on-stack buffer.  In addition, we eliminate the '/' that can appear in
      device names (e.g. "cciss/c0d0p9" --- see kernel bugzilla #11321) that
      can cause problems when creating proc directory names, and include the
      inode number to support ocfs2 which creates multiple journals with
      different inode numbers.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      05496769
  6. 05 8月, 2008 1 次提交
  7. 01 8月, 2008 1 次提交
    • H
      jbd2: don't abort if flushing file data failed · e9e34f4e
      Hidehiro Kawai 提交于
      In ordered mode, the current jbd2 aborts the journal if a file data buffer
      has an error.  But this behavior is unintended, and we found that it has
      been adopted accidentally.
      
      This patch undoes it and just calls printk() instead of aborting the
      journal.  Unlike a similar patch for ext3/jbd, file data buffers are
      written via generic_writepages().  But we also need to set AS_EIO
      into their mappings because wait_on_page_writeback_range() clears
      AS_EIO before a user process sees it.
      Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      e9e34f4e
  8. 12 7月, 2008 4 次提交
  9. 04 6月, 2008 1 次提交
  10. 16 5月, 2008 1 次提交
  11. 17 4月, 2008 2 次提交
    • J
      jbd2: fix possible journal overflow issues · 1dfc3220
      Josef Bacik 提交于
      There are several cases where the running transaction can get buffers
      added to its BJ_Metadata list which it never dirtied, which makes its
      t_nr_buffers counter end up larger than its t_outstanding_credits
      counter.
      
      This will cause issues when starting new transactions as while we are
      logging buffers we decrement t_outstanding_buffers, so when
      t_outstanding_buffers goes negative, we will report that we need less
      space in the journal than we actually need, so transactions will be
      started even though there may not be enough room for them.  In the worst
      case scenario (which admittedly is almost impossible to reproduce) this
      will result in the journal running out of space.
      
      The fix is to only refile buffers from the committing transaction to the
      running transactions BJ_Modified list when b_modified is set on that
      journal, which is the only way to be sure if the running transaction has
      modified that buffer.
      
      This patch also fixes an accounting error in journal_forget, it is
      possible that we can call journal_forget on a buffer without having
      modified it, only gotten write access to it, so instead of freeing a
      credit, we only do so if the buffer was modified.  The assert will help
      catch if this problem occurs.  Without these two patches I could hit
      this assert within minutes of running postmark, with them this issue no
      longer arises.
      
      Cc: <linux-ext4@vger.kernel.org>
      Cc: Jan Kara <jack@ucw.cz>
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      1dfc3220
    • J
      jbd2: fix the way the b_modified flag is cleared · 9fc7c63a
      Josef Bacik 提交于
      Currently at the start of a journal commit we loop through all of the buffers
      on the committing transaction and clear the b_modified flag (the flag that is
      set when a transaction modifies the buffer) under the j_list_lock.
      
      The problem is that everywhere else this flag is modified only under the jbd2
      lock buffer flag, so it will race with a running transaction who could
      potentially set it, and have it unset by the committing transaction.
      
      This is also a big waste, you can have several thousands of buffers that you
      are clearing the modified flag on when you may not need to.  This patch
      removes this code and instead clears the b_modified flag upon entering
      do_get_write_access/journal_get_create_access, so if that transaction does
      indeed use the buffer then it will be accounted for properly, and if it does
      not then we know we didn't use it.
      
      That will be important for the next patch in this series.  Tested thoroughly
      by myself using postmark/iozone/bonnie++.
      
      Cc: <linux-ext4@vger.kernel.org>
      Cc: Jan Kara <jack@ucw.cz>
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      9fc7c63a
  12. 10 2月, 2008 1 次提交
    • D
      JBD2: Clear buffer_ordered flag for barried IO request on success · c4e35e07
      Dave Kleikamp 提交于
      In JBD2 jbd2_journal_write_commit_record(), clear the buffer_ordered
      flag for the bh after barried IO has succeed. This prevents later, if
      the same buffer head were submitted to the underlying device, which has
      been reconfigured to not support barrier request, the JBD2 commit code
      could treat it as a normal IO (without barrier).
      
      This is a port from JBD/ext3 fix from Neil Brown.
      
      More details from Neil:
      
      Some devices - notably dm and md - can change their behaviour in
      response to BIO_RW_BARRIER requests.  They might start out accepting
      such requests but on reconfiguration, they find out that they cannot
      any more. JBD2 deal with this by always testing if BIO_RW_BARRIER
      requests fail with EOPNOTSUPP, and retrying the write
      requests without the barrier (probably after waiting for any pending
      writes to complete).
      
      However there is a bug in the handling this in JBD2 for ext4 .
      
      When ext4/JBD2 to submit a BIO_RW_BARRIER request,
      it sets the buffer_ordered flag on the buffer head.
      If the request completes successfully, the flag STAYS SET.
      
      Other code might then write the same buffer_head after the device has
      been reconfigured to not accept barriers.  This write will then fail,
      but the "other code" is not ready to handle EOPNOTSUPP errors and the
      error will be treated as fatal.
      
      Cc:  Neil Brown <neilb@suse.de>
      Signed-off-by: NDave Kleikamp <shaggy@linux.vnet.ibm.com>
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      c4e35e07
  13. 05 2月, 2008 3 次提交
  14. 30 1月, 2008 1 次提交
    • N
      spinlock: lockbreak cleanup · 95c354fe
      Nick Piggin 提交于
      The break_lock data structure and code for spinlocks is quite nasty.
      Not only does it double the size of a spinlock but it changes locking to
      a potentially less optimal trylock.
      
      Put all of that under CONFIG_GENERIC_LOCKBREAK, and introduce a
      __raw_spin_is_contended that uses the lock data itself to determine whether
      there are waiters on the lock, to be used if CONFIG_GENERIC_LOCKBREAK is
      not set.
      
      Rename need_lockbreak to spin_needbreak, make it use spin_is_contended to
      decouple it from the spinlock implementation, and make it typesafe (rwlocks
      do not have any need_lockbreak sites -- why do they even get bloated up
      with that break_lock then?).
      Signed-off-by: NNick Piggin <npiggin@suse.de>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
      95c354fe
  15. 29 1月, 2008 3 次提交
    • G
      ext4: Add the journal checksum feature · 818d276c
      Girish Shilamkar 提交于
      The journal checksum feature adds two new flags i.e
      JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT and JBD2_FEATURE_COMPAT_CHECKSUM.
      
      JBD2_FEATURE_CHECKSUM flag indicates that the commit block contains the
      checksum for the blocks described by the descriptor blocks.
      Due to checksums, writing of the commit record no longer needs to be
      synchronous. Now commit record can be sent to disk without waiting for
      descriptor blocks to be written to disk. This behavior is controlled
      using JBD2_FEATURE_ASYNC_COMMIT flag. Older kernels/e2fsck should not be
      able to recover the journal with _ASYNC_COMMIT hence it is made
      incompat.
      The commit header has been extended to hold the checksum along with the
      type of the checksum.
      
      For recovery in pass scan checksums are verified to ensure the sanity
      and completeness(in case of _ASYNC_COMMIT) of every transaction.
      Signed-off-by: NAndreas Dilger <adilger@clusterfs.com>
      Signed-off-by: NGirish Shilamkar <girish@clusterfs.com>
      Signed-off-by: NDave Kleikamp <shaggy@linux.vnet.ibm.com>
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      818d276c
    • J
      jbd2: jbd2 stats through procfs · 8e85fb3f
      Johann Lombardi 提交于
      The patch below updates the jbd stats patch to 2.6.20/jbd2.
      The initial patch was posted by Alex Tomas in December 2005
      (http://marc.info/?l=linux-ext4&m=113538565128617&w=2).
      It provides statistics via procfs such as transaction lifetime and size.
      
      Sometimes, investigating performance problems, i find useful to have
      stats from jbd about transaction's lifetime, size, etc. here is a
      patch for review and inclusion probably.
      
      for example, stats after creation of 3M files in htree directory:
      
      [root@bob ~]# cat /proc/fs/jbd/sda/history
      R/C  tid   wait  run   lock  flush log   hndls  block inlog ctime write drop  close
      R    261   8260  2720  0     0     750   9892   8170  8187
      C    259                                                    750   0     4885  1
      R    262   20    2200  10    0     770   9836   8170  8187
      R    263   30    2200  10    0     3070  9812   8170  8187
      R    264   0     5000  10    0     1340  0      0     0
      C    261                                                    8240  3212  4957  0
      R    265   8260  1470  0     0     4640  9854   8170  8187
      R    266   0     5000  10    0     1460  0      0     0
      C    262                                                    8210  2989  4868  0
      R    267   8230  1490  10    0     4440  9875   8171  8188
      R    268   0     5000  10    0     1260  0      0     0
      C    263                                                    7710  2937  4908  0
      R    269   7730  1470  10    0     3330  9841   8170  8187
      R    270   0     5000  10    0     830   0      0     0
      C    265                                                    8140  3234  4898  0
      C    267                                                    720   0     4849  1
      R    271   8630  2740  20    0     740   9819   8170  8187
      C    269                                                    800   0     4214  1
      R    272   40    2170  10    0     830   9716   8170  8187
      R    273   40    2280  0     0     3530  9799   8170  8187
      R    274   0     5000  10    0     990   0      0     0
      
      
      where,
      
      R     - line for transaction's life from T_RUNNING to T_FINISHED
      C     - line for transaction's checkpointing
      tid   - transaction's id
      wait  - for how long we were waiting for new transaction to start
               (the longest period journal_start() took in this transaction)
      run   - real transaction's lifetime (from T_RUNNING to T_LOCKED
      lock  - how long we were waiting for all handles to close
               (time the transaction was in T_LOCKED)
      flush - how long it took to flush all data (data=ordered)
      log   - how long it took to write the transaction to the log
      hndls - how many handles got to the transaction
      block - how many blocks got to the transaction
      inlog - how many blocks are written to the log (block + descriptors)
      ctime - how long it took to checkpoint the transaction
      write - how many blocks have been written during checkpointing
      drop  - how many blocks have been dropped during checkpointing
      close - how many running transactions have been closed to checkpoint this one
      
      all times are in msec.
      
      
      [root@bob ~]# cat /proc/fs/jbd/sda/info
      280 transaction, each upto 8192 blocks
      average:
        1633ms waiting for transaction
        3616ms running transaction
        5ms transaction was being locked
        1ms flushing data (in ordered mode)
        1799ms logging transaction
        11781 handles per transaction
        5629 blocks per transaction
        5641 logged blocks per transaction
      Signed-off-by: NJohann Lombardi <johann.lombardi@bull.net>
      Signed-off-by: NMariusz Kozlowski <m.kozlowski@tuxland.pl>
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      Signed-off-by: NEric Sandeen <sandeen@redhat.com>
      8e85fb3f
    • J
      jbd2: Fix assertion failure in fs/jbd2/checkpoint.c · f5a7a6b0
      Jan Kara 提交于
      Before we start committing a transaction, we call
      __journal_clean_checkpoint_list() to cleanup transaction's written-back
      buffers.
      
      If this call happens to remove all of them (and there were already some
      buffers), __journal_remove_checkpoint() will decide to free the transaction
      because it isn't (yet) a committing transaction and soon we fail some
      assertion - the transaction really isn't ready to be freed :).
      
      We change the check in __journal_remove_checkpoint() to free only a
      transaction in T_FINISHED state.  The locking there is subtle though (as
      everywhere in JBD ;().  We use j_list_lock to protect the check and a
      subsequent call to __journal_drop_transaction() and do the same in the end
      of journal_commit_transaction() which is the only place where a transaction
      can get to T_FINISHED state.
      
      Probably I'm too paranoid here and such locking is not really necessary -
      checkpoint lists are processed only from log_do_checkpoint() where a
      transaction must be already committed to be processed or from
      __journal_clean_checkpoint_list() where kjournald itself calls it and thus
      transaction cannot change state either.  Better be safe if something
      changes in future...
      Signed-off-by: NJan Kara <jack@suse.cz>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      f5a7a6b0
  16. 18 10月, 2007 3 次提交
  17. 17 7月, 2007 1 次提交
  18. 09 5月, 2007 1 次提交
  19. 08 12月, 2006 1 次提交
  20. 12 10月, 2006 5 次提交
  21. 04 10月, 2006 1 次提交
  22. 26 9月, 2006 1 次提交
    • J
      [PATCH] jbd: fix commit of ordered data buffers · 3998b930
      Jan Kara 提交于
      Original commit code assumes, that when a buffer on BJ_SyncData list is
      locked, it is being written to disk.  But this is not true and hence it can
      lead to a potential data loss on crash.  Also the code didn't count with
      the fact that journal_dirty_data() can steal buffers from committing
      transaction and hence could write buffers that no longer belong to the
      committing transaction.  Finally it could possibly happen that we tried
      writing out one buffer several times.
      
      The patch below tries to solve these problems by a complete rewrite of the
      data commit code.  We go through buffers on t_sync_datalist, lock buffers
      needing write out and store them in an array.  Buffers are also immediately
      refiled to BJ_Locked list or unfiled (if the write out is completed).  When
      the array is full or we have to block on buffer lock, we submit all
      accumulated buffers for IO.
      
      [suitable for 2.6.18.x around the 2.6.19-rc2 timeframe]
      Signed-off-by: NJan Kara <jack@suse.cz>
      Cc: Badari Pulavarty <pbadari@us.ibm.com>
      Cc: <stable@kernel.org>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      3998b930
  23. 28 8月, 2006 1 次提交
  24. 23 6月, 2006 1 次提交
    • J
      [PATCH] jbd: fix BUG in journal_commit_transaction() · 9ada7340
      Jan Kara 提交于
      Fix possible assertion failure in journal_commit_transaction() on
      jh->b_next_transaction == NULL (when we are processing BJ_Forget list and
      buffer is not jbddirty).
      
      !jbddirty buffers can be placed on BJ_Forget list for example by
      journal_forget() or by __dispose_buffer() - generally such buffer means
      that it has been freed by this transaction.
      
      Freed buffers should not be reallocated until the transaction has committed
      (that's why we have the assertion there) but they *can* be reallocated when
      the transaction has already been committed to disk and we are just
      processing the BJ_Forget list (as soon as we remove b_committed_data from
      the bitmap bh, ext3 will be able to reallocate buffers freed by the
      committing transaction).  So we have to also count with the case that the
      buffer has been reallocated and b_next_transaction has been already set.
      
      And one more subtle point: it can happen that we manage to reallocate the
      buffer and also mark it jbddirty.  Then we also add the freed buffer to the
      checkpoint list of the committing trasaction.  But that should do no harm.
      
      Non-jbddirty buffers should be filed to BJ_Reserved and not BJ_Metadata
      list.  It can actually happen that we refile such buffers during the commit
      phase when we reallocate in the running transaction blocks deleted in
      committing transaction (and that can happen if the committing transaction
      already wrote all the data and is just cleaning up BJ_Forget list).
      Signed-off-by: NJan Kara <jack@suse.cz>
      Acked-by: N"Stephen C. Tweedie" <sct@redhat.com>
      Signed-off-by: NAndrew Morton <akpm@osdl.org>
      Signed-off-by: NLinus Torvalds <torvalds@osdl.org>
      9ada7340
  25. 15 2月, 2006 1 次提交