1. 11 1月, 2011 1 次提交
  2. 28 10月, 2010 1 次提交
    • B
      jbd2: Fix I/O hang in jbd2_journal_release_jbd_inode · 39e3ac25
      Brian King 提交于
      This fixes a hang seen in jbd2_journal_release_jbd_inode
      on a lot of Power 6 systems running with ext4. When we get
      in the hung state, all I/O to the disk in question gets blocked
      where we stay indefinitely. Looking at the task list, I can see
      we are stuck in jbd2_journal_release_jbd_inode waiting on a
      wake up. I added some debug code to detect this scenario and
      dump additional data if we were stuck in jbd2_journal_release_jbd_inode
      for longer than 30 minutes. When it hit, I was able to see that
      i_flags was 0, suggesting we missed the wake up.
      
      This patch changes i_flags to be an unsigned long, uses bit operators
      to access it, and adds barriers around the accesses. Prior to applying
      this patch, we were regularly hitting this hang on numerous systems
      in our test environment. After applying the patch, the hangs no longer
      occur.
      Signed-off-by: NBrian King <brking@linux.vnet.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      39e3ac25
  3. 04 8月, 2010 2 次提交
  4. 02 8月, 2010 1 次提交
  5. 27 7月, 2010 1 次提交
    • T
      jbd2: Remove __GFP_NOFAIL from jbd2 layer · 47def826
      Theodore Ts'o 提交于
      __GFP_NOFAIL is going away, so add our own retry loop.  Also add
      jbd2__journal_start() and jbd2__journal_restart() which take a gfp
      mask, so that file systems can optionally (re)start transaction
      handles using GFP_KERNEL.  If they do this, then they need to be
      prepared to handle receiving an PTR_ERR(-ENOMEM) error, and be ready
      to reflect that error up to userspace.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      47def826
  6. 16 7月, 2010 1 次提交
    • J
      jbd2/ocfs2: Fix block checksumming when a buffer is used in several transactions · 13ceef09
      Jan Kara 提交于
      OCFS2 uses t_commit trigger to compute and store checksum of the just
      committed blocks. When a buffer has b_frozen_data, checksum is computed
      for it instead of b_data but this can result in an old checksum being
      written to the filesystem in the following scenario:
      
      1) transaction1 is opened
      2) handle1 is opened
      3) journal_access(handle1, bh)
          - This sets jh->b_transaction to transaction1
      4) modify(bh)
      5) journal_dirty(handle1, bh)
      6) handle1 is closed
      7) start committing transaction1, opening transaction2
      8) handle2 is opened
      9) journal_access(handle2, bh)
          - This copies off b_frozen_data to make it safe for transaction1 to commit.
            jh->b_next_transaction is set to transaction2.
      10) jbd2_journal_write_metadata() checksums b_frozen_data
      11) the journal correctly writes b_frozen_data to the disk journal
      12) handle2 is closed
          - There was no dirty call for the bh on handle2, so it is never queued for
            any more journal operation
      13) Checkpointing finally happens, and it just spools the bh via normal buffer
      writeback.  This will write b_data, which was never triggered on and thus
      contains a wrong (old) checksum.
      
      This patch fixes the problem by calling the trigger at the moment data is
      frozen for journal commit - i.e., either when b_frozen_data is created by
      do_get_write_access or just before we write a buffer to the log if
      b_frozen_data does not exist. We also rename the trigger to t_frozen as
      that better describes when it is called.
      Signed-off-by: NJan Kara <jack@suse.cz>
      Signed-off-by: NMark Fasheh <mfasheh@suse.com>
      Signed-off-by: NJoel Becker <joel.becker@oracle.com>
      13ceef09
  7. 30 3月, 2010 1 次提交
    • T
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking... · 5a0e3ad6
      Tejun Heo 提交于
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
      
      percpu.h is included by sched.h and module.h and thus ends up being
      included when building most .c files.  percpu.h includes slab.h which
      in turn includes gfp.h making everything defined by the two files
      universally available and complicating inclusion dependencies.
      
      percpu.h -> slab.h dependency is about to be removed.  Prepare for
      this change by updating users of gfp and slab facilities include those
      headers directly instead of assuming availability.  As this conversion
      needs to touch large number of source files, the following script is
      used as the basis of conversion.
      
        http://userweb.kernel.org/~tj/misc/slabh-sweep.py
      
      The script does the followings.
      
      * Scan files for gfp and slab usages and update includes such that
        only the necessary includes are there.  ie. if only gfp is used,
        gfp.h, if slab is used, slab.h.
      
      * When the script inserts a new include, it looks at the include
        blocks and try to put the new include such that its order conforms
        to its surrounding.  It's put in the include block which contains
        core kernel includes, in the same order that the rest are ordered -
        alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
        doesn't seem to be any matching order.
      
      * If the script can't find a place to put a new include (mostly
        because the file doesn't have fitting include block), it prints out
        an error message indicating which .h file needs to be added to the
        file.
      
      The conversion was done in the following steps.
      
      1. The initial automatic conversion of all .c files updated slightly
         over 4000 files, deleting around 700 includes and adding ~480 gfp.h
         and ~3000 slab.h inclusions.  The script emitted errors for ~400
         files.
      
      2. Each error was manually checked.  Some didn't need the inclusion,
         some needed manual addition while adding it to implementation .h or
         embedding .c file was more appropriate for others.  This step added
         inclusions to around 150 files.
      
      3. The script was run again and the output was compared to the edits
         from #2 to make sure no file was left behind.
      
      4. Several build tests were done and a couple of problems were fixed.
         e.g. lib/decompress_*.c used malloc/free() wrappers around slab
         APIs requiring slab.h to be added manually.
      
      5. The script was run on all .h files but without automatically
         editing them as sprinkling gfp.h and slab.h inclusions around .h
         files could easily lead to inclusion dependency hell.  Most gfp.h
         inclusion directives were ignored as stuff from gfp.h was usually
         wildly available and often used in preprocessor macros.  Each
         slab.h inclusion directive was examined and added manually as
         necessary.
      
      6. percpu.h was updated not to include slab.h.
      
      7. Build test were done on the following configurations and failures
         were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
         distributed build env didn't work with gcov compiles) and a few
         more options had to be turned off depending on archs to make things
         build (like ipr on powerpc/64 which failed due to missing writeq).
      
         * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
         * powerpc and powerpc64 SMP allmodconfig
         * sparc and sparc64 SMP allmodconfig
         * ia64 SMP allmodconfig
         * s390 SMP allmodconfig
         * alpha SMP allmodconfig
         * um on x86_64 SMP allmodconfig
      
      8. percpu.h modifications were reverted so that it could be applied as
         a separate patch and serve as bisection point.
      
      Given the fact that I had only a couple of failures from tests on step
      6, I'm fairly confident about the coverage of this conversion patch.
      If there is a breakage, it's likely to be something in one of the arch
      headers which should be easily discoverable easily on most builds of
      the specific arch.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
      5a0e3ad6
  8. 05 3月, 2010 1 次提交
  9. 23 12月, 2009 1 次提交
    • T
      ext4, jbd2: Add barriers for file systems with exernal journals · cc3e1bea
      Theodore Ts'o 提交于
      This is a bit complicated because we are trying to optimize when we
      send barriers to the fs data disk.  We could just throw in an extra
      barrier to the data disk whenever we send a barrier to the journal
      disk, but that's not always strictly necessary.
      
      We only need to send a barrier during a commit when there are data
      blocks which are must be written out due to an inode written in
      ordered mode, or if fsync() depends on the commit to force data blocks
      to disk.  Finally, before we drop transactions from the beginning of
      the journal during a checkpoint operation, we need to guarantee that
      any blocks that were flushed out to the data disk are firmly on the
      rust platter before we drop the transaction from the journal.
      
      Thanks to Oleg Drokin for pointing out this flaw in ext3/ext4.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      cc3e1bea
  10. 07 12月, 2009 1 次提交
    • T
      ext4: Use slab allocator for sub-page sized allocations · d2eecb03
      Theodore Ts'o 提交于
      Now that the SLUB seems to be fixed so that it respects the requested
      alignment, use kmem_cache_alloc() to allocator if the block size of
      the buffer heads to be allocated is less than the page size.
      Previously, we were using 16k page on a Power system for each buffer,
      even when the file system was using 1k or 4k block size.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      
      d2eecb03
  11. 30 9月, 2009 1 次提交
  12. 18 8月, 2009 1 次提交
  13. 17 6月, 2009 1 次提交
  14. 14 4月, 2009 1 次提交
  15. 26 3月, 2009 1 次提交
  16. 11 2月, 2009 1 次提交
    • J
      jbd2: Avoid possible NULL dereference in jbd2_journal_begin_ordered_truncate() · 7f5aa215
      Jan Kara 提交于
      If we race with commit code setting i_transaction to NULL, we could
      possibly dereference it.  Proper locking requires the journal pointer
      (to access journal->j_list_lock), which we don't have.  So we have to
      change the prototype of the function so that filesystem passes us the
      journal pointer.  Also add a more detailed comment about why the
      function jbd2_journal_begin_ordered_truncate() does what it does and
      how it should be used.
      
      Thanks to Dan Carpenter <error27@gmail.com> for pointing to the
      suspitious code.
      Signed-off-by: NJan Kara <jack@suse.cz>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      Acked-by: NJoel Becker <joel.becker@oracle.com>
      CC: linux-ext4@vger.kernel.org
      CC: ocfs2-devel@oss.oracle.com
      CC: mfasheh@suse.de
      CC: Dan Carpenter <error27@gmail.com>
      7f5aa215
  17. 17 1月, 2009 1 次提交
  18. 06 1月, 2009 2 次提交
    • J
      jbd2: Add buffer triggers · e06c8227
      Joel Becker 提交于
      Filesystems often to do compute intensive operation on some
      metadata.  If this operation is repeated many times, it can be very
      expensive.  It would be much nicer if the operation could be performed
      once before a buffer goes to disk.
      
      This adds triggers to jbd2 buffer heads.  Just before writing a metadata
      buffer to the journal, jbd2 will optionally call a commit trigger associated
      with the buffer.  If the journal is aborted, an abort trigger will be
      called on any dirty buffers as they are dropped from pending
      transactions.
      
      ocfs2 will use this feature.
      
      Initially I tried to come up with a more generic trigger that could be
      used for non-buffer-related events like transaction completion.  It
      doesn't tie nicely, because the information a buffer trigger needs
      (specific to a journal_head) isn't the same as what a transaction
      trigger needs (specific to a tranaction_t or perhaps journal_t).  So I
      implemented a buffer set, with the understanding that
      journal/transaction wide triggers should be implemented separately.
      
      There is only one trigger set allowed per buffer.  I can't think of any
      reason to attach more than one set.  Contrast this with a journal or
      transaction in which multiple places may want to watch the entire
      transaction separately.
      
      The trigger sets are considered static allocation from the jbd2
      perspective.  ocfs2 will just have one trigger set per block type,
      setting the same set on every bh of the same type.
      Signed-off-by: NJoel Becker <joel.becker@oracle.com>
      Cc: "Theodore Ts'o" <tytso@mit.edu>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NMark Fasheh <mfasheh@suse.com>
      e06c8227
    • M
      jbd2: Add BH_JBDPrivateStart · e97fcd95
      Mark Fasheh 提交于
      Add this so that file systems using JBD2 can safely allocate unused b_state
      bits.
      
      In this case, we add it so that Ocfs2 can define a single bit for tracking
      the validation state of a buffer.
      Acked-by: N"Theodore Ts'o" <tytso@mit.edu>
      Signed-off-by: NMark Fasheh <mfasheh@suse.com>
      e97fcd95
  19. 07 1月, 2009 1 次提交
  20. 07 11月, 2008 1 次提交
  21. 05 11月, 2008 1 次提交
  22. 04 1月, 2009 1 次提交
  23. 26 11月, 2008 2 次提交
    • J
      jbd2: improve jbd2 fsync batching · e07f7183
      Josef Bacik 提交于
      This patch removes the static sleep time in favor of a more self
      optimizing approach where we measure the average amount of time it
      takes to commit a transaction to disk and the ammount of time a
      transaction has been running.  If somebody does a sync write or an
      fsync() traditionally we would sleep for 1 jiffies, which depending on
      the value of HZ could be a significant amount of time compared to how
      long it takes to commit a transaction to the underlying storage.  With
      this patch instead of sleeping for a jiffie, we check to see if the
      amount of time this transaction has been running is less than the
      average commit time, and if it is we sleep for the delta using
      schedule_hrtimeout to give us a higher precision sleep time.  This
      greatly benefits high end storage where you could end up sleeping for
      longer than it takes to commit the transaction and therefore sitting
      idle instead of allowing the transaction to be committed by keeping
      the sleep time to a minimum so you are sure to always be doing
      something.
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      e07f7183
    • M
      jbd2: Add BH_JBDPrivateStart · 171bbfbe
      Mark Fasheh 提交于
      Add this so that file systems using JBD2 can safely allocate unused b_state
      bits.
      
      In this case, we add it so that Ocfs2 can define a single bit for tracking
      the validation state of a buffer.
      Signed-off-by: NMark Fasheh <mfasheh@suse.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      171bbfbe
  24. 17 10月, 2008 2 次提交
  25. 11 10月, 2008 2 次提交
    • H
      ext4: add an option to control error handling on file data · 5bf5683a
      Hidehiro Kawai 提交于
      If the journal doesn't abort when it gets an IO error in file data
      blocks, the file data corruption will spread silently.  Because
      most of applications and commands do buffered writes without fsync(),
      they don't notice the IO error.  It's scary for mission critical
      systems.  On the other hand, if the journal aborts whenever it gets
      an IO error in file data blocks, the system will easily become
      inoperable.  So this patch introduces a filesystem option to
      determine whether it aborts the journal or just call printk() when
      it gets an IO error in file data.
      
      If you mount an ext4 fs with data_err=abort option, it aborts on file
      data write error.  If you mount it with data_err=ignore, it doesn't
      abort, just call printk().  data_err=ignore is the default.
      
      Here is the corresponding patch of the ext3 version:
      http://kerneltrap.org/mailarchive/linux-kernel/2008/9/9/3239374Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      5bf5683a
    • H
      jbd2: fix error handling for checkpoint io · 44519faf
      Hidehiro Kawai 提交于
      When a checkpointing IO fails, current JBD2 code doesn't check the
      error and continue journaling.  This means latest metadata can be
      lost from both the journal and filesystem.
      
      This patch leaves the failed metadata blocks in the journal space
      and aborts journaling in the case of jbd2_log_do_checkpoint().
      To achieve this, we need to do:
      
      1. don't remove the failed buffer from the checkpoint list where in
         the case of __try_to_free_cp_buf() because it may be released or
         overwritten by a later transaction
      2. jbd2_log_do_checkpoint() is the last chance, remove the failed
         buffer from the checkpoint list and abort the journal
      3. when checkpointing fails, don't update the journal super block to
         prevent the journaled contents from being cleaned.  For safety,
         don't update j_tail and j_tail_sequence either
      4. when checkpointing fails, notify this error to the ext4 layer so
         that ext4 don't clear the needs_recovery flag, otherwise the
         journaled contents are ignored and cleaned in the recovery phase
      5. if the recovery fails, keep the needs_recovery flag
      6. prevent jbd2_cleanup_journal_tail() from being called between
         __jbd2_journal_drop_transaction() and jbd2_journal_abort()
         (a possible race issue between jbd2_log_do_checkpoint()s called by
         jbd2_journal_flush() and __jbd2_log_wait_for_space())
      Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      44519faf
  26. 17 9月, 2008 1 次提交
    • T
      jbd2: clean up how the journal device name is printed · 05496769
      Theodore Ts'o 提交于
      Calculate the journal device name once and stash it away in the
      journal_s structure.  This avoids needing to call bdevname()
      everywhere and reduces stack usage by not needing to allocate an
      on-stack buffer.  In addition, we eliminate the '/' that can appear in
      device names (e.g. "cciss/c0d0p9" --- see kernel bugzilla #11321) that
      can cause problems when creating proc directory names, and include the
      inode number to support ocfs2 which creates multiple journals with
      different inode numbers.
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      05496769
  27. 12 7月, 2008 3 次提交
  28. 07 6月, 2008 1 次提交
  29. 19 4月, 2008 1 次提交
  30. 29 1月, 2008 4 次提交
    • M
      jbd2: add lockdep support · 7b751066
      Mingming Cao 提交于
      Ported from similar patch for the jbd layer.
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      7b751066
    • G
      ext4: Add the journal checksum feature · 818d276c
      Girish Shilamkar 提交于
      The journal checksum feature adds two new flags i.e
      JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT and JBD2_FEATURE_COMPAT_CHECKSUM.
      
      JBD2_FEATURE_CHECKSUM flag indicates that the commit block contains the
      checksum for the blocks described by the descriptor blocks.
      Due to checksums, writing of the commit record no longer needs to be
      synchronous. Now commit record can be sent to disk without waiting for
      descriptor blocks to be written to disk. This behavior is controlled
      using JBD2_FEATURE_ASYNC_COMMIT flag. Older kernels/e2fsck should not be
      able to recover the journal with _ASYNC_COMMIT hence it is made
      incompat.
      The commit header has been extended to hold the checksum along with the
      type of the checksum.
      
      For recovery in pass scan checksums are verified to ensure the sanity
      and completeness(in case of _ASYNC_COMMIT) of every transaction.
      Signed-off-by: NAndreas Dilger <adilger@clusterfs.com>
      Signed-off-by: NGirish Shilamkar <girish@clusterfs.com>
      Signed-off-by: NDave Kleikamp <shaggy@linux.vnet.ibm.com>
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      818d276c
    • J
      jbd2: jbd2 stats through procfs · 8e85fb3f
      Johann Lombardi 提交于
      The patch below updates the jbd stats patch to 2.6.20/jbd2.
      The initial patch was posted by Alex Tomas in December 2005
      (http://marc.info/?l=linux-ext4&m=113538565128617&w=2).
      It provides statistics via procfs such as transaction lifetime and size.
      
      Sometimes, investigating performance problems, i find useful to have
      stats from jbd about transaction's lifetime, size, etc. here is a
      patch for review and inclusion probably.
      
      for example, stats after creation of 3M files in htree directory:
      
      [root@bob ~]# cat /proc/fs/jbd/sda/history
      R/C  tid   wait  run   lock  flush log   hndls  block inlog ctime write drop  close
      R    261   8260  2720  0     0     750   9892   8170  8187
      C    259                                                    750   0     4885  1
      R    262   20    2200  10    0     770   9836   8170  8187
      R    263   30    2200  10    0     3070  9812   8170  8187
      R    264   0     5000  10    0     1340  0      0     0
      C    261                                                    8240  3212  4957  0
      R    265   8260  1470  0     0     4640  9854   8170  8187
      R    266   0     5000  10    0     1460  0      0     0
      C    262                                                    8210  2989  4868  0
      R    267   8230  1490  10    0     4440  9875   8171  8188
      R    268   0     5000  10    0     1260  0      0     0
      C    263                                                    7710  2937  4908  0
      R    269   7730  1470  10    0     3330  9841   8170  8187
      R    270   0     5000  10    0     830   0      0     0
      C    265                                                    8140  3234  4898  0
      C    267                                                    720   0     4849  1
      R    271   8630  2740  20    0     740   9819   8170  8187
      C    269                                                    800   0     4214  1
      R    272   40    2170  10    0     830   9716   8170  8187
      R    273   40    2280  0     0     3530  9799   8170  8187
      R    274   0     5000  10    0     990   0      0     0
      
      
      where,
      
      R     - line for transaction's life from T_RUNNING to T_FINISHED
      C     - line for transaction's checkpointing
      tid   - transaction's id
      wait  - for how long we were waiting for new transaction to start
               (the longest period journal_start() took in this transaction)
      run   - real transaction's lifetime (from T_RUNNING to T_LOCKED
      lock  - how long we were waiting for all handles to close
               (time the transaction was in T_LOCKED)
      flush - how long it took to flush all data (data=ordered)
      log   - how long it took to write the transaction to the log
      hndls - how many handles got to the transaction
      block - how many blocks got to the transaction
      inlog - how many blocks are written to the log (block + descriptors)
      ctime - how long it took to checkpoint the transaction
      write - how many blocks have been written during checkpointing
      drop  - how many blocks have been dropped during checkpointing
      close - how many running transactions have been closed to checkpoint this one
      
      all times are in msec.
      
      
      [root@bob ~]# cat /proc/fs/jbd/sda/info
      280 transaction, each upto 8192 blocks
      average:
        1633ms waiting for transaction
        3616ms running transaction
        5ms transaction was being locked
        1ms flushing data (in ordered mode)
        1799ms logging transaction
        11781 handles per transaction
        5629 blocks per transaction
        5641 logged blocks per transaction
      Signed-off-by: NJohann Lombardi <johann.lombardi@bull.net>
      Signed-off-by: NMariusz Kozlowski <m.kozlowski@tuxland.pl>
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      Signed-off-by: NEric Sandeen <sandeen@redhat.com>
      8e85fb3f
    • J
      jbd2: Fix assertion failure in fs/jbd2/checkpoint.c · f5a7a6b0
      Jan Kara 提交于
      Before we start committing a transaction, we call
      __journal_clean_checkpoint_list() to cleanup transaction's written-back
      buffers.
      
      If this call happens to remove all of them (and there were already some
      buffers), __journal_remove_checkpoint() will decide to free the transaction
      because it isn't (yet) a committing transaction and soon we fail some
      assertion - the transaction really isn't ready to be freed :).
      
      We change the check in __journal_remove_checkpoint() to free only a
      transaction in T_FINISHED state.  The locking there is subtle though (as
      everywhere in JBD ;().  We use j_list_lock to protect the check and a
      subsequent call to __journal_drop_transaction() and do the same in the end
      of journal_commit_transaction() which is the only place where a transaction
      can get to T_FINISHED state.
      
      Probably I'm too paranoid here and such locking is not really necessary -
      checkpoint lists are processed only from log_do_checkpoint() where a
      transaction must be already committed to be processed or from
      __journal_clean_checkpoint_list() where kjournald itself calls it and thus
      transaction cannot change state either.  Better be safe if something
      changes in future...
      Signed-off-by: NJan Kara <jack@suse.cz>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      f5a7a6b0