1. 13 6月, 2013 1 次提交
  2. 05 6月, 2013 1 次提交
    • J
      jbd2: refine waiting for shadow buffers · b34090e5
      Jan Kara 提交于
      Currently when we add a buffer to a transaction, we wait until the
      buffer is removed from BJ_Shadow list (so that we prevent any changes
      to the buffer that is just written to the journal).  This can take
      unnecessarily long as a lot happens between the time the buffer is
      submitted to the journal and the time when we remove the buffer from
      BJ_Shadow list.  (e.g.  We wait for all data buffers in the
      transaction, we issue a cache flush, etc.)  Also this creates a
      dependency of do_get_write_access() on transaction commit (namely
      waiting for data IO to complete) which we want to avoid when
      implementing transaction reservation.
      
      So we modify commit code to set new BH_Shadow flag when temporary
      shadowing buffer is created and we clear that flag once IO on that
      buffer is complete.  This allows do_get_write_access() to wait only
      for BH_Shadow bit and thus removes the dependency on data IO
      completion.
      Reviewed-by: NZheng Liu <wenqing.lz@taobao.com>
      Signed-off-by: NJan Kara <jack@suse.cz>
      Signed-off-by: N"Theodore Ts'o" <tytso@mit.edu>
      b34090e5
  3. 22 5月, 2013 1 次提交
  4. 25 4月, 2013 1 次提交
  5. 16 5月, 2012 2 次提交
    • J
      jbd: Write journal superblock with WRITE_FUA after checkpointing · fd2cbd4d
      Jan Kara 提交于
      If journal superblock is written only in disk's caches and other transaction
      starts reusing space of the transaction cleaned from the log, it can happen
      blocks of a new transaction reach the disk before journal superblock. When
      power failure happens in such case, subsequent journal replay would still try
      to replay the old transaction but some of it's blocks may be already
      overwritten by the new transaction. For this reason we must use WRITE_FUA when
      updating log tail and we must first write new log tail to disk and update
      in-memory information only after that.
      Signed-off-by: NJan Kara <jack@suse.cz>
      fd2cbd4d
    • J
      jbd: Split updating of journal superblock and marking journal empty · 9754e39c
      Jan Kara 提交于
      There are three case of updating journal superblock. In the first case, we want
      to mark journal as empty (setting s_sequence to 0), in the second case we want
      to update log tail, in the third case we want to update s_errno. Split these
      cases into separate functions. It makes the code slightly more straightforward
      and later patches will make the distinction even more important.
      Signed-off-by: NJan Kara <jack@suse.cz>
      9754e39c
  6. 11 4月, 2012 1 次提交
    • J
      jbd: Refine commit writeout logic · 2db938be
      Jan Kara 提交于
      Currently we write out all journal buffers in WRITE_SYNC mode. This improves
      performance for fsync heavy workloads but hinders performance when writes
      are mostly asynchronous, most noticably it slows down readers and users
      complain about slow desktop response etc.
      
      So submit writes as asynchronous in the normal case and only submit writes as
      WRITE_SYNC if we detect someone is waiting for current transaction commit.
      
      I've gathered some numbers to back this change. The first is the read latency
      test. It measures time to read 1 MB after several seconds of sleeping in
      presence of streaming writes.
      
      Top 10 times (out of 90) in us:
      Before		After
      2131586		697473
      1709932		557487
      1564598		535642
      1480462		347573
      1478579		323153
      1408496		222181
      1388960		181273
      1329565		181070
      1252486		172832
      1223265		172278
      
      Average:
      619377		82180
      
      So the improvement in both maximum and average latency is massive.
      
      I've measured fsync throughput by:
      fs_mark -n 100 -t 1 -s 16384 -d /mnt/fsync/ -S 1 -L 4
      
      in presence of streaming reader. The numbers (fsyncs/s) are:
      Before		After
      9.9		6.3
      6.8		6.0
      6.3		6.2
      5.8		6.1
      
      So fsync performance seems unharmed by this change.
      Signed-off-by: NJan Kara <jack@suse.cz>
      2db938be
  7. 09 1月, 2012 1 次提交
    • J
      jbd: Remove j_barrier mutex · 00482785
      Jan Kara 提交于
      j_barrier mutex is used for serializing different journal lock operations.  The
      problem with it is that e.g. FIFREEZE ioctl results in process leaving kernel
      with j_barrier mutex held which makes lockdep freak out. Also hibernation code
      wants to freeze filesystem but it cannot do so because it then cannot hibernate
      the system because of mutex being locked.
      
      So we remove j_barrier mutex and use direct wait on j_barrier_count instead.
      Since locking journal is a rare operation we don't have to care about fairness
      or such things.
      
      CC: Andrew Morton <akpm@linux-foundation.org>
      Acked-by: NJoel Becker <jlbec@evilplan.org>
      Signed-off-by: NJan Kara <jack@suse.cz>
      00482785
  8. 22 11月, 2011 1 次提交
    • Y
      jbd: clear revoked flag on buffers before a new transaction started · 8c111b3f
      Yongqiang Yang 提交于
      Currently, we clear revoked flag only when a block is reused.  However,
      this can tigger a false journal error.  Consider a situation when a block
      is used as a meta block and is deleted(revoked) in ordered mode, then the
      block is allocated as a data block to a file.  At this moment, user changes
      the file's journal mode from ordered to journaled and truncates the file.
      The block will be considered re-revoked by journal because it has revoked
      flag still pending from the last transaction and an assertion triggers.
      
      We fix the problem by keeping the revoked status more uptodate - we clear
      revoked flag when switching revoke tables to reflect there is no revoked
      buffers in current transaction any more.
      Signed-off-by: NYongqiang Yang <xiaoqiangnk@gmail.com>
      Signed-off-by: NJan Kara <jack@suse.cz>
      8c111b3f
  9. 27 10月, 2011 1 次提交
  10. 27 6月, 2011 1 次提交
    • J
      jbd: Fix oops in journal_remove_journal_head() · bb189247
      Jan Kara 提交于
      journal_remove_journal_head() can oops when trying to access journal_head
      returned by bh2jh(). This is caused for example by the following race:
      
      	TASK1					TASK2
        journal_commit_transaction()
          ...
          processing t_forget list
            __journal_refile_buffer(jh);
            if (!jh->b_transaction) {
              jbd_unlock_bh_state(bh);
      					journal_try_to_free_buffers()
      					  journal_grab_journal_head(bh)
      					  jbd_lock_bh_state(bh)
      					  __journal_try_to_free_buffer()
      					  journal_put_journal_head(jh)
              journal_remove_journal_head(bh);
      
      journal_put_journal_head() in TASK2 sees that b_jcount == 0 and buffer is not
      part of any transaction and thus frees journal_head before TASK1 gets to doing
      so. Note that even buffer_head can be released by try_to_free_buffers() after
      journal_put_journal_head() which adds even larger opportunity for oops (but I
      didn't see this happen in reality).
      
      Fix the problem by making transactions hold their own journal_head reference
      (in b_jcount). That way we don't have to remove journal_head explicitely via
      journal_remove_journal_head() and instead just remove journal_head when
      b_jcount drops to zero. The result of this is that [__]journal_refile_buffer(),
      [__]journal_unfile_buffer(), and __journal_remove_checkpoint() can free
      journal_head which needs modification of a few callers. Also we have to be
      careful because once journal_head is removed, buffer_head might be freed as
      well. So we have to get our own buffer_head reference where it matters.
      Signed-off-by: NJan Kara <jack@suse.cz>
      bb189247
  11. 22 5月, 2010 1 次提交
  12. 30 3月, 2010 1 次提交
    • T
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking... · 5a0e3ad6
      Tejun Heo 提交于
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
      
      percpu.h is included by sched.h and module.h and thus ends up being
      included when building most .c files.  percpu.h includes slab.h which
      in turn includes gfp.h making everything defined by the two files
      universally available and complicating inclusion dependencies.
      
      percpu.h -> slab.h dependency is about to be removed.  Prepare for
      this change by updating users of gfp and slab facilities include those
      headers directly instead of assuming availability.  As this conversion
      needs to touch large number of source files, the following script is
      used as the basis of conversion.
      
        http://userweb.kernel.org/~tj/misc/slabh-sweep.py
      
      The script does the followings.
      
      * Scan files for gfp and slab usages and update includes such that
        only the necessary includes are there.  ie. if only gfp is used,
        gfp.h, if slab is used, slab.h.
      
      * When the script inserts a new include, it looks at the include
        blocks and try to put the new include such that its order conforms
        to its surrounding.  It's put in the include block which contains
        core kernel includes, in the same order that the rest are ordered -
        alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
        doesn't seem to be any matching order.
      
      * If the script can't find a place to put a new include (mostly
        because the file doesn't have fitting include block), it prints out
        an error message indicating which .h file needs to be added to the
        file.
      
      The conversion was done in the following steps.
      
      1. The initial automatic conversion of all .c files updated slightly
         over 4000 files, deleting around 700 includes and adding ~480 gfp.h
         and ~3000 slab.h inclusions.  The script emitted errors for ~400
         files.
      
      2. Each error was manually checked.  Some didn't need the inclusion,
         some needed manual addition while adding it to implementation .h or
         embedding .c file was more appropriate for others.  This step added
         inclusions to around 150 files.
      
      3. The script was run again and the output was compared to the edits
         from #2 to make sure no file was left behind.
      
      4. Several build tests were done and a couple of problems were fixed.
         e.g. lib/decompress_*.c used malloc/free() wrappers around slab
         APIs requiring slab.h to be added manually.
      
      5. The script was run on all .h files but without automatically
         editing them as sprinkling gfp.h and slab.h inclusions around .h
         files could easily lead to inclusion dependency hell.  Most gfp.h
         inclusion directives were ignored as stuff from gfp.h was usually
         wildly available and often used in preprocessor macros.  Each
         slab.h inclusion directive was examined and added manually as
         necessary.
      
      6. percpu.h was updated not to include slab.h.
      
      7. Build test were done on the following configurations and failures
         were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
         distributed build env didn't work with gcov compiles) and a few
         more options had to be turned off depending on archs to make things
         build (like ipr on powerpc/64 which failed due to missing writeq).
      
         * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
         * powerpc and powerpc64 SMP allmodconfig
         * sparc and sparc64 SMP allmodconfig
         * ia64 SMP allmodconfig
         * s390 SMP allmodconfig
         * alpha SMP allmodconfig
         * um on x86_64 SMP allmodconfig
      
      8. percpu.h modifications were reverted so that it could be applied as
         a separate patch and serve as bisection point.
      
      Given the fact that I had only a couple of failures from tests on step
      6, I'm fairly confident about the coverage of this conversion patch.
      If there is a breakage, it's likely to be something in one of the arch
      headers which should be easily discoverable easily on most builds of
      the specific arch.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
      5a0e3ad6
  13. 05 3月, 2010 1 次提交
  14. 23 9月, 2009 1 次提交
  15. 16 9月, 2009 1 次提交
  16. 14 4月, 2009 1 次提交
  17. 30 3月, 2009 1 次提交
  18. 28 3月, 2009 1 次提交
  19. 16 1月, 2009 1 次提交
  20. 09 1月, 2009 1 次提交
    • J
      jbd: improve fsync batching · f420d4dc
      Josef Bacik 提交于
      There is a flaw with the way jbd handles fsync batching.  If we fsync() a
      file and we were not the last person to run fsync() on this fs then we
      automatically sleep for 1 jiffie in order to wait for new writers to join
      into the transaction before forcing the commit.  The problem with this is
      that with really fast storage (ie a Clariion) the time it takes to commit
      a transaction to disk is way faster than 1 jiffie in most cases, so
      sleeping means waiting longer with nothing to do than if we just committed
      the transaction and kept going.  Ric Wheeler noticed this when using
      fs_mark with more than 1 thread, the throughput would plummet as he added
      more threads.
      
      This patch attempts to fix this problem by recording the average time in
      nanoseconds that it takes to commit a transaction to disk, and what time
      we started the transaction.  If we run an fsync() and we have been running
      for less time than it takes to commit the transaction to disk, we sleep
      for the delta amount of time and then commit to disk.  We acheive
      sub-jiffie sleeping using schedule_hrtimeout.  This means that the wait
      time is auto-tuned to the speed of the underlying disk, instead of having
      this static timeout.  I weighted the average according to somebody's
      comments (Andreas Dilger I think) in order to help normalize random
      outliers where we take way longer or way less time to commit than the
      average.  I also have a min() check in there to make sure we don't sleep
      longer than a jiffie in case our storage is super slow, this was requested
      by Andrew.
      
      I unfortunately do not have access to a Clariion, so I had to use a
      ramdisk to represent a super fast array.  I tested with a SATA drive with
      barrier=1 to make sure there was no regression with local disks, I tested
      with a 4 way multipathed Apple Xserve RAID array and of course the
      ramdisk.  I ran the following command
      
      fs_mark -d /mnt/ext3-test -s 4096 -n 2000 -D 64 -t $i
      
      where $i was 2, 4, 8, 16 and 32.  I mkfs'ed the fs each time.  Here are my
      results
      
      type	threads		with patch	without patch
      sata	2		24.6		26.3
      sata	4		49.2		48.1
      sata	8		70.1		67.0
      sata	16		104.0		94.1
      sata	32		153.6		142.7
      
      xserve	2		246.4		222.0
      xserve	4		480.0		440.8
      xserve	8		829.5		730.8
      xserve	16		1172.7		1026.9
      xserve	32		1816.3		1650.5
      
      ramdisk	2		2538.3		1745.6
      ramdisk	4		2942.3		661.9
      ramdisk	8		2882.5		999.8
      ramdisk	16		2738.7		1801.9
      ramdisk	32		2541.9		2394.0
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Cc: Andreas Dilger <adilger@sun.com>
      Cc: Arjan van de Ven <arjan@infradead.org>
      Cc: Ric Wheeler <rwheeler@redhat.com>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      f420d4dc
  21. 23 10月, 2008 1 次提交
    • H
      jbd: fix error handling for checkpoint io · 4afe9785
      Hidehiro Kawai 提交于
      When a checkpointing IO fails, current JBD code doesn't check the error
      and continue journaling.  This means latest metadata can be lost from both
      the journal and filesystem.
      
      This patch leaves the failed metadata blocks in the journal space and
      aborts journaling in the case of log_do_checkpoint().  To achieve this, we
      need to do:
      
      1. don't remove the failed buffer from the checkpoint list where in
         the case of __try_to_free_cp_buf() because it may be released or
         overwritten by a later transaction
      2. log_do_checkpoint() is the last chance, remove the failed buffer
         from the checkpoint list and abort the journal
      3. when checkpointing fails, don't update the journal super block to
         prevent the journaled contents from being cleaned.  For safety,
         don't update j_tail and j_tail_sequence either
      4. when checkpointing fails, notify this error to the ext3 layer so
         that ext3 don't clear the needs_recovery flag, otherwise the
         journaled contents are ignored and cleaned in the recovery phase
      5. if the recovery fails, keep the needs_recovery flag
      6. prevent cleanup_journal_tail() from being called between
         __journal_drop_transaction() and journal_abort() (a race issue
         between journal_flush() and __log_wait_for_space()
      Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Acked-by: NJan Kara <jack@suse.cz>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      4afe9785
  22. 20 10月, 2008 1 次提交
    • H
      ext3: add an option to control error handling on file data · 0e4fb5e2
      Hidehiro Kawai 提交于
      If the journal doesn't abort when it gets an IO error in file data blocks,
      the file data corruption will spread silently.  Because most of
      applications and commands do buffered writes without fsync(), they don't
      notice the IO error.  It's scary for mission critical systems.  On the
      other hand, if the journal aborts whenever it gets an IO error in file
      data blocks, the system will easily become inoperable.  So this patch
      introduces a filesystem option to determine whether it aborts the journal
      or just call printk() when it gets an IO error in file data.
      
      If you mount a ext3 fs with data_err=abort option, it aborts on file data
      write error.  If you mount it with data_err=ignore, it doesn't abort, just
      call printk().  data_err=ignore is the default.
      Signed-off-by: NHidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
      Cc: Jan Kara <jack@ucw.cz>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      0e4fb5e2
  23. 17 10月, 2008 1 次提交
  24. 19 4月, 2008 1 次提交
  25. 20 3月, 2008 1 次提交
    • R
      fs: fix kernel-doc notation warnings · a6b91919
      Randy Dunlap 提交于
      Fix kernel-doc notation warnings in fs/.
      
      Warning(mmotm-2008-0314-1449//fs/super.c:560): missing initial short description on line:
       *	mark_files_ro
      Warning(mmotm-2008-0314-1449//fs/locks.c:1277): missing initial short description on line:
       *	lease_get_mtime
      Warning(mmotm-2008-0314-1449//fs/locks.c:1277): missing initial short description on line:
       *	lease_get_mtime
      Warning(mmotm-2008-0314-1449//fs/namei.c:1368): missing initial short description on line:
       * lookup_one_len:  filesystem helper to lookup single pathname component
      Warning(mmotm-2008-0314-1449//fs/buffer.c:3221): missing initial short description on line:
       * bh_uptodate_or_lock: Test whether the buffer is uptodate
      Warning(mmotm-2008-0314-1449//fs/buffer.c:3240): missing initial short description on line:
       * bh_submit_read: Submit a locked buffer for reading
      Warning(mmotm-2008-0314-1449//fs/fs-writeback.c:30): missing initial short description on line:
       * writeback_acquire: attempt to get exclusive writeback access to a device
      Warning(mmotm-2008-0314-1449//fs/fs-writeback.c:47): missing initial short description on line:
       * writeback_in_progress: determine whether there is writeback in progress
      Warning(mmotm-2008-0314-1449//fs/fs-writeback.c:58): missing initial short description on line:
       * writeback_release: relinquish exclusive writeback access against a device.
      Warning(mmotm-2008-0314-1449//include/linux/jbd.h:351): contents before sections
      Warning(mmotm-2008-0314-1449//include/linux/jbd.h:561): contents before sections
      Warning(mmotm-2008-0314-1449//fs/jbd/transaction.c:1935): missing initial short description on line:
       * void journal_invalidatepage()
      Signed-off-by: NRandy Dunlap <randy.dunlap@oracle.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      a6b91919
  26. 07 2月, 2008 2 次提交
  27. 06 12月, 2007 1 次提交
    • J
      jbd: Fix assertion failure in fs/jbd/checkpoint.c · d4beaf4a
      Jan Kara 提交于
      Before we start committing a transaction, we call
      __journal_clean_checkpoint_list() to cleanup transaction's written-back
      buffers.
      
      If this call happens to remove all of them (and there were already some
      buffers), __journal_remove_checkpoint() will decide to free the transaction
      because it isn't (yet) a committing transaction and soon we fail some
      assertion - the transaction really isn't ready to be freed :).
      
      We change the check in __journal_remove_checkpoint() to free only a
      transaction in T_FINISHED state.  The locking there is subtle though (as
      everywhere in JBD ;().  We use j_list_lock to protect the check and a
      subsequent call to __journal_drop_transaction() and do the same in the end
      of journal_commit_transaction() which is the only place where a transaction
      can get to T_FINISHED state.
      
      Probably I'm too paranoid here and such locking is not really necessary -
      checkpoint lists are processed only from log_do_checkpoint() where a
      transaction must be already committed to be processed or from
      __journal_clean_checkpoint_list() where kjournald itself calls it and thus
      transaction cannot change state either.  Better be safe if something
      changes in future...
      Signed-off-by: NJan Kara <jack@suse.cz>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      d4beaf4a
  28. 20 10月, 2007 2 次提交
  29. 18 10月, 2007 2 次提交
  30. 16 10月, 2007 1 次提交
    • R
      docbook: fix filesystems content · e6716b87
      Randy Dunlap 提交于
      Fix filesystems docbook warnings.
      
      Warning(linux-2.6.23-git8//fs/debugfs/file.c:241): No description found for parameter 'name'
      Warning(linux-2.6.23-git8//fs/debugfs/file.c:241): No description found for parameter 'mode'
      Warning(linux-2.6.23-git8//fs/debugfs/file.c:241): No description found for parameter 'parent'
      Warning(linux-2.6.23-git8//fs/debugfs/file.c:241): No description found for parameter 'value'
      Warning(linux-2.6.23-git8//include/linux/jbd.h:404): No description found for parameter 'h_lockdep_map'
      Signed-off-by: NRandy Dunlap <randy.dunlap@oracle.com>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      e6716b87
  31. 12 10月, 2007 1 次提交
  32. 08 12月, 2006 2 次提交
  33. 30 9月, 2006 1 次提交
  34. 27 9月, 2006 2 次提交