1. 12 9月, 2012 1 次提交
    • J
      jbd: Fix assertion failure in commit code due to lacking transaction credits · 09e05d48
      Jan Kara 提交于
      ext3 users of data=journal mode with blocksize < pagesize were occasionally
      hitting assertion failure in journal_commit_transaction() checking whether the
      transaction has at least as many credits reserved as buffers attached.  The
      core of the problem is that when a file gets truncated, buffers that still need
      checkpointing or that are attached to the committing transaction are left with
      buffer_mapped set. When this happens to buffers beyond i_size attached to a
      page stradding i_size, subsequent write extending the file will see these
      buffers and as they are mapped (but underlying blocks were freed) things go
      awry from here.
      
      The assertion failure just coincidentally (and in this case luckily as we would
      start corrupting filesystem) triggers due to journal_head not being properly
      cleaned up as well.
      
      Under some rare circumstances this bug could even hit data=ordered mode users.
      There the assertion won't trigger and we would end up corrupting the
      filesystem.
      
      We fix the problem by unmapping buffers if possible (in lots of cases we just
      need a buffer attached to a transaction as a place holder but it must not be
      written out anyway). And in one case, we just have to bite the bullet and wait
      for transaction commit to finish.
      Reviewed-by: NJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: NJan Kara <jack@suse.cz>
      09e05d48
  2. 11 4月, 2012 1 次提交
    • J
      jbd: Refine commit writeout logic · 2db938be
      Jan Kara 提交于
      Currently we write out all journal buffers in WRITE_SYNC mode. This improves
      performance for fsync heavy workloads but hinders performance when writes
      are mostly asynchronous, most noticably it slows down readers and users
      complain about slow desktop response etc.
      
      So submit writes as asynchronous in the normal case and only submit writes as
      WRITE_SYNC if we detect someone is waiting for current transaction commit.
      
      I've gathered some numbers to back this change. The first is the read latency
      test. It measures time to read 1 MB after several seconds of sleeping in
      presence of streaming writes.
      
      Top 10 times (out of 90) in us:
      Before		After
      2131586		697473
      1709932		557487
      1564598		535642
      1480462		347573
      1478579		323153
      1408496		222181
      1388960		181273
      1329565		181070
      1252486		172832
      1223265		172278
      
      Average:
      619377		82180
      
      So the improvement in both maximum and average latency is massive.
      
      I've measured fsync throughput by:
      fs_mark -n 100 -t 1 -s 16384 -d /mnt/fsync/ -S 1 -L 4
      
      in presence of streaming reader. The numbers (fsyncs/s) are:
      Before		After
      9.9		6.3
      6.8		6.0
      6.3		6.2
      5.8		6.1
      
      So fsync performance seems unharmed by this change.
      Signed-off-by: NJan Kara <jack@suse.cz>
      2db938be
  3. 20 3月, 2012 1 次提交
  4. 09 1月, 2012 1 次提交
    • J
      jbd: Remove j_barrier mutex · 00482785
      Jan Kara 提交于
      j_barrier mutex is used for serializing different journal lock operations.  The
      problem with it is that e.g. FIFREEZE ioctl results in process leaving kernel
      with j_barrier mutex held which makes lockdep freak out. Also hibernation code
      wants to freeze filesystem but it cannot do so because it then cannot hibernate
      the system because of mutex being locked.
      
      So we remove j_barrier mutex and use direct wait on j_barrier_count instead.
      Since locking journal is a rare operation we don't have to care about fairness
      or such things.
      
      CC: Andrew Morton <akpm@linux-foundation.org>
      Acked-by: NJoel Becker <jlbec@evilplan.org>
      Signed-off-by: NJan Kara <jack@suse.cz>
      00482785
  5. 27 6月, 2011 1 次提交
    • J
      jbd: Fix oops in journal_remove_journal_head() · bb189247
      Jan Kara 提交于
      journal_remove_journal_head() can oops when trying to access journal_head
      returned by bh2jh(). This is caused for example by the following race:
      
      	TASK1					TASK2
        journal_commit_transaction()
          ...
          processing t_forget list
            __journal_refile_buffer(jh);
            if (!jh->b_transaction) {
              jbd_unlock_bh_state(bh);
      					journal_try_to_free_buffers()
      					  journal_grab_journal_head(bh)
      					  jbd_lock_bh_state(bh)
      					  __journal_try_to_free_buffer()
      					  journal_put_journal_head(jh)
              journal_remove_journal_head(bh);
      
      journal_put_journal_head() in TASK2 sees that b_jcount == 0 and buffer is not
      part of any transaction and thus frees journal_head before TASK1 gets to doing
      so. Note that even buffer_head can be released by try_to_free_buffers() after
      journal_put_journal_head() which adds even larger opportunity for oops (but I
      didn't see this happen in reality).
      
      Fix the problem by making transactions hold their own journal_head reference
      (in b_jcount). That way we don't have to remove journal_head explicitely via
      journal_remove_journal_head() and instead just remove journal_head when
      b_jcount drops to zero. The result of this is that [__]journal_refile_buffer(),
      [__]journal_unfile_buffer(), and __journal_remove_checkpoint() can free
      journal_head which needs modification of a few callers. Also we have to be
      careful because once journal_head is removed, buffer_head might be freed as
      well. So we have to get our own buffer_head reference where it matters.
      Signed-off-by: NJan Kara <jack@suse.cz>
      bb189247
  6. 25 6月, 2011 2 次提交
    • D
      jbd: fix a bug of leaking jh->b_jcount · bd5c9e18
      Ding Dinghua 提交于
      journal_get_create_access should drop jh->b_jcount in error handling path
      Signed-off-by: NDing Dinghua <dingdinghua@nrchpc.ac.cn>
      Signed-off-by: NJan Kara <jack@suse.cz>
      bd5c9e18
    • J
      jbd: remove dependency on __GFP_NOFAIL · 05713082
      Jan Kara 提交于
      The callers of start_this_handle() (or better ext3_journal_start()) are not
      really prepared to handle allocation failures. Such failures can for example
      result in silent data loss when it happens in ext3_..._writepage().  OTOH
      __GFP_NOFAIL is going away so we just retry allocation in start_this_handle().
      
      This loop is potentially dangerous because the oom killer cannot be invoked
      for GFP_NOFS allocation, so there is a potential for infinitely looping.
      But still this is better than silent data loss.
      Signed-off-by: NJan Kara <jack@suse.cz>
      05713082
  7. 24 5月, 2011 1 次提交
  8. 31 3月, 2011 1 次提交
  9. 10 12月, 2010 1 次提交
  10. 28 10月, 2010 3 次提交
  11. 05 3月, 2010 1 次提交
    • J
      jbd: Delay discarding buffers in journal_unmap_buffer · 86963918
      Jan Kara 提交于
      Delay discarding buffers in journal_unmap_buffer until
      we know that "add to orphan" operation has definitely been
      committed, otherwise the log space of committing transation
      may be freed and reused before truncate get committed, updates
      may get lost if crash happens.
      
      This patch is a backport of JBD2 fix by dingdinghua <dingdinghua@nrchpc.ac.cn>.
      Signed-off-by: NJan Kara <jack@suse.cz>
      86963918
  12. 09 2月, 2010 1 次提交
  13. 16 9月, 2009 2 次提交
  14. 16 7月, 2009 1 次提交
    • J
      jbd: Fix a race between checkpointing code and journal_get_write_access() · 1e9fd53b
      Jan Kara 提交于
      The following race can happen:
      
        CPU1                          CPU2
                                      checkpointing code checks the buffer, adds
                                        it to an array for writeback
      do_get_write_access()
        ...
        lock_buffer()
        unlock_buffer()
                                        flush_batch() submits the buffer for IO
        __jbd_journal_file_buffer()
      
        So a buffer under writeout is returned from do_get_write_access(). Since
      the filesystem code relies on the fact that journaled buffers cannot be
      written out, it does not take the buffer lock and so it can modify buffer
      while it is under writeout. That can lead to a filesystem corruption
      if we crash at the right moment. The similar problem can happen with
      the journal_get_create_access() path.
        We fix the problem by clearing the buffer dirty bit under buffer_lock
      even if the buffer is on BJ_None list. Actually, we clear the dirty bit
      regardless the list the buffer is in and warn about the fact if
      the buffer is already journalled.
      
      Thanks for spotting the problem goes to dingdinghua <dingdinghua85@gmail.com>.
      Reported-by: Ndingdinghua <dingdinghua85@gmail.com>
      Signed-off-by: NJan Kara <jack@suse.cz>
      1e9fd53b
  15. 19 6月, 2009 1 次提交
  16. 28 3月, 2009 1 次提交
  17. 09 1月, 2009 2 次提交
    • R
      jbd: remove excess kernel-doc notation · 1579c3a1
      Randy Dunlap 提交于
      Remove excess kernel-doc from fs/jbd/transaction.c:
      
      Warning(linux-2.6.28-git5//fs/jbd/transaction.c:764): Excess function parameter 'credits' description in 'journal_get_write_access'
      Signed-off-by: NRandy Dunlap <randy.dunlap@oracle.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      1579c3a1
    • J
      jbd: improve fsync batching · f420d4dc
      Josef Bacik 提交于
      There is a flaw with the way jbd handles fsync batching.  If we fsync() a
      file and we were not the last person to run fsync() on this fs then we
      automatically sleep for 1 jiffie in order to wait for new writers to join
      into the transaction before forcing the commit.  The problem with this is
      that with really fast storage (ie a Clariion) the time it takes to commit
      a transaction to disk is way faster than 1 jiffie in most cases, so
      sleeping means waiting longer with nothing to do than if we just committed
      the transaction and kept going.  Ric Wheeler noticed this when using
      fs_mark with more than 1 thread, the throughput would plummet as he added
      more threads.
      
      This patch attempts to fix this problem by recording the average time in
      nanoseconds that it takes to commit a transaction to disk, and what time
      we started the transaction.  If we run an fsync() and we have been running
      for less time than it takes to commit the transaction to disk, we sleep
      for the delta amount of time and then commit to disk.  We acheive
      sub-jiffie sleeping using schedule_hrtimeout.  This means that the wait
      time is auto-tuned to the speed of the underlying disk, instead of having
      this static timeout.  I weighted the average according to somebody's
      comments (Andreas Dilger I think) in order to help normalize random
      outliers where we take way longer or way less time to commit than the
      average.  I also have a min() check in there to make sure we don't sleep
      longer than a jiffie in case our storage is super slow, this was requested
      by Andrew.
      
      I unfortunately do not have access to a Clariion, so I had to use a
      ramdisk to represent a super fast array.  I tested with a SATA drive with
      barrier=1 to make sure there was no regression with local disks, I tested
      with a 4 way multipathed Apple Xserve RAID array and of course the
      ramdisk.  I ran the following command
      
      fs_mark -d /mnt/ext3-test -s 4096 -n 2000 -D 64 -t $i
      
      where $i was 2, 4, 8, 16 and 32.  I mkfs'ed the fs each time.  Here are my
      results
      
      type	threads		with patch	without patch
      sata	2		24.6		26.3
      sata	4		49.2		48.1
      sata	8		70.1		67.0
      sata	16		104.0		94.1
      sata	32		153.6		142.7
      
      xserve	2		246.4		222.0
      xserve	4		480.0		440.8
      xserve	8		829.5		730.8
      xserve	16		1172.7		1026.9
      xserve	32		1816.3		1650.5
      
      ramdisk	2		2538.3		1745.6
      ramdisk	4		2942.3		661.9
      ramdisk	8		2882.5		999.8
      ramdisk	16		2738.7		1801.9
      ramdisk	32		2541.9		2394.0
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Cc: Andreas Dilger <adilger@sun.com>
      Cc: Arjan van de Ven <arjan@infradead.org>
      Cc: Ric Wheeler <rwheeler@redhat.com>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      f420d4dc
  18. 31 10月, 2008 1 次提交
  19. 20 10月, 2008 1 次提交
  20. 11 8月, 2008 2 次提交
  21. 26 7月, 2008 1 次提交
    • M
      jbd: fix race between free buffer and commit transaction · 3f31fddf
      Mingming Cao 提交于
      journal_try_to_free_buffers() could race with jbd commit transaction when
      the later is holding the buffer reference while waiting for the data
      buffer to flush to disk.  If the caller of journal_try_to_free_buffers()
      request tries hard to release the buffers, it will treat the failure as
      error and return back to the caller.  We have seen the directo IO failed
      due to this race.  Some of the caller of releasepage() also expecting the
      buffer to be dropped when passed with GFP_KERNEL mask to the
      releasepage()->journal_try_to_free_buffers().
      
      With this patch, if the caller is passing the __GFP_WAIT and __GFP_FS to
      indicating this call could wait, in case of try_to_free_buffers() failed,
      let's waiting for journal_commit_transaction() to finish commit the
      current committing transaction, then try to free those buffers again.
      
      [akpm@linux-foundation.org: coding-style fixes]
      Signed-off-by: NMingming Cao <cmm@us.ibm.com>
      Reviewed-by: NBadari Pulavarty <pbadari@us.ibm.com>
      Acked-by: NJan Kara <jack@suse.cz>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      3f31fddf
  22. 28 4月, 2008 3 次提交
    • H
      jbd: replace remaining __FUNCTION__ occurrences · 08fc99bf
      Harvey Harrison 提交于
      __FUNCTION__ is gcc-specific, use __func__
      Signed-off-by: NHarvey Harrison <harvey.harrison@gmail.com>
      Cc: <linux-ext4@vger.kernel.org>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      08fc99bf
    • J
      jbd: fix possible journal overflow issues · 5b9a499d
      Josef Bacik 提交于
      There are several cases where the running transaction can get buffers added to
      its BJ_Metadata list which it never dirtied, which makes its t_nr_buffers
      counter end up larger than its t_outstanding_credits counter.
      
      This will cause issues when starting new transactions as while we are logging
      buffers we decrement t_outstanding_buffers, so when t_outstanding_buffers goes
      negative, we will report that we need less space in the journal than we
      actually need, so transactions will be started even though there may not be
      enough room for them.  In the worst case scenario (which admittedly is almost
      impossible to reproduce) this will result in the journal running out of space.
      
      The fix is to only
      refile buffers from the committing transaction to the running transactions
      BJ_Modified list when b_modified is set on that journal, which is the only way
      to be sure if the running transaction has modified that buffer.
      
      This patch also fixes an accounting error in journal_forget, it is possible
      that we can call journal_forget on a buffer without having modified it, only
      gotten write access to it, so instead of freeing a credit, we only do so if
      the buffer was modified.  The assert will help catch if this problem occurs.
      Without these two patches I could hit this assert within minutes of running
      postmark, with them this issue no longer arises.  Thank you,
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Cc: <linux-ext4@vger.kernel.org>
      Acked-by: NJan Kara <jack@ucw.cz>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      5b9a499d
    • J
      jbd: fix the way the b_modified flag is cleared · 5bc833fe
      Josef Bacik 提交于
      Currently at the start of a journal commit we loop through all of the buffers
      on the committing transaction and clear the b_modified flag (the flag that is
      set when a transaction modifies the buffer) under the j_list_lock.
      
      The problem is that everywhere else this flag is modified only under the jbd
      lock buffer flag, so it will race with a running transaction who could
      potentially set it, and have it unset by the committing transaction.
      
      This is also a big waste, you can have several thousands of buffers that you
      are clearing the modified flag on when you may not need to.  This patch
      removes this code and instead clears the b_modified flag upon entering
      do_get_write_access/journal_get_create_access, so if that transaction does
      indeed use the buffer then it will be accounted for properly, and if it does
      not then we know we didn't use it.
      
      That will be important for the next patch in this series.  Tested thoroughly
      by myself using postmark/iozone/bonnie++.
      Signed-off-by: NJosef Bacik <jbacik@redhat.com>
      Cc: <linux-ext4@vger.kernel.org>
      Acked-by: NJan Kara <jack@ucw.cz>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      5bc833fe
  23. 20 3月, 2008 2 次提交
    • R
      fs: fix kernel-doc notation warnings · a6b91919
      Randy Dunlap 提交于
      Fix kernel-doc notation warnings in fs/.
      
      Warning(mmotm-2008-0314-1449//fs/super.c:560): missing initial short description on line:
       *	mark_files_ro
      Warning(mmotm-2008-0314-1449//fs/locks.c:1277): missing initial short description on line:
       *	lease_get_mtime
      Warning(mmotm-2008-0314-1449//fs/locks.c:1277): missing initial short description on line:
       *	lease_get_mtime
      Warning(mmotm-2008-0314-1449//fs/namei.c:1368): missing initial short description on line:
       * lookup_one_len:  filesystem helper to lookup single pathname component
      Warning(mmotm-2008-0314-1449//fs/buffer.c:3221): missing initial short description on line:
       * bh_uptodate_or_lock: Test whether the buffer is uptodate
      Warning(mmotm-2008-0314-1449//fs/buffer.c:3240): missing initial short description on line:
       * bh_submit_read: Submit a locked buffer for reading
      Warning(mmotm-2008-0314-1449//fs/fs-writeback.c:30): missing initial short description on line:
       * writeback_acquire: attempt to get exclusive writeback access to a device
      Warning(mmotm-2008-0314-1449//fs/fs-writeback.c:47): missing initial short description on line:
       * writeback_in_progress: determine whether there is writeback in progress
      Warning(mmotm-2008-0314-1449//fs/fs-writeback.c:58): missing initial short description on line:
       * writeback_release: relinquish exclusive writeback access against a device.
      Warning(mmotm-2008-0314-1449//include/linux/jbd.h:351): contents before sections
      Warning(mmotm-2008-0314-1449//include/linux/jbd.h:561): contents before sections
      Warning(mmotm-2008-0314-1449//fs/jbd/transaction.c:1935): missing initial short description on line:
       * void journal_invalidatepage()
      Signed-off-by: NRandy Dunlap <randy.dunlap@oracle.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      a6b91919
    • R
      jbd: fix jbd kernel-doc notation · 0cf01f66
      Randy Dunlap 提交于
      Fix kernel-doc notation in jbd.
      Signed-off-by: NRandy Dunlap <randy.dunlap@oracle.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      0cf01f66
  24. 04 3月, 2008 1 次提交
  25. 18 1月, 2008 1 次提交
  26. 20 10月, 2007 1 次提交
  27. 19 10月, 2007 1 次提交
  28. 18 10月, 2007 2 次提交
  29. 12 10月, 2007 1 次提交
  30. 09 5月, 2007 1 次提交