1. 18 8月, 2017 1 次提交
  2. 31 7月, 2017 1 次提交
  3. 06 7月, 2017 1 次提交
    • T
      ext4: fix __ext4_new_inode() journal credits calculation · af65207c
      Tahsin Erdogan 提交于
      ea_inode feature allows creating extended attributes that are up to
      64k in size. Update __ext4_new_inode() to pick increased credit limits.
      
      To avoid overallocating too many journal credits, update
      __ext4_xattr_set_credits() to make a distinction between xattr create
      vs update. This helps __ext4_new_inode() because all attributes are
      known to be new, so we can save credits that are normally needed to
      delete old values.
      
      Also, have fscrypt specify its maximum context size so that we don't
      end up allocating credits for 64k size.
      Signed-off-by: NTahsin Erdogan <tahsin@google.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      af65207c
  4. 24 6月, 2017 1 次提交
  5. 23 6月, 2017 2 次提交
    • E
      ext4: forbid encrypting root directory · 9ce0151a
      Eric Biggers 提交于
      Currently it's possible to encrypt all files and directories on an ext4
      filesystem by deleting everything, including lost+found, then setting an
      encryption policy on the root directory.  However, this is incompatible
      with e2fsck because e2fsck expects to find, create, and/or write to
      lost+found and does not have access to any encryption keys.  Especially
      problematic is that if e2fsck can't find lost+found, it will create it
      without regard for whether the root directory is encrypted.  This is
      wrong for obvious reasons, and it causes a later run of e2fsck to
      consider the lost+found directory entry to be corrupted.
      
      Encrypting the root directory may also be of limited use because it is
      the "all-or-nothing" use case, for which dm-crypt can be used instead.
      (By design, encryption policies are inherited and cannot be overridden;
      so the root directory having an encryption policy implies that all files
      and directories on the filesystem have that same encryption policy.)
      
      In any case, encrypting the root directory is broken currently and must
      not be allowed; so start returning an error if userspace requests it.
      For now only do this in ext4, because f2fs and ubifs do not appear to
      have the lost+found requirement.  We could move it into
      fscrypt_ioctl_set_policy() later if desired, though.
      Signed-off-by: NEric Biggers <ebiggers@google.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      Reviewed-by: NAndreas Dilger <adilger@dilger.ca>
      9ce0151a
    • D
      ext4: send parallel discards on commit completions · a0154344
      Daeho Jeong 提交于
      Now, when we mount ext4 filesystem with '-o discard' option, we have to
      issue all the discard commands for the blocks to be deallocated and
      wait for the completion of the commands on the commit complete phase.
      Because this procedure might involve a lot of sequential combinations of
      issuing discard commands and waiting for that, the delay of this
      procedure might be too much long, even to 17.0s in our test,
      and it results in long commit delay and fsync() performance degradation.
      
      To reduce this kind of delay, instead of adding callback for each
      extent and handling all of them in a sequential manner on commit phase,
      we instead add a separate list of extents to free to the superblock and
      then process this list at once after transaction commits so that
      we can issue all the discard commands in a parallel manner like XFS
      filesystem.
      
      Finally, we could enhance the discard command handling performance.
      The result was such that 17.0s delay of a single commit in the worst
      case has been enhanced to 4.8s.
      Signed-off-by: NDaeho Jeong <daeho.jeong@samsung.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      Tested-by: NHobin Woo <hobin.woo@samsung.com>
      Tested-by: NKitae Lee <kitae87.lee@samsung.com>
      Reviewed-by: NJan Kara <jack@suse.cz>
      a0154344
  6. 22 6月, 2017 5 次提交
    • T
      ext4: add nombcache mount option · cdb7ee4c
      Tahsin Erdogan 提交于
      The main purpose of mb cache is to achieve deduplication in
      extended attributes. In use cases where opportunity for deduplication
      is unlikely, it only adds overhead.
      
      Add a mount option to explicitly turn off mb cache.
      Suggested-by: NAndreas Dilger <adilger@dilger.ca>
      Signed-off-by: NTahsin Erdogan <tahsin@google.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      cdb7ee4c
    • T
      quota: add get_inode_usage callback to transfer multi-inode charges · 7a9ca53a
      Tahsin Erdogan 提交于
      Ext4 ea_inode feature allows storing xattr values in external inodes to
      be able to store values that are bigger than a block in size. Ext4 also
      has deduplication support for these type of inodes. With deduplication,
      the actual storage waste is eliminated but the users of such inodes are
      still charged full quota for the inodes as if there was no sharing
      happening in the background.
      
      This design requires ext4 to manually charge the users because the
      inodes are shared.
      
      An implication of this is that, if someone calls chown on a file that
      has such references we need to transfer the quota for the file and xattr
      inodes. Current dquot_transfer() function implicitly transfers one inode
      charge. With ea_inode feature, we would like to transfer multiple inode
      charges.
      
      Add get_inode_usage callback which can interrogate the total number of
      inodes that were charged for a given inode.
      
      [ Applied fix from Colin King to make sure the 'ret' variable is
        initialized on the successful return path.  Detected by
        CoverityScan, CID#1446616 ("Uninitialized scalar variable") --tytso]
      Signed-off-by: NTahsin Erdogan <tahsin@google.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      Signed-off-by: NColin Ian King <colin.king@canonical.com>
      Acked-by: NJan Kara <jack@suse.cz>
      7a9ca53a
    • T
      ext4: xattr inode deduplication · dec214d0
      Tahsin Erdogan 提交于
      Ext4 now supports xattr values that are up to 64k in size (vfs limit).
      Large xattr values are stored in external inodes each one holding a
      single value. Once written the data blocks of these inodes are immutable.
      
      The real world use cases are expected to have a lot of value duplication
      such as inherited acls etc. To reduce data duplication on disk, this patch
      implements a deduplicator that allows sharing of xattr inodes.
      
      The deduplication is based on an in-memory hash lookup that is a best
      effort sharing scheme. When a xattr inode is read from disk (i.e.
      getxattr() call), its crc32c hash is added to a hash table. Before
      creating a new xattr inode for a value being set, the hash table is
      checked to see if an existing inode holds an identical value. If such an
      inode is found, the ref count on that inode is incremented. On value
      removal the ref count is decremented and if it reaches zero the inode is
      deleted.
      
      The quota charging for such inodes is manually managed. Every reference
      holder is charged the full size as if there was no sharing happening.
      This is consistent with how xattr blocks are also charged.
      
      [ Fixed up journal credits calculation to handle inline data and the
        rare case where an shared xattr block can get freed when two thread
        race on breaking the xattr block sharing. --tytso ]
      Signed-off-by: NTahsin Erdogan <tahsin@google.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      dec214d0
    • T
      ext2, ext4: make mb block cache names more explicit · 47387409
      Tahsin Erdogan 提交于
      There will be a second mb_cache instance that tracks ea_inodes. Make
      existing names more explicit so that it is clear that they refer to
      xattr block cache.
      Signed-off-by: NTahsin Erdogan <tahsin@google.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      47387409
    • T
      ext4: improve journal credit handling in set xattr paths · c1a5d5f6
      Tahsin Erdogan 提交于
      Both ext4_set_acl() and ext4_set_context() need to be made aware of
      ea_inode feature when it comes to credits calculation.
      
      Also add a sufficient credits check in ext4_xattr_set_handle() right
      after xattr write lock is grabbed. Original credits calculation is done
      outside the lock so there is a possiblity that the initially calculated
      credits are not sufficient anymore.
      Signed-off-by: NTahsin Erdogan <tahsin@google.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      c1a5d5f6
  7. 05 6月, 2017 1 次提交
  8. 25 5月, 2017 1 次提交
  9. 22 5月, 2017 1 次提交
  10. 09 5月, 2017 2 次提交
    • M
      mm: introduce kv[mz]alloc helpers · a7c3e901
      Michal Hocko 提交于
      Patch series "kvmalloc", v5.
      
      There are many open coded kmalloc with vmalloc fallback instances in the
      tree.  Most of them are not careful enough or simply do not care about
      the underlying semantic of the kmalloc/page allocator which means that
      a) some vmalloc fallbacks are basically unreachable because the kmalloc
      part will keep retrying until it succeeds b) the page allocator can
      invoke a really disruptive steps like the OOM killer to move forward
      which doesn't sound appropriate when we consider that the vmalloc
      fallback is available.
      
      As it can be seen implementing kvmalloc requires quite an intimate
      knowledge if the page allocator and the memory reclaim internals which
      strongly suggests that a helper should be implemented in the memory
      subsystem proper.
      
      Most callers, I could find, have been converted to use the helper
      instead.  This is patch 6.  There are some more relying on __GFP_REPEAT
      in the networking stack which I have converted as well and Eric Dumazet
      was not opposed [2] to convert them as well.
      
      [1] http://lkml.kernel.org/r/20170130094940.13546-1-mhocko@kernel.org
      [2] http://lkml.kernel.org/r/1485273626.16328.301.camel@edumazet-glaptop3.roam.corp.google.com
      
      This patch (of 9):
      
      Using kmalloc with the vmalloc fallback for larger allocations is a
      common pattern in the kernel code.  Yet we do not have any common helper
      for that and so users have invented their own helpers.  Some of them are
      really creative when doing so.  Let's just add kv[mz]alloc and make sure
      it is implemented properly.  This implementation makes sure to not make
      a large memory pressure for > PAGE_SZE requests (__GFP_NORETRY) and also
      to not warn about allocation failures.  This also rules out the OOM
      killer as the vmalloc is a more approapriate fallback than a disruptive
      user visible action.
      
      This patch also changes some existing users and removes helpers which
      are specific for them.  In some cases this is not possible (e.g.
      ext4_kvmalloc, libcfs_kvzalloc) because those seems to be broken and
      require GFP_NO{FS,IO} context which is not vmalloc compatible in general
      (note that the page table allocation is GFP_KERNEL).  Those need to be
      fixed separately.
      
      While we are at it, document that __vmalloc{_node} about unsupported gfp
      mask because there seems to be a lot of confusion out there.
      kvmalloc_node will warn about GFP_KERNEL incompatible (which are not
      superset) flags to catch new abusers.  Existing ones would have to die
      slowly.
      
      [sfr@canb.auug.org.au: f2fs fixup]
        Link: http://lkml.kernel.org/r/20170320163735.332e64b7@canb.auug.org.au
      Link: http://lkml.kernel.org/r/20170306103032.2540-2-mhocko@kernel.orgSigned-off-by: NMichal Hocko <mhocko@suse.com>
      Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
      Reviewed-by: Andreas Dilger <adilger@dilger.ca>	[ext4 part]
      Acked-by: NVlastimil Babka <vbabka@suse.cz>
      Cc: John Hubbard <jhubbard@nvidia.com>
      Cc: David Miller <davem@davemloft.net>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      a7c3e901
    • D
      block, dax: move "select DAX" from BLOCK to FS_DAX · ef510424
      Dan Williams 提交于
      For configurations that do not enable DAX filesystems or drivers, do not
      require the DAX core to be built.
      
      Given that the 'direct_access' method has been removed from
      'block_device_operations', we can also go ahead and remove the
      block-related dax helper functions from fs/block_dev.c to
      drivers/dax/super.c. This keeps dax details out of the block layer and
      lets the DAX core be built as a module in the FS_DAX=n case.
      
      Filesystems need to include dax.h to call bdev_dax_supported().
      
      Cc: linux-xfs@vger.kernel.org
      Cc: Jens Axboe <axboe@kernel.dk>
      Cc: "Theodore Ts'o" <tytso@mit.edu>
      Cc: Matthew Wilcox <mawilcox@microsoft.com>
      Cc: Alexander Viro <viro@zeniv.linux.org.uk>
      Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
      Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
      Reviewed-by: NJan Kara <jack@suse.com>
      Reported-by: NGeert Uytterhoeven <geert@linux-m68k.org>
      Signed-off-by: NDan Williams <dan.j.williams@intel.com>
      ef510424
  11. 04 5月, 2017 1 次提交
    • J
      ext4: mark superblock writes synchronous for nobarrier mounts · 00473374
      Jan Kara 提交于
      Commit b685d3d6 "block: treat REQ_FUA and REQ_PREFLUSH as
      synchronous" removed REQ_SYNC flag from WRITE_FUA implementation.
      generic_make_request_checks() however strips REQ_FUA flag from a bio
      when the storage doesn't report volatile write cache and thus write
      effectively becomes asynchronous which can lead to performance
      regressions. This affects superblock writes for ext4. Fix the problem
      by marking superblock writes always as synchronous.
      
      Fixes: b685d3d6
      CC: linux-ext4@vger.kernel.org
      Signed-off-by: NJan Kara <jack@suse.cz>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      00473374
  12. 30 4月, 2017 3 次提交
  13. 24 4月, 2017 1 次提交
  14. 19 4月, 2017 1 次提交
    • J
      ext4: Set flags on quota files directly · 957153fc
      Jan Kara 提交于
      Currently immutable and noatime flags on quota files are set by quota
      code which requires us to copy inode->i_flags to our on disk version of
      quota flags in GETFLAGS ioctl and ext4_do_update_inode(). Move to
      setting / clearing these on-disk flags directly to save that copying.
      Signed-off-by: NJan Kara <jack@suse.cz>
      957153fc
  15. 16 3月, 2017 1 次提交
    • E
      fscrypt: eliminate ->prepare_context() operation · 94840e3c
      Eric Biggers 提交于
      The only use of the ->prepare_context() fscrypt operation was to allow
      ext4 to evict inline data from the inode before ->set_context().
      However, there is no reason why this cannot be done as simply the first
      step in ->set_context(), and in fact it makes more sense to do it that
      way because then the policy modes and flags get validated before any
      real work is done.  Therefore, merge ext4_prepare_context() into
      ext4_set_context(), and remove ->prepare_context().
      Signed-off-by: NEric Biggers <ebiggers@google.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      94840e3c
  16. 15 2月, 2017 1 次提交
  17. 10 2月, 2017 1 次提交
  18. 08 2月, 2017 1 次提交
  19. 06 2月, 2017 1 次提交
  20. 05 2月, 2017 3 次提交
  21. 12 1月, 2017 1 次提交
  22. 08 1月, 2017 1 次提交
  23. 25 12月, 2016 1 次提交
  24. 11 12月, 2016 1 次提交
    • S
      ext4: do not perform data journaling when data is encrypted · 73b92a2a
      Sergey Karamov 提交于
      Currently data journalling is incompatible with encryption: enabling both
      at the same time has never been supported by design, and would result in
      unpredictable behavior. However, users are not precluded from turning on
      both features simultaneously. This change programmatically replaces data
      journaling for encrypted regular files with ordered data journaling mode.
      
      Background:
      Journaling encrypted data has not been supported because it operates on
      buffer heads of the page in the page cache. Namely, when the commit
      happens, which could be up to five seconds after caching, the commit
      thread uses the buffer heads attached to the page to copy the contents of
      the page to the journal. With encryption, it would have been required to
      keep the bounce buffer with ciphertext for up to the aforementioned five
      seconds, since the page cache can only hold plaintext and could not be
      used for journaling. Alternatively, it would be required to setup the
      journal to initiate a callback at the commit time to perform deferred
      encryption - in this case, not only would the data have to be written
      twice, but it would also have to be encrypted twice. This level of
      complexity was not justified for a mode that in practice is very rarely
      used because of the overhead from the data journalling.
      
      Solution:
      If data=journaled has been set as a mount option for a filesystem, or if
      journaling is enabled on a regular file, do not perform journaling if the
      file is also encrypted, instead fall back to the data=ordered mode for the
      file.
      
      Rationale:
      The intent is to allow seamless and proper filesystem operation when
      journaling and encryption have both been enabled, and have these two
      conflicting features gracefully resolved by the filesystem.
      
      Fixes: 44614711Signed-off-by: NSergey Karamov <skaramov@google.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      Cc: stable@vger.kernel.org
      73b92a2a
  25. 06 12月, 2016 1 次提交
  26. 04 12月, 2016 1 次提交
  27. 02 12月, 2016 1 次提交
    • E
      ext4: validate s_first_meta_bg at mount time · 3a4b77cd
      Eryu Guan 提交于
      Ralf Spenneberg reported that he hit a kernel crash when mounting a
      modified ext4 image. And it turns out that kernel crashed when
      calculating fs overhead (ext4_calculate_overhead()), this is because
      the image has very large s_first_meta_bg (debug code shows it's
      842150400), and ext4 overruns the memory in count_overhead() when
      setting bitmap buffer, which is PAGE_SIZE.
      
      ext4_calculate_overhead():
        buf = get_zeroed_page(GFP_NOFS);  <=== PAGE_SIZE buffer
        blks = count_overhead(sb, i, buf);
      
      count_overhead():
        for (j = ext4_bg_num_gdb(sb, grp); j > 0; j--) { <=== j = 842150400
                ext4_set_bit(EXT4_B2C(sbi, s++), buf);   <=== buffer overrun
                count++;
        }
      
      This can be reproduced easily for me by this script:
      
        #!/bin/bash
        rm -f fs.img
        mkdir -p /mnt/ext4
        fallocate -l 16M fs.img
        mke2fs -t ext4 -O bigalloc,meta_bg,^resize_inode -F fs.img
        debugfs -w -R "ssv first_meta_bg 842150400" fs.img
        mount -o loop fs.img /mnt/ext4
      
      Fix it by validating s_first_meta_bg first at mount time, and
      refusing to mount if its value exceeds the largest possible meta_bg
      number.
      Reported-by: NRalf Spenneberg <ralf@os-t.de>
      Signed-off-by: NEryu Guan <guaneryu@gmail.com>
      Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
      Reviewed-by: NAndreas Dilger <adilger@dilger.ca>
      3a4b77cd
  28. 27 11月, 2016 1 次提交
  29. 22 11月, 2016 1 次提交
    • E
      ext4: avoid lockdep warning when inheriting encryption context · 2f8f5e76
      Eric Biggers 提交于
      On a lockdep-enabled kernel, xfstests generic/027 fails due to a lockdep
      warning when run on ext4 mounted with -o test_dummy_encryption:
      
          xfs_io/4594 is trying to acquire lock:
           (jbd2_handle
          ){++++.+}, at:
          [<ffffffff813096ef>] jbd2_log_wait_commit+0x5/0x11b
      
          but task is already holding lock:
           (jbd2_handle
          ){++++.+}, at:
          [<ffffffff813000de>] start_this_handle+0x354/0x3d8
      
      The abbreviated call stack is:
      
       [<ffffffff813096ef>] ? jbd2_log_wait_commit+0x5/0x11b
       [<ffffffff8130972a>] jbd2_log_wait_commit+0x40/0x11b
       [<ffffffff813096ef>] ? jbd2_log_wait_commit+0x5/0x11b
       [<ffffffff8130987b>] ? __jbd2_journal_force_commit+0x76/0xa6
       [<ffffffff81309896>] __jbd2_journal_force_commit+0x91/0xa6
       [<ffffffff813098b9>] jbd2_journal_force_commit_nested+0xe/0x18
       [<ffffffff812a6049>] ext4_should_retry_alloc+0x72/0x79
       [<ffffffff812f0c1f>] ext4_xattr_set+0xef/0x11f
       [<ffffffff812cc35b>] ext4_set_context+0x3a/0x16b
       [<ffffffff81258123>] fscrypt_inherit_context+0xe3/0x103
       [<ffffffff812ab611>] __ext4_new_inode+0x12dc/0x153a
       [<ffffffff812bd371>] ext4_create+0xb7/0x161
      
      When a file is created in an encrypted directory, ext4_set_context() is
      called to set an encryption context on the new file.  This calls
      ext4_xattr_set(), which contains a retry loop where the journal is
      forced to commit if an ENOSPC error is encountered.
      
      If the task actually were to wait for the journal to commit in this
      case, then it would deadlock because a handle remains open from
      __ext4_new_inode(), so the running transaction can't be committed yet.
      Fortunately, __jbd2_journal_force_commit() avoids the deadlock by not
      allowing the running transaction to be committed while the current task
      has it open.  However, the above lockdep warning is still triggered.
      
      This was a false positive which was introduced by: 1eaa566d: jbd2:
      track more dependencies on transaction commit
      
      Fix the problem by passing the handle through the 'fs_data' argument to
      ext4_set_context(), then using ext4_xattr_set_handle() instead of
      ext4_xattr_set().  And in the case where no journal handle is specified
      and ext4_set_context() has to open one, add an ENOSPC retry loop since
      in that case it is the outermost transaction.
      Signed-off-by: NEric Biggers <ebiggers@google.com>
      2f8f5e76
  30. 21 11月, 2016 1 次提交