1. 26 7月, 2020 1 次提交
  2. 09 7月, 2020 1 次提交
  3. 08 7月, 2020 1 次提交
  4. 09 6月, 2020 1 次提交
  5. 19 5月, 2020 1 次提交
    • E
      fscrypt: support test_dummy_encryption=v2 · ed318a6c
      Eric Biggers 提交于
      v1 encryption policies are deprecated in favor of v2, and some new
      features (e.g. encryption+casefolding) are only being added for v2.
      
      Therefore, the "test_dummy_encryption" mount option (which is used for
      encryption I/O testing with xfstests) needs to support v2 policies.
      
      To do this, extend its syntax to be "test_dummy_encryption=v1" or
      "test_dummy_encryption=v2".  The existing "test_dummy_encryption" (no
      argument) also continues to be accepted, to specify the default setting
      -- currently v1, but the next patch changes it to v2.
      
      To cleanly support both v1 and v2 while also making it easy to support
      specifying other encryption settings in the future (say, accepting
      "$contents_mode:$filenames_mode:v2"), make ext4 and f2fs maintain a
      pointer to the dummy fscrypt_context rather than using mount flags.
      
      To avoid concurrency issues, don't allow test_dummy_encryption to be set
      or changed during a remount.  (The former restriction is new, but
      xfstests doesn't run into it, so no one should notice.)
      
      Tested with 'gce-xfstests -c {ext4,f2fs}/encrypt -g auto'.  On ext4,
      there are two regressions, both of which are test bugs: ext4/023 and
      ext4/028 fail because they set an xattr and expect it to be stored
      inline, but the increase in size of the fscrypt_context from
      24 to 40 bytes causes this xattr to be spilled into an external block.
      
      Link: https://lore.kernel.org/r/20200512233251.118314-4-ebiggers@kernel.orgAcked-by: NJaegeuk Kim <jaegeuk@kernel.org>
      Reviewed-by: NTheodore Ts'o <tytso@mit.edu>
      Signed-off-by: NEric Biggers <ebiggers@google.com>
      ed318a6c
  6. 12 5月, 2020 1 次提交
  7. 18 4月, 2020 2 次提交
  8. 17 4月, 2020 1 次提交
  9. 04 4月, 2020 1 次提交
  10. 20 3月, 2020 2 次提交
  11. 24 1月, 2020 1 次提交
    • H
      f2fs: Add f2fs stats to sysfs · fc7100ea
      Hridya Valsaraju 提交于
      Currently f2fs stats are only available from /d/f2fs/status. This patch
      adds some of the f2fs stats to sysfs so that they are accessible even
      when debugfs is not mounted.
      
      The following sysfs nodes are added:
      -/sys/fs/f2fs/<disk>/free_segments
      -/sys/fs/f2fs/<disk>/cp_foreground_calls
      -/sys/fs/f2fs/<disk>/cp_background_calls
      -/sys/fs/f2fs/<disk>/gc_foreground_calls
      -/sys/fs/f2fs/<disk>/gc_background_calls
      -/sys/fs/f2fs/<disk>/moved_blocks_foreground
      -/sys/fs/f2fs/<disk>/moved_blocks_background
      -/sys/fs/f2fs/<disk>/avg_vblocks
      Signed-off-by: NHridya Valsaraju <hridya@google.com>
      [Jaegeuk Kim: allow STAT_FS without DEBUG_FS]
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      fc7100ea
  12. 18 1月, 2020 2 次提交
    • C
      f2fs: fix memleak of kobject · fe396ad8
      Chao Yu 提交于
      If kobject_init_and_add() failed, caller needs to invoke kobject_put()
      to release kobject explicitly.
      Signed-off-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      fe396ad8
    • C
      f2fs: support data compression · 4c8ff709
      Chao Yu 提交于
      This patch tries to support compression in f2fs.
      
      - New term named cluster is defined as basic unit of compression, file can
      be divided into multiple clusters logically. One cluster includes 4 << n
      (n >= 0) logical pages, compression size is also cluster size, each of
      cluster can be compressed or not.
      
      - In cluster metadata layout, one special flag is used to indicate cluster
      is compressed one or normal one, for compressed cluster, following metadata
      maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
      data including compress header and compressed data.
      
      - In order to eliminate write amplification during overwrite, F2FS only
      support compression on write-once file, data can be compressed only when
      all logical blocks in file are valid and cluster compress ratio is lower
      than specified threshold.
      
      - To enable compression on regular inode, there are three ways:
      * chattr +c file
      * chattr +c dir; touch dir/file
      * mount w/ -o compress_extension=ext; touch file.ext
      
      Compress metadata layout:
                                   [Dnode Structure]
                   +-----------------------------------------------+
                   | cluster 1 | cluster 2 | ......... | cluster N |
                   +-----------------------------------------------+
                   .           .                       .           .
             .                       .                .                      .
        .         Compressed Cluster       .        .        Normal Cluster            .
      +----------+---------+---------+---------+  +---------+---------+---------+---------+
      |compr flag| block 1 | block 2 | block 3 |  | block 1 | block 2 | block 3 | block 4 |
      +----------+---------+---------+---------+  +---------+---------+---------+---------+
                 .                             .
               .                                           .
             .                                                           .
            +-------------+-------------+----------+----------------------------+
            | data length | data chksum | reserved |      compressed data       |
            +-------------+-------------+----------+----------------------------+
      
      Changelog:
      
      20190326:
      - fix error handling of read_end_io().
      - remove unneeded comments in f2fs_encrypt_one_page().
      
      20190327:
      - fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
      - don't jump into loop directly to avoid uninitialized variables.
      - add TODO tag in error path of f2fs_write_cache_pages().
      
      20190328:
      - fix wrong merge condition in f2fs_read_multi_pages().
      - check compressed file in f2fs_post_read_required().
      
      20190401
      - allow overwrite on non-compressed cluster.
      - check cluster meta before writing compressed data.
      
      20190402
      - don't preallocate blocks for compressed file.
      
      - add lz4 compress algorithm
      - process multiple post read works in one workqueue
        Now f2fs supports processing post read work in multiple workqueue,
        it shows low performance due to schedule overhead of multiple
        workqueue executing orderly.
      
      20190921
      - compress: support buffered overwrite
      C: compress cluster flag
      V: valid block address
      N: NEW_ADDR
      
      One cluster contain 4 blocks
      
       before overwrite   after overwrite
      
      - VVVV		->	CVNN
      - CVNN		->	VVVV
      
      - CVNN		->	CVNN
      - CVNN		->	CVVV
      
      - CVVV		->	CVNN
      - CVVV		->	CVVV
      
      20191029
      - add kconfig F2FS_FS_COMPRESSION to isolate compression related
      codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
      note that: will remove lzo backend if Jaegeuk agreed that too.
      - update codes according to Eric's comments.
      
      20191101
      - apply fixes from Jaegeuk
      
      20191113
      - apply fixes from Jaegeuk
      - split workqueue for fsverity
      
      20191216
      - apply fixes from Jaegeuk
      
      20200117
      - fix to avoid NULL pointer dereference
      
      [Jaegeuk Kim]
      - add tracepoint for f2fs_{,de}compress_pages()
      - fix many bugs and add some compression stats
      - fix overwrite/mmap bugs
      - address 32bit build error, reported by Geert.
      - bug fixes when handling errors and i_compressed_blocks
      
      Reported-by: <noreply@ellerman.id.au>
      Signed-off-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      4c8ff709
  13. 16 1月, 2020 1 次提交
  14. 26 11月, 2019 1 次提交
    • J
      f2fs: expose main_blkaddr in sysfs · a4db59ac
      Jaegeuk Kim 提交于
      Expose in /sys/fs/f2fs/<blockdev>/main_blkaddr the block address where the
      main area starts. This allows user mode programs to determine:
      
      - That pinned files that are made exclusively of fully allocated 2MB
        segments will never be unpinned by the file system.
      
      - Where the main area starts. This is required by programs that want to
        verify if a file is made exclusively of 2MB f2fs segments, the alignment
        boundary for segments starts at this address. Testing for 2MB alignment
        relative to the start of the device is incorrect, because for some
        filesystems main_blkaddr is not at a 2MB boundary relative to the start
        of the device.
      
      The entry will be used when validating reliable pinning file feature proposed
      by "f2fs: support aligned pinned file".
      Signed-off-by: NRamon Pantin <pantin@google.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      a4db59ac
  15. 08 11月, 2019 1 次提交
  16. 23 8月, 2019 1 次提交
    • D
      f2fs: include charset encoding information in the superblock · 5aba5430
      Daniel Rosenberg 提交于
      Add charset encoding to f2fs to support casefolding. It is modeled after
      the same feature introduced in commit c83ad55e ("ext4: include charset
      encoding information in the superblock")
      
      Currently this is not compatible with encryption, similar to the current
      ext4 imlpementation. This will change in the future.
      
      >From the ext4 patch:
      """
      The s_encoding field stores a magic number indicating the encoding
      format and version used globally by file and directory names in the
      filesystem.  The s_encoding_flags defines policies for using the charset
      encoding, like how to handle invalid sequences.  The magic number is
      mapped to the exact charset table, but the mapping is specific to ext4.
      Since we don't have any commitment to support old encodings, the only
      encoding I am supporting right now is utf8-12.1.0.
      
      The current implementation prevents the user from enabling encoding and
      per-directory encryption on the same filesystem at the same time.  The
      incompatibility between these features lies in how we do efficient
      directory searches when we cannot be sure the encryption of the user
      provided fname will match the actual hash stored in the disk without
      decrypting every directory entry, because of normalization cases.  My
      quickest solution is to simply block the concurrent use of these
      features for now, and enable it later, once we have a better solution.
      """
      Signed-off-by: NDaniel Rosenberg <drosen@google.com>
      Reviewed-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      5aba5430
  17. 13 8月, 2019 1 次提交
    • E
      f2fs: add fs-verity support · 95ae251f
      Eric Biggers 提交于
      Add fs-verity support to f2fs.  fs-verity is a filesystem feature that
      enables transparent integrity protection and authentication of read-only
      files.  It uses a dm-verity like mechanism at the file level: a Merkle
      tree is used to verify any block in the file in log(filesize) time.  It
      is implemented mainly by helper functions in fs/verity/.  See
      Documentation/filesystems/fsverity.rst for the full documentation.
      
      The f2fs support for fs-verity consists of:
      
      - Adding a filesystem feature flag and an inode flag for fs-verity.
      
      - Implementing the fsverity_operations to support enabling verity on an
        inode and reading/writing the verity metadata.
      
      - Updating ->readpages() to verify data as it's read from verity files
        and to support reading verity metadata pages.
      
      - Updating ->write_begin(), ->write_end(), and ->writepages() to support
        writing verity metadata pages.
      
      - Calling the fs-verity hooks for ->open(), ->setattr(), and ->ioctl().
      
      Like ext4, f2fs stores the verity metadata (Merkle tree and
      fsverity_descriptor) past the end of the file, starting at the first 64K
      boundary beyond i_size.  This approach works because (a) verity files
      are readonly, and (b) pages fully beyond i_size aren't visible to
      userspace but can be read/written internally by f2fs with only some
      relatively small changes to f2fs.  Extended attributes cannot be used
      because (a) f2fs limits the total size of an inode's xattr entries to
      4096 bytes, which wouldn't be enough for even a single Merkle tree
      block, and (b) f2fs encryption doesn't encrypt xattrs, yet the verity
      metadata *must* be encrypted when the file is because it contains hashes
      of the plaintext data.
      Acked-by: NJaegeuk Kim <jaegeuk@kernel.org>
      Acked-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NEric Biggers <ebiggers@google.com>
      95ae251f
  18. 03 7月, 2019 1 次提交
  19. 22 6月, 2019 1 次提交
  20. 13 6月, 2019 1 次提交
  21. 04 6月, 2019 1 次提交
    • D
      f2fs: Add option to limit required GC for checkpoint=disable · 4d3aed70
      Daniel Rosenberg 提交于
      This extends the checkpoint option to allow checkpoint=disable:%u[%]
      This allows you to specify what how much of the disk you are willing
      to lose access to while mounting with checkpoint=disable. If the amount
      lost would be higher, the mount will return -EAGAIN. This can be given
      as a percent of total space, or in blocks.
      
      Currently, we need to run garbage collection until the amount of holes
      is smaller than the OVP space. With the new option, f2fs can mark
      space as unusable up front instead of requiring garbage collection until
      the number of holes is small enough.
      Signed-off-by: NDaniel Rosenberg <drosen@google.com>
      Reviewed-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      4d3aed70
  22. 05 2月, 2019 1 次提交
  23. 24 1月, 2019 1 次提交
  24. 23 1月, 2019 1 次提交
    • S
      f2fs: UBSAN: set boolean value iostat_enable correctly · ac929858
      Sheng Yong 提交于
      When setting /sys/fs/f2fs/<DEV>/iostat_enable with non-bool value, UBSAN
      reports the following warning.
      
      [ 7562.295484] ================================================================================
      [ 7562.296531] UBSAN: Undefined behaviour in fs/f2fs/f2fs.h:2776:10
      [ 7562.297651] load of value 64 is not a valid value for type '_Bool'
      [ 7562.298642] CPU: 1 PID: 7487 Comm: dd Not tainted 4.20.0-rc4+ #79
      [ 7562.298653] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
      [ 7562.298662] Call Trace:
      [ 7562.298760]  dump_stack+0x46/0x5b
      [ 7562.298811]  ubsan_epilogue+0x9/0x40
      [ 7562.298830]  __ubsan_handle_load_invalid_value+0x72/0x90
      [ 7562.298863]  f2fs_file_write_iter+0x29f/0x3f0
      [ 7562.298905]  __vfs_write+0x115/0x160
      [ 7562.298922]  vfs_write+0xa7/0x190
      [ 7562.298934]  ksys_write+0x50/0xc0
      [ 7562.298973]  do_syscall_64+0x4a/0xe0
      [ 7562.298992]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
      [ 7562.299001] RIP: 0033:0x7fa45ec19c00
      [ 7562.299004] Code: 73 01 c3 48 8b 0d 88 92 2c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 0f 1f 44 00 00 83 3d dd eb 2c 00 00 75 10 b8 01 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 ce 8f 01 00 48 89 04 24
      [ 7562.299044] RSP: 002b:00007ffca52b49e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
      [ 7562.299052] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fa45ec19c00
      [ 7562.299059] RDX: 0000000000000400 RSI: 000000000093f000 RDI: 0000000000000001
      [ 7562.299065] RBP: 000000000093f000 R08: 0000000000000004 R09: 0000000000000000
      [ 7562.299071] R10: 00007ffca52b47b0 R11: 0000000000000246 R12: 0000000000000400
      [ 7562.299077] R13: 000000000093f000 R14: 000000000093f400 R15: 0000000000000000
      [ 7562.299091] ================================================================================
      
      So, if iostat_enable is enabled, set its value as true.
      Signed-off-by: NSheng Yong <shengyong1@huawei.com>
      Reviewed-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      ac929858
  25. 09 1月, 2019 1 次提交
  26. 27 11月, 2018 2 次提交
  27. 01 10月, 2018 1 次提交
  28. 20 9月, 2018 1 次提交
  29. 13 9月, 2018 1 次提交
  30. 21 8月, 2018 1 次提交
  31. 14 8月, 2018 1 次提交
  32. 02 8月, 2018 2 次提交
    • Y
      f2fs: add proc entry to show victim_secmap bitmap · 970e348d
      Yunlong Song 提交于
      This patch adds a new proc entry to show victim_secmap information in
      more detail, which is very helpful to know the get_victim candidate
      status clearly, and helpful to debug problems (e.g., some sections can
      not gc all of its blocks, since some blocks belong to atomic file,
      leaving victim_secmap with section bit setting, in extrem case, this
      will lead all bytes of victim_secmap setting with 0xff).
      Signed-off-by: NYunlong Song <yunlong.song@huawei.com>
      Reviewed-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      970e348d
    • R
      f2fs: fix defined but not used build warnings · cb15d1e4
      Randy Dunlap 提交于
      Fix build warnings in f2fs when CONFIG_PROC_FS is not enabled
      by marking the unused functions as __maybe_unused.
      
      ../fs/f2fs/sysfs.c:519:12: warning: 'segment_info_seq_show' defined but not used [-Wunused-function]
      ../fs/f2fs/sysfs.c:546:12: warning: 'segment_bits_seq_show' defined but not used [-Wunused-function]
      ../fs/f2fs/sysfs.c:570:12: warning: 'iostat_info_seq_show' defined but not used [-Wunused-function]
      Signed-off-by: NRandy Dunlap <rdunlap@infradead.org>
      Cc: Jaegeuk Kim <jaegeuk@kernel.org>
      Cc: Chao Yu <yuchao0@huawei.com>
      Cc: linux-f2fs-devel@lists.sourceforge.net
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      cb15d1e4
  33. 27 7月, 2018 1 次提交
    • J
      f2fs: avoid potential deadlock in f2fs_sbi_store · a1933c09
      Jaegeuk Kim 提交于
      [  155.018460] ======================================================
      [  155.021431] WARNING: possible circular locking dependency detected
      [  155.024339] 4.18.0-rc3+ #5 Tainted: G           OE
      [  155.026879] ------------------------------------------------------
      [  155.029783] umount/2901 is trying to acquire lock:
      [  155.032187] 00000000c4282f1f (kn->count#130){++++}, at: kernfs_remove+0x1f/0x30
      [  155.035439]
      [  155.035439] but task is already holding lock:
      [  155.038892] 0000000056e4307b (&type->s_umount_key#41){++++}, at: deactivate_super+0x33/0x50
      [  155.042602]
      [  155.042602] which lock already depends on the new lock.
      [  155.042602]
      [  155.047465]
      [  155.047465] the existing dependency chain (in reverse order) is:
      [  155.051354]
      [  155.051354] -> #1 (&type->s_umount_key#41){++++}:
      [  155.054768]        f2fs_sbi_store+0x61/0x460 [f2fs]
      [  155.057083]        kernfs_fop_write+0x113/0x1a0
      [  155.059277]        __vfs_write+0x36/0x180
      [  155.061250]        vfs_write+0xbe/0x1b0
      [  155.063179]        ksys_write+0x55/0xc0
      [  155.065068]        do_syscall_64+0x60/0x1b0
      [  155.067071]        entry_SYSCALL_64_after_hwframe+0x49/0xbe
      [  155.069529]
      [  155.069529] -> #0 (kn->count#130){++++}:
      [  155.072421]        __kernfs_remove+0x26f/0x2e0
      [  155.074452]        kernfs_remove+0x1f/0x30
      [  155.076342]        kobject_del.part.5+0xe/0x40
      [  155.078354]        f2fs_put_super+0x12d/0x290 [f2fs]
      [  155.080500]        generic_shutdown_super+0x6c/0x110
      [  155.082655]        kill_block_super+0x21/0x50
      [  155.084634]        kill_f2fs_super+0x9c/0xc0 [f2fs]
      [  155.086726]        deactivate_locked_super+0x3f/0x70
      [  155.088826]        cleanup_mnt+0x3b/0x70
      [  155.090584]        task_work_run+0x93/0xc0
      [  155.092367]        exit_to_usermode_loop+0xf0/0x100
      [  155.094466]        do_syscall_64+0x162/0x1b0
      [  155.096312]        entry_SYSCALL_64_after_hwframe+0x49/0xbe
      [  155.098603]
      [  155.098603] other info that might help us debug this:
      [  155.098603]
      [  155.102418]  Possible unsafe locking scenario:
      [  155.102418]
      [  155.105134]        CPU0                    CPU1
      [  155.107037]        ----                    ----
      [  155.108910]   lock(&type->s_umount_key#41);
      [  155.110674]                                lock(kn->count#130);
      [  155.113010]                                lock(&type->s_umount_key#41);
      [  155.115608]   lock(kn->count#130);
      Reviewed-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      a1933c09
  34. 01 6月, 2018 2 次提交
    • C
      f2fs: clean up symbol namespace · 4d57b86d
      Chao Yu 提交于
      As Ted reported:
      
      "Hi, I was looking at f2fs's sources recently, and I noticed that there
      is a very large number of non-static symbols which don't have a f2fs
      prefix.  There's well over a hundred (see attached below).
      
      As one example, in fs/f2fs/dir.c there is:
      
      unsigned char get_de_type(struct f2fs_dir_entry *de)
      
      This function is clearly only useful for f2fs, but it has a generic
      name.  This means that if any other file system tries to have the same
      symbol name, there will be a symbol conflict and the kernel would not
      successfully build.  It also means that when someone is looking f2fs
      sources, it's not at all obvious whether a function such as
      read_data_page(), invalidate_blocks(), is a generic kernel function
      found in the fs, mm, or block layers, or a f2fs specific function.
      
      You might want to fix this at some point.  Hopefully Kent's bcachefs
      isn't similarly using genericly named functions, since that might
      cause conflicts with f2fs's functions --- but just as this would be a
      problem that we would rightly insist that Kent fix, this is something
      that we should have rightly insisted that f2fs should have fixed
      before it was integrated into the mainline kernel.
      
      acquire_orphan_inode
      add_ino_entry
      add_orphan_inode
      allocate_data_block
      allocate_new_segments
      alloc_nid
      alloc_nid_done
      alloc_nid_failed
      available_free_memory
      ...."
      
      This patch adds "f2fs_" prefix for all non-static symbols in order to:
      a) avoid conflict with other kernel generic symbols;
      b) to indicate the function is f2fs specific one instead of generic
      one;
      Reported-by: NTheodore Ts'o <tytso@mit.edu>
      Signed-off-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      4d57b86d
    • C
      f2fs: fix to avoid race during access gc_thread pointer · 250dbf51
      Chao Yu 提交于
      Thread A			Thread B
      - f2fs_remount
       - stop_gc_thread
      				- f2fs_sbi_store
         sbi->gc_thread = NULL;
      				  access sbi->gc_thread->gc_*
      
      Previously, we allocate memory for sbi->gc_thread based on background
      gc thread mount option, the memory can be released if we turn off
      that mount option, but still there are several places access gc_thread
      pointer without considering race condition, result in NULL point
      dereference.
      
      In order to fix this issue, use sb->s_umount to exclude those operations.
      Signed-off-by: NChao Yu <yuchao0@huawei.com>
      Signed-off-by: NJaegeuk Kim <jaegeuk@kernel.org>
      250dbf51