1. 13 9月, 2013 1 次提交
  2. 12 9月, 2013 1 次提交
    • M
      mm/page-writeback.c: add strictlimit feature · 5a537485
      Maxim Patlasov 提交于
      The feature prevents mistrusted filesystems (ie: FUSE mounts created by
      unprivileged users) to grow a large number of dirty pages before
      throttling.  For such filesystems balance_dirty_pages always check bdi
      counters against bdi limits.  I.e.  even if global "nr_dirty" is under
      "freerun", it's not allowed to skip bdi checks.  The only use case for now
      is fuse: it sets bdi max_ratio to 1% by default and system administrators
      are supposed to expect that this limit won't be exceeded.
      
      The feature is on if a BDI is marked by BDI_CAP_STRICTLIMIT flag.  A
      filesystem may set the flag when it initializes its BDI.
      
      The problematic scenario comes from the fact that nobody pays attention to
      the NR_WRITEBACK_TEMP counter (i.e.  number of pages under fuse
      writeback).  The implementation of fuse writeback releases original page
      (by calling end_page_writeback) almost immediately.  A fuse request queued
      for real processing bears a copy of original page.  Hence, if userspace
      fuse daemon doesn't finalize write requests in timely manner, an
      aggressive mmap writer can pollute virtually all memory by those temporary
      fuse page copies.  They are carefully accounted in NR_WRITEBACK_TEMP, but
      nobody cares.
      
      To make further explanations shorter, let me use "NR_WRITEBACK_TEMP
      problem" as a shortcut for "a possibility of uncontrolled grow of amount
      of RAM consumed by temporary pages allocated by kernel fuse to process
      writeback".
      
      The problem was very easy to reproduce.  There is a trivial example
      filesystem implementation in fuse userspace distribution: fusexmp_fh.c.  I
      added "sleep(1);" to the write methods, then recompiled and mounted it.
      Then created a huge file on the mount point and run a simple program which
      mmap-ed the file to a memory region, then wrote a data to the region.  An
      hour later I observed almost all RAM consumed by fuse writeback.  Since
      then some unrelated changes in kernel fuse made it more difficult to
      reproduce, but it is still possible now.
      
      Putting this theoretical happens-in-the-lab thing aside, there is another
      thing that really hurts real world (FUSE) users.  This is write-through
      page cache policy FUSE currently uses.  I.e.  handling write(2), kernel
      fuse populates page cache and flushes user data to the server
      synchronously.  This is excessively suboptimal.  Pavel Emelyanov's patches
      ("writeback cache policy") solve the problem, but they also make resolving
      NR_WRITEBACK_TEMP problem absolutely necessary.  Otherwise, simply copying
      a huge file to a fuse mount would result in memory starvation.  Miklos,
      the maintainer of FUSE, believes strictlimit feature the way to go.
      
      And eventually putting FUSE topics aside, there is one more use-case for
      strictlimit feature.  Using a slow USB stick (mass storage) in a machine
      with huge amount of RAM installed is a well-known pain.  Let's make simple
      computations.  Assuming 64GB of RAM installed, existing implementation of
      balance_dirty_pages will start throttling only after 9.6GB of RAM becomes
      dirty (freerun == 15% of total RAM).  So, the command "cp 9GB_file
      /media/my-usb-storage/" may return in a few seconds, but subsequent
      "umount /media/my-usb-storage/" will take more than two hours if effective
      throughput of the storage is, to say, 1MB/sec.
      
      After inclusion of strictlimit feature, it will be trivial to add a knob
      (e.g.  /sys/devices/virtual/bdi/x:y/strictlimit) to enable it on demand.
      Manually or via udev rule.  May be I'm wrong, but it seems to be quite a
      natural desire to limit the amount of dirty memory for some devices we are
      not fully trust (in the sense of sustainable throughput).
      
      [akpm@linux-foundation.org: fix warning in page-writeback.c]
      Signed-off-by: NMaxim Patlasov <MPatlasov@parallels.com>
      Cc: Jan Kara <jack@suse.cz>
      Cc: Miklos Szeredi <miklos@szeredi.hu>
      Cc: Wu Fengguang <fengguang.wu@intel.com>
      Cc: Pavel Emelyanov <xemul@parallels.com>
      Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      5a537485
  3. 06 9月, 2013 3 次提交
  4. 04 9月, 2013 1 次提交
  5. 03 9月, 2013 4 次提交
    • M
      fuse: readdir: check for slash in names · efeb9e60
      Miklos Szeredi 提交于
      Userspace can add names containing a slash character to the directory
      listing.  Don't allow this as it could cause all sorts of trouble.
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      Cc: stable@vger.kernel.org
      efeb9e60
    • M
      fuse: hotfix truncate_pagecache() issue · 06a7c3c2
      Maxim Patlasov 提交于
      The way how fuse calls truncate_pagecache() from fuse_change_attributes()
      is completely wrong. Because, w/o i_mutex held, we never sure whether
      'oldsize' and 'attr->size' are valid by the time of execution of
      truncate_pagecache(inode, oldsize, attr->size). In fact, as soon as we
      released fc->lock in the middle of fuse_change_attributes(), we completely
      loose control of actions which may happen with given inode until we reach
      truncate_pagecache. The list of potentially dangerous actions includes
      mmap-ed reads and writes, ftruncate(2) and write(2) extending file size.
      
      The typical outcome of doing truncate_pagecache() with outdated arguments
      is data corruption from user point of view. This is (in some sense)
      acceptable in cases when the issue is triggered by a change of the file on
      the server (i.e. externally wrt fuse operation), but it is absolutely
      intolerable in scenarios when a single fuse client modifies a file without
      any external intervention. A real life case I discovered by fsx-linux
      looked like this:
      
      1. Shrinking ftruncate(2) comes to fuse_do_setattr(). The latter sends
      FUSE_SETATTR to the server synchronously, but before getting fc->lock ...
      2. fuse_dentry_revalidate() is asynchronously called. It sends FUSE_LOOKUP
      to the server synchronously, then calls fuse_change_attributes(). The
      latter updates i_size, releases fc->lock, but before comparing oldsize vs
      attr->size..
      3. fuse_do_setattr() from the first step proceeds by acquiring fc->lock and
      updating attributes and i_size, but now oldsize is equal to
      outarg.attr.size because i_size has just been updated (step 2). Hence,
      fuse_do_setattr() returns w/o calling truncate_pagecache().
      4. As soon as ftruncate(2) completes, the user extends file size by
      write(2) making a hole in the middle of file, then reads data from the hole
      either by read(2) or mmap-ed read. The user expects to get zero data from
      the hole, but gets stale data because truncate_pagecache() is not executed
      yet.
      
      The scenario above illustrates one side of the problem: not truncating the
      page cache even though we should. Another side corresponds to truncating
      page cache too late, when the state of inode changed significantly.
      Theoretically, the following is possible:
      
      1. As in the previous scenario fuse_dentry_revalidate() discovered that
      i_size changed (due to our own fuse_do_setattr()) and is going to call
      truncate_pagecache() for some 'new_size' it believes valid right now. But
      by the time that particular truncate_pagecache() is called ...
      2. fuse_do_setattr() returns (either having called truncate_pagecache() or
      not -- it doesn't matter).
      3. The file is extended either by write(2) or ftruncate(2) or fallocate(2).
      4. mmap-ed write makes a page in the extended region dirty.
      
      The result will be the lost of data user wrote on the fourth step.
      
      The patch is a hotfix resolving the issue in a simplistic way: let's skip
      dangerous i_size update and truncate_pagecache if an operation changing
      file size is in progress. This simplistic approach looks correct for the
      cases w/o external changes. And to handle them properly, more sophisticated
      and intrusive techniques (e.g. NFS-like one) would be required. I'd like to
      postpone it until the issue is well discussed on the mailing list(s).
      
      Changed in v2:
       - improved patch description to cover both sides of the issue.
      Signed-off-by: NMaxim Patlasov <mpatlasov@parallels.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      Cc: stable@vger.kernel.org
      06a7c3c2
    • A
      fuse: invalidate inode attributes on xattr modification · d331a415
      Anand Avati 提交于
      Calls like setxattr and removexattr result in updation of ctime.
      Therefore invalidate inode attributes to force a refresh.
      Signed-off-by: NAnand Avati <avati@redhat.com>
      Reviewed-by: NBrian Foster <bfoster@redhat.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      Cc: stable@vger.kernel.org
      d331a415
    • M
      fuse: postpone end_page_writeback() in fuse_writepage_locked() · 4a4ac4eb
      Maxim Patlasov 提交于
      The patch fixes a race between ftruncate(2), mmap-ed write and write(2):
      
      1) An user makes a page dirty via mmap-ed write.
      2) The user performs shrinking truncate(2) intended to purge the page.
      3) Before fuse_do_setattr calls truncate_pagecache, the page goes to
         writeback. fuse_writepage_locked fills FUSE_WRITE request and releases
         the original page by end_page_writeback.
      4) fuse_do_setattr() completes and successfully returns. Since now, i_mutex
         is free.
      5) Ordinary write(2) extends i_size back to cover the page. Note that
         fuse_send_write_pages do wait for fuse writeback, but for another
         page->index.
      6) fuse_writepage_locked proceeds by queueing FUSE_WRITE request.
         fuse_send_writepage is supposed to crop inarg->size of the request,
         but it doesn't because i_size has already been extended back.
      
      Moving end_page_writeback to the end of fuse_writepage_locked fixes the
      race because now the fact that truncate_pagecache is successfully returned
      infers that fuse_writepage_locked has already called end_page_writeback.
      And this, in turn, infers that fuse_flush_writepages has already called
      fuse_send_writepage, and the latter used valid (shrunk) i_size. write(2)
      could not extend it because of i_mutex held by ftruncate(2).
      Signed-off-by: NMaxim Patlasov <mpatlasov@parallels.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      Cc: stable@vger.kernel.org
      4a4ac4eb
  6. 27 7月, 2013 1 次提交
  7. 17 7月, 2013 5 次提交
  8. 04 7月, 2013 1 次提交
  9. 29 6月, 2013 2 次提交
  10. 18 6月, 2013 1 次提交
    • M
      fuse: hold i_mutex in fuse_file_fallocate() · 14c14414
      Maxim Patlasov 提交于
      Changing size of a file on server and local update (fuse_write_update_size)
      should be always protected by inode->i_mutex. Otherwise a race like this is
      possible:
      
      1. Process 'A' calls fallocate(2) to extend file (~FALLOC_FL_KEEP_SIZE).
      fuse_file_fallocate() sends FUSE_FALLOCATE request to the server.
      2. Process 'B' calls ftruncate(2) shrinking the file. fuse_do_setattr()
      sends shrinking FUSE_SETATTR request to the server and updates local i_size
      by i_size_write(inode, outarg.attr.size).
      3. Process 'A' resumes execution of fuse_file_fallocate() and calls
      fuse_write_update_size(inode, offset + length). But 'offset + length' was
      obsoleted by ftruncate from previous step.
      
      Changed in v2 (thanks Brian and Anand for suggestions):
       - made relation between mutex_lock() and fuse_set_nowrite(inode) more
         explicit and clear.
       - updated patch description to use ftruncate(2) in example
      Signed-off-by: NMaxim V. Patlasov <MPatlasov@parallels.com>
      Reviewed-by: NBrian Foster <bfoster@redhat.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      14c14414
  11. 03 6月, 2013 3 次提交
    • M
      fuse: fix alignment in short read optimization for async_dio · e5c5f05d
      Maxim Patlasov 提交于
      The bug was introduced with async_dio feature: trying to optimize short reads,
      we cut number-of-bytes-to-read to i_size boundary. Hence the following example:
      
      	truncate --size=300 /mnt/file
      	dd if=/mnt/file of=/dev/null iflag=direct
      
      led to FUSE_READ request of 300 bytes size. This turned out to be problem
      for userspace fuse implementations who rely on assumption that kernel fuse
      does not change alignment of request from client FS.
      
      The patch turns off the optimization if async_dio is disabled. And, if it's
      enabled, the patch fixes adjustment of number-of-bytes-to-read to preserve
      alignment.
      
      Note, that we cannot throw out short read optimization entirely because
      otherwise a direct read of a huge size issued on a tiny file would generate
      a huge amount of fuse requests and most of them would be ACKed by userspace
      with zero bytes read.
      Signed-off-by: NMaxim Patlasov <MPatlasov@parallels.com>
      Reviewed-by: NBrian Foster <bfoster@redhat.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      e5c5f05d
    • B
      fuse: return -EIOCBQUEUED from fuse_direct_IO() for all async requests · c9ecf989
      Brian Foster 提交于
      If request submission fails for an async request (i.e.,
      get_user_pages() returns -ERESTARTSYS), we currently skip the
      -EIOCBQUEUED return and drop into wait_for_sync_kiocb() forever.
      
      Avoid this by always returning -EIOCBQUEUED for async requests. If
      an error occurs, the error is passed into fuse_aio_complete(),
      returned via aio_complete() and thus propagated to userspace via
      io_getevents().
      Signed-off-by: NBrian Foster <bfoster@redhat.com>
      Reviewed-by: NMaxim Patlasov <MPatlasov@parallels.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      c9ecf989
    • M
      fuse: fix readdirplus Oops in fuse_dentry_revalidate · 28420dad
      Miklos Szeredi 提交于
      Fix bug introduced by commit 4582a4ab "FUSE: Adapt readdirplus to application
      usage patterns".
      
      We need to check for a positive dentry; negative dentries are not added by
      readdirplus.  Secondly we need to advise the use of readdirplus on the *parent*,
      otherwise the whole thing is useless.  Thirdly all this is only relevant if
      "readdirplus_auto" mode is selected by the filesystem.
      
      We advise the use of readdirplus only if the dentry was still valid.  If we had
      to redo the lookup then there was no use in doing the -plus version.
      Reported-by: NBernd Schubert <bernd.schubert@itwm.fraunhofer.de>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      CC: Feng Shuo <steve.shuo.feng@gmail.com>
      CC: stable@vger.kernel.org
      28420dad
  12. 20 5月, 2013 2 次提交
    • B
      fuse: update inode size and invalidate attributes on fallocate · bee6c307
      Brian Foster 提交于
      An fallocate request without FALLOC_FL_KEEP_SIZE set can extend the
      size of a file. Update the inode size after a successful fallocate.
      
      Also invalidate the inode attributes after a successful fallocate
      to ensure we pick up the latest attribute values (i.e., i_blocks).
      Signed-off-by: NBrian Foster <bfoster@redhat.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      bee6c307
    • B
      fuse: truncate pagecache range on hole punch · 3634a632
      Brian Foster 提交于
      fuse supports hole punch via the fallocate() FALLOC_FL_PUNCH_HOLE
      interface. When a hole punch is passed through, the page cache
      is not cleared and thus allows reading stale data from the cache.
      
      This is easily demonstrable (using FOPEN_KEEP_CACHE) by reading a
      smallish random data file into cache, punching a hole and creating
      a copy of the file. Drop caches or remount and observe that the
      original file no longer matches the file copied after the hole
      punch. The original file contains a zeroed range and the latter
      file contains stale data.
      
      Protect against writepage requests in progress and punch out the
      associated page cache range after a successful client fs hole
      punch.
      Signed-off-by: NBrian Foster <bfoster@redhat.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      3634a632
  13. 15 5月, 2013 1 次提交
  14. 08 5月, 2013 1 次提交
  15. 01 5月, 2013 1 次提交
  16. 18 4月, 2013 7 次提交
    • M
      fuse: truncate file if async dio failed · efb9fa9e
      Maxim Patlasov 提交于
      The patch improves error handling in fuse_direct_IO(): if we successfully
      submitted several fuse requests on behalf of synchronous direct write
      extending file and some of them failed, let's try to do our best to clean-up.
      
      Changed in v2: reuse fuse_do_setattr(). Thanks to Brian for suggestion.
      Signed-off-by: NMaxim Patlasov <mpatlasov@parallels.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      efb9fa9e
    • M
      fuse: optimize short direct reads · 439ee5f0
      Maxim Patlasov 提交于
      If user requested direct read beyond EOF, we can skip sending fuse requests
      for positions beyond EOF because userspace would ACK them with zero bytes read
      anyway. We can trust to i_size in fuse_direct_IO for such cases because it's
      called from fuse_file_aio_read() and the latter updates fuse attributes
      including i_size.
      Signed-off-by: NMaxim Patlasov <mpatlasov@parallels.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      439ee5f0
    • M
      fuse: enable asynchronous processing direct IO · bcba24cc
      Maxim Patlasov 提交于
      In case of synchronous DIO request (i.e. read(2) or write(2) for a file
      opened with O_DIRECT), the patch submits fuse requests asynchronously, but
      waits for their completions before return from fuse_direct_IO().
      
      In case of asynchronous DIO request (i.e. libaio io_submit() or a file opened
      with O_DIRECT), the patch submits fuse requests asynchronously and return
      -EIOCBQUEUED immediately.
      
      The only special case is async DIO extending file. Here the patch falls back
      to old behaviour because we can't return -EIOCBQUEUED and update i_size later,
      without i_mutex hold. And we have no method to wait on real async I/O
      requests.
      
      The patch also clean __fuse_direct_write() up: it's better to update i_size
      in its callers. Thanks Brian for suggestion.
      Signed-off-by: NMaxim Patlasov <mpatlasov@parallels.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      bcba24cc
    • M
      fuse: make fuse_direct_io() aware about AIO · 36cf66ed
      Maxim Patlasov 提交于
      The patch implements passing "struct fuse_io_priv *io" down the stack up to
      fuse_send_read/write where it is used to submit request asynchronously.
      io->async==0 designates synchronous processing.
      
      Non-trivial part of the patch is changes in fuse_direct_io(): resources
      like fuse requests and user pages cannot be released immediately in async
      case.
      Signed-off-by: NMaxim Patlasov <mpatlasov@parallels.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      36cf66ed
    • M
      fuse: add support of async IO · 01e9d11a
      Maxim Patlasov 提交于
      The patch implements a framework to process an IO request asynchronously. The
      idea is to associate several fuse requests with a single kiocb by means of
      fuse_io_priv structure. The structure plays the same role for FUSE as 'struct
      dio' for direct-io.c.
      
      The framework is supposed to be used like this:
       - someone (who wants to process an IO asynchronously) allocates fuse_io_priv
         and initializes it setting 'async' field to non-zero value.
       - as soon as fuse request is filled, it can be submitted (in non-blocking way)
         by fuse_async_req_send()
       - when all submitted requests are ACKed by userspace, io->reqs drops to zero
         triggering aio_complete()
      
      In case of IO initiated by libaio, aio_complete() will finish processing the
      same way as in case of dio_complete() calling aio_complete(). But the
      framework may be also used for internal FUSE use when initial IO request
      was synchronous (from user perspective), but it's beneficial to process it
      asynchronously. Then the caller should wait on kiocb explicitly and
      aio_complete() will wake the caller up.
      Signed-off-by: NMaxim Patlasov <mpatlasov@parallels.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      01e9d11a
    • M
      fuse: move fuse_release_user_pages() up · 187c5c36
      Maxim Patlasov 提交于
      fuse_release_user_pages() will be indirectly used by fuse_send_read/write
      in future patches.
      Signed-off-by: NMaxim Patlasov <mpatlasov@parallels.com>
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      187c5c36
    • M
      fuse: optimize wake_up · 3c18ef81
      Miklos Szeredi 提交于
      Normally blocked_waitq will be inactive, so optimize this case.
      Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
      3c18ef81
  17. 17 4月, 2013 4 次提交
  18. 10 4月, 2013 1 次提交
    • A
      get rid of pipe->inode · 6447a3cf
      Al Viro 提交于
      it's used only as a flag to distinguish normal pipes/FIFOs from the
      internal per-task one used by file-to-file splice.  And pipe->files
      would work just as well for that purpose...
      Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
      6447a3cf