1. 03 4月, 2009 3 次提交
    • D
      FS-Cache: Add netfs registration · 726dd7ff
      David Howells 提交于
      Add functions to register and unregister a network filesystem or other client
      of the FS-Cache service.  This allocates and releases the cookie representing
      the top-level index for a netfs, and makes it available to the netfs.
      
      If the FS-Cache facility is disabled, then the calls are optimised away at
      compile time.
      
      Note that whilst this patch may appear to work with FS-Cache enabled and a
      netfs attempting to use it, it will leak the cookie it allocates for the netfs
      as fscache_relinquish_cookie() is implemented in a later patch.  This will
      cause the slab code to emit a warning when the module is removed.
      Signed-off-by: NDavid Howells <dhowells@redhat.com>
      Acked-by: NSteve Dickson <steved@redhat.com>
      Acked-by: NTrond Myklebust <Trond.Myklebust@netapp.com>
      Acked-by: NAl Viro <viro@zeniv.linux.org.uk>
      Tested-by: NDaire Byrne <Daire.Byrne@framestore.com>
      726dd7ff
    • D
      FS-Cache: Add cache tag handling · 0e04d4ce
      David Howells 提交于
      Implement two features of FS-Cache:
      
       (1) The ability to request and release cache tags - names by which a cache may
           be known to a netfs, and thus selected for use.
      
       (2) An internal function by which a cache is selected by consulting the netfs,
           if the netfs wishes to be consulted.
      Signed-off-by: NDavid Howells <dhowells@redhat.com>
      Acked-by: NSteve Dickson <steved@redhat.com>
      Acked-by: NTrond Myklebust <Trond.Myklebust@netapp.com>
      Acked-by: NAl Viro <viro@zeniv.linux.org.uk>
      Tested-by: NDaire Byrne <Daire.Byrne@framestore.com>
      0e04d4ce
    • D
      FS-Cache: Add the FS-Cache netfs API and documentation · 2d6fff63
      David Howells 提交于
      Add the API for a generic facility (FS-Cache) by which filesystems (such as AFS
      or NFS) may call on local caching capabilities without having to know anything
      about how the cache works, or even if there is a cache:
      
      	+---------+
      	|         |                        +--------------+
      	|   NFS   |--+                     |              |
      	|         |  |                 +-->|   CacheFS    |
      	+---------+  |   +----------+  |   |  /dev/hda5   |
      	             |   |          |  |   +--------------+
      	+---------+  +-->|          |  |
      	|         |      |          |--+
      	|   AFS   |----->| FS-Cache |
      	|         |      |          |--+
      	+---------+  +-->|          |  |
      	             |   |          |  |   +--------------+
      	+---------+  |   +----------+  |   |              |
      	|         |  |                 +-->|  CacheFiles  |
      	|  ISOFS  |--+                     |  /var/cache  |
      	|         |                        +--------------+
      	+---------+
      
      General documentation and documentation of the netfs specific API are provided
      in addition to the header files.
      
      As this patch stands, it is possible to build a filesystem against the facility
      and attempt to use it.  All that will happen is that all requests will be
      immediately denied as if no cache is present.
      
      Further patches will implement the core of the facility.  The facility will
      transfer requests from networking filesystems to appropriate caches if
      possible, or else gracefully deny them.
      
      If this facility is disabled in the kernel configuration, then all its
      operations will trivially reduce to nothing during compilation.
      
      WHY NOT I_MAPPING?
      ==================
      
      I have added my own API to implement caching rather than using i_mapping to do
      this for a number of reasons.  These have been discussed a lot on the LKML and
      CacheFS mailing lists, but to summarise the basics:
      
       (1) Most filesystems don't do hole reportage.  Holes in files are treated as
           blocks of zeros and can't be distinguished otherwise, making it difficult
           to distinguish blocks that have been read from the network and cached from
           those that haven't.
      
       (2) The backing inode must be fully populated before being exposed to
           userspace through the main inode because the VM/VFS goes directly to the
           backing inode and does not interrogate the front inode's VM ops.
      
           Therefore:
      
           (a) The backing inode must fit entirely within the cache.
      
           (b) All backed files currently open must fit entirely within the cache at
           	 the same time.
      
           (c) A working set of files in total larger than the cache may not be
           	 cached.
      
           (d) A file may not grow larger than the available space in the cache.
      
           (e) A file that's open and cached, and remotely grows larger than the
           	 cache is potentially stuffed.
      
       (3) Writes go to the backing filesystem, and can only be transferred to the
           network when the file is closed.
      
       (4) There's no record of what changes have been made, so the whole file must
           be written back.
      
       (5) The pages belong to the backing filesystem, and all metadata associated
           with that page are relevant only to the backing filesystem, and not
           anything stacked atop it.
      
      OVERVIEW
      ========
      
      FS-Cache provides (or will provide) the following facilities:
      
       (1) Caches can be added / removed at any time, even whilst in use.
      
       (2) Adds a facility by which tags can be used to refer to caches, even if
           they're not available yet.
      
       (3) More than one cache can be used at once.  Caches can be selected
           explicitly by use of tags.
      
       (4) The netfs is provided with an interface that allows either party to
           withdraw caching facilities from a file (required for (1)).
      
       (5) A netfs may annotate cache objects that belongs to it.  This permits the
           storage of coherency maintenance data.
      
       (6) Cache objects will be pinnable and space reservations will be possible.
      
       (7) The interface to the netfs returns as few errors as possible, preferring
           rather to let the netfs remain oblivious.
      
       (8) Cookies are used to represent indices, files and other objects to the
           netfs.  The simplest cookie is just a NULL pointer - indicating nothing
           cached there.
      
       (9) The netfs is allowed to propose - dynamically - any index hierarchy it
           desires, though it must be aware that the index search function is
           recursive, stack space is limited, and indices can only be children of
           indices.
      
      (10) Indices can be used to group files together to reduce key size and to make
           group invalidation easier.  The use of indices may make lookup quicker,
           but that's cache dependent.
      
      (11) Data I/O is effectively done directly to and from the netfs's pages.  The
           netfs indicates that page A is at index B of the data-file represented by
           cookie C, and that it should be read or written.  The cache backend may or
           may not start I/O on that page, but if it does, a netfs callback will be
           invoked to indicate completion.  The I/O may be either synchronous or
           asynchronous.
      
      (12) Cookies can be "retired" upon release.  At this point FS-Cache will mark
           them as obsolete and the index hierarchy rooted at that point will get
           recycled.
      
      (13) The netfs provides a "match" function for index searches.  In addition to
           saying whether a match was made or not, this can also specify that an
           entry should be updated or deleted.
      
      FS-Cache maintains a virtual index tree in which all indices, files, objects
      and pages are kept.  Bits of this tree may actually reside in one or more
      caches.
      
                                                 FSDEF
                                                   |
                              +------------------------------------+
                              |                                    |
                             NFS                                  AFS
                              |                                    |
                 +--------------------------+                +-----------+
                 |                          |                |           |
              homedir                     mirror          afs.org   redhat.com
                 |                          |                            |
           +------------+           +---------------+              +----------+
           |            |           |               |              |          |
         00001        00002       00007           00125        vol00001   vol00002
           |            |           |               |                         |
       +---+---+     +-----+      +---+      +------+------+            +-----+----+
       |   |   |     |     |      |   |      |      |      |            |     |    |
      PG0 PG1 PG2   PG0  XATTR   PG0 PG1   DIRENT DIRENT DIRENT        R/W   R/O  Bak
                           |                                            |
                          PG0                                       +-------+
                                                                    |       |
                                                                  00001   00003
                                                                    |
                                                                +---+---+
                                                                |   |   |
                                                               PG0 PG1 PG2
      
      In the example above, two netfs's can be seen to be backed: NFS and AFS.  These
      have different index hierarchies:
      
       (*) The NFS primary index will probably contain per-server indices.  Each
           server index is indexed by NFS file handles to get data file objects.
           Each data file objects can have an array of pages, but may also have
           further child objects, such as extended attributes and directory entries.
           Extended attribute objects themselves have page-array contents.
      
       (*) The AFS primary index contains per-cell indices.  Each cell index contains
           per-logical-volume indices.  Each of volume index contains up to three
           indices for the read-write, read-only and backup mirrors of those volumes.
           Each of these contains vnode data file objects, each of which contains an
           array of pages.
      
      The very top index is the FS-Cache master index in which individual netfs's
      have entries.
      
      Any index object may reside in more than one cache, provided it only has index
      children.  Any index with non-index object children will be assumed to only
      reside in one cache.
      
      The FS-Cache overview can be found in:
      
      	Documentation/filesystems/caching/fscache.txt
      
      The netfs API to FS-Cache can be found in:
      
      	Documentation/filesystems/caching/netfs-api.txt
      Signed-off-by: NDavid Howells <dhowells@redhat.com>
      Acked-by: NSteve Dickson <steved@redhat.com>
      Acked-by: NTrond Myklebust <Trond.Myklebust@netapp.com>
      Acked-by: NAl Viro <viro@zeniv.linux.org.uk>
      Tested-by: NDaire Byrne <Daire.Byrne@framestore.com>
      2d6fff63