Before introducing redirect_dir feature, the condition
!ovl_lower_positive(dentry) for a directory, implied that it is a pure
upper directory, which may be removed if empty.

Now that directory can be redirect, it is possible that upper does not
cover any lower (i.e. !ovl_lower_positive(dentry)), but the directory is a
merge (with redirected path) and maybe non empty.

Check for this case in ovl_remove_upper().

This change fixes the following test case from rename-pop-dir.py
of unionmount-testsuite:

    """Remove dir and rename old name"""
    d = ctx.non_empty_dir()
    d2 = ctx.no_dir()

    ctx.rmdir(d, err=ENOTEMPTY)
    ctx.rename(d, d2)
    ctx.rmdir(d, err=ENOENT)
    ctx.rmdir(d2, err=ENOTEMPTY)

./run --ov rename-pop-dir
/mnt/a/no_dir103: Expected error (Directory not empty) was not produced
Signed-off-by: NAmir Goldstein <amir73il@gmail.com>
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

Current code returns EXDEV when a directory would need to be copied up to
move.  We could copy up the directory tree in this case, but there's
another, simpler solution: point to old lower directory from moved upper
directory.

This is achieved with a "trusted.overlay.redirect" xattr storing the path
relative to the root of the overlay.  After such attribute has been set,
the directory can be moved without further actions required.

This is a backward incompatible feature, old kernels won't be able to
correctly mount an overlay containing redirected directories.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

If a directory has the "trusted.overlay.redirect" xattr, it means that the
value of the xattr should be used to find the underlying directory on the
next lower layer.

The redirect may be relative or absolute.  Absolute redirects begin with a
slash.

A relative redirect means: instead of the current dentry's name use the
value of the redirect to find the directory in the next lower
layer. Relative redirects must not contain a slash.

An absolute redirect means: look up the directory relative to the root of
the overlay using the value of the redirect in the next lower layer.

Redirects work on lower layers as well.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

Use a common helper for lookup of upper and lower layers.  This paves the
way for looking up directory redirects.

No functional change.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

When the upper overlayfs checks "trusted.overlay.*" xattr on the underlying
overlayfs mount, it gets -EPERM, which confuses the upper overlayfs.

Fix this by returning -EOPNOTSUPP instead of -EPERM from
ovl_own_xattr_get() and ovl_own_xattr_set().  This behavior is consistent
with the behavior of ovl_listxattr(), which filters out the private
overlayfs xattrs.

Note: nested overlays are deprecated.  But this change makes sense
regardless: these xattrs are private to the overlay and should always be
hidden.  Hence getting and setting them should indicate this.

[SzMi: Use EOPNOTSUPP instead of ENODATA and use it for both getting and
setting "trusted.overlay." xattrs.  This is a perfectly valid error code
for "we don't support this prefix", which is the case here.]
Signed-off-by: NAmir Goldstein <amir73il@gmail.com>
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

We already calculate f_namelen in statfs as the maximum of the name lengths
provided by the filesystems taking part in the overlay.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

fs/overlayfs/super.c is the biggest of the overlayfs source files and it
contains various utility functions as well as the rather complicated lookup
code.  Split these parts out to separate files.

Before:

 1446 fs/overlayfs/super.c

After:

  919 fs/overlayfs/super.c
  267 fs/overlayfs/namei.c
  235 fs/overlayfs/util.c
   51 fs/overlayfs/ovl_entry.h
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

If encountering a non-directory, then stop looking at lower layers.

In this case the oe->opaque flag is not set anymore, which doesn't matter
since existence of lower file is now checked at remove/rename time.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

Check if something exists on the lower layer(s) under the target or rename
to decide if directory needs to be marked "opaque".

Marking opaque is done before the rename, and on failure the marking was
undone.  Also the opaque xattr was removed if the target didn't cover
anything.

This patch changes behavior so that removal of "opaque" is not done in
either of the above cases.  This means that directory may have the opaque
flag even if it doesn't cover anything.  However this shouldn't affect the
performance or semantics of the overalay, while simplifying the code.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

d_is_dir() is safe to call on a negative dentry.  Use this fact to simplify
handling of the lower or merged directories.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

The remainging uses of __OVL_PATH_PURE can be replaced by
ovl_dentry_is_opaque().
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

Currently ovl_lookup() checks existence of lower file even if there's a
non-directory on upper (which is always opaque).  This is done so that
remove can decide whether a whiteout is needed or not.

It would be better to defer this check to unlink, since most of the time
the gathered information about opaqueness will be unused.

This adds a helper ovl_lower_positive() that checks if there's anything on
the lower layer(s).

The following patches also introduce changes to how the "opaque" attribute
is updated on directories: this attribute is added when the directory is
creted or moved over a whiteout or object covering something on the lower
layer.  However following changes will allow the attribute to remain on the
directory after being moved, even if the new location doesn't cover
anything.  Because of this, we need to check lower layers even for opaque
directories, so that whiteout is only created when necessary.

This function will later be also used to decide about marking a directory
opaque, so deal with negative dentries as well.  When dealing with
negative, it's enough to check for being a whiteout

If the dentry is positive but not upper then it also obviously needs
whiteout/opaque.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

And use it instead of ovl_dentry_is_opaque() where appropriate.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

Since commit 07a2daab ("ovl: Copy up underlying inode's ->i_mode to
overlay inode") sticky checking on overlay inode is performed by the vfs,
so checking against sticky on underlying inode is not needed.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

This is redundant, the vfs already performed this check (and was broken,
see commit 9409e22a ("vfs: rename: check backing inode being equal")).
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

No sense in opening special files on the underlying layers, they work just
as well if opened on the overlay.

Side effect is that it's no longer possible to connect one side of a pipe
opened on overlayfs with the other side opened on the underlying layer.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

When copying up within the same fs, try to use vfs_clone_file_range().
This is very efficient when lower and upper are on the same fs
with file reflink support. If vfs_clone_file_range() fails for any
reason, copy up falls back to the regular data copy code.

Tested correct behavior when lower and upper are on:
1. same ext4 (copy)
2. same xfs + reflink patches + mkfs.xfs (copy)
3. same xfs + reflink patches + mkfs.xfs -m reflink=1 (reflink)
4. different xfs + reflink patches + mkfs.xfs -m reflink=1 (copy)

For comparison, on my laptop, xfstest overlay/001 (copy up of large
sparse files) takes less than 1 second in the xfs reflink setup vs.
25 seconds on the rest of the setups.
Signed-off-by: NAmir Goldstein <amir73il@gmail.com>
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

This reverts commit 03bea604.

Commit 4d0c5ba2 ("vfs: do get_write_access() on upper layer of
overlayfs") makes the writecount checks inside overlayfs superfluous, the
file is already copied up and write access acquired on the upper inode when
ovl_setattr is called with ATTR_SIZE.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

With overlayfs, it is wrong to compare file_inode(inode)->i_sb
of regular files with those of non-regular files, because the
former reference the real (upper/lower) sb and the latter reference
the overlayfs sb.

Move the test for same super block after the sanity tests for
clone range of directory and non-regular file.

This change fixes xfstest generic/157, which returned EXDEV instead
of EISDIR/EINVAL in the following test cases over overlayfs:

  echo "Try to reflink a dir"
  _reflink_range $testdir1/dir1 0 $testdir1/file2 0 $blksz

  echo "Try to reflink a device"
  _reflink_range $testdir1/dev1 0 $testdir1/file2 0 $blksz
Signed-off-by: NAmir Goldstein <amir73il@gmail.com>
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

Move sb_start_write()/sb_end_write() out of the vfs helper and up into the
ioctl handler.
Signed-off-by: NAmir Goldstein <amir73il@gmail.com>
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

FICLONE/FICLONERANGE ioctls return -EXDEV if src and dest
files are not on the same mount point.
Practically, clone only requires that src and dest files
are on the same file system.

Move the check for same mount point to ioctl handler and keep
only the check for same super block in the vfs helper.

A following patch is going to use the vfs_clone_file_range()
helper in overlayfs to copy up between lower and upper
mount points on the same file system.
Signed-off-by: NAmir Goldstein <amir73il@gmail.com>
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

We've checked for file_out being opened for write.  This ensures that we
already have mnt_want_write() on target.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

This reverts commit 9409e22a.

Since commit 51f7e52d ("ovl: share inode for hard link") there's no
need to call d_real_inode() to check two overlay inodes for equality.
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>

The ER records are printed without explicit log level presuming line
continuation until "\n".  After the commit 4bcc595c (printk:
reinstate KERN_CONT for printing continuation lines), the ER records are
printed a character per line.

Adding KERN_CONT to appropriate printk statements restores the printout
behavior.
Signed-off-by: NMike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Handling of recursion in d_real() is completely broken.  Recursion is only
done in the 'inode != NULL' case.  But when opening the file we have
'inode == NULL' hence d_real() will return an overlay dentry.  This won't
work since overlayfs doesn't define its own file operations, so all file
ops will fail.

Fix by doing the recursion first and the check against the inode second.

Bash script to reproduce the issue written by Quentin:

 - 8< - - - - - 8< - - - - - 8< - - - - - 8< - - - -
tmpdir=$(mktemp -d)
pushd ${tmpdir}

mkdir -p {upper,lower,work}
echo -n 'rocks' > lower/ksplice
mount -t overlay level_zero upper -o lowerdir=lower,upperdir=upper,workdir=work
cat upper/ksplice

tmpdir2=$(mktemp -d)
pushd ${tmpdir2}

mkdir -p {upper,work}
mount -t overlay level_one upper -o lowerdir=${tmpdir}/upper,upperdir=upper,workdir=work
ls -l upper/ksplice
cat upper/ksplice
 - 8< - - - - - 8< - - - - - 8< - - - - - 8< - - - - 
Reported-by: NQuentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: NMiklos Szeredi <mszeredi@redhat.com>
Fixes: 2d902671 ("vfs: merge .d_select_inode() into .d_real()")
Cc: <stable@vger.kernel.org> # v4.8+

Commit 2211d5ba ("posix_acl: xattr representation cleanups")
removes the typedefs and the zero-length a_entries array in struct
posix_acl_xattr_header, and uses bare struct posix_acl_xattr_header
and struct posix_acl_xattr_entry directly.

But it failed to iterate over posix acl slots when converting posix
acls to CIFS format, which results in several test failures in
xfstests (generic/053 generic/105) when testing against a samba v1
server, starting from v4.9-rc1 kernel. e.g.

  [root@localhost xfstests]# diff -u tests/generic/105.out /root/xfstests/results//generic/105.out.bad
  --- tests/generic/105.out       2016-09-19 16:33:28.577962575 +0800
  +++ /root/xfstests/results//generic/105.out.bad 2016-10-22 15:41:15.201931110 +0800
  @@ -1,3 +1,4 @@
   QA output created by 105
   -rw-r--r-- root
  +setfacl: subdir: Invalid argument
   -rw-r--r-- root

Fix it by introducing a new "ace" var, like what
cifs_copy_posix_acl() does, and iterating posix acl xattr entries
over it in the for loop.
Signed-off-by: NEryu Guan <guaneryu@gmail.com>
Signed-off-by: NSteve French <smfrench@gmail.com>

Commit 4fcd1813 ("Fix reconnect to not defer smb3 session reconnect
long after socket reconnect") changes the behaviour of the SMB2 echo
service and causes it to renegotiate after a socket reconnect. However
under default settings, the echo service could take up to 120 seconds to
be scheduled.

The patch forces the echo service to be called immediately resulting a
negotiate call being made immediately on reconnect.
Signed-off-by: NSachin Prabhu <sprabhu@redhat.com>
Reviewed-by: NPavel Shilovsky <pshilov@microsoft.com>
Signed-off-by: NSteve French <smfrench@gmail.com>

Andy Lutromirski's new virtually mapped kernel stack allocations moves
kernel stacks the vmalloc area. This triggers the bug
 kernel BUG at ./include/linux/scatterlist.h:140!
at calc_seckey()->sg_init()
Signed-off-by: NSachin Prabhu <sprabhu@redhat.com>
Signed-off-by: NSteve French <smfrench@gmail.com>
Reviewed-by: NJeff Layton <jlayton@redhat.com>

Botched calculation of number of pages.  As the result,
we were dropping pieces when doing splice to pipe from
e.g. 9p.
Reported-by: NAlexei Starovoitov <ast@kernel.org>
Tested-by: NAlexei Starovoitov <ast@kernel.org>
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

A correct bugfix introduced a harmless warning that shows up with gcc-7:

fs/nfs/callback.c: In function 'nfs_callback_up':
fs/nfs/callback.c:214:14: error: array subscript is outside array bounds [-Werror=array-bounds]

What happens here is that the 'minorversion == 0' check tells the
compiler that we assume minorversion can be something other than 0,
but when CONFIG_NFS_V4_1 is disabled that would be invalid and
result in an out-of-bounds access.

The added check for IS_ENABLED(CONFIG_NFS_V4_1) tells gcc that this
really can't happen, which makes the code slightly smaller and also
avoids the warning.

The bugfix that introduced the warning is marked for stable backports,
we want this one backported to the same releases.

Fixes: 98b0f80c ("NFSv4.x: Fix a refcount leak in nfs_callback_up_net")
Cc: stable@vger.kernel.org # v3.7+
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NAnna Schumaker <Anna.Schumaker@Netapp.com>

While walking the list of lock_states, keep a reference on each
nfs4_lock_state to be checked, otherwise the lock state could be removed
while the check performs TEST_STATEID and possible FREE_STATEID.
Signed-off-by: NBenjamin Coddington <bcodding@redhat.com>
Signed-off-by: NAnna Schumaker <Anna.Schumaker@Netapp.com>

If the block size or cluster size is insane, reject the mount.  This
is important for security reasons (although we shouldn't be just
depending on this check).

Ref: http://www.securityfocus.com/archive/1/539661
Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1332506Reported-by: NBorislav Petkov <bp@alien8.de>
Reported-by: NNikolay Borisov <kernel@kyup.com>
Signed-off-by: NTheodore Ts'o <tytso@mit.edu>
Cc: stable@vger.kernel.org

With the new (in 4.9) option to use a virtually-mapped stack
(CONFIG_VMAP_STACK), stack buffers cannot be used as input/output for
the scatterlist crypto API because they may not be directly mappable to
struct page.  get_crypt_info() was using a stack buffer to hold the
output from the encryption operation used to derive the per-file key.
Fix it by using a heap buffer.

This bug could most easily be observed in a CONFIG_DEBUG_SG kernel
because this allowed the BUG in sg_set_buf() to be triggered.

Cc: stable@vger.kernel.org
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NTheodore Ts'o <tytso@mit.edu>

With the new (in 4.9) option to use a virtually-mapped stack
(CONFIG_VMAP_STACK), stack buffers cannot be used as input/output for
the scatterlist crypto API because they may not be directly mappable to
struct page.  For short filenames, fname_encrypt() was encrypting a
stack buffer holding the padded filename.  Fix it by encrypting the
filename in-place in the output buffer, thereby making the temporary
buffer unnecessary.

This bug could most easily be observed in a CONFIG_DEBUG_SG kernel
because this allowed the BUG in sg_set_buf() to be triggered.

Cc: stable@vger.kernel.org
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NTheodore Ts'o <tytso@mit.edu>

Now that we're doing TEST_STATEID in nfs4_reclaim_open_state(), we can have
a NFS4ERR_OLD_STATEID returned from nfs41_open_expired() .  Instead of
marking state recovery as failed, mark the state for recovery again.
Signed-off-by: NBenjamin Coddington <bcodding@redhat.com>
Signed-off-by: NAnna Schumaker <Anna.Schumaker@Netapp.com>

Ensure we test to see if the open stateid is actually set, before we
send a CLOSE.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: NAnna Schumaker <Anna.Schumaker@Netapp.com>

If the reply to a successful CLOSE call races with an OPEN to the same
file, we can end up scribbling over the stateid that represents the
new open state.
The race looks like:

  Client				Server
  ======				======

  CLOSE stateid A on file "foo"
					CLOSE stateid A, return stateid C
  OPEN file "foo"
					OPEN "foo", return stateid B
  Receive reply to OPEN
  Reset open state for "foo"
  Associate stateid B to "foo"

  Receive CLOSE for A
  Reset open state for "foo"
  Replace stateid B with C

The fix is to examine the argument of the CLOSE, and check for a match
with the current stateid "other" field. If the two do not match, then
the above race occurred, and we should just ignore the CLOSE.
Reported-by: NBenjamin Coddington <bcodding@redhat.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: NAnna Schumaker <Anna.Schumaker@Netapp.com>

We don't want to call nfs4_free_revoked_stateid() in the case where
the delegreturn was successful.
Reported-by: NBenjamin Coddington <bcodding@redhat.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: NAnna Schumaker <Anna.Schumaker@Netapp.com>