Otherwise we are risking a hard error where nonlazy restart would be the right
thing to do; it's a very narrow race with mount --move and most of the time it
ends up being completely harmless, but it's possible to construct a case when
we'll get a bogus hard error instead of falling back to non-lazy walk...

For one thing, when crossing _into_ overmount of parent we need to check for
mount_lock bumps when we get NULL from __lookup_mnt() as well.

For another, and less exotically, we need to make sure that the data fetched
in follow_up_rcu() had been consistent.  ->mnt_mountpoint is pinned for as
long as it is a mountpoint, but we need to check mount_lock after fetching
to verify that.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

now that we have ->root_seq, legitimize_path(&nd->root, nd->root_seq)
will do just fine...
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

touch_atime is not RCU-safe, and so cannot be called on an RCU walk.
However, in situations where RCU-walk makes a difference, the symlink
will likely to accessed much more often than it is useful to update
the atime.

So split out the test of "Does the atime actually need to be updated"
into  atime_needs_update(), and have get_link() unlazy if it finds that
it will need to do that update.
Signed-off-by: NNeilBrown <neilb@suse.de>
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

We are almost done - primitives for leaving RCU mode are aware of nd->stack
now, a new primitive for going to non-RCU mode when we have a symlink on hands
added.

The thing we are heavily relying upon is that *any* unlazy failure will be
shortly followed by terminate_walk(), with no access to nameidata in between.
So it's enough to leave the things in a state terminate_walk() would cope with.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

we'll need them for unlazy_walk()
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

We *can't* call that audit garbage in RCU mode - it's doing a weird
mix of allocations (GFP_NOFS, immediately followed by GFP_KERNEL)
and I'm not touching that... thing again.

So if this security sclero^Whardening feature gets triggered when
we are in RCU mode, tough - we'll fail with -ECHILD and have
everything restarted in non-RCU mode.  Only to hit the same test
and fail, this time with EACCES and with (oh, rapture) an audit spew
produced.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

very simple - just make path_put() conditional on !RCU.
Note that right now it doesn't get called in RCU mode -
we leave it before getting anything into stack.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

only one instance looks at that argument at all; that sole
exception wants inode rather than dentry.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

inode_follow_link now takes an inode and rcu flag as well as the
dentry.

inode is used in preference to d_backing_inode(dentry), particularly
in RCU-walk mode.

selinux_inode_follow_link() gets dentry_has_perm() and
inode_has_perm() open-coded into it so that it can call
avc_has_perm_flags() in way that is safe if LOOKUP_RCU is set.

Calling avc_has_perm_flags() with rcu_read_lock() held means
that when avc_has_perm_noaudit calls avc_compute_av(), the attempt
to rcu_read_unlock() before calling security_compute_av() will not
actually drop the RCU read-lock.

However as security_compute_av() is completely in a read_lock()ed
region, it should be safe with the RCU read-lock held.
Signed-off-by: NNeilBrown <neilb@suse.de>
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

no need to refetch (and once we move unlazy out of there, recheck ->d_seq).
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Make use of d_backing_inode() in pathwalk to gain access to an
inode or dentry that's on a lower layer.
Signed-off-by: NDavid Howells <dhowells@redhat.com>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Lift it from link_path_walk(), trailing_symlink(), lookup_last(),
mountpoint_last(), complete_walk() and do_last().  A _lot_ of
those suckers merge.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Make trailing_symlink() return the pathname to traverse or ERR_PTR(-E...).
A subtle point is that for "magic" symlinks it returns "" now - that
leads to link_path_walk("", nd), which is immediately returning 0 and
we are back to the treatment of the last component, at whereever the
damn thing has left us.

Reduces the stack footprint - link_path_walk() called on more shallow
stack now.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

* lift link_path_walk() into callers; moving it down into path_init()
had been a mistake.  Stack footprint, among other things...
* do _not_ call path_cleanup() after path_init() failure; on all failure
exits out of it we have nothing for path_cleanup() to do
* have path_init() return pathname or ERR_PTR(-E...)
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

we can do fdput() under rcu_read_lock() just fine; all we need to take
care of is fetching nd->inode value first.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

new functions: filename_parentat() and path_parentat() resp.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Makes the situation much more regular - we avoid a strange state
when the element just after the top of stack is used to store
struct path of symlink, but isn't counted in nd->depth.  This
is much more regular, so the normal failure exits, etc., work
fine.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Just store it in nd->stack[nd->depth].link right in pick_link().
Now that we make sure of stack expansion in pick_link(), we can
do so...
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

update the failure cleanup in may_follow_link() to match that.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

... and don't open-code unlazy_walk() in there - the only reason
for that is to avoid verfication of cached nd->root, which is
trivially avoided by discarding said cached nd->root first.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

rather than letting the callers handle the jump-to-root part of
semantics, do it right in get_link() and return the rest of the
body for the caller to deal with - at that point it's treated
the same way as relative symlinks would be.  And return NULL
when there's no "rest of the body" - those are treated the same
as pure jump symlink would be.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Instead of saving name and branching to OK:, where we'll immediately restore
it, and call walk_component() with WALK_PUT|WALK_GET and nd->last_type being
LAST_BIND, which is equivalent to put_link(nd), err = 0, we can just treat
that the same way we'd treat procfs-style "jump" symlinks - do put_link(nd)
and move on.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

when cookie is NULL, put_link() is equivalent to path_put(), so
as soon as we'd set last->cookie to NULL, we can bump nd->depth and
let the normal logics in terminate_walk() to take care of cleanups.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

its only use is getting passed to nd_jump_link(), which can obtain
it from current->nameidata
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

now that it gets nameidata, no reason to have setting LOOKUP_JUMPED on
mountpoint crossing and calling path_put_conditional() on failures
done in every caller.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

task_struct currently contains two ad-hoc members for use by the VFS:
link_count and total_link_count.  These are only interesting to fs/namei.c,
so exposing them explicitly is poor layering.  Incidentally, link_count
isn't used anymore, so it can just die.

This patches replaces those with a single pointer to 'struct nameidata'.
This structure represents the current filename lookup of which
there can only be one per process, and is a natural place to
store total_link_count.

This will allow the current "nameidata" argument to all
follow_link operations to be removed as current->nameidata
can be used instead in the _very_ few instances that care about
it at all.

As there are occasional circumstances where pathname lookup can
recurse, such as through kern_path_locked, we always save and old
current->nameidata (if there is one) when setting a new value, and
make sure any active link_counts are preserved.

follow_mount and follow_automount now get a 'struct nameidata *'
rather than 'int flags' so that they can directly access
total_link_count, rather than going through 'current'.
Suggested-by: NAl Viro <viro@ZenIV.linux.org.uk>
Signed-off-by: NNeilBrown <neilb@suse.de>
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

... if it decides to follow, that is.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

instead of a single flag (!= 0 => we want to follow symlinks) pass
two bits - WALK_GET (want to follow symlinks) and WALK_PUT (put_link()
once we are done looking at the name).  The latter matters only for
success exits - on failure the caller will discard everything anyway.

Suggestions for better variant are welcome; what this thing aims for
is making sure that pending put_link() is done *before* walk_component()
decides to pick a symlink up, rather than between picking it up and
acting upon it.  See the next commit for payoff.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

fewer labels that way; in particular, resuming after the end of
nested symlink is straight-line.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

All callers of terminate_walk() are followed by more or less
open-coded eqiuvalent of "do put_link() on everything left
in nd->stack".  Better done in terminate_walk() itself, and
when we go for RCU symlink traversal we'll have to do it
there anyway.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>