The sequence number we store in the sequence id is the last one we received
from the client.  So on the next operation we'll check that the client gives
us the next higher number.

We increment sequence id's at the last moment, in encode, so that we're sure
of knowing the right error return.  (The decision to increment the sequence id
depends on the exact error returned.)

However on the *first* use of a sequence number, if we set the sequence number
to the one received from the client and then let the increment happen on
encode, we'll be left with a sequence number one to high.

For that reason, ENCODE_SEQID_OP_TAIL only increments the sequence id on
*confirmed* stateowners.

This creates a problem for open reclaims, which are confirmed on first use.
Therefore the open reclaim code, as a special exception, *decrements* the
sequence id, cancelling out the undesired increment on encode.  But this
prevents the sequence id from ever being incremented in the case where
multiple reclaims are sent with the same openowner.  Yuch!

We could add another exception to the open reclaim code, decrementing the
sequence id only if this is the first use of the open owner.

But it's simpler by far to modify the meaning of the op_seqid field: instead
of representing the previous value sent by the client, we take op_seqid, after
encoding, to represent the *next* sequence id that we expect from the client.
This eliminates the need for special-case handling of the first use of a
stateowner.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Yeah, it's trivial, but this drives me up the wall....
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

A misreading of the spec lead us to convert all errors on open and lock
reclaims to RECLAIM_BAD.  This causes problems--for example, a reboot within
the grace period could lead to reclaims with stale stateid's, and we'd like to
return STALE errors in those cases.

What rfc3530 actually says about RECLAIM_BAD: "The reclaim provided by the
client does not match any of the server's state consistency checks and is
bad." I'm assuming that "state consistency checks" refers to checks for
consistency with the state recorded to stable storage, and that the error
should be reserved for that case.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

A GRACE or NOGRACE response to a lock request should also bump the sequence
id.  So we delay the handling of grace period errors till after we've found
the relevant owner.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

The GRACE and NOGRACE errors should bump the sequence id on open.  So we delay
the handling of these errors until nfsd4_process_open2, at which point we've
set the open owner, so the encode routine will be able to bump the sequence
id.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

We oops in list_for_each_entry(), because release_stateowner frees something
on the list we're traversing.
Signed-off-by: NAndy Adamson <andros@citi.umich.edu>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

We need to remove the recovery directory here too.  (This chunk just got lost
somehow in the process of commuting the reboot recovery patches past the other
patches.)
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Set the recovery directory via /proc/fs/nfsd/nfs4recoverydir.

It may be changed any time, but is used only on startup.
Signed-off-by: NAndy Adamson <andros@citi.umich.edu>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

This patch adds the code to create and remove client subdirectories from the
recovery directory, as described in the previous patch comment.
Signed-off-by: NAndy Adamson <andros@citi.umich.edu>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

NFSv4 clients are required to know what state they have on the server so that
they can reclaim it on server reboot.  However, it is possible for
pathalogical combinations of server reboots and network partitions to leave a
client in a state where it cannot know whether it has lost its state on the
server.

For this reason, rfc3530 requires that we store some information about clients
to stable storage.

So we maintain a directory /var/lib/nfs/v4recovery with a subdirectory for
each client with active state.  We leave open the possibility of including
files underneath each such subdirectory with information about the client, but
for now the subdirectories are empty.

We create a client subdirectory whenever a client makes its first non-reclaim
open_confirm.

We remove a client subdirectory whenever either
        a) its lease expires, or
	b) the grace period ends without it reclaiming anything.
When handling reclaims, we allow the reclaim if and only if the client doing
the reclaim has a subdirectory.

This patch adds just the code to scan the recovery directory on nfsd startup.
Signed-off-by: NAndy Adamson <andros@citi.umich.edu>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

The cb_parsed field is only used by probe_callback, to determine whether the
callback information has been filled in by setclientid.  But there is no way
that probe_callback() can be called without that having already happened, so
that check is superfluous, as is cb_parsed.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

>From the language of rfc3530 section 8.1.3 (e.g., the suggestion that a
"process id" might be a reasonable lockowner value) it's conceivable that a
client might want to use the same lockowner string on multiple files, so we may
as well allow that.  We expect each use of open_to_lockowner to create a
distinct seqid stream, though.

For now we're also allowing multiple uses of open_to_lockowner with the same
open, though it seems unlikely clients would actually do that.

Also add a comment reminding myself of some very non-scalable data structures.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Trivial renaming patch:

I can never remember, while looking at various lists relating the nfsd4 state
structures, which are the "heads" and which are items on other lists, or which
structures are actually on the various lists.  The following convention helps
me: given structures foo and bar, with foo containing the head of a list of
bars, use "bars" for the name of the head of the list contained in the struct
foo, and use "per_foo" for the entries in the struct bars.

Already done for struct nfs4_file; go ahead and do it for the other nfsd4
state structures.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Minor cleanup, remove some unnecessary printk's.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Trivial whitespace and comment fixes.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Change from "goto" to "else if" format in setclientid_confirm.

From: Fred Isaman
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

NFS4_INVAL is not a valid error for setclientid_confirm, and INUSE is the more
logical error here anyway.

From: Fred Isaman
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Setclientid_confirm code confused states 1 and 3 (numbering from the
IMPLEMENTATION section of rfc3530, section 14.2.33).  Fix this.

State 1 allows the client to change the callback channel on the fly.  We don't
implement this currently, so just turn off the callback channel in this case.

From: Fred Isaman
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Setclientid code assumes there is only one match in unconfirmed list.
Make sure that assumption holds.

From: Fred Isaman
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

This patch contains the following possible cleanups:

- make needlessly global code static
Signed-off-by: NAdrian Bunk <bunk@stusta.de>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

For the purposes of reboot recovery, we want to do some work during the
transition period at the end of the grace period.  Some of that work must be
guaranteed to have a certain relationship with the end of the grace period, so
we want to control the transition there.

Our approach is to modify the in_grace() checks to consult a global variable
instead of checking the time directly, to schedule the first run of the
laundromat thread at the end of the grace period, and to set the global
end-of-grace-period there.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Minor setclientid cleanup
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

For the purposes of reboot recovery we keep a directory with subdirectories
each having a name that is the ascii hex representation of the md5 sum of a
client identifier for an active client.

This adds the code to calculate that name.  We also use it for the purposes of
comparing clients, so if someone ever manages to find two client names that
are md5 collisions, then we'll return clid_inuse to the second.
Signed-off-by: NAndy Adamson <andros@citi.umich.edu>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

We can be a little more concise here.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

nfs4_reclaim_init is no longer performing any useful function.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Separate out stuff that needs initialization on startup from stuff that only
needs initialization on module init from static data.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Somewhat gratuitous rename to simplify following patch.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Allow recovery of delegations after reboot.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

The only way the protocol gives to change the lease time on the fly is to
simulate a reboot.  We don't have that completely right in the current code;
among other things, we should probably put lockd in grace too while we do
this.

For now, let's just keep this simple, and wait till the next time nfsd starts
to register any changes in lease time.  If the administrator really wants to
change the lease time *now*, they can go ahead and bring nfsd down and then
back up again after changing the lease time.

Also remove the "if (reclaim_str_hashtbl_size == 0)" case, a shortcut which
skips the grace period if we know of no clients in need of recovery.  This
isn't going to work well with nlm.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

We're running the laundromat work on the default kevent worker thread.  But
the laundromat takes the nfsv4 state semaphore, which is used for way too much
stuff, and the potential for deadlocks is high.  Better to have this on a
separate workqueue.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Minor cleanup.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Add a struct kref to each nfs4_file and take a reference to it from each
stateid and delegation that refers to it.  The atomicity guarantees are
overkill given that all this stuff is done under the single nfsd4 state lock,
but a) we'd like finer-grained locking some day, and b) this simplifies the
cleanup of the structures a bit, something that has previously been a bit
complicated and bug-prone.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Trivial renaming patch:

I can never remember, while looking at various lists relating the nfsd4 state
structures, which are the "heads" and which are items on other lists, or which
structures are actually on the various lists.  The following convention helps
me: given structures foo and bar, with foo containing the head of a list of
bars, use "bars" for the name of the head of the list contained in the struct
foo, and use "per_foo" for the entries in the struct bars.

Go ahead and do this for struct nfs4_file.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

These remaining debugging counters haven't proved that useful.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Allocate delegations from a slab cache.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Allocate stateid's from a slab cache.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

The structures the server uses to keep track of various pieces of nfsv4 state
(open files, outstanding delegations, etc.) are likely to be allocated and
deallocated frequently and seem reasonable candidates for slab caches.

While we're at it, the slab code keeps statistics that help catch leaks and
such, so we may as well take this chance to eliminate some debugging counters
that we've been keeping ourselves.

Start with the struct nfs4_file.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

State logic for OPEN with claim type CLAIM_DELEGATE_CUR, which the NFSv4
client uses to report local OPENs on a delegated file back to the NFSv4
server.

nfs4_check_deleg() performs input delegation stateid lookup and sanity check.
Signed-off-by: NAndy Adamson <andros@citi.umich.edu>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

We don't really need to be doing a separate open for every stateid.  And in
the case of an open from a client that already has a delegation on a file, it
unnecessarily results in a delegation recall.
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Additional minor code reshuffling to prepare for claim_deleg_cur support.
Signed-off-by: NAndy Adamson <andros@citi.umich.edu>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NNeil Brown <neilb@cse.unsw.edu.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>