Eliminate the persistent use of automatic storage in all parts of the
NFS client's direct write path to pave the way for introducing support
for aio against files opened with the O_DIRECT flag.

Test plan:
Compile the kernel with CONFIG_NFS and CONFIG_NFS_DIRECTIO enabled.
Millions of fsx-odirect ops.  OraSim.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Duplicate infrastructure from direct read path that will allow write
path to generate multiple write requests concurrently.  This will
enable us to add support for aio in this path.

Temporarily we will lose the ability to do UNSTABLE writes followed by
a COMMIT in the direct write path.  However, all applications I am
aware of that use NFS O_DIRECT currently write in relatively small
chunks, so this should not be inconvenient in any way.

Test plan:
Millions of fsx-odirect ops. OraSim.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Factor out the common piece of completing an NFS direct I/O request.

Test plan:
Compile kernel with CONFIG_NFS and CONFIG_NFS_DIRECTIO enabled.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Factor out a small common piece of the path that allocate nfs_direct_req
structures.

Test plan:
Compile kernel with CONFIG_NFS and CONFIG_NFS_DIRECTIO enabled.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

We're about to add asynchrony to the NFS direct write path.  Begin by
abstracting out the common pieces in the read path.

The first piece is nfs_direct_read_wait, which works the same whether the
process is waiting for a read or a write.

Test plan:
Compile kernel with CONFIG_NFS and CONFIG_NFS_DIRECTIO enabled.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

For async iocb's, the NFS direct read path should return EIOCBQUEUED and
call aio_complete when all the requested reads are finished.  The
synchronous part of the NFS direct read path behaves exactly as it was
before.

Test plan:
aio-stress with "-O".  OraSim.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Pass the iocb argument all the way down to the direct read request
scheduler, and make it available in nfs_direct_read_result.

Test plan:
Compile the kernel with CONFIG_NFS and CONFIG_NFS_DIRECTIO enabled.
Millions of fsx-odirect ops.  OraSim.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Eliminate the persistent use of automatic storage in all parts of the NFS
client's direct read path to pave the way for introducing support for aio
against files opened with the O_DIRECT flag.

Test plan:
Compile the kernel with CONFIG_NFS and CONFIG_NFS_DIRECTIO enabled.
Millions of fsx-odirect ops.  OraSim.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

size_t is used for holding byte counts, so use it for variables storing rsize.
Note that the write path will be updated as we add support for async
O_DIRECT writes.

Test plan:
Need to verify that existing comparisons against new size_t variables behave
correctly.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Update to latest coding style standards.  Remove block comments on
statically defined functions, and place function definitions all on
one line.

Test plan:
Compile kernel with CONFIG_NFS and CONFIG_NFS_DIRECTIO.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

The NFS client's a_ops->direct_IO method, nfs_direct_IO, is required to
be present to allow NFS files to be opened with O_DIRECT, but is never
called because the NFS client shunts reads and writes to files opened
with O_DIRECT directly to its own routines.

Gut the nfs_direct_IO function.  This eliminates the only part of the
NFS client's direct I/O path that requires support for multi-segment
iovs, allowing further simplification in subsequent patches.

Test plan:
Compile the kernel with CONFIG_NFS and CONFIG_NFS_DIRECTIO enabled.  Millions
of fsx-odirect ops.  OraSim.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Same callback hierarchy inversion as for the NFS write calls. This patch is
not strictly speaking needed by the O_DIRECT code, but avoids confusing
differences between the asynchronous read and write code.
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

This patch inverts the callback hierarchy for NFS write calls.

Instead of having the NFSv2/v3/v4-specific code set up the RPC callback
ops, we allow the original caller to do so. This allows for more
flexibility w.r.t. how to set up and tear down the nfs_write_data
structure while still allowing the NFSv3/v4 code to perform error
handling.

The greater flexibility is needed by the asynchronous O_DIRECT code, which
wants to be able to hold on to the original nfs_write_data structures after
the WRITE RPC call has completed in order to be able to replay them if the
COMMIT call determines that the server has rebooted.
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

posix_test_lock() returns a pointer to a struct file_lock which is unprotected
and can be removed while in use by the caller.  Move the conflicting lock from
the return to a parameter, and copy the conflicting lock.

In most cases the caller ends up putting the copy of the conflicting lock on
the stack.  On i386, sizeof(struct file_lock) appears to be about 100 bytes.
We're assuming that's reasonable.
Signed-off-by: NAndy Adamson <andros@citi.umich.edu>
Signed-off-by: NJ. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Reuse NFSDBG_DIRCACHE and NFSDBG_LOOKUPCACHE to provide additional
diagnostic messages that trace the operation of the NFS client's
directory behavior.  A few new messages are now generated when NFSDBG_VFS
is active, as well, to trace normal VFS activity.  This compromise
provides better trace debugging for those who use pre-built kernels,
without adding a lot of extra noise to the standard debug settings.

Test-plan:
Enable NFS trace debugging with flags 1, 2, or 4.  You should be able to
see different types of trace messages with each flag setting.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Clean-up: replace rpc_call() helper with direct call to rpc_call_sync.

This makes NFSv2 and NFSv3 synchronous calls more computationally
efficient, and reduces stack consumption in functions that used to
invoke rpc_call more than once.

Test plan:
Compile kernel with CONFIG_NFS enabled.  Connectathon on NFS version 2,
version 3, and version 4 mount points.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Add fields to the rpc_procinfo struct that allow the display of a
human-readable name for each procedure in the rpc_iostats output.

Also fix it so that the NFSv4 stats are broken up correctly by
sub-procedure number.  NFSv4 uses only two real RPC procedures:
NULL, and COMPOUND.

Test plan:
Mount with NFSv2, NFSv3, and NFSv4, and do "cat /proc/self/mountstats".
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

NFS client now shows various RPC I/O metrics in /proc/self/mountstats.

Test plan:
Mount/umount while doing "cat /proc/self/mountstats", multiple iterations
of connectathon locking suite.  Test with NFS version 2, 3, and 4.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Add a field in nfs_server to record a timestamp when a mount succeeds.
Report the number of seconds the file system has been mounted via
nfs_show_stats().

Test plan:
Mount an NFS file system, watch the mountstats reports and compare with
clock time.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Make an inode or an nfs_server struct available in the logic that handles
JUKEBOX/DELAY type errors so the NFS client can account for them.

This patch is split out from the main nfs iostat patch to highlight minor
architectural changes required to support this statistic.

Test plan:
None.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Invoke the byte and event counter macros where we want to count bytes and
events.

Clean-up: fix a possible NULL dereference in nfs_lock, and simplify
nfs_file_open.

Test-plan:
fsx and iozone on UP and SMP systems, with and without pre-emption.  Watch
for memory overwrite bugs, and performance loss (significantly more CPU
required per op).
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Add a per-superblock performance counter facility to the NFS client.  This
facility mimics the counters available for block devices and for
networking.  Expose these new counters via the new /proc/self/mountstats
interface.

Thanks to Andrew Morton and Trond Myklebust for their review and comments.

Test plan:
fsx and iozone on UP and SMP systems, with and without pre-emption.  Watch
for memory overwrite bugs, and performance loss (significantly more CPU
required per op).
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Get rid of "lock" and "posix", and spell out "vers=".

Test plan:
None.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Sometimes it's important to know the exact RPC retransmit settings the
kernel is using for an NFS mount point.  Add this facility to the NFS
client's show_options method.

Test plan:
Set various retransmit settings via the mount command, and check that the
settings are reflected in /proc/mounts.
Signed-off-by: NChuck Lever <cel@netapp.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

semaphore to mutex conversion.

the conversion was generated via scripts, and the result was validated
automatically via a script as well.

build and boot tested.
Signed-off-by: NIngo Molnar <mingo@elte.hu>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

this converts fs/nfs to kzalloc() usage.
compile tested with make allyesconfig
Signed-off-by: NEric Sesterhenn <snakebyte@gmx.de>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

nfs4_open_revalidate: 'res' may be used uninitialized
nfs4_callback_compound: ‘hdr_res.nops’ may be used uninitialized
			'op_nr’ may be used uninitialized
encode_getattr_res: ‘savep’ may be used uninitialized
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

The reason is that the idmapper cleanup may call flush_workqueue() on
rpciod_workqueue.
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

If not, we cannot guarantee that idmap->idmap_dentry, gss_auth->dentry and
clnt->cl_dentry are valid dentries.
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

My previous "const static" vs "static const" cleanup missed a single case,
patch below takes care of it.
Signed-off-by: NJesper Juhl <jesper.juhl@gmail.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Ensure that we flush out writes in the case when someone calls utimes() in
order to set the file times.
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

I've been reading through fs/nfs/write.c trying to track down a bug
that seems to be related to pages loosing a refcount and getting
freed too early (you interested in detail??) and I spotted a little
bug which the following patch should fix.
Signed-off-by: NNeil Brown <neilb@suse.de>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

Currently, there is no serialisation between NFS asynchronous writebacks
and truncation at the page level due to the fact that nfs_sync_inode()
cannot lock the pages that it is about to write out.

This means that it is possible to be flushing out data (and calling something
like set_page_writeback()) while the page cache is busy evicting the page.
Oops...

Use the hooks provided in try_to_release_page() to ensure that dirty pages
are always written back to storage before we evict them.
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

The nfs_open_context may live longer than the file descriptor that spawned
it, so it needs to carry a reference to the vfsmount. If not, then
generic_shutdown_super() may end up being called before reads and writes
have been flushed out.

Make a couple of functions static while we're at it...
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>

It turns out that nfs4_proc_get_root() may return raw NFSv4 errors instead of
mapping them to kernel errors.  Problem spotted by Neil Horman
<nhorman@tuxdriver.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Based on an original patch by Mike O'Connor and Greg Banks of SGI.

Mike states:

A normal user can panic an NFS client and cause a local DoS with
'judicious'(?) use of O_DIRECT.  Any O_DIRECT write to an NFS file where the
user buffer starts with a valid mapped page and contains an unmapped page,
will crash in this way.  I haven't followed the code, but O_DIRECT reads with
similar user buffers will probably also crash albeit in different ways.

Details: when nfs_get_user_pages() calls get_user_pages(), it detects and
correctly handles get_user_pages() returning an error, which happens if the
first page covered by the user buffer's address range is unmapped.  However,
if the first page is mapped but some subsequent page isn't, get_user_pages()
will return a positive number which is less than the number of pages requested
(this behaviour is sort of analagous to a short write() call and appears to be
intentional).  nfs_get_user_pages() doesn't detect this and hands off the
array of pages (whose last few elements are random rubbish from the newly
allocated array memory) to it's caller, whence they go to
nfs_direct_write_seg(), which then totally ignores the nr_pages it's given,
and calculates its own idea of how many pages are in the array from the user
buffer length.  Needless to say, when it comes to transmit those uninitialised
page* pointers, we see a crash in the network stack.
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Direct backport of 2.4 fix that didn't get propagated to 2.6; original
comment follows:
<quote>
   When I specify the NFS port for nfsroot (e.g.,
   nfsroot=<dir>,port=2049), the
   kernel uses the wrong port. In my case it tries to use 264 (0x108)
   instead
   of 2049 (0x801).

   This patch adds the missing htons().

   Eric
</quote>

Patch got applied in 2.4.21-pre6.  Author: Eric Lammerts (<eric@lammerts.org>,
AFAICS).
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

 Only do a sync_retry if the memcmp failed.
Signed-off-by: NDirk Mueller <dmueller@suse.com>
Signed-off-by: NTrond Myklebust <Trond.Myklebust@netapp.com>