Once we've started sending unstable NFS writes, we do not want to
clear pg_moreio, or we may end up sending the very last request as
a stable write if the commit lists are still empty.

Do, however, reset pg_moreio in the case where we end up having to
recoalesce the write if an attempt to use pNFS failed.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Cc: Weston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Use nfs_lock_and_join_requests to merge all subrequests into the head request -
this cancels and dereferences all subrequests.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Change nfs_find_and_lock_request so nfs_page_async_flush can handle multiple
requests in a page. There is only one request for a page the first time
nfs_page_async_flush is called, but if a write or commit fails, async_flush
is called again and there may be multiple requests associated with the page.
The solution is to merge all the requests in a page group into a single
request before calling nfs_pageio_add_request.

Rename nfs_find_and_lock_request to nfs_lock_and_join_requests and
change it to first lock all requests for the page, then cancel and merge
all subrequests into the head request.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

nfs_page_find_request_locked* should find the head request for that page.
Rename the functions and add comments to make this clear, and fix a bug
that could return a subrequest when page_private isn't set on the page.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

nfs_pages that aren't the the head of a group must take a reference on the
head as long as ->wb_head is set to it. This stops the head from hitting
a refcount of 0 while there is still an active nfs_page for the page group.

This avoids kref warnings in the writeback code when the page group head
is found and referenced.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Change the use of PG_INODE_REF - set it when taking extra reference on
subrequests and take care to only release once for each request.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

The big ACL switched nfs to use generic_listxattr, which calls all existing
->list handlers.  Add a custom .listxattr implementation that only lists
the ACLs if they actually are present on the given inode.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reported-by: NPhilippe Troin <phil@fifi.org>
Tested-by: NPhilippe Troin <phil@fifi.org>
Fixes: 013cdf10 (nfs: use generic posix ACL infrastructure ...)
Cc: stable@vger.kernel.org # 3.14+
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Fix nfs4_negotiate_security to create an rpc_clnt used to test each SECINFO
returned pseudoflavor. Check credential creation  (and gss_context creation)
which is important for RPC_AUTH_GSS pseudoflavors which can fail for multiple
reasons including mis-configuration.

Don't call nfs4_negotiate in nfs4_submount as it was just called by
nfs4_proc_lookup_mountpoint (nfs4_proc_lookup_common)
Signed-off-by: NAndy Adamson <andros@netapp.com>
[Trond: fix corrupt return value from nfs_find_best_sec()]
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Do not return RPC_AUTH_UNIX if SEINFO reply tests fail. This
prevents an infinite loop of NFS4ERR_WRONGSEC for non RPC_AUTH_UNIX mounts.

Without this patch, a mount with no sec= option to a server
that does not include RPC_AUTH_UNIX in the
SECINFO return can be presented with an attemtp to use RPC_AUTH_UNIX
which will result in an NFS4ERR_WRONG_SEC which will prompt the SECINFO
call which will again try RPC_AUTH_UNIX....
Signed-off-by: NAndy Adamson <andros@netapp.com>
Tested-By: NSteve Dickson <steved@redhat.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: NAndy Adamson <andros@netapp.com>
Tested-By: NSteve Dickson <steved@redhat.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Now that we have functions such as nfs_write_pageuptodate() that use
the cache_validity flags to check if the data cache is valid or not,
it is a little more important to keep the flags in sync with the
state of the data cache.
In particular, we'd like to ensure that if the data cache is empty, we
don't start marking it as needing revalidation.
Reported-by: NScott Mayhew <smayhew@redhat.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

In nfs_update_inode(), if the change attribute is seen to change on
the server, then we set NFS_INO_REVAL_PAGECACHE in order to make
sure that we check the file size.
However, if we also update the file size in the same function, we
don't need to check it again. So make sure that we clear the
NFS_INO_REVAL_PAGECACHE that was set earlier.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

NFS_INO_INVALID_DATA cannot be ignored, even if we have a delegation.

We're still having some problems with data corruption when multiple
clients are appending to a file and those clients are being granted
write delegations on open.

To reproduce:

Client A:
vi /mnt/`hostname -s`
while :; do echo "XXXXXXXXXXXXXXX" >>/mnt/file; sleep $(( $RANDOM % 5 )); done

Client B:
vi /mnt/`hostname -s`
while :; do echo "YYYYYYYYYYYYYYY" >>/mnt/file; sleep $(( $RANDOM % 5 )); done

What's happening is that in nfs_update_inode() we're recognizing that
the file size has changed and we're setting NFS_INO_INVALID_DATA
accordingly, but then we ignore the cache_validity flags in
nfs_write_pageuptodate() because we have a delegation.  As a result,
in nfs_updatepage() we're extending the write to cover the full page
even though we've not read in the data to begin with.
Signed-off-by: NScott Mayhew <smayhew@redhat.com>
Cc: <stable@vger.kernel.org> # v3.11+
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Otherwise the kernel oopses when remounting with IPv6 server because
net is dereferenced in dev_get_by_name.

Use net ns of current thread so that dev_get_by_name does not operate on
foreign ns. Changing the address is prohibited anyway so this should not
affect anything.
Signed-off-by: NMateusz Guzik <mguzik@redhat.com>
Cc: linux-nfs@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: stable@vger.kernel.org # 3.4+
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

end_offset and req_offset both return u64 - avoid casting to u32
until it's needed, when it's less than the (u32) size returned by
nfs_generic_pg_test.

Also, fix the comments in pnfs_generic_pg_test.

Running the cthon04 special tests caused this lockup in the
"write/read at 2GB, 4GB edges" test when running against a file layout server:

BUG: soft lockup - CPU#0 stuck for 22s! [bigfile2:823]
Modules linked in: nfs_layout_nfsv41_files rpcsec_gss_krb5 nfsv4 nfs fscache ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_mangle ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ppdev crc32c_intel aesni_intel aes_x86_64 glue_helper lrw gf128mul ablk_helper cryptd serio_raw e1000 shpchp i2c_piix4 i2c_core parport_pc parport nfsd auth_rpcgss oid_registry exportfs nfs_acl lockd sunrpc btrfs xor zlib_deflate raid6_pq mptspi scsi_transport_spi mptscsih mptbase ata_generic floppy autofs4
irq event stamp: 205958
hardirqs last  enabled at (205957): [<ffffffff814a62dc>] restore_args+0x0/0x30
hardirqs last disabled at (205958): [<ffffffff814ad96a>] apic_timer_interrupt+0x6a/0x80
softirqs last  enabled at (205956): [<ffffffff8103ffb2>] __do_softirq+0x1ea/0x2ab
softirqs last disabled at (205951): [<ffffffff8104026d>] irq_exit+0x44/0x9a
CPU: 0 PID: 823 Comm: bigfile2 Not tainted 3.15.0-rc1-branch-pgio_plus+ #3
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
task: ffff8800792ec480 ti: ffff880078c4e000 task.ti: ffff880078c4e000
RIP: 0010:[<ffffffffa02ce51f>]  [<ffffffffa02ce51f>] nfs_page_group_unlock+0x3e/0x4b [nfs]
RSP: 0018:ffff880078c4fab0  EFLAGS: 00000202
RAX: 0000000000000fff RBX: ffff88006bf83300 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff88006bf83300
RBP: ffff880078c4fab8 R08: 0000000000000001 R09: 0000000000000000
R10: ffffffff8249840c R11: 0000000000000000 R12: 0000000000000035
R13: ffff88007ffc72d8 R14: 0000000000000001 R15: 0000000000000000
FS:  00007f45f11b7740(0000) GS:ffff88007f200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3a8cb632d0 CR3: 000000007931c000 CR4: 00000000001407f0
Stack:
 ffff88006bf832c0 ffff880078c4fb00 ffffffffa02cec22 ffff880078c4fad8
 00000fff810f9d99 ffff880078c4fca0 ffff88006bf832c0 ffff88006bf832c0
 ffff880078c4fca0 ffff880078c4fd60 ffff880078c4fb28 ffffffffa02cee34
Call Trace:
 [<ffffffffa02cec22>] __nfs_pageio_add_request+0x298/0x34f [nfs]
 [<ffffffffa02cee34>] nfs_pageio_add_request+0x1f/0x42 [nfs]
 [<ffffffffa02d1722>] nfs_do_writepage+0x1b5/0x1e4 [nfs]
 [<ffffffffa02d1764>] nfs_writepages_callback+0x13/0x25 [nfs]
 [<ffffffffa02d1751>] ? nfs_do_writepage+0x1e4/0x1e4 [nfs]
 [<ffffffff810eb32d>] write_cache_pages+0x254/0x37f
 [<ffffffffa02d1751>] ? nfs_do_writepage+0x1e4/0x1e4 [nfs]
 [<ffffffff8149cf9e>] ? printk+0x54/0x56
 [<ffffffff810eacca>] ? __set_page_dirty_nobuffers+0x22/0xe9
 [<ffffffffa016d864>] ? put_rpccred+0x38/0x101 [sunrpc]
 [<ffffffffa02d1ae1>] nfs_writepages+0xb4/0xf8 [nfs]
 [<ffffffff810ec59c>] do_writepages+0x21/0x2f
 [<ffffffff810e36e8>] __filemap_fdatawrite_range+0x55/0x57
 [<ffffffff810e374a>] filemap_write_and_wait_range+0x2d/0x5b
 [<ffffffffa030ba0a>] nfs4_file_fsync+0x3a/0x98 [nfsv4]
 [<ffffffff8114ee3c>] vfs_fsync_range+0x18/0x20
 [<ffffffff810e40c2>] generic_file_aio_write+0xa7/0xbd
 [<ffffffffa02c5c6b>] nfs_file_write+0xf0/0x170 [nfs]
 [<ffffffff81129215>] do_sync_write+0x59/0x78
 [<ffffffff8112956c>] vfs_write+0xab/0x107
 [<ffffffff81129c8b>] SyS_write+0x49/0x7f
 [<ffffffff814acd12>] system_call_fastpath+0x16/0x1b
Reported-by: NAnna Schumaker <Anna.Schumaker@netapp.com>
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: NTom Haynes <Thomas.Haynes@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

The save of the write offset was removed some time ago, so that
part of the comment is bogus.

The remainder is pretty self-evident.

So off with it!
Signed-off-by: NTom Haynes <Thomas.Haynes@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

This constant has the wrong value.  And we don't use it.  And it's been
removed from the 4.2 spec anyway.
Signed-off-by: NJ. Bruce Fields <bfields@redhat.com>

This typedef is unnecessary and should just be removed.
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The addition of lockdep code to write_seqcount_begin/end has lead to
a bunch of false positive claims of ABBA deadlocks with the so_lock
spinlock. Audits show that this simply cannot happen because the
read side code does not spin while holding so_lock.

Cc: <stable@vger.kernel.org> # 3.13.x
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Currently, the fl_owner isn't set for flock locks. Some filesystems use
byte-range locks to simulate flock locks and there is a common idiom in
those that does:

    fl->fl_owner = (fl_owner_t)filp;
    fl->fl_start = 0;
    fl->fl_end = OFFSET_MAX;

Since flock locks are generally "owned" by the open file description,
move this into the common flock lock setup code. The fl_start and fl_end
fields are already set appropriately, so remove the unneeded setting of
that in flock ops in those filesystems as well.

Finally, the lease code also sets the fl_owner as if they were owned by
the process and not the open file description. This is incorrect as
leases have the same ownership semantics as flock locks. Set them the
same way. The lease code doesn't actually use the fl_owner value for
anything, so this is more for consistency's sake than a bugfix.
Reported-by: NTrond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: NJeff Layton <jlayton@poochiereds.net>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> (Staging portion)
Acked-by: NJ. Bruce Fields <bfields@fieldses.org>

The object and block layouts already exist in their own
subdirectories. This patch completes the set!

Note that as a layout denotes nfs4 already, I stripped
that prefix out of the file names.
Signed-off-by: NTom Haynes <Thomas.Haynes@primarydata.com>
Acked-by: NJeff Layton <jlayton@poochiereds.net>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Return the NULL pointer when the allocation fails.

Cc: Boaz Harrosh <bharrosh@panasas.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Return the NULL pointer when the allocation fails.
Reported-by: NFengguang Wu <fengguang.wu@intel.com>
Cc: <stable@vger.kernel.org> # 3.5.x
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Those flags are obsolete and checking them can incorrectly cause
remount operations to fail.
Signed-off-by: NScott Mayhew <smayhew@redhat.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Place the call to resend the failed GETATTR under the error handler so that
when appropriate, the GETATTR is retried more than once.

The server can fail the GETATTR op in the OPEN compound with a recoverable
error such as NFS4ERR_DELAY. In the case of an O_EXCL open, the server has
created the file, so a retrans of the OPEN call will fail with NFS4ERR_EXIST.
Signed-off-by: NAndy Adamson <andros@netapp.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

We cannot allow nfs_page_group_lock to use TASK_KILLABLE here, since
the loop would cause a busy wait if somebody kills the task.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Handle the case where nfs_create_request() returns an error.
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

nfs_read_completion relied on the fact that there was a 1:1 mapping
of page to nfs_request, but this has now changed.

Regions not covered by a request have already been zeroed elsewhere.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Use the new pg_test interface to adjust requests to fit in the current
stripe / segment.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Remove alignment checks that would revert to MDS and change pg_test
to return the max ammount left in the segment (or other pg_test call)
up to size of passed request, or 0 if no space is left.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Support direct requests that span multiple pnfs data servers by
comparing nfs_pgio_header->verf to a cached verf in pnfs_commit_bucket.
Continue to use dreq->verf if the MDS is used / non-pNFS.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Since the ability to split pages into subpage requests has been added,
nfs_pgio_header->rpc_list only ever has one pgio data.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Use the newly added support for multiple requests per page for
rsize/wsize < PAGE_SIZE, instead of having multiple read / write
data structures per pageio header.

This allows us to get rid of nfs_pgio_multi.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Now that pg_test can change the size of the request (by returning a non-zero
size smaller than the request), pg_test functions that call other
pg_test functions must return the minimum of the result - or 0 if any fail.

Also clean up the logic of some pg_test functions so that all checks are
for contitions where coalescing is not possible.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Remove check that the request covers a whole page.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Remove unneeded else statement and clean up how commit info
dataserver buckets are replaced.
Suggested-by: NTrond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Change how nfs_mark_uptodate checks to see if writes cover a whole page.

This patch should have no effect yet since all page groups currently
have one request, but will come into play when pg_test functions are
modified to split pages into sub-page regions.
Signed-off-by: NWeston Andros Adamson <dros@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>