I hit the following oops out of the blue while testing with flexfiles:

BUG: unable to handle kernel NULL pointer dereference at 00000000000000e8
IP: [<ffffffffa048f6b8>] nfs4_ff_find_or_create_ds_client+0x48/0x50 [nfs_layout_flexfiles]
PGD 44031067 PUD 5062d067 PMD 0
Oops: 0000 [#1] SMP
Modules linked in: nfsv3 nfs_layout_flexfiles tun rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache dcdbas nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw bonding ipmi_devintf ipmi_msghandler snd_hda_codec_generic virtio_balloon ppdev snd_hda_intel snd_hda_controller snd_hda_codec iosf_mbi crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hda_core parport_pc snd_hwdep parport snd_seq snd_seq_device snd_pcm snd_timer acpi_cpufreq
 snd soundcore i2c_piix4 xfs libcrc32c joydev virtio_net virtio_console qxl drm_kms_helper ttm crc32c_intel drm virtio_pci serio_raw ata_generic virtio_ring virtio pata_acpi
CPU: 0 PID: 19138 Comm: test5 Not tainted 4.1.9-100.pd.90.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014
task: ffff88007b70cf00 ti: ffff88004cc44000 task.ti: ffff88004cc44000
RIP: 0010:[<ffffffffa048f6b8>]  [<ffffffffa048f6b8>] nfs4_ff_find_or_create_ds_client+0x48/0x50 [nfs_layout_flexfiles]
RSP: 0018:ffff88004cc47890  EFLAGS: 00010246
RAX: 0000000000000003 RBX: ffff880050932300 RCX: ffff88006978f488
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88003e0e8540
RBP: ffff88004cc47908 R08: 0000000000000000 R09: 0000000000000000
R10: ffff88007ff8c758 R11: 0000000000000005 R12: ffff88003e0e8540
R13: 0000000000000000 R14: ffff88006978f488 R15: ffff88004431cc80
FS:  00007fea40c7c740(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000000000e8 CR3: 0000000044318000 CR4: 00000000000406f0
Stack:
 ffffffffa048c934 ffff880050932310 0000000100000001 ffff88006978f510
 ffff88006978f3c8 ffff88003e56cd90 ffff88004cc479d0 00000020a052aff0
 000000000004b000 ffff88004cc47908 ffff880050932300 ffff88004cc479d0
Call Trace:
 [<ffffffffa048c934>] ? ff_layout_write_pagelist+0x64/0x220 [nfs_layout_flexfiles]
 [<ffffffffa057a3bf>] pnfs_generic_pg_writepages+0xaf/0x1b0 [nfsv4]
 [<ffffffffa051ab57>] nfs_pageio_doio+0x27/0x60 [nfs]
 [<ffffffffa051bfe4>] nfs_pageio_complete_mirror+0x54/0xa0 [nfs]
 [<ffffffffa051c7ad>] nfs_pageio_complete+0x2d/0x90 [nfs]
 [<ffffffffa052032d>] nfs_writepage_locked+0x8d/0xe0 [nfs]
 [<ffffffff811e4630>] ? page_referenced_one+0x1a0/0x1a0
 [<ffffffffa05210e7>] nfs_wb_single_page+0xf7/0x190 [nfs]
 [<ffffffffa05108d1>] nfs_launder_page+0x41/0x90 [nfs]
 [<ffffffff811b8930>] invalidate_inode_pages2_range+0x340/0x3a0
 [<ffffffff811b89a7>] invalidate_inode_pages2+0x17/0x20
 [<ffffffffa0513e1e>] nfs_release+0x9e/0xb0 [nfs]
 [<ffffffffa050fa1d>] nfs_file_release+0x3d/0x60 [nfs]
 [<ffffffff8122481c>] __fput+0xdc/0x1e0
 [<ffffffff8122496e>] ____fput+0xe/0x10
 [<ffffffff810bde67>] task_work_run+0xa7/0xe0
 [<ffffffff810af735>] get_signal+0x565/0x600
 [<ffffffff811a9815>] ? __filemap_fdatawrite_range+0x65/0x90
 [<ffffffff810144a7>] do_signal+0x37/0x730
 [<ffffffffa0569921>] ? nfs4_file_fsync+0x81/0x150 [nfsv4]
 [<ffffffff81254dbb>] ? vfs_fsync_range+0x3b/0xb0
 [<ffffffff811446a6>] ? __audit_syscall_exit+0x1e6/0x280
 [<ffffffff81014bff>] do_notify_resume+0x5f/0xa0
 [<ffffffff8178ec3c>] int_signal+0x12/0x17
Code: 48 8b 40 70 8b 00 83 f8 03 74 20 83 f8 04 75 13 55 48 89 ce 48 89 d7 48 89 e5 e8 14 0f 0e 00 5d c3 66 90 0f 0b 66 0f 1f 44 00 00 <48> 8b 82 e8 00 00 00 c3 66 66 66 66 90 55 48 89 e5 41 57 41 56
RIP  [<ffffffffa048f6b8>] nfs4_ff_find_or_create_ds_client+0x48/0x50 [nfs_layout_flexfiles]
 RSP <ffff88004cc47890>
CR2: 00000000000000e8

When the DS connection attempt fails, nfs4_ff_layout_prepare_ds marks it
for the error but then just returns the ds as if it were usable. The
comments though say:

  /* Upon return, either ds is connected, or ds is NULL */

Ensure that we set the return pointer to NULL in the event that the
connection attempt fails.
Signed-off-by: NJeff Layton <jeff.layton@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Just call inode_dio_wait directly instead of through a pointless wrapper.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

The only difference to nfs_file_fsync is the call to pnfs_sync_inode.  But
pnfs_sync_inode is just an inline that calls a pNFS layout driver method
if CONFIG_PNFS is designed, and thus can be called just fine from the core
NFS module.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Replace another case where the layout 'plh_block_lgets' can trigger
infinite loops in send_layoutget().
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

If the server reboots while there is a layoutget outstanding, then
the call to pnfs_choose_layoutget_stateid() will fail with an EAGAIN
error, which causes an infinite loop in send_layoutget(). The reason
why we never break out of the loop is that the layout 'plh_block_lgets'
field is never cleared.

Fix is to replace plh_block_lgets with NFS_LAYOUT_INVALID_STID, which
can be reset after a new layoutget.

Fixes: ab7d763e ("pNFS: Ensure nfs4_layoutget_prepare returns...")
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

unreferenced object 0xffffc90000abf000 (size 16900):
  comm "fsync02", pid 15765, jiffies 4297431627 (age 423.772s)
  hex dump (first 32 bytes):
    00 00 00 00 00 00 00 00 00 a0 c2 19 00 88 ff ff  ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [<ffffffff8174d54e>] kmemleak_alloc+0x4e/0xb0
    [<ffffffff811b9b91>] __vmalloc_node_range+0x231/0x280
    [<ffffffff811b9c2a>] __vmalloc+0x4a/0x50
    [<ffffffffa02c9ec1>] ext_tree_prepare_commit+0x231/0x2e0 [blocklayoutdriver]
    [<ffffffffa02c700e>] bl_prepare_layoutcommit+0xe/0x10 [blocklayoutdriver]
    [<ffffffffa0596a6c>] pnfs_layoutcommit_inode+0x29c/0x330 [nfsv4]
    [<ffffffffa0596b13>] pnfs_generic_sync+0x13/0x20 [nfsv4]
    [<ffffffffa0585188>] nfs4_file_fsync+0x58/0x150 [nfsv4]
    [<ffffffff81228e5b>] vfs_fsync_range+0x4b/0xb0
    [<ffffffff81228f1d>] do_fsync+0x3d/0x70
    [<ffffffff812291d0>] SyS_fsync+0x10/0x20
    [<ffffffff81757def>] entry_SYSCALL_64_fastpath+0x12/0x76
    [<ffffffffffffffff>] 0xffffffffffffffff

v2, add missing include header
Signed-off-by: NKinglong Mee <kinglongmee@gmail.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

The newly added NFS v4.2 operations (ALLOCATE, DEALLOCATE, SEEK and CLONE)
use a helper called nfs42_set_rw_stateid to select a stateid that is sent
to the server.  But they don't set the inode and state fields in the
nfs4_exception structure, and this don't partake in the stateid recovery
protocol.  Because of this they will simply return errors insted of trying
to recover a stateid when the server return a BAD_STATEID error.

Additionally CLONE has the problem that it operates on two files and thus
two stateids, and thus needs to call the exception handler twice to
recover stateids.

While we're at it stop grabbing an addititional reference to the open
context in all these operations - having the file open guarantees that
the open context won't go away.

All this can be produces with the generic/168 and generic/170 tests in
xfstests which stress the CLONE stateid handling.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

In the case where d_add_unique() finds an appropriate alias to use it will
have already incremented the reference count.  An additional dget() to swap
the open context's dentry is unnecessary and will leak a reference.
Signed-off-by: NBenjamin Coddington <bcodding@redhat.com>
Fixes: 275bb307 ("NFSv4: Move dentry instantiation into the NFSv4-...")
Cc: stable@vger.kernel.org # 3.10+
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

When setting the layout return mode, we must always also set the
NFS_LAYOUT_RETURN_REQUESTED flag to ensure that we send a layoutreturn.
Otherwise pnfs_error_mark_layout_for_return() could set the mode, but
fail to send the layoutreturn because another is already in flight.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

We don't need to schedule a layoutreturn if the layout segment can
be freed immediately.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Fix oops when NULL callback_ops pointer accessed in rpc_init_task
Signed-off-by: NAndy Adamson <andros@netapp.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

This adds multipathing to pNFS over NFSv3 as described in the flexfiles
draft spec.
Ideally, we'd like to do the same for pNFS files, but the NFSv4.1 protocol
requires a call to EXCHANGE_ID in order to test that the connection can do
session trunking.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Use the new helper to ensure that nfs4_proc_bind_conn_to_session() is called
for all connections.
However ensure that we only set the backchannel flag for the connection
pointed to by rpc_clnt->cl_xprt.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

We need to initialize cb_sequenceres information when reporting a
NFS4ERR_RETRY_UNCACHED_REP error, since that will apply to the
next operation, not to the CB_SEQUENCE itself.
Reported-by: NKinglong Mee <kinglongmee@gmail.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

NFS_LAYOUT_RETURN_BEFORE_CLOSE is being used to signal that a
layoutreturn is needed, either due to a layout recall or to a
layout error. Rename it to NFS_LAYOUT_RETURN_REQUESTED in order
to clarify its purpose.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

The layoutreturn code currently relies on pnfs_put_lseg() to initiate the
RPC call when conditions are right. A problem arises when we want to
free the layout segment from inside an inode->i_lock section (e.g. in
pnfs_clear_request_commit()), since we cannot sleep.

The workaround is to move the actual call to pnfs_send_layoutreturn()
to pnfs_put_layout_hdr(), which doesn't have this restriction.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Hook the callback channel into the same session management machinery
as we use for the forward channel.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

We need to make sure that we don't allow args->csa_sequenceid == 0.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

We have no duplicate reply cache, so we always set the back channel
ca_maxresponsesize_cached to zero when negotiating the session.
That means we should always error out as soon as we see the server
set args->csa_cachethis.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

See RFC5661 Section 2.10.6.2: if retrying a request, and the old one is
still in progress, we must return NFS4ERR_DELAY as the reply to sequence.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Instead, use the values encoded in the slot table itself.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

parallel to mutex_{lock,unlock,trylock,is_locked,lock_nested},
inode_foo(inode) being mutex_foo(&inode->i_mutex).

Please, use those for access to ->i_mutex; over the coming cycle
->i_mutex will become rwsem, with ->lookup() done with it held
only shared.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

We must not skip encoding the statistics, or the server will see an
XDR encoding error.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>
Cc: stable@vger.kernel.org # 4.0+

I noticed that all the callers of this function pass cinfo->mds->list as
an argument in addition to the cinfo structure itself.  Let's get rid of
the extra argument, since it doesn't seem to be adding anything.
Signed-off-by: NAnna Schumaker <Anna.Schumaker@Netapp.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

When we hit 22 errors, we start to overflow the memory buffers allocated
to the LAYOUTRETURN errors. The issue is that currently, RPC call reply
ordering determines how successful we are in merging errors that refer
to contiguous READ or WRITE requests.

Fix is to use an insertion sort to help detect contiguity.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Mark those kmem allocations that are known to be easily triggered from
userspace as __GFP_ACCOUNT/SLAB_ACCOUNT, which makes them accounted to
memcg.  For the list, see below:

 - threadinfo
 - task_struct
 - task_delay_info
 - pid
 - cred
 - mm_struct
 - vm_area_struct and vm_region (nommu)
 - anon_vma and anon_vma_chain
 - signal_struct
 - sighand_struct
 - fs_struct
 - files_struct
 - fdtable and fdtable->full_fds_bits
 - dentry and external_name
 - inode for all filesystems. This is the most tedious part, because
   most filesystems overwrite the alloc_inode method.

The list is far from complete, so feel free to add more objects.
Nevertheless, it should be close to "account everything" approach and
keep most workloads within bounds.  Malevolent users will be able to
breach the limit, but this was possible even with the former "account
everything" approach (simply because it did not account everything in
fact).

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: NVladimir Davydov <vdavydov@virtuozzo.com>
Acked-by: NJohannes Weiner <hannes@cmpxchg.org>
Acked-by: NMichal Hocko <mhocko@suse.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Greg Thelen <gthelen@google.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

inode_nohighmem() is sufficient to make sure that page_get_link()
won't try to allocate a highmem page.  Moreover, it is sufficient
to make sure that page_symlink/__page_symlink won't do the same
thing.  However, any filesystem that manually preseeds the symlink's
page cache upon symlink(2) needs to make sure that the page it
inserts there won't be a highmem one.

Fortunately, only nfs and shmem have run afoul of that...
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

The use of wait_on_atomic_t() for waiting on I/O to complete before
unlocking allows us to git rid of the NFS_IO_INPROGRESS flag, and thus the
nfs_iocounter's flags member, and finally the nfs_iocounter altogether.
The count of I/O is moved to the lock context, and the counter
increment/decrement functions become simple enough to open-code.
Signed-off-by: NBenjamin Coddington <bcodding@redhat.com>
[Trond: Fix up conflict with existing function nfs_wait_atomic_killable()]
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Make it more obvious what we're returning...
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Peng Tao points out that the call to pnfs_mark_matching_lsegs_return()
could race with pnfs_put_lseg(), in which case the layout segment is
cleared, but no layoutreturn will be sent.
Fix is to replace the call to pnfs_mark_matching_lsegs_invalid().
Reported-by: NPeng Tao <tao.peng@primarydata.com>
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Fix a bug whereby if all the layout segments could be immediately freed,
the call to pnfs_error_mark_layout_for_return() would never result in
a layoutreturn.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

If pnfs_mark_matching_lsegs_return() needs to mark a layout segment for
return, then it must also set the return iomode.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

A stateid is a structure, pass it as a pointer.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

If two processes share the same credentials and NFSv4 open stateid, then
allow them both to dirty the same page, even if their nfs_open_context
differs.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>

If the layout segment is invalid, then we should not be adding more
write requests to the commit list. Instead, those writes should be
replayed after requesting a new layout.
Signed-off-by: NTrond Myklebust <trond.myklebust@primarydata.com>