Add ioctl to query previous versions of file

Allows listing snapshots on files on SMB3 mounts.
Signed-off-by: NSteve French <smfrench@gmail.com>

Add mount option "max_credits" to allow setting maximum SMB3
credits to any value from 10 to 64000 (default is 32000).
This can be useful to workaround servers with problems allocating
credits, or to throttle the client to use smaller amount of
simultaneous i/o or to workaround server performance issues.

Also adds a cap, so that even if the server granted us more than
65000 credits due to a server bug, we would not use that many.
Signed-off-by: NSteve French <steve.french@primarydata.com>

When we open a durable handle we give a Globally Unique
Identifier (GUID) to the server which we must keep for later reference
e.g. when reopening persistent handles on reconnection.

Without this the GUID generated for a new persistent handle was lost and
16 zero bytes were used instead on re-opening.
Signed-off-by: NAurelien Aptel <aaptel@suse.com>
CC: Stable <stable@vger.kernel.org>
Signed-off-by: NSteve French <smfrench@gmail.com>

GUIDs although random, and 16 bytes, need to be generated as
proper uuids.
Signed-off-by: NSteve French <steve.french@primarydata.com>
Reviewed-by: NAurelien Aptel <aaptel@suse.com>
Reported-by: NDavid Goebels <davidgoe@microsoft.com>
CC: Stable <stable@vger.kernel.org>

During following a symbolic link we received err_buf from SMB2_open().
While the validity of SMB2 error response is checked previously
in smb2_check_message() a symbolic link payload is not checked at all.
Fix it by adding such checks.

Cc: Dan Carpenter <dan.carpenter@oracle.com>
CC: Stable <stable@vger.kernel.org>
Signed-off-by: NPavel Shilovsky <pshilovsky@samba.org>
Signed-off-by: NSteve French <smfrench@gmail.com>

We should be able to use the same helper functions used for SMB 2.1 and
later versions.
Signed-off-by: NSachin Prabhu <sprabhu@redhat.com>
Signed-off-by: NSteve French <smfrench@gmail.com>

Prepare for encryption support (first part). Add decryption and encryption key generation. Thanks to Metze for helping with this.
Reviewed-by: NStefan Metzmacher <metze@samba.org>
Signed-off-by: NSteve French <steve.french@primarydata.com>

Validate "persistenthandles" and "nopersistenthandles" mount options against
the support the server claims in negotiate and tree connect SMB3 responses.
Signed-off-by: NSteve French <steve.french@primarydata.com>
Reviewed-by: NPavel Shilovsky <pshilovsky@samba.org>

Enable duplicate extents (cp --reflink) ioctl for SMB3.0 not just
SMB3.1.1 since have verified that this works to Windows 2016
(REFS) and additional testing done at recent plugfest with
SMB3.0 not just SMB3.1.1  This will also make it easier
for Samba.
Signed-off-by: NSteve French <steve.french@primarydata.com>
Reviewed-by: NDavid Disseldorp <ddiss@suse.de>

leases (oplocks) were always requested for SMB2/SMB3 even when oplocks
disabled in the cifs.ko module.
Signed-off-by: NSteve French <steve.french@primarydata.com>
Reviewed-by: NChandrika Srinivasan <chandrika.srinivasan@citrix.com>
CC: Stable <stable@vger.kernel.org>

set integrity increases reliability of files stored on SMB3 servers.
Add ioctl to allow setting this on files on SMB3 and later mounts.
Signed-off-by: NSteve French <steve.french@primarydata.com>

 Getting fantastic copy performance with cp --reflink over SMB3.11
 using the new FSCTL_DUPLICATE_EXTENTS.

 This FSCTL was added in the SMB3.11 dialect (testing was
 against REFS file system) so have put it as a 3.11 protocol
 specific operation ("vers=3.1.1" on the mount).  Tested at
 the SMB3 plugfest in Redmond.

 It depends on the new FS Attribute (BLOCK_REFCOUNTING) which
 is used to advertise support for the ability to do this ioctl
 (if you can support multiple files pointing to the same block
 than this refcounting ability or equivalent is needed to
 support the new reflink-like duplicate extent SMB3 ioctl.
Signed-off-by: NSteve French <steve.french@primarydata.com>

Most people think of SMB 3.1.1 as SMB version 3.11 so add synonym
for "vers=3.1.1" of "vers=3.11" on mount.

Also make sure that unlike SMB3.0 and 3.02 we don't send
validate negotiate on mount (it is handled by negotiate contexts) -
add list of SMB3.11 specific functions (distinct from 3.0 dialect).

Signed-off-by: Steve French <steve.french@primarydata.com>w

Parses and recognizes "vers=3.1.1" on cifs mount and allows sending
0x0311 as a new CIFS/SMB3 dialect. Subsequent patches will add
the new negotiate contexts and updated session setup
Reviewed-by: NJeff Layton <jlayton@primarydata.com>
Signed-off-by: NSteve French <steve.french@primarydata.com>

that's the bulk of filesystem drivers dealing with inodes of their own
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

While attempting to clone a file on a samba server, we receive a
STATUS_INVALID_DEVICE_REQUEST. This is mapped to -EOPNOTSUPP which
isn't handled in smb2_clone_range(). We end up looping in the while loop
making same call to the samba server over and over again.

The proposed fix is to exit and return the error value when encountered
with an unhandled error.

Cc: <stable@vger.kernel.org>
Signed-off-by: NSachin Prabhu <sprabhu@redhat.com>
Signed-off-by: NSteve French <steve.french@primarydata.com>
Signed-off-by: NSteve French <smfrench@gmail.com>

We have encountered failures when When testing smb2 mounts on ppc64
machines when using both Samba as well as Windows 2012.

On poking around, the problem was determined to be caused by the
high endian MessageID passed in the header for smb2. On checking the
corresponding MID for smb1 is converted to LE before being sent on the
wire.

We have tested this patch successfully on a ppc64 machine.
Signed-off-by: NSachin Prabhu <sprabhu@redhat.com>

Replace all __constant_foo to foo() except in smb2status.h (1700 lines to
update).
Signed-off-by: NFabian Frederick <fabf@skynet.be>
Cc: Steve French <sfrench@samba.org>
Cc: Jeff Layton <jlayton@poochiereds.net>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

In many cases the simple fallocate call is
a no op (since the file is already not sparse) or
can simply be converted from a sparse to a non-sparse
file if we are fallocating the whole file and keeping
the size.
Signed-off-by: NSteven French <smfrench@gmail.com>

Acked-by: NShirish Pargaonkar <shirishpargaonkar@gmail.com>
Signed-off-by: NSteve French <smfrench@gmail.com>

Adds support on SMB2.1 and SMB3 mounts for emulation of symlinks
via the "Minshall/French" symlink format already used for cifs
mounts when mfsymlinks mount option is used (and also used by Apple).
  http://wiki.samba.org/index.php/UNIX_Extensions#Minshall.2BFrench_symlinks
This second patch adds support to query them (recognize them as symlinks
and read them).  Third version of patch makes minor corrections
to error handling.
Signed-off-by: NSteve French <smfrench@gmail.com>
Reviewed-by: NStefan Metzmacher <metze@samba.org>

Adds support on SMB2.1 and SMB3 mounts for emulation of symlinks
via the "Minshall/French" symlink format already used for cifs
mounts when mfsymlinks mount option is used (and also used by Apple).
http://wiki.samba.org/index.php/UNIX_Extensions#Minshall.2BFrench_symlinks
This first patch adds support to create them.  The next patch will
add support for recognizing them and reading them.  Although CIFS/SMB3
have other types of symlinks, in the many use cases they aren't
practical (e.g. either require cifs only mounts with unix extensions
to Samba, or require the user to be Administrator to Windows for SMB3).
This also helps enable running additional xfstests over SMB3 (since some
xfstests directly or indirectly require symlink support).
Signed-off-by: NSteve French <smfrench@gmail.com>
CC: Stefan Metzmacher <metze@samba.org>

The "sfu" mount option did not work on SMB2/SMB3 mounts.
With these changes when the "sfu" mount option is passed in
on an smb2/smb2.1/smb3 mount the client can emulate (and
recognize) fifo and device (character and device files).

In addition the "sfu" mount option should not conflict
with "mfsymlinks" (symlink emulation) as we will never
create "sfu" style symlinks, but using "sfu" mount option
will allow us to recognize existing symlinks, created with
Microsoft "Services for Unix" (SFU and SUA).

To enable the "sfu" mount option for SMB2/SMB3 the calling
syntax of the generic cifs/smb2/smb3 sync_read and sync_write
protocol dependent function needed to be changed (we
don't have a file struct in all cases), but this actually
ended up simplifying the code a little.
Signed-off-by: NSteve French <smfrench@gmail.com>

The existing code uses the old MAX_NAME constant. This causes
XFS test generic/013 to fail. Fix it by replacing MAX_NAME with
PATH_MAX that SMB1 uses. Also remove an unused MAX_NAME constant
definition.

Cc: <stable@vger.kernel.org> # v3.7+
Signed-off-by: NPavel Shilovsky <pshilovsky@samba.org>
Signed-off-by: NSteve French <smfrench@gmail.com>

Signed-off-by: NSteve French <smfrench@gmail.com>

fallocate -z (FALLOC_FL_ZERO_RANGE) can map to SMB3
FSCTL_SET_ZERO_DATA SMB3 FSCTL but FALLOC_FL_ZERO_RANGE
when called without the FALLOC_FL_KEEPSIZE flag set could want
the file size changed so we can not support that subcase unless
the file is cached (and thus we know the file size).
Signed-off-by: NSteve French <smfrench@gmail.com>
Reviewed-by: NPavel Shilovsky <pshilovsky@samba.org>

Implement FALLOC_FL_PUNCH_HOLE (which does not change the file size
fortunately so this matches the behavior of the equivalent SMB3
fsctl call) for SMB3 mounts.  This allows "fallocate -p" to work.
It requires that the server support setting files as sparse
(which Windows allows).
Signed-off-by: NSteve French <smfrench@gmail.com>

SMB2 servers indicates the end of a directory search with
STATUS_NO_MORE_FILE error code that is not processed now.
This causes generic/257 xfstest to fail. Fix this by triggering
the end of search by this error code in SMB2_query_directory.

Also when negotiating CIFS protocol we tell the server to close
the search automatically at the end and there is no need to do
it itself. In the case of SMB2 protocol, we need to close it
explicitly - separate close directory checks for different
protocols.

Cc: <stable@vger.kernel.org>
Signed-off-by: NPavel Shilovsky <pshilovsky@samba.org>
Signed-off-by: NSteve French <smfrench@gmail.com>

Simply move code to new function (for clarity). Function sets or clears
the sparse file attribute flag.
Signed-off-by: NSteve French <smfrench@gmail.com>
Reviewed-by: NDavid Disseldorp <ddiss@samba.org>

Many Linux filesystes make a file "sparse" when extending
a file with ftruncate. This does work for CIFS to Samba
(only) but not for SMB2/SMB3 (to Samba or Windows) since
there is a "set sparse" fsctl which is supposed to be
sent to mark a file as sparse.

This patch marks a file as sparse by sending this simple
set sparse fsctl if it is extended more than 2 pages.
It has been tested to Windows 8.1, Samba and various
SMB2/SMB3 servers which do support setting sparse (and
MacOS which does not appear to support the fsctl yet).
If a server share does not support setting a file
as sparse, then we do not retry setting sparse on that
share.

The disk space savings for sparse files can be quite
large (even more significant on Windows servers than Samba).
Signed-off-by: NSteve French <smfrench@gmail.com>
Reviewed-by: NShirish Pargaonkar <spargaonkar@suse.com>

Adds setinfo worker function for SMB2/SMB3 support of SET_ALLOCATION_INFORMATION
Signed-off-by: NSteve French <smfrench@gmail.com>
Reviewed-by: NPavel Shilovsky <pshilovsky@samba.org>

If we negotiate SMB 2.1 and higher version of the protocol and
a server supports large read buffer size, we need to consume 1
credit per 65536 bytes. So, we need to know how many credits
we have and obtain the required number of them before constructing
a readdata structure in readpages and user read.
Reviewed-by: NShirish Pargaonkar <spargaonkar@suse.com>
Signed-off-by: NPavel Shilovsky <pshilovsky@samba.org>
Signed-off-by: NSteve French <smfrench@gmail.com>

If we negotiate SMB 2.1 and higher version of the protocol and
a server supports large write buffer size, we need to consume 1
credit per 65536 bytes. So, we need to know how many credits
we have and obtain the required number of them before constructing
a writedata structure in writepages and iovec write.
Reviewed-by: NShirish Pargaonkar <spargaonkar@suse.com>
Signed-off-by: NPavel Shilovsky <pshilovsky@samba.org>
Signed-off-by: NSteve French <smfrench@gmail.com>

If wsize changes on reconnect we need to use new writedata structure
that for retrying.
Reviewed-by: NShirish Pargaonkar <spargaonkar@suse.com>
Signed-off-by: NPavel Shilovsky <pshilovsky@samba.org>
Signed-off-by: NSteve French <smfrench@gmail.com>

Clarify comments for create contexts which we do send,
and fix typo in one create context definition and add
newer SMB3 create contexts to the list.
Signed-off-by: NSteve French <smfrench@gmail.com>

Problem reported in Red Hat bz 1040329 for strict writes where we cache
only when we hold oplock and write direct to the server when we don't.

When we receive an oplock break, we first change the oplock value for
the inode in cifsInodeInfo->oplock to indicate that we no longer hold
the oplock before we enqueue a task to flush changes to the backing
device. Once we have completed flushing the changes, we return the
oplock to the server.

There are 2 ways here where we can have data corruption
1) While we flush changes to the backing device as part of the oplock
break, we can have processes write to the file. These writes check for
the oplock, find none and attempt to write directly to the server.
These direct writes made while we are flushing from cache could be
overwritten by data being flushed from the cache causing data
corruption.
2) While a thread runs in cifs_strict_writev, the machine could receive
and process an oplock break after the thread has checked the oplock and
found that it allows us to cache and before we have made changes to the
cache. In that case, we end up with a dirty page in cache when we
shouldn't have any. This will be flushed later and will overwrite all
subsequent writes to the part of the file represented by this page.

Before making any writes to the server, we need to confirm that we are
not in the process of flushing data to the server and if we are, we
should wait until the process is complete before we attempt the write.
We should also wait for existing writes to complete before we process
an oplock break request which changes oplock values.

We add a version specific  downgrade_oplock() operation to allow for
differences in the oplock values set for the different smb versions.

Cc: stable@vger.kernel.org
Signed-off-by: NSachin Prabhu <sprabhu@redhat.com>
Reviewed-by: NJeff Layton <jlayton@redhat.com>
Reviewed-by: NPavel Shilovsky <piastry@etersoft.ru>
Signed-off-by: NSteve French <smfrench@gmail.com>

SMB3 servers can respond with MaxTransactSize of more than 4M
that can cause a memory allocation error returned from kmalloc
in a lock codepath. Also the client doesn't support multicredit
requests now and allows buffer sizes of 65536 bytes only. Set
MaxTransactSize to this maximum supported value.

Cc: stable@vger.kernel.org # 3.7+
Signed-off-by: NPavel Shilovsky <piastry@etersoft.ru>
Acked-by: NJeff Layton <jlayton@redhat.com>
Signed-off-by: NSteve French <smfrench@gmail.com>

When we are running SMB3 or SMB3.02 connections which are signed
we need to validate the protocol negotiation information,
to ensure that the negotiate protocol response was not tampered with.

Add the missing FSCTL which is sent at mount time (immediately after
the SMB3 Tree Connect) to validate that the capabilities match
what we think the server sent.

"Secure dialect negotiation is introduced in SMB3 to protect against
man-in-the-middle attempt to downgrade dialect negotiation.
The idea is to prevent an eavesdropper from downgrading the initially
negotiated dialect and capabilities between the client and the server."

For more explanation see 2.2.31.4 of MS-SMB2 or
http://blogs.msdn.com/b/openspecification/archive/2012/06/28/smb3-secure-dialect-negotiation.aspxReviewed-by: NPavel Shilovsky <piastry@etersoft.ru>
Signed-off-by: NSteve French <smfrench@gmail.com>

This third version of the patch, incorparating feedback from David Disseldorp
extends the ability of copychunk (refcopy) over smb2/smb3 mounts to
handle servers with smaller than usual maximum chunk sizes
and also fixes it to handle files bigger than the maximum chunk sizes

In the future this can be extended further to handle sending
multiple chunk requests in on SMB2 ioctl request which will
further improve performance, but even with one 1MB chunk per
request the speedup on cp is quite large.
Reviewed-by: NDavid Disseldorp <ddiss@samba.org>
Signed-off-by: NSteve French <smfrench@gmail.com>

This first patch adds the ability for us to do a server side copy
(ie fast copy offloaded to the server to perform, aka refcopy)

"cp --reflink"

of one file to another located on the same server.  This
is much faster than traditional copy (which requires
reading and writing over the network and extra
memcpys).

This first version is not going to be copy
files larger than about 1MB (to Samba) until I add
support for multiple chunks and for autoconfiguring
the chunksize.

It includes:
1) processing of the ioctl
2) marshalling and sending the SMB2/SMB3 fsctl over the network
3) simple parsing of the response

It does not include yet (these will be in followon patches to come soon):
1) support for multiple chunks
2) support for autoconfiguring and remembering the chunksize
3) Support for the older style copychunk which Samba 4.1 server supports
(because this requires write permission on the target file, which
cp does not give you, apparently per-posix).  This may require
a distinct tool (other than cp) and other ioctl to implement.
Reviewed-by: NPavel Shilovsky <piastry@etersoft.ru>
Signed-off-by: NSteve French <smfrench@gmail.com>