Lots of filesystems calls vmtruncate despite not implementing the old
->truncate method.  Switch them to use simple_setsize and add some
comments about the truncate code where it seems fitting.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNick Piggin <npiggin@suse.de>
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: NTejun Heo <tj@kernel.org>
Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

Improve 'dbg_dump_lprop()' and print dark and dead space there,
decode flags, and journal heads.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

The journal head names and numbers are part of the UBIFS format, so
they should be in the ubifs-media.h.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

At the moment UBIFS print large and scary error messages and
flash dumps in case of nearly any corruption, even if it is
a recoverable corruption. For example, if the master node is
corrupted, ubifs_scan() prints error dumps, then UBIFS recovers
just fine and goes on.

This patch makes UBIFS print scary error messages only in
real cases, which are not recoverable. It adds 'quiet' argument
to the 'ubifs_scan()' function, so the caller may ask 'ubi_scan()'
not to print error messages if the caller is able to do recovery.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>
Reviewed-by: NAdrian Hunter <Adrian.Hunter@nokia.com>

Signed-off-by: NAdrian Hunter <adrian.hunter@nokia.com>

This patch cleans up write-buffer timeout initialization and
sets it to 3-5 interval.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch adds the following minor optimization:

1. If write-buffer does not use the timer, indicate it with the
   wbuf->no_timer variable, instead of using the wbuf->softlimit
   variable. This is better because wbuf->softlimit is of ktime_t
   type, and the ktime_to_ns function contains 64-bit multiplication.

2. Do not call the 'hrtimer_cancel()' function for write-buffers
   which do not use timers.

3. Do not cancel the timer in 'ubifs_put_super()' because the
   synchronization function does this.

This patch also removes a confusing comment.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBIFS uses timers for write-buffer write-back. It is not
crucial for us to write-back exactly on time. We are fine
to write-back a little earlier or later. And this means
we may optimize UBIFS timer so that it could be groped
with a close timer event, so that the CPU would not be
waken up just to do the write back. This is optimization
to lessen power consumption, which is important in
embedded devices UBIFS is used for.

hrtimers have a nice feature: they are effectively range
timers, and we may defind the soft and hard limits for
it. Standard timers do not have these feature. They may
only be made deferrable, but this means there is effectively
no hard limit. So, we will better use hrtimers.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Now UBIFS is supported by u-boot. If we ever decide to change the
media format, then people will have to upgrade their u-boots to
mount new format images. However, very often it is possible to
preserve R/O forward-compatibility, even though the write
forward-compatibility is not preserved.

This patch introduces a new super-block field which stores the
R/O compatibility version.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>
Acked-by: NAdrian Hunter <Adrian.Hunter@nokia.com>

This patch introduces a helpful @c->idx_leb_size variable.
The patch also fixes some spelling issues and makes comments
use "LEB" instead of "eraseblock", which is more correct.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Make 'ubifs_find_free_space()' return offset where free space starts,
rather than the amount of free space. This is just more appropriat
for its caller.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

This UBIFS feature has never worked properly, and it was a mistake
to add it because we simply have no use-cases. So, lets still accept
the fast_unmount mount option, but ignore it. This does not change
much, because UBIFS commit in sync_fs anyway, and sync_fs is called
while unmounting.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

- preserve the idx_gc list - it will be needed in the same
state, should UBIFS be remounted rw again
- prevent remounting ro if we have switched to read only
mode (due to a fatal error)
Signed-off-by: NAdrian Hunter <ext-adrian.hunter@nokia.com>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

When data CRC checking is disabled, UBIFS returns incorrect return
code from the 'try_read_node()' function (0 instead of 1, which means
CRC error), which make the caller re-read the data node again, but using
a different code patch, so the second read is fine. Thus, we read the
same node twice. And the result of this is that UBIFS is slower
with no_chk_data_crc option than it is with chk_data_crc option.
This patches fixes the problem.
Reported-by: NReuben Dowle <Reuben.Dowle@navico.com>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

When mounting read-only the orphan area head is
not initialized.  It must be initialized when
remounting read/write, but it was not.  This patch
fixes that.

[Artem: sorry, added comment tweaking noise]
Signed-off-by: NAdrian Hunter <ext-adrian.hunter@nokia.com>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

We observe space corrupted accounting when re-mounting. So add some
debbugging checks to catch problems like this.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

When freeing the c->idx_lebs list, we have to release the LEBs as well,
because we might be called from mount to read-only mode code. Otherwise
the LEBs stay taken forever, which may cause problems when we re-mount
back ro RW mode.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Just add more commentaries. Also some commentary fixes for
lprops flags.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Mark super, file, and inode operation structcutes with 'const'.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

These are mostly long lines and wrong indentation warning
fixes. But also there are two volatile variables and
checkpatch.pl complains about them:

WARNING: Use of volatile is usually wrong: see Documentation/volatile-considered-harmful.txt
+       volatile int gc_seq;

WARNING: Use of volatile is usually wrong: see Documentation/volatile-considered-harmful.txt
+       volatile int gced_lnum;

Well, we anyway use smp_wmb() for c->gc_seq and c->gced_lnum, so
these 'volatile' modifiers can be just dropped.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Instead of using do_div(), use better primitives from
linux/math64.h.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Take into account that 2 eraseblocks are never available because
they are reserved for the index. This gives more realistic count
of FS blocks.

To avoid future confusions like this, introduce a constant.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Dump available space calculated by budgeting subsystem.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch fixes the following section mismatch:

WARNING: fs/ubifs/ubifs.o(.init.text+0xec): Section mismatch in reference from the function init_module() to the function .exit.text:ubifs_compressors_exit()
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

We need to have a possibility to see various UBIFS variables
and ask UBIFS to dump various information. Debugfs is what
we need.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Introduce a new data structure which contains all debugging
stuff inside. This is cleaner than having debugging stuff
directly in 'c'.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

It is very handy to be able to change default UBIFS compressor
via mount options. Introduce -o compr=<name> mount option support.
Currently only "none", "lzo" and "zlib" compressors are supported.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Save a 4 bytes of RAM per 'struct inode' by stroring inode
compression type in bit-filed, instead of using 'int'.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

To avoid memory allocation failure during bulk-read, pre-allocate
a bulk-read buffer, so that if there is only one bulk-reader at
a time, it would just use the pre-allocated buffer and would not
do any memory allocation. However, if there are more than 1 bulk-
reader, then only one reader would use the pre-allocated buffer,
while the other reader would allocate the buffer for itself.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Bulk-read allocates 128KiB or more using kmalloc. The allocation
starts failing often when the memory gets fragmented. UBIFS still
works fine in this case, because it falls-back to standard
(non-optimized) read method, though. This patch teaches bulk-read
to allocate exactly the amount of memory it needs, instead of
allocating 128KiB every time.

This patch is also a preparation to the further fix where we'll
have a pre-allocated bulk-read buffer as well. For example, now
the @bu object is prepared in 'ubifs_bulk_read()', so we could
path either pre-allocated or allocated information to
'ubifs_do_bulk_read()' later. Or teaching 'ubifs_do_bulk_read()'
not to allocate 'bu->buf' if it is already there.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Bulk-read allocates a lot of memory with 'kmalloc()', and when it
is/gets fragmented 'kmalloc()' fails with a scarry warning. But
because bulk-read is just an optimization, UBIFS keeps working fine.
Supress the warning by passing __GFP_NOWARN option to 'kmalloc()'.

This patch also introduces a macro for the magic 128KiB constant.
This is just neater.

Note, this is not really fixes the problem we had, but just hides
the warnings. The further patches fix the problem.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Also add debugging checks for LPT size and separate
out c->check_lpt_free from unrelated bitfields.
Signed-off-by: NAdrian Hunter <ext-adrian.hunter@nokia.com>

Znode may refer both data nodes and indexing nodes
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

We cannot store bit-fields together if the processes which
change them may race, unless we serialize them.

Thus, move the nospc and nospc_rp bit-fields eway from
the mount option/constant bit-fields, to avoid races.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

The "bulk_read" and "no_chk_data_crc" have only 2 values -
0 and 1. We already have bit-fields in corresponding data
structers, so make "bulk_read" and "no_chk_data_crc"
bit-fields as well.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>