Fix the followong compilation warnings introduced by commit
095751a6:

drivers/mtd/ubi/scan.c: In function 'check_what_we_have':
drivers/mtd/ubi/scan.c:960: warning: passing argument 1 of 'get_random_bytes' discards qualifiers from pointer target type

Fix the following compilation warnings introduced by commit
1a49af2c:

drivers/mtd/ubi/io.c: In function 'ubi_io_read':
drivers/mtd/ubi/io.c:153: warning: initialization makes integer from pointer without a cast
drivers/mtd/ubi/io.c:170: warning: format '%s' expects type 'char *', but argument 5 has type 'int'
drivers/mtd/ubi/io.c:177: warning: format '%zd' expects type 'signed size_t', but argument 7 has type 'int'
drivers/mtd/ubi/io.c:177: warning: too many arguments for format

Also, amend the ECC error code string and add brackets and whitespace
there - this should make the message readable.
Reported-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

ECC errors are quite typical errors on NAND, so it is worth improving
the UBI message and print something like

ubi_io_read: error -74 (ECC error) while reading 4096 bytes from PEB 1:4 ...

rather than

ubi_io_read: error -74 while reading 4096 bytes from PEB 1:4 ...
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch introduces the %UBI_IO_BAD_HDR_READ return code for
the I/O level function. We will use this code in order to distinguish
between "corrupted header possibly because this is non-ubi data" and
"corrupted header possibly because of real data corruption and ECC error".

So far this patch does not introduce any functional change, just a
preparation.

This patch is pased on a patch from
Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>
Reviewed-by: NSebastian Andrzej Siewior <sebastian@breakpoint.cc>
Tested-by: NSebastian Andrzej Siewior <sebastian@breakpoint.cc>

We do not really need 2 separate error codes for indicating bad VID
and bad EC headers (UBI_IO_BAD_EC_HDR, UBI_IO_BAD_VID_HDR), it is
enough to have only one UBI_IO_BAD_HDR return code.

This patch does not introduce any functional change, only some
code simplification.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>
Reviewed-by: NSebastian Andrzej Siewior <sebastian@breakpoint.cc>
Tested-by: NSebastian Andrzej Siewior <sebastian@breakpoint.cc>

Signed-off-by: NShinya Kuribayashi <shinya.kuribayashi.px@renesas.com>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Signed-off-by: NShinya Kuribayashi <shinya.kuribayashi.px@renesas.com>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: NTejun Heo <tj@kernel.org>
Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

Add an extra debugging check function which validates writes.
After every write it reads the data back, compares it with the
original data, and complains if they mismatch.

Useful for debugging. No-op if extra debugging checks are disabled.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI debugging functions were a little bit over-engineered and
returned more error codes than needed, and the callers had to
do useless checks. Simplify the return codes.

Impact: only debugging code is affected, which means that for
        non-developers this is a no-op patch.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

More testing of NOR flash against power cuts showed that sometimes
eraseblocks may be unwritable, and we cannot really invalidate
them before erasure. But in this case the eraseblock probably
contains garbage anyway, and we do not have to invalidate the
headers. This assumption might be not true, but this is at least
what I have observed. So if we cannot invalidate the headers,
we make sure that the PEB does not contain valid VID header.
If this is true, everything is fine, otherwise we panic.

Useful for debugging problems, compiled in only if UBI debugging
is enabled. This patch also makes the UBI writing function dump
the flash if it fails to write.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

In case of NOR flash, UBI zeroes EC and VID headers' magic,
in order to detect interrupted erasures. It first zeroes out
the EC magic, then VID magic. However, if a power cut happens
in between, we'll end up with a corrupted EC header and a valid
VID header, in which case UBI accepts the PEB, but prints a
warning. This patch makes sure we first zero out the VID
magic, then the EC magic, not vice versa. This is just a
small amendment to prevent warning messages.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

The recent "UBI: fix NOR flash recovery" introduced compilation
warnings which were immediately spotted by our linux-next keeper.
This patch fixes them.
Reported-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

This commit fixes NOR flash recovery issues observed with Spansion
S29GL512N NOR.

When NOR erases, it first fills PEBs with zeroes, then sets all bytes
to 0xFF. Filling with zeroes starts from the end of the PEB. And when
power is cut, this results in PEBs containing correct EC and VID headers
but corrupted with zeros at the end. This confuses UBI and it mistakinly
accepts these PEBs and associate them with LEBs.

Fis this issue by zeroing EC and VID magics before erasing PEBs, to
make UBI later refuse zem.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Move the image seq. number handling from I/O level to the scanning
lever, where it really belongs to. Move the @image_seq_set variable
to the @struct ubi_scan_info structure, which exists only during
scanning.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

An image sequence number is added to the UBI erase-counter header
to be able determine if the root file system contains a mixture
of old and new images (because the flashing failed to complete).

A change to nolo is also needed for this to take effect.
Signed-off-by: NAdrian Hunter <adrian.hunter@nokia.com>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

The 'paranoid_check_empty()' is bogus because, which is easilly
seen on NOR flash, which has long erase cycles, and which may
easilly end-up with half-erased eraseblocks. In this case the
paranoid check fails. I is just wrong to assume that PEBs which
do not have EC headers always contain all 0xFF. Such assumption
should not be made on the I/O level, which is quite low.

Thus, just kill the check.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch adds code which makes sure eraseblocks contain all 0xFF
bytes before starting using them. The verification is done only when
debugging checks are enabled.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Some of the typos were indicated by Adrian Hunter,
some by 'aspell'.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

When paranoid checs are enabled, the 'io_paral' test from the
'mtd-utils' package fails. The symptoms are:

UBI error: paranoid_check_all_ff: flash region at PEB 3973:512, length 15872 does not contain all 0xFF bytes
UBI error: paranoid_check_all_ff: paranoid check failed for PEB 3973
UBI: hex dump of the 512-16384 region

It turned out to be a bug in the checking function. Suppose there
are 2 tasks - A and B. Task A is the wear-levelling working
('wear_leveling_worker()'). It is reading the VID header to find
which LEB this PEB belongs to. Say, task A is reading header
of PEB X. Suppose PEB X is unmapped, and has no VID header.
Task B is trying to write to PEB X.

Task A: in 'ubi_io_read_vid_hdr()': reads the VID header from PEB X.
        The read data contain all 0xFF bytes.
Task B: writes VID header and some data to PEB X
Task A: assumes PEB X is empty, calls 'paranoid_check_all_ff()', which
        fails.

The solution for this problem is to make 'paranoid_check_all_ff()'
re-read the VID header, re-check it, and only if it is not there,
check the rest. This now implemented by the 'paranoid_check_empty()'
function.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

- (better, more, bigger ...) then -> (...) than
Signed-off-by: NFrederik Schwarzer <schwarzerf@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Just minor indentation and "over 80 characters" fixes.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

The 'ubi_io_read_vid_hdr()' and 'ubi_io_read_ec_hdr()' function
have the 'verbose' argument which controls whether they should
print a warning if the VID/EC header was not found or was corrupted.
Some callers require the headers to be OK, and pass 1. Some allow
a corrupted/not present header, and pass 0.

       if (UBI_IO_DEBUG)
               verbose = 1;

And UBI_IO_DEBUG is 1 if CONFIG_MTD_UBI_DEBUG_MSG_BLD is true. So in
this case the warning is printed all the time. This confuses people.

Thus, do not print the messages as warnings if UBI_IO_DEBUG is true,
but print them as debugging messages instead.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

No functional changes, just tweak comments to make kernel-doc
work fine and stop complaining.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Just out or curiousity ran checkpatch.pl for whole UBI,
and discovered there are quite a few of stylistic issues.
Fix them.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

If bit-flips happen often, UBI prints to many messages. Lessen
the amount by only printing the messages when the PEB has been
scrubbed. Also, print torturing messages.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Hch asked not to use "unit" for sub-systems, let it be so.
Also some other commentaries modifications.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Make I/O function to be always verbose when about CRC errors
and magic number errors when I/O debugging is enabled.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

When NAND detects an ECC error, it returns -EBADMSG. It does not
stop reading requested data if one page has an ECC error, it keeps
going and reads all the requested data. If it fails to read all
the data, it does not return -EBADMSG, but returns the error code
which reflects the reason of the failure.

But some drivers may have bugs (e.g., OneNAND had) and stop reading
after the first ECC error, so it returns -EBADMSG. In turn, UBI
propagates this up to the caller. The caller will treat this as
"all the requested data was read, but there was an ECC error".

So we change the error code to -EIO if it is -EBADMSG and the read
length is less then the requested length. We also add an assertion,
so if UBI debugging is enabled, UBI will bug.
Pointed-to-by: NAdrian Hunter <ext-adrian.hunter@nokia.com>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

More handy since word hexdump prints in host endian.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Similar reason as in case of the previous patch: it causes
deadlocks if a filesystem with writeback support works on top
of UBI. So pre-allocate needed buffers when attaching MTD device.
We also need mutexes to protect the buffers, but they do not
cause much contantion because they are used in recovery, torture,
and WL copy routines, which are called seldom.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Use GFP_NOFS flag when allocating memory on I/O path, because otherwise
we may deadlock the filesystem which works on top of us. We observed
the deadlocks with UBIFS. Example:

VFS->FS lock a lock->UBI->kmalloc()->VFS writeback->FS locks the same
lock again.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

I hit those situations and found out lack of print messages. Add more prints
when erase problems occur.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

There is signed multiplication assigned to unsigned ei.addr in io.c.
This causes wrong addresses for big multiplication.This patch solves the
problem.
Signed-off-by: NBrijesh Singh <brijesh.s.singh@gmail.com>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

Kill UBI's homegrown endianess handling and replace it with
the standard kernel endianess handling.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI allocates temporary buffers of PEB size, which may be 256KiB.
Use vmalloc instead of kmalloc for such big temporary buffers.
Signed-off-by: NArtem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI (Latin: "where?") manages multiple logical volumes on a single
flash device, specifically supporting NAND flash devices. UBI provides
a flexible partitioning concept which still allows for wear-levelling
across the whole flash device.

In a sense, UBI may be compared to the Logical Volume Manager
(LVM). Whereas LVM maps logical sector numbers to physical HDD sector
numbers, UBI maps logical eraseblocks to physical eraseblocks.

More information may be found at
http://www.linux-mtd.infradead.org/doc/ubi.html

Partitioning/Re-partitioning

  An UBI volume occupies a certain number of erase blocks. This is
  limited by a configured maximum volume size, which could also be
  viewed as the partition size. Each individual UBI volume's size can
  be changed independently of the other UBI volumes, provided that the
  sum of all volume sizes doesn't exceed a certain limit.

  UBI supports dynamic volumes and static volumes. Static volumes are
  read-only and their contents are protected by CRC check sums.

Bad eraseblocks handling

  UBI transparently handles bad eraseblocks. When a physical
  eraseblock becomes bad, it is substituted by a good physical
  eraseblock, and the user does not even notice this.

Scrubbing

  On a NAND flash bit flips can occur on any write operation,
  sometimes also on read. If bit flips persist on the device, at first
  they can still be corrected by ECC, but once they accumulate,
  correction will become impossible. Thus it is best to actively scrub
  the affected eraseblock, by first copying it to a free eraseblock
  and then erasing the original. The UBI layer performs this type of
  scrubbing under the covers, transparently to the UBI volume users.

Erase Counts

  UBI maintains an erase count header per eraseblock. This frees
  higher-level layers (like file systems) from doing this and allows
  for centralized erase count management instead. The erase counts are
  used by the wear-levelling algorithm in the UBI layer. The algorithm
  itself is exchangeable.

Booting from NAND

  For booting directly from NAND flash the hardware must at least be
  capable of fetching and executing a small portion of the NAND
  flash. Some NAND flash controllers have this kind of support. They
  usually limit the window to a few kilobytes in erase block 0. This
  "initial program loader" (IPL) must then contain sufficient logic to
  load and execute the next boot phase.

  Due to bad eraseblocks, which may be randomly scattered over the
  flash device, it is problematic to store the "secondary program
  loader" (SPL) statically. Also, due to bit-flips it may become
  corrupted over time. UBI allows to solve this problem gracefully by
  storing the SPL in a small static UBI volume.

UBI volumes vs. static partitions

  UBI volumes are still very similar to static MTD partitions:

    * both consist of eraseblocks (logical eraseblocks in case of UBI
      volumes, and physical eraseblocks in case of static partitions;
    * both support three basic operations - read, write, erase.

  But UBI volumes have the following advantages over traditional
  static MTD partitions:

    * there are no eraseblock wear-leveling constraints in case of UBI
      volumes, so the user should not care about this;
    * there are no bit-flips and bad eraseblocks in case of UBI volumes.

  So, UBI volumes may be considered as flash devices with relaxed
  restrictions.

Where can it be found?

  Documentation, kernel code and applications can be found in the MTD
  gits.

What are the applications for?

  The applications help to create binary flash images for two purposes: pfi
  files (partial flash images) for in-system update of UBI volumes, and plain
  binary images, with or without OOB data in case of NAND, for a manufacturing
  step. Furthermore some tools are/and will be created that allow flash content
  analysis after a system has crashed..

Who did UBI?

  The original ideas, where UBI is based on, were developed by Andreas
  Arnez, Frank Haverkamp and Thomas Gleixner. Josh W. Boyer and some others
  were involved too. The implementation of the kernel layer was done by Artem
  B. Bityutskiy. The user-space applications and tools were written by Oliver
  Lohmann with contributions from Frank Haverkamp, Andreas Arnez, and Artem.
  Joern Engel contributed a patch which modifies JFFS2 so that it can be run on
  a UBI volume. Thomas Gleixner did modifications to the NAND layer. Alexander
  Schmidt made some testing work as well as core functionality improvements.
Signed-off-by: NArtem B. Bityutskiy <dedekind@linutronix.de>
Signed-off-by: NFrank Haverkamp <haver@vnet.ibm.com>