spl: Lightweight UBI and UBI fastmap support

Booting a payload out of NAND FLASH from the SPL is a crux today, as
it requires hard partioned FLASH. Not a brilliant idea with the
reliability of todays NAND FLASH chips.

The upstream UBI + UBI fastmap implementation which is about to
brought to u-boot is too heavy weight for SPLs as it provides way more
functionality than needed for a SPL and does not even fit into the
restricted SPL areas which are loaded from the SoC boot ROM.

So this provides a fast and lightweight implementation of UBI scanning
and UBI fastmap attach. The scan and logical to physical block mapping
code is developed from scratch, while the fastmap implementation is
lifted from the linux kernel source and stripped down to fit the SPL
needs.

The text foot print on the board which I used for development is:

6854	0	0	6854	1abd
drivers/mtd/ubispl/built-in.o

Attaching a NAND chip with 4096 physical eraseblocks (4 blocks are
reserved for the SPL) takes:

In full scan mode:      1172ms
In fastmap mode:          95ms

The code requires quite some storage. The largest and unknown part of
it is the number of fastmap blocks to read. Therefor the data
structure is not put into the BSS. The code requires a pointer to free
memory handed in which is initialized by the UBI attach code itself.

See doc/README.ubispl for further information on how to use it.

This shares the ubi-media.h and crc32 implementation of drivers/mtd/ubi
There is no way to share the fastmap code, as UBISPL only utilizes the
slightly modified functions ubi_attach_fastmap() and ubi_scan_fastmap()
from the original kernel ubi fastmap implementation.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NLadislav Michl <ladis@linux-mips.org>
Acked-by: NHeiko Schocher <hs@denx.de>
Reviewed-by: NTom Rini <trini@konsulko.com>

README

@@ -3583,6 +3583,10 @@ FIT uImage format:
 		Support for NAND boot using simple NAND drivers that
 		expose the cmd_ctrl() interface.
 
+		CONFIG_SPL_UBI
+		Support for a lightweight UBI (fastmap) scanner and
+		loader
+
 		CONFIG_SPL_MTD_SUPPORT
 		Support for the MTD subsystem within SPL.  Useful for
 		environment on NAND support within SPL.

doc/README.ubispl

+Lightweight UBI and UBI fastmap support
+
+# Copyright (C) Thomas Gleixner <tglx@linutronix.de>
+#
+# SPDX-License-Identifier: GPL 2.0+ BSD-3-Clause
+
+Scans the UBI information and loads the requested static volumes into
+memory.
+
+Configuration Options:
+
+   CONFIG_SPL_UBI
+     Enables the SPL UBI support
+
+   CONFIG_SPL_UBI_MAX_VOL_LEBS
+     The maximum number of logical eraseblocks which a static volume
+     to load can contain. Used for sizing the scan data structure
+
+   CONFIG_SPL_UBI_MAX_PEB_SIZE
+     The maximum physical erase block size. Either a compile time
+     constant or runtime detection. Used for sizing the scan data
+     structure
+
+   CONFIG_SPL_UBI_MAX_PEBS
+     The maximum physical erase block count. Either a compile time
+     constant or runtime detection. Used for sizing the scan data
+     structure
+
+   CONFIG_SPL_UBI_VOL_IDS
+     The maximum volume ids which can be loaded. Used for sizing the
+     scan data structure.
+
+Usage notes:
+
+In the board config file define for example:
+
+#define CONFIG_SPL_UBI
+#define CONFIG_SPL_UBI_MAX_VOL_LEBS	256
+#define CONFIG_SPL_UBI_MAX_PEB_SIZE	(256*1024)
+#define CONFIG_SPL_UBI_MAX_PEBS		4096
+#define CONFIG_SPL_UBI_VOL_IDS		8
+
+The size requirement is roughly as follows:
+
+    2k for the basic data structure
+  + CONFIG_SPL_UBI_VOL_IDS * CONFIG_SPL_UBI_MAX_VOL_LEBS * 8
+  + CONFIG_SPL_UBI_MAX_PEBS * 64
+  + CONFIG_SPL_UBI_MAX_PEB_SIZE * UBI_FM_MAX_BLOCKS
+
+The last one is big, but I really don't care in that stage. Real world
+implementations only use the first couple of blocks, but the code
+handles up to UBI_FM_MAX_BLOCKS.
+
+Given the above configuration example the requirement is about 5M
+which is usually not a problem to reserve in the RAM along with the
+other areas like the kernel/dts load address.
+
+So something like this will do the trick:
+
+#define SPL_FINFO_ADDR			0x80800000
+#define SPL_DTB_LOAD_ADDR		0x81800000
+#define SPL_KERNEL_LOAD_ADDR		0x82000000
+
+In the board file, implement the following:
+
+static struct ubispl_load myvolumes[] = {
+	{
+		.vol_id		= 0,	/* kernel volume */
+		.load_addr	= (void *)SPL_KERNEL_LOAD_ADDR,
+	},
+	{
+		.vol_id		= 1,	/* DT blob */
+		.load_addr	= (void *)SPL_DTB_LOAD_ADDR,
+	}
+};
+
+int spl_start_uboot(void)
+{
+	struct ubispl_info info;
+
+	info.ubi = (struct ubi_scan_info *) SPL_FINFO_ADDR;
+	info.fastmap = 1;
+	info.read = nand_spl_read_flash;
+
+#if COMPILE_TIME_DEFINED
+	/*
+	 * MY_NAND_NR_SPL_PEBS is the number of physical erase blocks
+	 * in the FLASH which are reserved for the SPL. Think about
+	 * mtd partitions:
+	 *
+	 * part_spl { .start = 0, .end = 4 }
+	 * part_ubi { .start = 4, .end = NR_PEBS }
+	 */
+	info.peb_offset = MY_NAND_NR_SPL_PEBS;
+	info.peb_size = CONFIG_SYS_NAND_BLOCK_SIZE;
+	info.vid_offset = MY_NAND_UBI_VID_OFFS;
+	info.leb_start = MY_NAND_UBI_DATA_OFFS;
+	info.peb_count = MY_NAND_UBI_NUM_PEBS;
+#else
+	get_flash_info(&flash_info);
+	info.peb_offset = MY_NAND_NR_SPL_PEBS;
+	info.peb_size = flash_info.peb_size;
+
+	/*
+	 * The VID and Data offset depend on the capability of the
+	 * FLASH chip to do subpage writes.
+	 *
+	 * If the flash chip supports subpage writes, then the VID
+	 * header starts at the second subpage. So for 2k pages size
+	 * with 4 subpages the VID offset is 512. The DATA offset is 2k.
+	 *
+	 * If the flash chip does not support subpage writes then the
+	 * VID offset is FLASH_PAGE_SIZE and the DATA offset
+	 * 2 * FLASH_PAGE_SIZE
+	 */
+	info.vid_offset = flash_info.vid_offset;
+	info.leb_start = flash_info.data_offset;
+
+	/*
+	 * The flash reports the total number of erase blocks, so
+	 * we need to subtract the number of blocks which are reserved
+	 * for the SPL itself and not managed by UBI.
+	 */
+	info.peb_count = flash_info.peb_count - MY_NAND_NR_SPL_PEBS;
+#endif
+
+	ret = ubispl_load_volumes(&info, myvolumes, ARRAY_SIZE(myvolumes);
+
+	....
+
+}
+
+Note: you can load any payload that way. You can even load u-boot from
+UBI, so the only non UBI managed FLASH area is the one which is
+reserved for the SPL itself and read from the SoC ROM.
+
+And you can do fallback scenarios:
+
+    if (ubispl_load_volumes(&info, volumes0, ARRAY_SIZE(volumes0)))
+        if (ubispl_load_volumes(&info, volumes1, ARRAY_SIZE(volumes1)))
+	    ubispl_load_volumes(&info, vol_uboot, ARRAY_SIZE(vol_uboot));

drivers/Makefile

@@ -27,6 +27,7 @@ obj-$(CONFIG_SPL_MTD_SUPPORT) += mtd/
 obj-$(CONFIG_SPL_NAND_SUPPORT) += mtd/nand/
 obj-$(CONFIG_SPL_ONENAND_SUPPORT) += mtd/onenand/
 obj-$(CONFIG_SPL_SPI_FLASH_SUPPORT) += mtd/spi/
+obj-$(CONFIG_SPL_UBI) += mtd/ubispl/
 obj-$(CONFIG_SPL_DMA_SUPPORT) += dma/
 obj-$(CONFIG_SPL_ETH_SUPPORT) += net/
 obj-$(CONFIG_SPL_ETH_SUPPORT) += net/phy/

drivers/mtd/ubispl/Makefile

+obj-y += ubispl.o ../ubi/crc32.o

drivers/mtd/ubispl/ubi-wrapper.h

+/*
+ * The parts taken from the kernel implementation are:
+ *
+ * Copyright (c) International Business Machines Corp., 2006
+ *
+ * UBISPL specific defines:
+ *
+ * Copyright (c) Thomas Gleixner <tglx@linutronix.de>
+ *
+ * SPDX-License-Identifier: GPL 2.0+ BSD-3-Clause
+ */
+
+/*
+ * Contains various defines copy&pasted from ubi.h and ubi-user.h to make
+ * the upstream fastboot code happy.
+ */
+#ifndef __UBOOT_UBI_WRAPPER_H
+#define __UBOOT_UBI_WRAPPER_H
+
+/*
+ * Error codes returned by the I/O sub-system.
+ *
+ * UBI_IO_FF: the read region of flash contains only 0xFFs
+ * UBI_IO_FF_BITFLIPS: the same as %UBI_IO_FF, but also also there was a data
+ *                     integrity error reported by the MTD driver
+ *                     (uncorrectable ECC error in case of NAND)
+ * UBI_IO_BAD_HDR: the EC or VID header is corrupted (bad magic or CRC)
+ * UBI_IO_BAD_HDR_EBADMSG: the same as %UBI_IO_BAD_HDR, but also there was a
+ *                         data integrity error reported by the MTD driver
+ *                         (uncorrectable ECC error in case of NAND)
+ * UBI_IO_BITFLIPS: bit-flips were detected and corrected
+ *
+ * UBI_FASTMAP_ANCHOR:  u-boot SPL add on to tell the caller that the fastmap
+ *			anchor block has been found
+ *
+ * Note, it is probably better to have bit-flip and ebadmsg as flags which can
+ * be or'ed with other error code. But this is a big change because there are
+ * may callers, so it does not worth the risk of introducing a bug
+ */
+enum {
+	UBI_IO_FF = 1,
+	UBI_IO_FF_BITFLIPS,
+	UBI_IO_BAD_HDR,
+	UBI_IO_BAD_HDR_EBADMSG,
+	UBI_IO_BITFLIPS,
+	UBI_FASTMAP_ANCHOR,
+};
+
+/*
+ * UBI volume type constants.
+ *
+ * @UBI_DYNAMIC_VOLUME: dynamic volume
+ * @UBI_STATIC_VOLUME:  static volume
+ */
+enum {
+	UBI_DYNAMIC_VOLUME = 3,
+	UBI_STATIC_VOLUME  = 4,
+};
+
+/*
+ * Return codes of the fastmap sub-system
+ *
+ * UBI_NO_FASTMAP: No fastmap super block was found
+ * UBI_BAD_FASTMAP: A fastmap was found but it's unusable
+ */
+enum {
+	UBI_NO_FASTMAP = 1,
+	UBI_BAD_FASTMAP,
+};
+
+/**
+ * struct ubi_fastmap_layout - in-memory fastmap data structure.
+ * @e: PEBs used by the current fastmap
+ * @to_be_tortured: if non-zero tortured this PEB
+ * @used_blocks: number of used PEBs
+ * @max_pool_size: maximal size of the user pool
+ * @max_wl_pool_size: maximal size of the pool used by the WL sub-system
+ */
+struct ubi_fastmap_layout {
+	struct ubi_wl_entry *e[UBI_FM_MAX_BLOCKS];
+	int to_be_tortured[UBI_FM_MAX_BLOCKS];
+	int used_blocks;
+	int max_pool_size;
+	int max_wl_pool_size;
+};
+
+/**
+ * struct ubi_fm_pool - in-memory fastmap pool
+ * @pebs: PEBs in this pool
+ * @used: number of used PEBs
+ * @size: total number of PEBs in this pool
+ * @max_size: maximal size of the pool
+ *
+ * A pool gets filled with up to max_size.
+ * If all PEBs within the pool are used a new fastmap will be written
+ * to the flash and the pool gets refilled with empty PEBs.
+ *
+ */
+struct ubi_fm_pool {
+	int pebs[UBI_FM_MAX_POOL_SIZE];
+	int used;
+	int size;
+	int max_size;
+};
+
+#endif

drivers/mtd/ubispl/ubispl.h

+/*
+ * Copyright (c) Thomas Gleixner <tglx@linutronix.de>
+ *
+ * SPDX-License-Identifier: GPL 2.0+ BSD-3-Clause
+ */
+
+#ifndef _UBOOT_MTD_UBISPL_H
+#define _UBOOT_MTD_UBISPL_H
+
+#include "../ubi/ubi-media.h"
+#include "ubi-wrapper.h"
+
+/*
+ * The maximum number of volume ids we scan. So you can load volume id
+ * 0 to (CONFIG_SPL_UBI_VOL_ID_MAX - 1)
+ */
+#define UBI_SPL_VOL_IDS		CONFIG_SPL_UBI_VOL_IDS
+/*
+ * The size of the read buffer for the fastmap blocks. In theory up to
+ * UBI_FM_MAX_BLOCKS * CONFIG_SPL_MAX_PEB_SIZE. In practice today
+ * one or two blocks.
+ */
+#define UBI_FM_BUF_SIZE		(UBI_FM_MAX_BLOCKS*CONFIG_SPL_UBI_MAX_PEB_SIZE)
+/*
+ * The size of the bitmaps for the attach/ scan
+ */
+#define UBI_FM_BM_SIZE		((CONFIG_SPL_UBI_MAX_PEBS / BITS_PER_LONG) + 1)
+/*
+ * The maximum number of logical erase blocks per loadable volume
+ */
+#define UBI_MAX_VOL_LEBS	CONFIG_SPL_UBI_MAX_VOL_LEBS
+/*
+ * The bitmap size for the above to denote the found blocks inside the volume
+ */
+#define UBI_VOL_BM_SIZE		((UBI_MAX_VOL_LEBS / BITS_PER_LONG) + 1)
+
+/**
+ * struct ubi_vol_info - UBISPL internal volume represenation
+ * @last_block:		The last block (highest LEB) found for this volume
+ * @found:		Bitmap to mark found LEBS
+ * @lebs_to_pebs:	LEB to PEB translation table
+ */
+struct ubi_vol_info {
+	u32				last_block;
+	unsigned long			found[UBI_VOL_BM_SIZE];
+	u32				lebs_to_pebs[UBI_MAX_VOL_LEBS];
+};
+
+/**
+ * struct ubi_scan_info - UBISPL internal data for FM attach and full scan
+ *
+ * @read:		Read function to access the flash provided by the caller
+ * @peb_count:		Number of physical erase blocks in the UBI FLASH area
+ *			aka MTD partition.
+ * @peb_offset:		Offset of PEB0 in the UBI FLASH area (aka MTD partition)
+ *			to the real start of the FLASH in erase blocks.
+ * @fsize_mb:		Size of the scanned FLASH area in MB (stats only)
+ * @vid_offset:		Offset from the start of a PEB to the VID header
+ * @leb_start:		Offset from the start of a PEB to the data area
+ * @leb_size:		Size of the data area
+ *
+ * @fastmap_pebs:	Counter of PEBs "attached" by fastmap
+ * @fastmap_anchor:	The anchor PEB of the fastmap
+ * @fm_sb:		The fastmap super block data
+ * @fm_vh:		The fastmap VID header
+ * @fm:			Pointer to the fastmap layout
+ * @fm_layout:		The fastmap layout itself
+ * @fm_pool:		The pool of PEBs to scan at fastmap attach time
+ * @fm_wl_pool:		The pool of PEBs scheduled for wearleveling
+ *
+ * @fm_enabled:		Indicator whether fastmap attachment is enabled.
+ * @fm_used:		Bitmap to indicate the PEBS covered by fastmap
+ * @scanned:		Bitmap to indicate the PEBS of which the VID header
+ *			hase been physically scanned.
+ * @corrupt:		Bitmap to indicate corrupt blocks
+ * @toload:		Bitmap to indicate the volumes which should be loaded
+ *
+ * @blockinfo:		The vid headers of the scanned blocks
+ * @volinfo:		The volume information of the interesting (toload)
+ *			volumes
+ *
+ * @fm_buf:		The large fastmap attach buffer
+ */
+struct ubi_scan_info {
+	ubispl_read_flash		read;
+	unsigned int			fsize_mb;
+	unsigned int			peb_count;
+	unsigned int			peb_offset;
+
+	unsigned long			vid_offset;
+	unsigned long			leb_start;
+	unsigned long			leb_size;
+
+	/* Fastmap: The upstream required fields */
+	int				fastmap_pebs;
+	int				fastmap_anchor;
+	size_t				fm_size;
+	struct ubi_fm_sb		fm_sb;
+	struct ubi_vid_hdr		fm_vh;
+	struct ubi_fastmap_layout	*fm;
+	struct ubi_fastmap_layout	fm_layout;
+	struct ubi_fm_pool		fm_pool;
+	struct ubi_fm_pool		fm_wl_pool;
+
+	/* Fastmap: UBISPL specific data */
+	int				fm_enabled;
+	unsigned long			fm_used[UBI_FM_BM_SIZE];
+	unsigned long			scanned[UBI_FM_BM_SIZE];
+	unsigned long			corrupt[UBI_FM_BM_SIZE];
+	unsigned long			toload[UBI_FM_BM_SIZE];
+
+	/* Data for storing the VID and volume information */
+	struct ubi_vol_info		volinfo[UBI_SPL_VOL_IDS];
+	struct ubi_vid_hdr		blockinfo[CONFIG_SPL_UBI_MAX_PEBS];
+
+	/* The large buffer for the fastmap */
+	uint8_t				fm_buf[UBI_FM_BUF_SIZE];
+};
+
+#ifdef CFG_DEBUG
+#define ubi_dbg(fmt, ...) printf("UBI: debug:" fmt "\n", ##__VA_ARGS__)
+#else
+#define ubi_dbg(fmt, ...)
+#endif
+
+#ifdef CONFIG_UBI_SILENCE_MSG
+#define ubi_msg(fmt, ...)
+#else
+#define ubi_msg(fmt, ...) printf("UBI: " fmt "\n", ##__VA_ARGS__)
+#endif
+/* UBI warning messages */
+#define ubi_warn(fmt, ...) printf("UBI warning: " fmt "\n", ##__VA_ARGS__)
+/* UBI error messages */
+#define ubi_err(fmt, ...) printf("UBI error: " fmt "\n", ##__VA_ARGS__)
+
+#endif

include/ubispl.h

+/*
+ * Copyright (c) Thomas Gleixner <tglx@linutronix.de>
+ *
+ * SPDX-License-Identifier: GPL 2.0+ BSD-3-Clause
+ */
+#ifndef __UBOOT_UBISPL_H
+#define __UBOOT_UBISPL_H
+
+/*
+ * The following CONFIG options are relevant for UBISPL
+ *
+ * #define CONFIG_SPL_UBI_MAX_VOL_LEBS		256
+ *
+ * Defines the maximum number of logical erase blocks per loadable
+ * (static) volume to size the ubispl internal arrays.
+ *
+ * #define CONFIG_SPL_UBI_MAX_PEB_SIZE		(256*1024)
+ *
+ * Defines the maximum physical erase block size to size the fastmap
+ * buffer for ubispl.
+ *
+ * #define CONFIG_SPL_UBI_MAX_PEBS		4096
+ *
+ * Define the maximum number of physical erase blocks to size the
+ * ubispl internal arrays.
+ *
+ * #define CONFIG_SPL_UBI_VOL_IDS		8
+ *
+ * Defines the maximum number of volumes in which UBISPL is
+ * interested. Limits the amount of memory for the scan data and
+ * speeds up the scan process as we simply ignore stuff which we dont
+ * want to load from the SPL anyway. So the volumes which can be
+ * loaded in the above example are ids 0 - 7
+ */
+
+/*
+ * The struct definition is in drivers/mtd/ubispl/ubispl.h. It does
+ * not fit into the BSS due to the large buffer requirement of the
+ * upstream fastmap code. So the caller of ubispl_load_volumes needs
+ * to hand in a pointer to a free memory area where ubispl will place
+ * its data. The area is not required to be initialized.
+ */
+struct ubi_scan_info;
+
+typedef int (*ubispl_read_flash)(int pnum, int offset, int len, void *dst);
+
+/**
+ * struct ubispl_info - description structure for fast ubi scan
+ * @ubi:		Pointer to memory space for ubi scan info structure
+ * @peb_size:		Physical erase block size
+ * @vid_offset:		Offset of the VID header
+ * @leb_start:		Start of the logical erase block, i.e. offset of data
+ * @peb_count:		Number of physical erase blocks in the UBI FLASH area
+ *			aka MTD partition.
+ * @peb_offset:		Offset of PEB0 in the UBI FLASH area (aka MTD partition)
+ *			to the real start of the FLASH in erase blocks.
+ * @fastmap:		Enable fastmap attachment
+ * @read:		Read function to access the flash
+ */
+struct ubispl_info {
+	struct ubi_scan_info	*ubi;
+	u32			peb_size;
+	u32			vid_offset;
+	u32			leb_start;
+	u32			peb_count;
+	u32			peb_offset;
+	int			fastmap;
+	ubispl_read_flash	read;
+};
+
+/**
+ * struct ubispl_load - structure to describe a volume to load
+ * @vol_id:	Volume id
+ * @load_addr:	Load address of the volume
+ */
+struct ubispl_load {
+	int		vol_id;
+	void		*load_addr;
+};
+
+/**
+ * ubispl_load_volumes - Scan flash and load volumes
+ * @info:	Pointer to the ubi scan info structure
+ * @lovls:	Pointer to array of volumes to load
+ * @nrvols:	Array size of @lovls
+ */
+int ubispl_load_volumes(struct ubispl_info *info,
+			struct ubispl_load *lvols, int nrvols);
+
+#endif