lkml.org 
[lkml]   [2012]   [Sep]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectUBI: Fastmap request for inclusion (v19)
Date
UBI Fastmap is an optional feature which stores the physical to
logical eraseblock relations in a checkpoint (called fastmap) to reduce
the initialization time of UBI. The current init time of UBI is
proportional to the number of physical erase blocks on the FLASH
device. With fastmap enabled the scan time is limited to a fixed
number of blocks.

Fastmap does not affect any of the existing UBI robustness
features and in fact the fastmap code falls back to scanning
mode when the found fastmap is corrupted.

A fastmap consists of two elements:

1) A primary super block, which contains merrily a pointer to the
erase block(s) which hold the real fastmap data.

This super block is guaranteed to be held within the first 64
eraseblocks of a device.

2) The secondary fastmap blocks, which contain the real
fastmap data (physical to logical eraseblock relations,
erase counts, sequence numbers ...)

Aside of that the fastmap data contains a list of blocks
which belong to the active working pool. The active working pool is
a fixed number of blocks to be used by the WL sub-system itself and the
layer on top of UBI, which can be modified before the next fastmap
is written to FLASH. These blocks need to be scanned in the
conventional UBI scan mode.

The reason for these pool blocks is to reduce the fastmap
updates to the necessary minimum to avoid accelerated device
wearout in scenarios where data changes rapidly. The fastmap
data is updated whenever a working pool runs out of blocks.

So the fastmap scan consists of the following steps:

1) Find the fastmap super block block by scanning the start of the
device.

2) Read the real fastmap data and construct the UBI device info
structures.

3) Scan the pool blocks.

The fastmap functionality is fully compatible with existing UBI
deployments. If no fastmap blocks can be found then the device is
scanned and the fastmap blocks are created from the scanned
information.

Changes sinze v18:
- Aiaiai uncovered some issues, now the patch set is fully bisectable,
all compiler warnings have been fixed (set but unused variable),
coccinelle found a missing check for kmalloc()'s return value.

"drivers/mtd/ubi/wl.c:284:3-9: preceding lock on line 282 [coccinelle]"
is a false positive.

Changes since v17:

- UBI reserves now PEBs to hold two fastmaps. It is guaranteed that
each LEB has a corresponding PEB which can be used by an user.

- Fastmap is automatically disabled if the FLASH devices has less than 64 PEBs

- Fastmap is only available if CONFIG_MTD_UBI_FASTMAP is enabled.
It is per default N. If fastmap has proven stable over some time we can
remove CONFIG_MTD_UBI_FASTMAP and the corresponding #ifdefs.
Even with CONFIG_MTD_UBI_FASTMAP=Y UBI will not automatically
install a fastmap on existing images.
This behavior is controlled by the fm_autoconvert UBI module parameter.
Only if set to 1 UBI will install a fastmap automatically.
By default it will use the fastmap only if one is found.

So, if you want to test fastmap enable CONFIG_MTD_UBI_FASTMAP and
set fm_autoconvert to 1.
If the device was attached by a fastmap you'll observe the following log
message while attaching: "UBI: attached by fastmap".

State of fastmap:

We (linutronix gmbh) did numerous tests and are not aware of any issue or
regression so far. The tests have been done on real hardware and simulators
(nandsim and mtdram). Of course with and without bitflip emulation enabled
and on NAND "bad" devices with bad blocks, bitflips, IO erros, ...
The tests consisted of mtd-util's ubi-tests, my fastmap torture script[1]
and real world use cases.

Each test run was done with the following configurations:
- CONFIG_MTD_UBI_FASTMAP=n
To verify that traditional UBI works fine.

- CONFIG_MTD_UBI_FASTMAP=y, fm_autoconvert=0
In this configuration the fastmap pool mechanism is active but
no fastmap is written to non-fastmap devices.

- CONFIG_MTD_UBI_FASTMAP=y, fm_autoconvert=1
Fastmap is enabled and non-fastmap devices get fastmap enabled.
Either at detach time or while writing.

I hope the MTD community do will also test fastmap in many different situations.
Bug reports are welcome!

Future work:

- Currently fastmap's powercut policy is that it will fall back to scanning mode if
a power cut happened while writing the fastmap. If users really need fast attaching
also after a power cut we could use the adjacent blocks mechanism for writing fastmap.
- Add fastmap support to ubinize
- If fastmap has proven stable, we can remove CONFIG_MTD_UBI_FASTMAP
and enable it per default.

This patch set is based on artems ubi repository:
git://git.infradead.org/linux-ubi.git

Thanks to the folks at CELF who sponsored this work!

Thanks,
//richard

[1]
http://lists.infradead.org/pipermail/linux-mtd/2012-June/042347.html

MAINTAINERS | 6 +
drivers/mtd/ubi/Kconfig | 20 ++
drivers/mtd/ubi/Makefile | 1 +
drivers/mtd/ubi/attach.c | 386 +++++++++++++++++++---------
drivers/mtd/ubi/build.c | 70 +++++-
drivers/mtd/ubi/eba.c | 126 +++++++++-
drivers/mtd/ubi/ubi-media.h | 137 ++++++++++
drivers/mtd/ubi/ubi.h | 118 +++++++++-
drivers/mtd/ubi/wl.c | 599 ++++++++++++++++++++++++++++++++++++++-----
9 files changed, 1252 insertions(+), 211 deletions(-)

[PATCH 01/10] UBI: Add fastmap on-flash data structures
[PATCH 02/10] UBI: Add fastmap stuff to ubi.h
[PATCH 03/10] UBI: Export next_sqnum()
[PATCH 04/10] UBI: Add self_check_eba()
[PATCH 05/10] UBI: Add fastmap bits to build.c
[PATCH 06/10] UBI: Wire-up ->fm_sem
[PATCH 07/10] UBI: Add fastmap stuff to attach.c
[PATCH 08/10] UBI: Add fastmap support to the WL sub-system
[PATCH 09/10] UBI: Add fastmap core
[PATCH 10/10] UBI: Wire-up fastmap


\
 
 \ /
  Last update: 2012-09-26 18:41    [W:1.063 / U:0.012 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site