Messages in this thread | | | Date | Sun, 17 Jul 2005 15:41:48 +0900 | From | Hiroyuki Machida <> | Subject | [RFD] FAT robustness |
| |
Folks,
I'd like to have a discussion about FAT robustness. Please give your thought, comments and related issues.
About few years ago, we added some features to FAT, called xvfat, so that System and FAT have robustness against unexpected media hot unplug and ability to let applications correctly be aware the event.
Just for your reference, I put a patch to 2.4.20 kernel at http://www.celinuxforum.org/CelfPubWiki/XvFatDiscussion?action=AttachFile&do=get&target=20050715-xvfat-2.4.20.patch This includes following features;
Handle media removed during “mount” Notification of media removal to application Cancellation of I/O Elevator for Block device Block system calls until a completion of writing Control order of meta-data updates, using transaction control implemented in fs/xvfat/fwrq.c File syscall return “error”, except umount Japanese file name support possible 1-N mapping issues SJIS <-> UNICODE Dirty Flag support TIME ZONE support
On moving to 2.6, we consider and categorize issues, again. And we are planing to have open source project for these features to add 2.6 kernel. I'd like to open discussion about these features and how to implement on 2.6 kernel.
1. Issues to be addressed
- Issues around FAT with CE devices - Hot unplug issues - File System corruption on unplug media/storage device Almost same as power down without umount
- Notification of the event Application need to know the event precisely Need to more investigation
- System stability after unplug Almost same as I/O error recovery issues discussed at LKLM http://developer.osdl.jp/projects/doubt/fs-consistency-and-coherency/index.html
http://groups.google.co.jp/group/linux.kernel/browse_thread/thread/b9c11bccd59e0513/4a4dd84b411c6d32?q=[RFD]+FS+behavior+(I%2FO+failure)+in+kernel+summit++lkml&rnum=1&hl=ja#4a4dd84b411c6d32
- Other issues - Time stamp issues using always local time time resolution is 2sec unit
- Issues around mapping with UNICODE and local char code 1-N mapping SJIS<-> UNICODE Potential directory cache problem due to 1 –N mapping Possible inconsistency problems with application side
- Support file size over 2GB
- Support dirty flag
Q1 : First issue for discussion is "Do you have any other issues about this?" and "Do you have any other idea to categorize the issues?"
2. FAT corruption on unplug media/storage device
On starting the open source project, we focus to the following issue, first. - File System corruption on unplug media/storage device Almost same as power down without umount
And, we are planing to focus on HDD device and treat system power down instead of unplug media, because A. Damages and it's counter methods may depend on property of lower layer E.g. - Memory Card Some controller can guaranty atomicity of certain operations - Flush Memory (NAND, NOR) I/O operations may be constrained by Block Size (e,g, 128KB) or Page Size (e.g. 2KB) - HDD - Cache memory my resident inside in - Sector which is under writing on power down may be corrupted(can't read anymore)
B. It may make the problem easier - Sector size is 512 Byte - Many developers may check with PC
Q2 : Do you know any other storage devices and it's property, to be address later?
3. Features to be developed for FAT corruption.
We currently plan to add following features to address FAT corruption.
- Utilize standard 2.6 features as much as possible - Implement as options of fat, vfat and uvfat - Utilize existent journal block device (JBD) for transaction control - Utilize noop elevator to cancel unexpected operation reordering - Coordinate order of operations so that update data first, meta data later with transaction control - With O_SYNC, close() make flush all related data and meta-data, then wait completion of I/O
Q3 : I'm not sure JBD can be used for FAT improvements. Do you have any comments ?
Thanks, Hiroyuki Machida
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |