Messages in this thread | | | Date | Mon, 24 Aug 2009 21:26:38 -0400 | From | Theodore Tso <> | Subject | Re: Starting a grad project that may change kernel VFS. Early research |
| |
On Mon, Aug 24, 2009 at 04:54:52PM -0700, Jeff Shanab wrote: > I was thinking that a good way to handle this is that it starts with > a file change in a directory. The directory entry contains a sum already > for itself and all the subdirs and an adjustment is made immediately to > that, it should be in the cache. Then we queue up the change to be sent > to the parent(s?). These queued up events should be a low priority at a > more human time like 1 second. If a large number of changes come to a > directory, multiple adjustments hit the queue with the same (directory > name, inode #?) and early ones are thrown out. So levels above would see > at most a 1 per second low priority update.
Is this something that you want to be stored in the file system, or just cached in memory? If it is going to be stored on disk, which seems to be implied by your description, and it is only going to be updated once a second, what happens if there is a system crash? Over time, the values will go out of date. Fsck could fix this, sure, but that means you have to do the equivant of running "du -s" on the root directory of the filesystem after an unclean shutdown.
You could write the size changes in a journal, but that blows up the size of information that would need to be stored in a journal. It also slows down the very common operaton of writing to a file, all for the sake of speeding up the relatively uncommon "du -s" operation. It's not at all clear it's worthwhile tradeoff.
In addition, how will you handle hard links? An inode can have multiple hard links in different directories, and there is no way to find all of the directories which might contain a hard link to a particular inode, short of doing a brute force search. Hence if you have a file living in src/linux/v2.6.29/README, and it is a hard link to ~/hacker/linux/README, and a program appends data to the file ~/hacker/linux/README, this would also change the result of running du -s src/linux/v2.6.29; however, there's no way for your extension to know that.
> title: "User Metadata" aka "pet peeve reduction" > I would like to maintain a few classifications of metadata, most > optional and configurable.
Most Linux filesystems already have extended attributes that can be used to store your proposed metadata. Changing user application programs to store the keywords, etc., is an exercise in application-level programming; the kernel-side support is already there.
- Ted
| |