Messages in this thread | | | Date | Thu, 24 Jun 1999 00:18:44 -0700 | From | Mike Touloumtzis <> | Subject | file streams a la mac, ntfs, etc. |
| |
Some thoughts about the 'streams within files' problem:
-- I basically agree with Ted. -- You probably don't care since you don't know who I am and I haven't contributed to the kernel. -- I hope you read this anyway :-)
If an implementation of a feature like this is to go into the kernel, it should at least strive to be a reasonable abstraction of the problem rather than an imitation of another system's semantics.
The best abstraction that has emerged here is to allow a directory to be opened as a file containing all of the bytes of its component files. This is at least reasonably in keeping with the Unix 'file as a stream of bytes' precedent.
Problems with this:
-- Not supported by well established tools. -- Not supported by well established documentation. -- Linux only feature (for the time being at least), and only for some filesystems. -- Made complex by existing Unix metadata (update times, perms, etc). -- Works well for sequential read, not so well for random access and memory mapped files. -- Should it recurse into directories?
These costs are not worth it IMHO. The kernel already does the things it should do to implement its end of the problem: it allows fast, hierarchical access to garbage collected data streams (reiserfs takes it further and allows efficient small streams). The "data hiding" problem is a UI problem (well, not totally, but you have to admit that's the main reason the Mac folks went for it).
The Userspace Alternative -------------------------
Something that people might have been overlooking: what are the consumers of a feature like this? It's not 'ls' or 'cp'. It's applications (like MS Word) that use complex, structured, proprietary data formats. Such applications either:
-- are written as monolithic 'suites' that can be modified by a central entity to adhere to these conventions, or
-- are written using a modern desktop environment (Gnome, KDE, probably modern MS office) that provides centralized utility libraries for doing just this sort of thing.
So the right approach IMHO is to work on lib-gnome-datablob rather than to hack data streams support into the kernel and try to get developers to take advantage of a Linux-only feature.
Some people here want the huge, fancy office apps to see the files as what they really are (directories) and the lean, simple command line tools to be duped. This is the wrong way around; if desktop end users working with structured documents have enough expertise to use the command line tools, they can figure out the directory thing. Rather than trying to fool thousands of simple, straightforward tools, effort should be focused on an agreed-upon format for the higher-level tools.
For those who feel that sticking entirely userspace conventions between the kernel's view of files and the user's is a bad thing: I draw your attention to the highly successful dotfile convention.
Speed -----
In terms of speed: applications that both
(a) allow editing of complex blobs containing multiple data streams (b) require the resulting blobs to be highly user-friendly in format
tend to fit the 'office suite' usage profile: people open blobs for long periods of time and manipulate them at human speed rather than with back-to-back command line/batch processing type operations. Given that, such an app could just store the blob in tar-like format all the time and untar it as needed. This can be plenty fast enough for typical office automation users:
sarcastro:~$ du -c /usr/doc/HOWTO/*.gz | grep total 2953 total sarcastro:~$ mkdir test sarcastro:~$ cp /usr/doc/HOWTO/*.gz test/ * do a bunch of stuff to flush cache * sarcastro:~$ time tar cf test.tar test/ real 0m0.381s user 0m0.040s sys 0m0.340s sarcastro:~$ rm -r test sarcastro:~$ time tar xf test.tar real 0m0.423s user 0m0.020s sys 0m0.350s
(and this isn't a fast machine either)
This model uses filesystems for what they do very well today. Using the right extensions for the blob file and directory (extensions on directories are underused) would actually make it easier for the app to do recovery of partially edited files if it has been killed, and would help file managers do the right thing.
For power users who want more speed, a 'store in directory format' checkbox in the apps would solve their problems. And if a shared desktop-environment library is used to implement this convention, the file manager app could have 'pack blob/unpack blob' menu options.
Don't forget that people using office apps are often using them on desktop systems that are essentially single-user. Trading off a little speed for the increased user friendliness to the average desktop user is a pretty reasonable thing.
miket
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |