Messages in this thread | | | From | Dark Dragon <> | Subject | Directory virtualization / Cloud filesystem | Date | Wed, 13 Dec 2017 17:42:47 +0100 |
| |
Is anyone working on a filesystem driver or filesystem overlay with the following characteristics: - *Kernel space* for performance reasons (no FUSE) - *All data* is stored on some server - *All metadata* is locally available - *Some file data* is locally available - *Other file data* is automatically fetched on-demand -> *All files* can be accessed from *anywhere* (explorer, console, APIs, ...) independent of their location
Many projects currently work on their own (partly closed source) implementations - for performance reasons often in *kernel space* (instead of FUSE) in order to avoid context switches. In my opinion this is bad in terms of security and stability! Despite your personal opinion on whether (personal) data should be stored in the (private) cloud, it is unquestionable that there is a raising demand for these features (files are updated more often then used locally, limited fast storage in contrast to raising amounts of data, ...).
My suggestion is the following: - Create a virtual filesystem (where is data stored?) or overlay filesystem (store data in underlying filesystem) in the kernel - Agree on a data retrieval protocol - File selection for intelligent caching should be performed in userspace
Current projects working on this: - Seafile Drive client [1] - Git Virtual Filesystem (GVFS) [2, 3] - Dropbox Smart Sync (Project Infinite) [4, 5] - OneDrive On-Demand [6, 7]
I also suggest the following news coverage about some more details: - "Microsoft and GitHub team up to take Git virtual file system to macOS, Linux" [8] -> last two paragraphs about implementation thoughts - "OneDrive done right is back, and now it works properly" [9] -> lessons learned from Microsoft's first unsuccessful implementation
Links: [1]: https://blogs.seafile.com/2016/09/02/announcing-seafile-drive-client-a-new-way-to-map-seafile-storage-as-virtual-drive/ [2]: https://github.com/Microsoft/GVFS [3]: https://github.com/Microsoft/GVFS/issues/4 [4]: https://www.dropbox.com/smartsync [5]: https://blogs.dropbox.com/tech/2016/05/going-deeper-with-project-infinite/ [6]: https://support.office.com/en-us/article/Learn-about-OneDrive-Files-On-Demand-0e6860d3-d9f3-4971-b321-7092438fb38e [7]: https://blogs.office.com/en-us/2017/05/11/introducing-onedrive-files-on-demand-and-additional-features-making-it-easier-to-access-and-share-files/ [8]: https://arstechnica.com/gadgets/2017/11/microsoft-and-github-team-up-to-take-git-virtual-file-system-to-macos-linux/ [9]: https://arstechnica.com/information-technology/2017/06/onedrive-done-right-is-back-and-now-it-works-properly/
| |