lkml.org 
[lkml]   [2023]   [Feb]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: [PATCH v3 0/6] Composefs: an opportunistically sharing verified image filesystem
    On Sun, 5 Feb 2023 at 20:06, Amir Goldstein <amir73il@gmail.com> wrote:
    >
    > > >>> Apart from that, I still fail to get some thoughts (apart from
    > > >>> unprivileged
    > > >>> mounts) how EROFS + overlayfs combination fails on automative real
    > > >>> workloads
    > > >>> aside from "ls -lR" (readdir + stat).
    > > >>>
    > > >>> And eventually we still need overlayfs for most use cases to do
    > > >>> writable
    > > >>> stuffs, anyway, it needs some words to describe why such < 1s
    > > >>> difference is
    > > >>> very very important to the real workload as you already mentioned
    > > >>> before.
    > > >>>
    > > >>> And with overlayfs lazy lookup, I think it can be close to ~100ms or
    > > >>> better.
    > > >>>
    > > >>
    > > >> If we had an overlay.fs-verity xattr, then I think there are no
    > > >> individual features lacking for it to work for the automotive usecase
    > > >> I'm working on. Nor for the OCI container usecase. However, the
    > > >> possibility of doing something doesn't mean it is the better technical
    > > >> solution.
    > > >>
    > > >> The container usecase is very important in real world Linux use today,
    > > >> and as such it makes sense to have a technically excellent solution for
    > > >> it, not just a workable solution. Obviously we all have different
    > > >> viewpoints of what that is, but these are the reasons why I think a
    > > >> composefs solution is better:
    > > >>
    > > >> * It is faster than all other approaches for the one thing it actually
    > > >> needs to do (lookup and readdir performance). Other kinds of
    > > >> performance (file i/o speed, etc) is up to the backing filesystem
    > > >> anyway.
    > > >>
    > > >> Even if there are possible approaches to make overlayfs perform better
    > > >> here (the "lazy lookup" idea) it will not reach the performance of
    > > >> composefs, while further complicating the overlayfs codebase. (btw, did
    > > >> someone ask Miklos what he thinks of that idea?)
    > > >>
    > > >
    > > > Well, Miklos was CCed (now in TO:)
    > > > I did ask him specifically about relaxing -ouserxarr,metacopy,redirect:
    > > > https://lore.kernel.org/linux-unionfs/20230126082228.rweg75ztaexykejv@wittgenstein/T/#mc375df4c74c0d41aa1a2251c97509c6522487f96
    > > > but no response on that yet.
    > > >
    > > > TBH, in the end, Miklos really is the one who is going to have the most
    > > > weight on the outcome.
    > > >
    > > > If Miklos is interested in adding this functionality to overlayfs, you are going
    > > > to have a VERY hard sell, trying to merge composefs as an independent
    > > > expert filesystem. The community simply does not approve of this sort of
    > > > fragmentation unless there is a very good reason to do that.
    > > >
    > > >> For the automotive usecase we have strict cold-boot time requirements
    > > >> that make cold-cache performance very important to us. Of course, there
    > > >> is no simple time requirements for the specific case of listing files
    > > >> in an image, but any improvement in cold-cache performance for both the
    > > >> ostree rootfs and the containers started during boot will be worth its
    > > >> weight in gold trying to reach these hard KPIs.
    > > >>
    > > >> * It uses less memory, as we don't need the extra inodes that comes
    > > >> with the overlayfs mount. (See profiling data in giuseppes mail[1]).
    > > >
    > > > Understood, but we will need profiling data with the optimized ovl
    > > > (or with the single blob hack) to compare the relevant alternatives.
    > >
    > > My little request again, could you help benchmark on your real workload
    > > rather than "ls -lR" stuff? If your hard KPI is really what as you
    > > said, why not just benchmark the real workload now and write a detailed
    > > analysis to everyone to explain it's a _must_ that we should upstream
    > > a new stacked fs for this?
    > >
    >
    > I agree that benchmarking the actual KPI (boot time) will have
    > a much stronger impact and help to build a much stronger case
    > for composefs if you can prove that the boot time difference really matters.
    >
    > In order to test boot time on fair grounds, I prepared for you a POC
    > branch with overlayfs lazy lookup:
    > https://github.com/amir73il/linux/commits/ovl-lazy-lowerdata

    Sorry about being late to the party...

    Can you give a little detail about what exactly this does?

    Thanks,
    Miklos

    \
     
     \ /
      Last update: 2023-03-27 00:10    [W:4.242 / U:1.004 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site