Messages in this thread | | | Date | Thu, 9 Feb 2023 20:17:05 +0100 | Subject | Re: copy on write for splice() from file to pipe? | From | Stefan Metzmacher <> |
| |
Hi Linus,
> Adding Jens, because he's one of the main splice people. You do seem > to be stepping on his work ;) > > Jens, see > > https://lore.kernel.org/lkml/0cfd9f02-dea7-90e2-e932-c8129b6013c7@samba.org
Ok, thanks! Maybe Jens should apear in the output of:
scripts/get_maintainer.pl fs/splice.c
> On Thu, Feb 9, 2023 at 5:56 AM Stefan Metzmacher <metze@samba.org> wrote: >> >> So we have two cases: >> >> 1. network -> socket -> splice -> pipe -> splice -> file -> storage >> >> 2. storage -> file -> splice -> pipe -> splice -> socket -> network >> >> With 1. I guess everything can work reliable [..] >> >> But with 2. there's a problem, as the pages from the file, >> which are spliced into the pipe are still shared without >> copy on write with the file(system). > > Well, honestly, that's really the whole point of splice. It was > designed to be a way to share the storage data without having to go > through a copy.
>> I'm wondering if there's a possible way out of this, maybe triggered by a new >> flag passed to splice. > > Not really. > > So basically, you cannot do "copy on write" on a page cache page, > because that breaks sharing. > > You *want* the sharing to break, but that's because you're violating > what splice() was for, but think about all the cases where somebody is > just using mmap() and expects to see the file changes. > > You also aren't thinking of the case where the page is already mapped > writably, and user processes may be changing the data at any time.
I do because we're using that in our tdb library, but I hoped there would be a way out...
>> I looked through the code and noticed the existence of IOMAP_F_SHARED. > > Yeah, no. That's a hacky filesystem thing. It's not even a flag in > anything core like 'struct page', it's just entirely internal to the > filesystem itself.
Ok, I guess it's used for shared blocks in the filesystems, in order to support things like cow support in order to allow snapshots, correct?
>> Is there any other way we could archive something like this? > > I suspect you simply want to copy it at splice time, rather than push > the page itself into the pipe as we do in copy_page_to_iter_pipe(). > > Because the whole point of zero-copy really is that zero copy. And the > whole point of splice() was to *not* complicate the rest of the system > over-much, while allowing special cases. > > Linux is not the heap of bad ideas that is Hurd that does various > versioning etc, and that made copy-on-write a first-class citizen > because it uses the concept of "immutable mapped data" for reads and > writes.
Ok, thanks very much for the detailed feedback!
> Now, I do see a couple of possible alternatives to "just create a stable copy". > > For example, we very much have the notion of "confirm buffer data > before copying". It's used for things like "I started the IO on the > page, but the IO failed with an error, so even though I gave you a > splice buffer, it turns out you can't use it". > > And I do wonder if we could introduce a notion of "optimistic splice", > where the splice works exactly the way it does now (you get a page > reference), but the "confirm" phase could check whether something has > changed in that mapping (using the file versioning or whatever - I'm > hand-waving) and simply fail the confirm. > > That would mean that the "splice to socket" part would fail in your > chain, and you'd have to re-try it. But then the onus would be on > *you* as a splicer, not on the rest of the system to fix up your > special case. > > That idea sounds fairly far out there, and complicated and maybe not > usable. So I'm just throwing it out as a "let's try to think of > alternative solutions".
That sounds complicated and still racy.
Any comment about the idea of having a preadv2() flag that asks for a dma copy with something like async_memcpy() instead of the default that ends up in copy_user_enhanced_fast_string()? If that would be possible, a similar flag would also be possible for splice() in order to dma copy the pages into the pipe.
metze
| |