lkml.org 
[lkml]   [2013]   [Apr]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [RFC PATCH 0/4] Support vranges on files
On 04/03/2013 11:55 PM, Minchan Kim wrote:
> On Wed, Apr 03, 2013 at 04:52:19PM -0700, John Stultz wrote:
>> Next we introduce a parallel fvrange() syscall for creating
>> volatile ranges directly against files.
> Okay. It seems you want to replace ashmem interface with fvrange.
> I dobut we have to eat a slot for system call. Can't we add "int fd"
> in vrange systemcall without inventing new wheel?

Sure, that would be doable. I just added the new syscall to make the
differences in functionality clear.
Once the subtleties are understood, we can condense things down if we
think its best.


>> And finally, we change the range pruging logic to be able to
>> handle both anonymous and file volatile ranges.
> Okay. Then, what's the semantic file-vrange?
>
> There is a file F. Process A mapped some part of file into his
> address space. Then, Process B calls fvrange same part.
> As I looked over your code, it purges the range although process B
> is using now. Right? Is it your intention? Maybe isn't.

Not sure if you're example has a type-o and you meant "process A is
using it"? If so, yes. The point is the volatility is shared and
consistent across all users of the file, in the same way the data in the
file is shared. If process B punched a hole in the file, process A would
see the effect immediately. With volatile ranges, the hole punching is
just delayed and possibly done later by the kernel, in effect on behalf
of process B, so the behavior is the same.

Consider the case where we could have two processes mmap a tmpfs file in
order to create a circular buffer shared between them. You could then
have a producer/consumer relationship with two processes where any data
not between the head & tail offsets were marked volatile. The producer
would mark tail+size non-volatile, write the data, and update the tail
offset. The consumer would read data from the head offset, mark the
just-read range as volatile, and update the offset.

In this example, the producer would be the only process to mark data
non-volatile, while the consumer would be the only one marking ranges
volatile. Thus the state of volatility would need to be an attribute of
the file, not the process, in the same way the shared data is.

Is that clear?



> Let's define fvrange's semantic same with anon-vrange.
> If there is a process using range with non-volatile, at least,
> we shouldn't purge at all.

So this I'm not in agreement with.

Anonymous pages are for the most part not shared, except via COW. And
for the COW case, yes, I agree, we shouldn't purge those pages.

Similarly (and I have yet to handle this in the code), for private
mapped files, those pages shouldn't be purged either (or purging them
shouldn't affect the private mapped pages - not sure which direction to
go here).

But for shared mapped files, we need to keep the volatility state shared
as well.


>> Now there are some quirks still to be resolved with the approach
>> used here. The biggest one being the vrange() call can't be used to
>> create volatile ranges against mmapped files. Instead only the
> Why?

As explained above, the volatility is shared like the data. The current
vrange() code creates per-mm volatile ranges, which aren't shared.


>
>> fvrange() can be used to create file backed volatile ranges.
> I could't understand your point. It would be better to explain
> my thought firstly then, you could point out something I am missing
> now. Look below.
>
>> This could be overcome by iterating across all the process VMAs to
>> determine if they're anonymous or file based, and if file-based,
>> create a VMA sized volatile range on the mapping pointed to by the
>> VMA.
> It needs just when we start to discard pages. Simply, it is related
> to reclaim path, NOT system call path so it's not a problem.

The reason we can't defer this to only the reclaim path is if volatile
ranges on shared mappings are stored in the mm_struct, if process A sets
up a volatile range on a shared mapping, but stores the volatility in
its own mm, then process B wants to clear the volatility on the range,
process B would have to iterate over all processes that have those file
vmas mapped and change them.

Additionally if process A sets up a volatile range on a shared mapped
file, then quits, the volatility state dies with that process.

Either way, its not just a simple matter of handling data on your own
mm_struct. That's fine for the process' own anonymous memory, but
doesn't work for shared file mappings.


>
>> But this would have downsides, as Minchan has been clear that he wants
>> to optmize the vrange() calls so that it is very cheap to create and
>> destroy volatile ranges. Having simple per-process ranges be created
>> means we don't have to iterate across the vmas in the range to
>> determine if they're anonymous or file backed. Instead the current
>> vrange() code just creates per process ranges (which may or may not
>> cover mmapped file data), but will only purge anonymous pages in
>> that range. This keeps the vrange() call cheap.
> Right.
>
>> Additionally, just creating or destroying a single range is very
>> simple to do, and requires a fixed amount of memory known up front.
>> Thus we can allocate needed data prior to making any modifications.
>>
>> But If we were to create a range that crosses anonymous and file
>> backed pages, it must create or destroy multiple per-process or
>> per-file ranges. This could require an unknown number of allocations,
> This is a part I can fail to parse your opinion.

So if we were in the vrange() code to iterate over all the VMAs in the
range, creating VMA sizes ranges on either the mm_struct or the backing
address_space where appropriate, its possible that we could hit an
ENOMEM half way through the operation. This would leaving the range in
an inconsistent state: partially marked, and potentially causing us to
lose the purged state on the subranges.



>
>> opening the possibility of getting an ENOMEM half-way through the
>> operation, leaving the volatile range partially created or destroyed.
>>
>> So to keep this simple for this first pass, for now we have two
>> syscalls for two types of volatile ranges.
>
> My idea is following as
>
> vrange(fd, start, len, mode, behavior)
>
> A) fd = 0

Well we'd probably need to use -1 or something that would be an invalid
fd here.

And really, I think having separate interfaces might be good, just as
there are separate madvise() and fadvise() calls (and when all this is
done, we may need to re-visit the new syscall vs new madvise/fadvise
flags decision).

>
> 1) system call context - vrange system call registers new vrange
> in mm_struct.
> 2) Add new vrange into LRU
> 3) reclaim context - walk with rmap to confirm all processes make
> the range with volatile -> discard
>
> B) fd = 1
The fd would just need to be valid right, not 1.

> 1) system call context - vrange system call registers new vrange
> in address_space
> 2) Add new vrange into LRU
> 3) reclaim context - walk with rmap to confirm all processes make
> the range with volatile -> discard
>
> What's the problem in this logic?

The problem is only if in the first case, the volatile range being
created crosses over both anonymous and shared file mmap pages. In that
case we have to create appropriate sub-ranges on the mm_struct, and
sub-ranges on the address_space of the mmaped file.

This isn't impossible to do, but again, the handling of errors mid-way
through creating subranges is problematic (there may be yet a way around
it, I just haven't thought of it yet).


Thus with my patches, I simplified the problem a bit by partitioning it
into two separate problems and two separate interfaces: Volatile ranges
that are created by the vrange() call won't affect mmaped pages, only
anonymous pages. We may create a range that covers them, but the
volatility isn't shared with other processes and the purging logic still
skips file pages. If you want to to create a volatile range on file
pages, you have to use fvrange().

Of course, my patchset has its own inconsistencies too, since if a range
is marked non-volatile that covers a mmapped file that has been marked
volatile, that volatility would persist. So I probably should return an
error if the vrange call covers any mmapped files.


Also, to be clear, I'm not saying that we *have* to partition these
operations into two separate behaviors, but I think having two separate
behaviors at first helps makes clear the subtleties of the differences
between them.


Let me know if any of this helps your understanding. :)

thanks
-john


\
 
 \ /
  Last update: 2013-04-04 20:21    [W:0.066 / U:0.488 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site