[lkml]   [2013]   [Nov]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectMultiple copy-on-write branches of an anonymous memory segment

I am developing a data structure that resides in a large memory buffer. It is important that I be able to
mutate the data structure without interfering with concurrent reads, and then swap it in at a convenient time.

There is a set of initial data stored in a file. I use mmap(2) to map the file into memory. At some later point, I would like to run incremental updates. This involves copying the data structure, mutating it, then publishing it. Only a very small percentage of the data structure is changed, but it can change at random.

Currently, I accomplish this by mapping a new anonymous segment, copying the data, and modifying it. However this is wasteful both in time (copying all that data takes a fair amount of time) and memory (most of the data did not actually change, but now we have two copies).

I would like to utilize kernel support for copy-on-write to eliminate both of these bottlenecks. Ideally, I would be able to:

1) create a new COW mapping of an existing anonymous mapping that shares pages initially and I can write to without affecting the original
2) At the time of creating that new mapping, extend the buffer with initially-zero pages (in case I need more space to append to the end of the data structure in the new version)

The mmap and mremap documentation get me tantalizingly close — I can resize the segment I already have, I can allocate file-backed copy on write segments. Also, forking while holding a private mapping seems to provide other bits of the functionality (COW of anonymous segments). But I can’t seem to figure out how to fit the puzzle pieces together. Also, I’m running in the JVM, so forking is not a possibility.

There’s a couple of unanswered StackOverflow questions in a similar vein, so I hope I’m not just missing something obvious. The most relevant is:

And it looks like this dance is even possible on Mach, with some clever use of ‘vm_remap'.

I’m hoping that I’m just missing the magic incantation to pass to mmap to achieve this behavior. I wasn’t brave enough to subscribe to LKML just yet, so please CC me on replies!

Steven Schlansker

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2013-11-05 20:21    [W:0.034 / U:1.352 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site