Messages in this thread | | | Date | Fri, 10 Oct 2003 11:36:29 -0700 (PDT) | From | Linus Torvalds <> | Subject | Re: statfs() / statvfs() syscall ballsup... |
| |
On Fri, 10 Oct 2003, Andrea Arcangeli wrote: > > O_DIRECT only walk the pagetables, no pte mangling, no tlb flushes, the > TLB is preserved fully.
Yes. However, it's even _nicer_ if you don't need to walk the page tables at all.
Quite a lot of operations could be done directly on the page cache. I'm not a huge fan of mmap() myself - the biggest advantage of mmap is when you don't know your access patterns, and you have reasonably good locality. In many other cases mmap is just a total loss, because the page table walking is often more expensive than even a memcpy().
That's _especially_ true if you have to move mappings around, and you have to invalidate TLB's.
memcpy() often gets a bad name. Yeah, memory is slow, but especially if you copy something you just worked on, you're actually often better off letting the CPU cache do its job, rather than walking page tables and trying to be clever.
Just as an example: copying often means that you don't need nearly as much locking and synchronization - which in turn avoids one whole big mess (yes, the memcpy() will look very hot in profiles, but then doing extra work to avoid the memcpy() will cause spread-out overhead that is a lot worse and harder to think about).
This is why a simple read()/write() loop often _beats_ mmap approaches. And often it's actually better to not even have big buffers (ie the old "avoid system calls by aggregation" approach) because that just blows your cache away.
Right now, the fastest way to copy a file is apparently by doing lots of ~8kB read/write pairs (that data may be slightly stale, but it was true at some point). Never mind the system call overhead - just having the extra buffer stay in the L1 cache and avoiding page faults from mmap is a bigger win.
And I don't think mmap _can_ beat that. It's fundamental.
In contrast, direct page cache accesses really can do so. Exactly because they don't touch any page tables at all, and because they can take advantage of internal kernel data structure layout and move pages around without any cost..
Linus
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |