Messages in this thread | | | From | "Joseph Malicki" <> | Subject | Re: cache limit | Date | Wed, 27 Aug 2003 12:01:18 -0400 |
| |
I've had experience with unneeded, *mapped* pages that would be ideally flushed oppressing needed mapped and unmapped pages. Test case: grep --mmap SOME_STRING_I_WONT_FIND some_multi-GB-file
Sure, it's bad programming etc, but in that case, once those pages are mapped, they can't be forcibly unmapped even though in a utopian VM they would be discarded as unneeded.
This could very well be the problem?
-joe
----- Original Message ----- From: "William Lee Irwin III" <wli@holomorphy.com> To: "Takao Indoh" <indou.takao@soft.fujitsu.com> Cc: "Mike Fedyk" <mfedyk@matchmail.com>; <linux-kernel@vger.kernel.org> Sent: Wednesday, August 27, 2003 5:45 AM Subject: Re: cache limit
> On Wed, Aug 27, 2003 at 06:36:10PM +0900, Takao Indoh wrote: > > This problem happened a few month ago and the detailed data does not > > remain. Therefore it is difficult to know what is essential cause for > > this problem, but, I guessed that pagecache used as I/O cache grew > > gradually during system running, and finally it oppressed memory. > > But this doesn't make any sense; the only memory you could "oppress" > is pagecache. > > > On Wed, Aug 27, 2003 at 06:36:10PM +0900, Takao Indoh wrote: > > Besides this problem, there are many cases where increase of pagecache > > causes trouble, I think. > > For example, DBMS. > > DBMS caches index of DB in their process space. > > This index cache conflicts with the pagecache used by other applications, > > and index cache may be paged out. It cause uneven response of DBMS. > > In this case, limiting pagecache is effective. > > Why is it effective? You're describing pagecache vs. pagecache > competition and the DBMS outcompeting the cooperating applications for > memory to the detriment of the workload; this is a very different > scenario from what "limiting pagecache" sounds like. > > How do you know it would be effective? Have you written a patch to > limit it in some way and tried running it? > > > On Tue, 26 Aug 2003 02:46:34 -0700, William Lee Irwin III wrote: > >> One thing I thought of after the post was whether they actually had in > >> mind tunable hard limits on _unmapped_ pagecache, which is, in fact, > >> useful. OTOH that's largely speculation and we really need them to > >> articulate the true nature of their problem. > > On Wed, Aug 27, 2003 at 06:36:10PM +0900, Takao Indoh wrote: > > I also think that is effective. Empirically, in the case where pagecache > > causes memory shortage, most of pagecache is unmapped page. Of course > > real problem may not be pagecashe, as you or Mike said. > > How do you know most of it is unmapped? > > At any rate, the above assigns a meaningful definition to the words you > used; it does not necessarily have anything to do with the issue you're > trying to describe. If you could start from the very basics, reproduce > the problem, instrument the workload with top(1) and vmstat(1), and find > some way to describe how the performance is inadequate (e.g. performance > metrics for your running DBMS/whatever in MB/s or transactions/s etc.), > it would be much more helpful than proposing a solution up front. > Without any evidence, we can't know it is a solution at all, or that > it's the right solution. > > > -- wli > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ >
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |