Messages in this thread | | | Date | Wed, 28 Jan 2009 12:13:31 -0500 | From | Mathieu Desnoyers <> | Subject | Re: [BUG][kprobes][vunmap?]: kprobes may cause memory corruption |
| |
* Masami Hiramatsu (mhiramat@redhat.com) wrote: > Mathieu Desnoyers wrote: > > * Masami Hiramatsu (mhiramat@redhat.com) wrote: > >> Masami Hiramatsu wrote: > > Hi Masami, > > > > This would not surprise me if it came from bug in the new vmap() > > implementation done in this commit : > > > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=db64fe02258f1507e13fe5212a989922323685ce > > > > Especially because going from vmap -> vm_map_ram makes this behavior > > disappear. > > > > Looking at the commit, I notice that it delays vunmap so it's done in > > batch to minimize locking effect. I think it would be good to create a > > test case to try to isolate this, without any kprobes/text_poke > > involved, which does something like this : > > > > load module (this is also doing vmalloc, so it might be part of the > > problem) > > for i (i=0; i < 400; i++) { > > vmap() > > vfree() > ^^^^^ vunmap?
yep.
> > } > > unload module > > > > Another interesting test would be : > > > > for i (i=0; i < 400; i++) { > > vmalloc() > > vfree() > > } > > Hi Mathieu, > > Thank you for test ideas. > I made both of above two tests and run it. Both test modules > do NOT cause memory corruption... >
OK
> > All this called in a loop. This would help isolating the "vmap" part of > > the issue. If this test is not enough, then we should maybe try > > something like this in a kernel module (which does what text_poke does > > with vmalloc, more or less) in a loop : > > > > char somedata[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); > > char copydata[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); > > Should both of them have PAGE_SIZE*2? >
Yes.
> > > > void test_vmap(void) > > } > > struct page *pages[2]; > > char *vaddr; > > int i; > > > > for (i = 0; i < 2 * PAGE_SIZE; i++) > > copydata[i] = somedata[i]; > > page[0] = virt_to_page(&somedata); > > BUG_ON(!page[0]); > > page[1] = virt_to_page(&somedata + PAGE_SIZE); > > BUG_ON(!page[1]); > > vaddr = vmap(pages, 2, VM_MAP, PAGE_KERNEL); > > BUG_ON(!vaddr); > > > > for (i = 0; i < 2 * PAGE_SIZE; i++) > > vaddr[i] = copydata[i] + 1; > > > > vunmap(vaddr); > > > > for (i = 0; i < 2 * PAGE_SIZE; i++) > > BUG_ON(somedata[i] != copydata[i] + 1); > > } > > Hmm, when I ran above code, it hit the last BUG_ON(). > I checked that somedata[i] didn't updated. >
Do you hit the BUG_ON after the first loop ?
> > Given you don't seem to have hit the > > for (i = 0; i < len; i++) > > BUG_ON(((char *)addr)[i] != ((char *)opcode)[i]); > > test at the end of text_poke, > > However, when I ran kprobe-based test, it doesn't hit the BUG_ON() > in text_poke(). >
The variable declarations should have been 2*PAGE_SIZE, hopefully you fixed them.
There is also a sync_core() in text_poke. It should not matter, but maybe that could help ?
> > I suspect the write through the vmapped > > area is correctly done, but that the problem may lay in the mm layer. > > Maybe it's running out of pre-allocated vmap areas or something like > > this ? > > I haven't seen vmalloc failure message on 2.6.29-rc2. >
It could be because the available vmalloc space is slightly higher. Looking into the lazy vunmap threshold would be useful.
You could also try with loop values higher than 400.
Mathieu
> Thank you again, > > > -- > Masami Hiramatsu > > Software Engineer > Hitachi Computer Products (America) Inc. > Software Solutions Division > > e-mail: mhiramat@redhat.com >
-- Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
| |