Messages in this thread | | | Date | Sat, 1 Nov 2008 13:35:36 -0400 (EDT) | From | Steven Rostedt <> | Subject | Re: 2.6.28-rc2 hates my e1000e |
| |
[ Note, lots of family activities this weekend, so my response may be slow ]
On Sat, 1 Nov 2008, Linus Torvalds wrote: > On Sat, 1 Nov 2008, Jonathan Corbet wrote: > > > Networking is fine in the absence of NFS. I retried things and > > stress-tested it in a few ways with no trouble. I think your last patch > > fixes the network card just fine. > > > > Then I tried NFS again, watching more closely this time around. > > Everything locks up. In fact, the soft lockup watchdog starts to > > scream: > > Interesting. I wonder why it happens for NFS, but not apparently for all > your other modules. > > It does look very much like a ftrace issue, though, not NFS or > network-related. Steven? Is this something that you are aware of already, > with what looks like a lockup in ftrace_record_ip()? >
No, I have not seen this before. The code is now pretty straight forward.
Jon, could you do a gdb vmlinux and a li *ftrace_record_ip+0xcb to find the exact line that is?
Showing the call path of this we have:
in module.c:
/* sechdrs[0].sh_size is always zero */ mseg = section_objs(hdr, sechdrs, secstrings, "__mcount_loc", sizeof(*mseg), &num_mcount); ftrace_init_module(mseg, mseg + num_mcount);
Where we pass a table of mcount callers to the ftrace_init_module.
void ftrace_init_module(unsigned long *start, unsigned long *end) { if (ftrace_disabled || start == end) return; ftrace_convert_nops(start, end); }
I wonder if I should test to make sure start is < end :-/
ftrace_convert_nops does the following under a mutex.
while (p < end) { addr = ftrace_call_adjust(*p++); ftrace_record_ip(addr); }
And ftrace_record_ip does:
ftrace_record_ip(unsigned long ip) { struct dyn_ftrace *rec; if (!ftrace_enabled || ftrace_disabled) return NULL; rec = ftrace_alloc_dyn_node(ip); if (!rec) return NULL; rec->ip = ip; list_add(&rec->list, &ftrace_new_addrs); return rec; }
The ftrace_alloc_dyn_node does allocate a page if we are running low, but there's no other loops or locks that I can see us deadlocking on.
-- Steve
| |