Messages in this thread Patch in this message | | | From | Kaiwan N Billimoria <> | Date | Wed, 19 Jan 2022 09:27:18 +0530 | Subject | Re: Issue using faddr2line on kernel modules |
| |
On Tue, Jan 18, 2022 at 11:22 PM Josh Poimboeuf <jpoimboe@redhat.com> wrote: > > On Tue, Jan 18, 2022 at 08:10:28AM +0530, Kaiwan N Billimoria wrote: > > Hi Josh, > > > > Actually your first patch - the one you mentioned had other issues - > > worked perfectly when applied: > > > > scripts/faddr2line ./oops_tryv2.ko do_the_work+0x16f/0x194 > > do_the_work+0x16f/0x0000000000000194: > > do_the_work at <...>/oops_tryv2/oops_tryv2.c:62 > > > > The second one still failed in the same manner: > > > > scripts/faddr2line ./oops_tryv2.ko do_the_work+0x16f/0x194 > > bad symbol size: base: 0x0000000000000000 end: 0x0000000000000000 > > > > So, is it possible to fixup issues with the first version? > > What are these issues? > > The first patch basically reverts the fix in commit efdb4167e676 > ("scripts/faddr2line: Fix "size mismatch" error"). That would be nice > as it's simpler and more robust, but unfortunately it would cause a lot > of "size mismatch" errors with vmlinux symbols.
I see... if it comes to it, could there be a workaround where the script detects a vmlinux image and works in a given manner, else in this manner for other images - in effect, for modules?
> > Can you give the output of 'nm -n ./oops_tryv2.ko'? There must be some > text symbol immediately after the do_the_work() symbol which is either > out of order, or part of another section. > > Is do_the_work() in the .text section? Yes... Here's the output I get:
$ nm -n ./oops_tryv2.ko |grep -C5 do_the_work 0000000000000000 r __func__.24215 0000000000000000 r __param_bug_in_workq 0000000000000000 D __this_module 0000000000000000 r _note_7 0000000000000000 T cleanup_module 0000000000000000 t do_the_work 0000000000000000 t do_the_work.cold 0000000000000000 b gctx 0000000000000000 T init_module 0000000000000000 t try_oops_exit 0000000000000000 t try_oops_init 0000000000000008 b t1 $
BTW, here's the code: https://github.com/PacktPublishing/Linux-Kernel-Debugging/tree/main/ch7/oops_tryv2
Thanks, Kaiwan.
> > > On Tue, Jan 18, 2022 at 1:57 AM Josh Poimboeuf <jpoimboe@redhat.com> wrote: > > > > > > On Mon, Jan 17, 2022 at 11:48:39AM -0800, Josh Poimboeuf wrote: > > > > On Mon, Jan 17, 2022 at 10:27:14AM +0530, Kaiwan N Billimoria wrote: > > > > > Hi there, > > > > > > > > > > Am researching using the cool faddr2line script to help debug Oops'es > > > > > from kernel modules.. > > > > > I find it works just fine when used against the unstripped vmlinux > > > > > with debug symbols. > > > > > > > > > > My use case is for a kernel module which Oopses, though. Here's my scenario: > > > > > I built a module on a custom debug kernel (5.10.60) with most debug > > > > > options enabled... > > > > > KASLR is enabled by default as well. > > > > > > > > > > A test kernel module Oopses on my x86_64 guest running this kernel with: > > > > > RIP: 0010:do_the_work+0x15b/0x174 [oops_tryv2] > > > > > > > > > > So, i try this: > > > > > > > > > > $ <...>/linux-5.10.60/scripts/faddr2line ./oops_tryv2.ko do_the_work+0x15b/0x174 > > > > > bad symbol size: base: 0x0000000000000000 end: 0x0000000000000000 > > > > > $ > > > > > > > > > > (It works fine with addr2line though!). > > > > > Now I think I've traced the faddr2line script's failure to locate > > > > > anything down to this: > > > > > ... > > > > > done < <(${NM} -n $objfile | awk -v fn=$func -v end=$file_end '$3 == > > > > > fn { found=1; line =$0; start=$1; next } found == 1 { found=0; > > > > > print line, "0x"$1 } END {if (found == 1) print line, end; }') > > > > > > > > > > The nm output is: > > > > > $ nm -n ./oops_tryv2.ko |grep -i do_the_work > > > > > 0000000000000000 t do_the_work > > > > > $ > > > > > > > > > > nm shows the text addr as 0x0; this is obviously incorrect (same 0x0 > > > > > with objdump -d on the module). > > > > > Am I missing something? Any suggestions as to what I can try, to get > > > > > faddr2line working? > > > > > > > > Hi Kaiwan, > > > > > > > > Thanks for reporting this issue. The module text address of 0x0 is not > > > > necessarily incorrect, as the address is relative the the module, where > > > > all text usually starts at zero. > > > > > > > > I was able to recreate this problem using a module which only has a > > > > single function in .text. Does this fix it? > > > > > > Actually, that patch has other problems. Try this one? > > > > > > ---- > > > > > > From: Josh Poimboeuf <jpoimboe@redhat.com> > > > Subject: [PATCH] scripts/faddr2line: Only look for text symbols when > > > calculating function size > > > > > > With the following commit: > > > > > > efdb4167e676 ("scripts/faddr2line: Fix "size mismatch" error") > > > > > > ... it was discovered that faddr2line can't just read a function's ELF > > > size, because that wouldn't match the kallsyms function size which is > > > printed in the stack trace. The kallsyms size includes any padding > > > after the function, whereas the ELF size does not. > > > > > > So faddr2line has to manually calculate the size of a function similar > > > to how kallsyms does. It does so by starting with a sorted list of > > > symbols and subtracting the function address from the subsequent > > > symbol's address. > > > > > > That calculation is broken in the case where the function is the last > > > (or only) symbol in the .text section. The next symbol in the sorted > > > list might actually be a data symbol, which can break the function size > > > detection: > > > > > > $ scripts/faddr2line sound/soundcore.ko sound_devnode+0x5/0x35 > > > bad symbol size: base: 0x0000000000000000 end: 0x0000000000000000 > > > > > > Similar breakage can occur when reading from a .o file. > > > > > > Fix it by only looking for text symbols. > > > > > > Fixes: efdb4167e676 ("scripts/faddr2line: Fix "size mismatch" error") > > > Reported-by: Kaiwan N Billimoria <kaiwan.billimoria@gmail.com> > > > Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com> > > > --- > > > scripts/faddr2line | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/scripts/faddr2line b/scripts/faddr2line > > > index 6c6439f69a72..2a130134f1e6 100755 > > > --- a/scripts/faddr2line > > > +++ b/scripts/faddr2line > > > @@ -189,7 +189,7 @@ __faddr2line() { > > > > > > DONE=1 > > > > > > - done < <(${NM} -n $objfile | awk -v fn=$func -v end=$file_end '$3 == fn { found=1; line=$0; start=$1; next } found == 1 { found=0; print line, "0x"$1 } END {if (found == 1) print line, end; }') > > > + done < <(${NM} -n $objfile | awk -v fn=$func -v end=$file_end '$2 !~ /[Tt]/ {next} $3 == fn { found=1; line=$0; start=$1; next } found == 1 { found=0; print line, "0x"$1 } END {if (found == 1) print line, end; }') > > > } > > > > > > [[ $# -lt 2 ]] && usage > > > -- > > > 2.31.1 > > > > > > > -- > Josh >
| |