[lkml]   [2014]   [Feb]   [12]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [BUG] perf report/annotate: consuming too many file descriptors
Hi Stephane,

(I'd be better if you use my korg address).

2014-02-12 PM 11:32, Stephane Eranian wrote:
> Hi,
> I was testing 3.14-rc2 (tip.git) perf and ran into a serious problem
> with report/annotate on a collection with lots of shared libraries (500).
> Perf ran out of file descriptors (ulimit set to 1024). It did not print
> an error message, but simply refused to proceed to objdump.
> I ran strace and discovered it was running out of descriptors to
> my big surprise! I narrowed it down with strace -etrace=open.
> I saw an appalling result.
> My .so files were opened at least 4 times each, each
> allocated an fd that was kept open, each open incur a read of the ELF
> headers. So each dso was consuming 4 fd.
> Why?
> Because of what's going on in dso__load() when perf iterates
> over the binary_type_symtab[]. It tries a bunch of types. For
> each match, symsrc_init() is invoked to read the ELF image.
> The fd opened there is kept open (for later loading).
> The first problem is why is dso__read_binary_type_filename()
> blindly returning success on many types. For my files, I got matches
> I have a tendency to believe that the dso__read_binary_type_filename()
> meaning has been abused to stash things that do not really
> belong there.
> Furthermore, I believe many of the type matches do NOT need an ELF
> read and certainly not one that keeps a descriptor opened.
> This problem is not just about consuming too many fd, it is also about
> execution overhead. Why read the ELF headers 4 times?
> The current situation makes reporting on large collection impossible for
> regular users. I don't quite know how to best address this issue. One way
> I found would be for dso__read_binary_type_filename() to return a value
> meaning success but no ELF read. Not sure would be correct, though.

It looks like a case of stripped binary or static build. dso__load()
looked for both of symtab and dynsym, but it failed. So it iterated
through the binary types and continued to fail with fd open. :(

I think we should close/destroy the duplicate symsrcs. Does attached
patch fix your problem?


 \ /
  Last update: 2014-02-13 01:41    [W:0.047 / U:1.452 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site