Messages in this thread | | | Date | Thu, 11 Nov 2010 13:25:40 -0500 | From | Ted Ts'o <> | Subject | Re: Tracing Requirements (was: [RFC/Requirements/Design] h/w error reporting) |
| |
On Wed, Nov 10, 2010 at 06:20:27PM -0500, Steven Rostedt wrote: > On Thu, 2010-11-11 at 00:12 +0100, Thomas Gleixner wrote: > > > Cramming both into the same session is just insane. > > That just doubled the overhead of the tracer.
At least when I've used ftrace for the "flight recorder" use case, I'm not tracing as well. What I do is enable a bunch of trace points, maybe I've sprinkled in some "trace_printk()'s" into various kernel code paths, and then I run the workload which locks up the kernel. When locks up, I've used sysrq-z to dump out the ftrace ring buffer, and usually _exactly_ what I need to debug the lock up is waiting for me in the ring buffer.
So, this use case, is incredibly useful, and I hope whatever folks do with the new-fangled API, that somehow "overwrite mode" is supported. Even if for speed reasons, what you do is wait until for the head to overrun the tail, that the tail gets bumped up by 50% and we lose half the log (so that whatever expensive locking is necessary only happens once in a while), I at least would find that quite acceptable.
The other feature/requirements request I would make is that there should be a way that common kernel abstractions, such as converting a dev_t to either a MAJOR/MINOR number pair, or to a device name, be made available. For now I've changed the tracepoints to translate MAJOR/MINOR and drop integers into the ring buffer, and a generic workaround in the future is to always drop strings into the ring buffer instead of allowing the translation to be done in TP_printk (which doesn't work for perf; it causes the userspace perf client to fall over and die, without even skipping the problematic tracepoint record --- boo, hiss). But that can be relatively inefficient, because we're now having to drop potentially fairly large text strings into ring buffer, because of limitations that perf has in its output transformations step.
I know that because perf is doing its output transformation in userspace, there are fundamental limitations about what it can do. But it would be nice if it could be expanded at least _somewhat_, and either way, there needs to be some clear documentation about what it can and can not accept. And if these limitations means that I should just simply continue using ftrace, and not use perf, it would be nice if the tracepoints I create that work with ftrace don't cause perf to just die horribly when it tries to parse them.
- Ted
| |