Messages in this thread | | | From | He Kuang <> | Subject | [RFC PATCH 0/5] Make eBPF programs output data to perf event | Date | Wed, 1 Jul 2015 02:57:30 +0000 |
| |
The idea to let eBPF output data to perf sample event was first metioned in "[RFC PATCH v4 10/29] bpf tools: Collect map definitions from 'maps' section", and the output data is not limited to PMU counters but data like time latencies, cache misses or other things users want to record.
This patch adds an extra perf trace buffer for other utilities like bpf to fill extra data to perf events. A recursion flags which was first introduced by commit 444a2a3bcd6d ("tracing, perf_events: Protect the buffer from recursion in perf") should be got before saving data to buffer, to protect the percpu data buffer from being overwritten.
A bpf function for output data to perf event is added, the data is written to perf trace extra buffer which will be collected and appended to the end of the orignal perf event data. The new data does not change the original format and the generated event can still be processed by perf script.
The first patch in this series has been posted last week, Masami acked-by it but Alexei said there's some reason to call bpf_prog in the head of the funciton, so it may require more discussion.
Here is a sample for recording time interval of a probed functions in perf sample event, the bpf program is like this:
SEC("generic_perform_write=generic_perform_write") int NODE_generic_perform_write(struct pt_regs *ctx) { char fmt[] = "generic_perform_write, cur=0x%llx, del=0x%llx\n"; u64 cur_time, del_time; int ind =0; struct time_table output, *last = bpf_map_lookup_elem(&global_time_table, &ind); if (!last) return 0; cur_time = bpf_ktime_get_ns(); if (!last->last_time) del_time = 0; else del_time = cur_time - last->last_time; /* For debug */ bpf_trace_printk(fmt, sizeof(fmt), cur_time, del_time); /* Update time table */ output.last_time = cur_time; bpf_map_update_elem(&global_time_table, &ind, &output, BPF_ANY); /* This is a casual condition to show the funciton */ if (del_time < 1000) return 0; bpf_output_sample(&del_time, sizeof(del_time)); return 1; }
Debug output in /sys/kernel/debug/tracing/trace:
dd-1018 [000] d... 48512.533331: : generic_perform_write cur=0x2c15cc83436e, del=0x0 dd-1018 [000] d... 48512.534032: : generic_perform_write cur=0x2c15cc8e3c35, del=0xaf8c7 dd-1018 [000] d... 48512.534528: : generic_perform_write cur=0x2c15cc95ec0f, del=0x7afda
The raw data record in perf.data: $ perf-report -D
. ... raw event: size 80 bytes . 0000: 09 00 00 00 01 00 50 00 61 0b 14 81 ff ff ff ff ......P.a....... . 0010: fa 03 00 00 fa 03 00 00 81 3e 24 c8 15 2c 00 00 .........>$..,.. . 0020: 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 ................ . 0030: 1c 00 00 00 99 05 01 00 fa 03 00 00 60 0b 14 81 ............`... . 0040: ff ff ff ff c7 f8 0a 00 00 00 00 00 00 00 00 00 ................ _______ \ ----- del_time = 0x000af8c7
And the perf script result is untouched:
dd 1018 [000] 48472.063753: perf_bpf_probe:generic_perform_write: (ffffffff81140b60) dd 1018 [000] 48472.063753: perf_bpf_probe:generic_perform_write: (ffffffff81140b60)
Next step, we'll let the userspace perf tools to parse this extra data generated by eBPF.
Thank you.
He Kuang (5): bpf: Put perf_events check ahead of bpf prog perf/trace: Add perf extra percpu trace buffer tracing/kprobe: Separate inc recursion count out of perf_trace_buf_prepare bpf: Introduce function for outputing sample data to perf event tracing/kprobe: Combine extra trace buf into perf trace buf
include/linux/ftrace_event.h | 7 +++- include/linux/perf_event.h | 2 ++ include/uapi/linux/bpf.h | 1 + kernel/events/core.c | 6 ++++ kernel/events/internal.h | 17 ++++++---- kernel/trace/bpf_trace.c | 29 ++++++++++++++++ kernel/trace/trace_event_perf.c | 73 ++++++++++++++++++++++++++++++++++++++--- kernel/trace/trace_kprobe.c | 70 ++++++++++++++++++++++++++++++++------- samples/bpf/bpf_helpers.h | 2 ++ 9 files changed, 182 insertions(+), 25 deletions(-)
-- 1.8.5.2
| |