Messages in this thread | | | From | Arnaldo Carvalho de Melo <> | Date | Wed, 25 Mar 2020 10:05:48 -0300 | Subject | Re: [PATCHSET 0/9] perf: Improve cgroup profiling (v6) |
| |
Em Wed, Mar 25, 2020 at 09:45:27PM +0900, Namhyung Kim escreveu: > Hello, > > This work is to improve cgroup profiling in perf. Currently it only > supports profiling tasks in a specific cgroup and there's no way to > identify which cgroup the current sample belongs to. So I added > PERF_SAMPLE_CGROUP to add cgroup id into each sample. It's a 64-bit > integer having file handle of the cgroup. And kernel also generates > PERF_RECORD_CGROUP event for new groups to correlate the cgroup id and > cgroup name (path in the cgroup filesystem). The cgroup id can be > read from userspace by name_to_handle_at() system call so it can > synthesize the CGROUP event for existing groups.
Thanks for the refresh, I'll test this today and all going well push to Ingo soon,
- Arnaldo
> So why do we want this? Systems running a large number of jobs in > different cgroups want to profiling such jobs precisely. This includes > container hosting systems widely used today. Currently perf supports > namespace tracking but the systems may not use (cgroup) namespace for > their jobs. Also it'd be more intuitive to see cgroup names (as > they're given by user or sysadmin) rather than numeric > cgroup/namespace id even if they use the namespaces. > > From Stephane Eranian: > > In data centers you care about attributing samples to a job not such > > much to a process. A job may have multiple processes which may come > > and go. The cgroup on the other hand stays around for the entire > > lifetime of the job. It is much easier to map a cgroup name to a > > particular job than it is to map a pid back to a job name, > > especially for offline post-processing. > > Note that this only works for "perf_event" cgroups (obviously) so if > users are still using cgroup-v1 interface, they need to have same > hierarchy for subsystem(s) want to profile with it. > > * Changes from v5: > - add ACKs for the kernel changes > - fix sort column alignment when no cgroup is available > > * Changes from v4: > - use CONFIG_CGROUP_PERF > - move cgroup tree to perf_env > - move cgroup fs utility function to tools/lib/api/fs > - use a local buffer and check its size for cgroup systhesis > > * Changes from v3: > - staticize perf_event_cgroup() > - fix sample parsing test > > * Changes from v2: > - remove path_len from cgroup_event > - bail out if kernel doesn't support cgroup sampling > - add some description in the Kconfig > > * Changes from v1: > - use new cgroup id (= file handle) > > The code is available at perf/cgroup-v6 branch in > > git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git > > > The testing result looks something like this: > > [root@qemu build]# ./perf record --all-cgroups ./cgtest > [ perf record: Woken up 1 times to write data ] > [ perf record: Captured and wrote 0.009 MB perf.data (150 samples) ] > > [root@qemu build]# ./perf report -s cgroup,comm --stdio > # To display the perf.data header info, please use --header/--header-only options. > # > # > # Total Lost Samples: 0 > # > # Samples: 150 of event 'cpu-clock:pppH' > # Event count (approx.): 37500000 > # > # Overhead cgroup Command > # ........ .......... ....... > # > 32.00% /sub/cgrp2 looper2 > 28.00% /sub/cgrp1 looper1 > 25.33% /sub looper0 > 4.00% / cgtest > 4.00% /sub cgtest > 3.33% /sub/cgrp2 cgtest > 2.67% /sub/cgrp1 cgtest > 0.67% / looper0 > > > The test program (cgtest) follows. > > Thanks, > Namhyung > > > Cc: Tejun Heo <tj@kernel.org> > Cc: Li Zefan <lizefan@huawei.com> > Cc: Johannes Weiner <hannes@cmpxchg.org> > Cc: Adrian Hunter <adrian.hunter@intel.com> > > > -------8<-----------------------------------------8<---------------- > #define _GNU_SOURCE > #include <stdio.h> > #include <stdlib.h> > #include <string.h> > #include <unistd.h> > #include <fcntl.h> > #include <sched.h> > #include <errno.h> > #include <sys/stat.h> > #include <sys/wait.h> > #include <sys/prctl.h> > #include <sys/mount.h> > > char cgbase[] = "/sys/fs/cgroup/perf_event"; > > void mkcgrp(char *name) { > char buf[256]; > > snprintf(buf, sizeof(buf), "%s%s", cgbase, name); > if (mkdir(buf, 0755) < 0) > perror("mkdir"); > } > > void rmcgrp(char *name) { > char buf[256]; > > snprintf(buf, sizeof(buf), "%s%s", cgbase, name); > if (rmdir(buf) < 0) > perror("rmdir"); > } > > void setcgrp(char *name) { > char buf[256]; > int fd; > > snprintf(buf, sizeof(buf), "%s%s/tasks", cgbase, name); > > fd = open(buf, O_WRONLY); > if (fd > 0) { > if (write(fd, "0\n", 2) != 2) > perror("write"); > close(fd); > } > } > > void create_sub_cgroup(int idx) { > char buf[128]; > > snprintf(buf, sizeof(buf), "/sub/cgrp%d", idx); > mkcgrp(buf); > } > > void remove_sub_cgroup(int idx) { > char buf[128]; > > snprintf(buf, sizeof(buf), "/sub/cgrp%d", idx); > rmcgrp(buf); > } > > void set_sub_cgroup(int idx) { > char buf[128]; > > snprintf(buf, sizeof(buf), "/sub/cgrp%d", idx); > setcgrp(buf); > } > > void set_task_name(int idx) { > char buf[16]; > > snprintf(buf, sizeof(buf), "looper%d", idx); > prctl(PR_SET_NAME, buf, 0, 0, 0); > } > > void loop(unsigned max) { > volatile unsigned int count = 1; > > while (count++ != max) { > asm volatile ("pause"); > } > } > > void worker(int idx, unsigned cnt, int new_ns) { > int oldns; > > create_sub_cgroup(idx); > set_sub_cgroup(idx); > > if (new_ns) { > if (unshare(CLONE_NEWCGROUP) < 0) > perror("unshare"); > > #if 0 /* FIXME */ > if (unshare(CLONE_NEWNS) < 0) > perror("unshare"); > > if (mount("none", "/sys", NULL, MS_REMOUNT | MS_REC | MS_SLAVE, NULL) < 0) > perror("mount --make-rslave"); > > sleep(1); > if (umount("/sys/fs/cgroup/perf_event") < 0) > perror("umount"); > > if (mount("cgroup", "/sys/fs/cgroup/perf_event", "cgroup", > MS_NODEV | MS_NOEXEC | MS_NOSUID, "perf_event") < 0) > perror("mount again"); > #endif > } > > if (fork() == 0) { > set_task_name(idx); > loop(cnt); > exit(0); > } > wait(NULL); > } > > int main(int argc, char *argv[]) > { > int i, nr = 2; > int new_ns = 1; > unsigned cnt = 1000000; > int fd; > > if (argc > 1) > nr = atoi(argv[1]); > if (argc > 2) > cnt = atoi(argv[2]); > if (argc > 3) > new_ns = atoi(argv[3]); > > mkcgrp("/sub"); > setcgrp("/sub"); > > for (i = 0; i < nr; i++) { > if (fork() == 0) { > worker(i+1, cnt, new_ns); > exit(0); > } > } > > set_task_name(0); > loop(cnt); > > for (i = 0; i < nr; i++) > wait(NULL); > > for (i = 0; i < nr; i++) > remove_sub_cgroup(i+1); > > setcgrp("/"); > rmcgrp("/sub"); > > return 0; > } > -------8<-----------------------------------------8<---------------- > > > Namhyung Kim (9): > perf/core: Add PERF_RECORD_CGROUP event > perf/core: Add PERF_SAMPLE_CGROUP feature > perf tools: Basic support for CGROUP event > perf tools: Maintain cgroup hierarchy > perf report: Add 'cgroup' sort key > perf record: Support synthesizing cgroup events > perf record: Add --all-cgroups option > perf top: Add --all-cgroups option > perf script: Add --show-cgroup-events option > > include/linux/perf_event.h | 1 + > include/uapi/linux/perf_event.h | 16 ++- > init/Kconfig | 3 +- > kernel/events/core.c | 133 ++++++++++++++++++++++ > tools/include/uapi/linux/perf_event.h | 16 ++- > tools/lib/perf/include/perf/event.h | 7 ++ > tools/perf/Documentation/perf-record.txt | 5 +- > tools/perf/Documentation/perf-report.txt | 1 + > tools/perf/Documentation/perf-script.txt | 3 + > tools/perf/Documentation/perf-top.txt | 4 + > tools/perf/builtin-diff.c | 1 + > tools/perf/builtin-record.c | 10 ++ > tools/perf/builtin-report.c | 1 + > tools/perf/builtin-script.c | 41 +++++++ > tools/perf/builtin-top.c | 9 ++ > tools/perf/tests/sample-parsing.c | 6 +- > tools/perf/util/cgroup.c | 80 +++++++++++++ > tools/perf/util/cgroup.h | 17 ++- > tools/perf/util/env.c | 2 + > tools/perf/util/env.h | 6 + > tools/perf/util/event.c | 18 +++ > tools/perf/util/event.h | 6 + > tools/perf/util/evsel.c | 17 ++- > tools/perf/util/evsel.h | 1 + > tools/perf/util/hist.c | 13 +++ > tools/perf/util/hist.h | 1 + > tools/perf/util/machine.c | 19 ++++ > tools/perf/util/machine.h | 3 + > tools/perf/util/perf_event_attr_fprintf.c | 2 + > tools/perf/util/record.h | 1 + > tools/perf/util/session.c | 4 + > tools/perf/util/sort.c | 37 ++++++ > tools/perf/util/sort.h | 2 + > tools/perf/util/synthetic-events.c | 121 ++++++++++++++++++++ > tools/perf/util/synthetic-events.h | 1 + > tools/perf/util/tool.h | 2 + > 36 files changed, 598 insertions(+), 12 deletions(-) > > -- > 2.25.1.696.g5e7596f4ac-goog >
--
- Arnaldo
| |