Messages in this thread | | | Date | Wed, 17 Jun 2015 13:16:19 -0400 (EDT) | From | Abe Jakop <> | Subject | SystemTap 2.8 release |
| |
The SystemTap team announces release 2.8!
Syscall coverage, improved netfilter probes, namespace-aware tapsets, JSON tapsets, new and improved man pages, additional golang support
= Where to get it
https://sourceware.org/systemtap/ - our project page https://sourceware.org/systemtap/ftp/releases/systemtap-2.8.tar.gz https://koji.fedoraproject.org/koji/packageinfo?packageID=615 git tag release-2.8 (commit 08dc718)
There have been over 205 commits since the last release. There have been over 50 bugs fixed / features added since the last release.
= How to build it
See the README and NEWS files at https://sourceware.org/git/?p=systemtap.git;a=tree
Further information at https://sourceware.org/systemtap/wiki/
= SystemTap frontend (stap) changes
- Colorization of error string tokens is made more robust, especially in presence of $N/@N substitution.
- The function::*, probe::* and new macro::* man pages cross-references the enclosing tapset::* man page. For example:
function::pn(3stap) mentions tapset::pn(3stap) in the SEE ALSO section
- New stapref(1) man page provides a reference for the scripting language. The stapref page contains an overview of the features available in the language, such as keywords, data types, operators and more.
= SystemTap script language changes
- The @task macro performs the very common @cast to a task_struct.
The embedded-C bodies of task_current() and pid2task() are now wrapped by @task, which gives them a debuginfo type on the return value. With autocast type propagation, this removes the need for any explicit @cast in many places.
Other places which take untyped task pointers as parameters, for instance, now use @task as well to simplify their code.
= SystemTap runtime changes
- SystemTap has improved support for probing golang programs. Work has been done to be able to handle DWARF information, reporting file names, line numbers, and column numbers, and tolerance of odd characters in symbol names.
= SystemTap tapset changes
- New tapsets: json.stp provides probes, functions, and macros to generate a JSON metadata and data file. JSON metadata file is located in proc/systemtap/MODULE/metadata.json. JSON data file is located in /proc/systemtap/MODULE/data.json. json.stpm ditto.
- Changed tapsets: context.stp use new @task macro which gives a debuginfo type on the return value context-envvar.stp Ditto ip.stp Add "kernel<foo.h>" for all @cast()'s netfilter.stp Ditto. Exposed arp and bridge protocol info. nfs.stp Fix vfs.do_sync_read and vfs.do_sync_write probe aliases on rawhide kernels where the underlying probe points no longer exist proc_mem.stp Use @mm in _stp_get_mm_counter(). use new @task macro which gives a debuginfo type on the return value arm/registers.stp Allow argument 7 to be fetched in _stp_arg. handle arguments passed on the arm stack scheduler.stp Use new @task macro which gives a debuginfo type on the return value socket.stp Support socket_write_iter() and socket_read_iter(), the replacements for socket_aio_write() and socket_aio_read speculative.stp Optimize by replacing several iterations with array slices nd_syscalls.stp Made vm86_warning probes optional. Add arm fix for fadvise64_64. String representation of timex struct. Add 'env_str' convenience variable for the list of environment variables. For kernels < 3.7, move execve tapset support to arch-specific tapset code. syscalls.stp Ditto. Use __compat_[u]long() macros in syscall.migrate_pages and __keyctl_argstr(). Fix s390x compat task pointer retrieval. aux_syscalls.stp Updated ptrace functions. Print the failing address instead of "UNKNOWN". Rename all uses of syscall_get_nr() to _stp_syscall_get_nr(). Add SPLICE* defs. Improve formatting and report the difference between an invalid address and a NULL in __get_[compat_]argv. String representation of timex struct and arch_prctl option arg. nd_syscalls2.stp Improve argstr handling, 32-bit support, ptrace probes. Fix syscall.rt_sig{action,pending,procmask} for s390x. Call _stp_sigaltstack_u() or _stp_compat_sigaltstack_u() in syscall.sigalstack to decode the uss. syscalls2.stp Ditto. Fix s390x compat task pointer retrieval. Fix syscall.rt_sigsuspend compat probe. Minor improvement to syscall.nfsservctl for RHEL6. syscalls.stpm New __BIG_ENDIAN, @__quotactl_argstr(), and other macros. Use __compat_[u]long() macros in syscall.migrate_pages and __keyctl_argstr(). i386/syscalls.stp Decode struct user_desc. Fix iopl probe. i386/nd_syscalls.stp Ditto. [nd_]syscalls.stp Changed [compat_]execve[at] probes to use new __count[_compat]_envp functions to display the number of environment variables. Displaying the actual environment variables can easily overflow MAXSTRINGLEN and doesn't match what strace does. task.stp Use new @task macro which gives a debuginfo type on the return value. tty.stp Replace kernel_string calls with kernel_string_quoted task.stpm Add @mm and @task macros. task_time.stp Since @defined(task->real_start_time->tv_sec) doesn't work, use @defined(@task(0)->...). Use new @task macro which gives a debuginfo type on the return value. utrace.stp Rename syscall_get_nr() to _stp_syscall_get_nr(). vfs.stp Fix vfs.do_sync_read and vfs.do_sync_write probe aliases on rawhide kernels where the underlying probe points no longer exist target_set.stp Handle [nd_]syscall.fork probe's split into [nd_]syscall.{fork,vfork,clone} probes.
- Extensive work was done to provide coverage for syscalls in [nd_|aux_]syscalls[2].stp, along with continuous improvement to the types of the parameters in many syscall probe aliases. Work was also done to add compat and 32-bit support for syscall.
- New namespace-aware tapset functions [task_]ns_*() in task.stp and context.stp use option --target-namespaces=PID to denote a target set of namespaces corresponding to the PID's namespaces. The namespace-aware tapsets will return values relative to the target namespaces if specified, or the stap process' namespaces.
- Many side-effect free inline embedded-C blocks have been tagged with /* pure */ so that they can be optimized away.
- The following tapset variables and functions are deprecated in version 2.8: - The 'hostname_uaddr' variable in the syscall.setdomainname and nd_syscall.setdomainname probe aliases have been deprecated in favor of the new 'domainname_uaddr' variable. - The 'fd' and 'fd_str' variables in the syscall.execveat and nd_syscall.execveat probe aliases have been deprecated in favor of the new 'dirfd' and 'dirfd_str' variables.
= SystemTap sample scripts
- New samples: netfilter_summary_json.stp watches all IPv4 network traffic on the system. The data is output in JSON format. net_xmit_json.stp tracks time between packet queue and transmit. The information is provided to userspace via procfs in JSON format. proctop.stp top-like script that periodically prints out process info. stp_dump.stp prints out the packet contents. Each block contains the STP protocol ID, version ID, flags, root and bridge MAC addresses, and various times. whythefail.stp prints a statement-execution trace for a given function. works for failure analysis.
= Examples of tested kernel versions
2.6.9 (RHEL 4 x86) 2.6.18 (RHEL 5 x86 and x86_64) 2.6.32 (RHEL 6 x86 and x86_64) 3.10.0 (RHEL 7 x86_64) 3.17.7 (Fedora 20 x86_64) 3.19.3 (Fedora 21 x86_64) 4.0.4 (Fedora 22 and 21 x86_64)
= Known issues with this release
- Some kernel crashes continue to be reported when a script probes broad kernel function wildcards. (PR2725)
- 32-on-64 bit userspace unwinding is truncated on older kernels, such as 2.6.32 (PR15757)
- The dyninst backend is still very much a prototype, with a number of issues, limitations, and general teething woes. For instance: + lack of support for multiarch/cross-instrumentation + tapset functions are still incomplete relative to what is supported when the kernel backend is active + exception handling becomes completely broken in programs instrumented by the current version of dyninst (PR14702) + not all registers are made available on 32-bit x86 (PR15136)
See dyninst/README and the systemtap/dyninst Bugzilla component (http://tinyurl.com/stapdyn-PR-list) if you want all the gory details about the state of the feature.
- An upstream kernel commit #2062afb4f804a put "-fno-var-tracking-assignments" into KCFLAGS, reducing debuginfo quality which can cause debuginfo failures. A proposed workaround to this issue exists in: https://lkml.org/lkml/2014/11/21/505 . Fedora kernels are not affected by this issue.
= Contributors for this release
Abegail Jakop, *Athira, David Smith, *Felix Lu, Frank Ch. Eigler, *Hemant Kumar, Jonathan Lebon, Josh Stone, Lukas Berk, Mark Wielaard, Martin Cermak, Masanari Iida, Nathan Scott, *Qiao Nuohan, William Cohen
Special thanks to new contributors, marked with '*' above. Special thanks to Abegail and Felix for compiling these notes.
= Bugs fixed for this release <https://sourceware.org/PR#####>
6762 Some syscalls functions just wrappers for other syscalls 10488 sample script for unsampled ""true-top"" process monitor 13481 ARM register.stp arg() functions don't handle arguments passed on the stack 14164 netfilter.stp -- expose arp and bridge protocol info 14325 for arm, the nd_syscall tapsets need improvement 15987 generated function::* man pages should xref to enclosing tapset::* 16663 Provide synthetic pt_regs context for backtracing 16716 syscall wrappers causing syscall probes to get the wrong types 16974 excessive context size after {stmt;}->stmt; optimization 17638 Symbol resolution broken for PPC64 ABIv2 17660 @perf wont work with uprobes 17906 Wildcards in .nearest probes causing unprivileged_probes.exp to fail 17957 Systemtap rejects binaries probuced by the standard Go compiler (gc) 17958 Systemtap doesn't like variables with ~ in their name 17959 Something causes Systemtap to generate C code with uint0_t types 17986 unprivileged_[probes|myproc].exp regressed on el6 18000 kernel tracepoints not found without using the cache 18026 need timerfd_create, timerfd_gettime, timerfd_settime syscall support 18115 probe-condition mistranslated for array expressions 18120 no test coverage for adjtimex, arch_prctl, brk, capget, capset 18121 fallocate syscall support needs to be added 18122 [nd_]syscall.exec* probes need work 18143 target_set tapset does not track threads created with clone() 18151 chroot, getpid, getppid, gettid, iopl, lookup_dcookie, mincore need test coverage 18154 let's be more forgiving on ppc64le 18159 the [nd_]syscall.ptrace probes need improvement 18162 aarch64 compile server issue 18184 personality pivot_root quotactl need test coverage 18213 on arm, the runtime doesn't return correct syscall numbers 18262 The 'sync_file_range' and 'syncfs' syscalls need tapset support 18263 In tty tapset, driver_name can be null, fails when probing tty.write or tty.read 18264 the 'name_to_handle_at' and 'open_by_handle_at' syscalls need tapset support 18284 some of the rt_* syscalls need improved/added tapset support 18293 unshare, times, sysinfo, sysctl, syncfs, stime need test coverage 18309 [nd_]syscall.{sigpending,sigsuspend,sigaltstack} probes need improvement/testing 18310 remap_file_pages restart_syscall setsid uselib vhangup need test coverage 18337 [nd_]syscall.{kexec_load,set_tid_address} probes need improvement and test coverage 18343 Syscall clock_adjtime needs tapset handler and test coverage. 18361 systemtap doesn't realize RHEL7 kernels require secure-boot signed modules 18387 The setns syscall needs tapset handler and test coverage. 18389 gcc5 code folding breaks tracepoint queries 18395 Syscalls {get,set}_robust_list need tapset handler and test coverage 18398 The {get,set}_thread_area syscalls need tapset support and test coverage 18460 tracepoint_onthefly.exp kernel crash 18461 code generated by tapset-netfilter.cxx for nf_hook_ops does not compile with linux-4.1.0-rc5 kernel 18477 fix remaining kernel 3.10 syscall issues 18492 Syscalls sched_getattr and sched_setattr need tapset handler and test coverage 18501 The kcmp syscall needs tapset support and test coverage. 18518 vfs.do_sync_read and vfs.do_sync_write probe aliases no longer work on rawhide 18526 socket.aio_read and socket.aio_write probe aliases are broken on rawhide 18538 on rawhide, the scsi.iodispatching probe alias can't be resolved
| |