-----[[ DTrace ]]----- The textbook makes use of the DTrace tracing tool (which truss is built on): Start with this great summary of DTrace features: http://d3s.mff.cuni.cz/teaching/crash_dump_analysis/slides/10-dtrace.pdf Local copy of the DTrace User Guide: d/DTrace-User-Guide.pdf Cheatsheets: http://www.oracle.com/technetwork/server-storage/solaris11/documentation/dtrace-cheatsheet-1930738.pdf http://dtrace.org/resources/brendan/DTrace-cheatsheet.pdf About DTrace on MacOS X (it's always more fun to learn things about your actual computer :) -- but note that much of the MacOS is opaque to its version of DTrace, for proprietary reasons): http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-scripts-for-mac-os-x/ (this link from http://wiki.illumos.org/display/illumos/DTrace; you can find many tutorials there, but sun.com links are broken.) Also note that the new versions of Mac (after Yosemite) disabled much of its DTrace functionality as a part if their "SIP/rootless". Now you can only profile your processes, and not much of the rest of the system. This is sad. -----[[ Why DTrace ]]----- DTrace allows to observe unprecedented amounts of events happening on in the kernel, by placing "probes" throughout the kernel code and printing out and aggregating the information produced by them when execution reaches them. -----[ Probe names ]----- DTrace probes all have fixed interger IDs, but are normally referred to via human-readable names: provider:module:func:probe To list all available probes (omit pfexec if you are already root): pfexec dtrace -l -n ::: To list syscall-related probes: pfexec dtrace -l -n syscall::: (parts that match all can be * or simply omitted) To list only probes that fire on entry to a syscall: pfexec dtrace -l -n syscall:::entry To list probes matching pattern: pfexec dtrace -l -n syscall:*:o*:entry ID PROVIDER MODULE FUNCTION NAME 70776 syscall open entry 71172 syscall open64 entry Unlike "truss " which traces only system calls by the process created by , DTrace probes fire for events caused by all processes on the system, unless limited by predicates. Hint: as a rule, there is some event that happens a lot more often than others, and needs to be filtered out before the output becomes readable. -----[ Providers ]----- syscall -- entry & return of all syscalls (argument knowledge) proc -- process creation and lifecycle; signal-related events fbt -- all kernel functions' boundaries sdt -- statically defined tracing: programmer-placed probes sched -- scheduler events that happen to threads io -- I/O subsystem pid -- user-level functions in processes vminfo -- VM events (based on kstat) sysinfo -- sys kstat kernel statistics profile -- profiler, runs periodic "tick" actions (e.g., profile:::tick-5s) -----[ D language ]----- D, the language of DTraces, has its own built-in string type. Convert to it with stringof( char* ) for char* kernel strings. Note that strings arguments to syscalls are located in _userspace_ and must be copied into _kernel space_ before they can used, since all probe actions are executed within kernel space. Use "copyinstr". For example: # Files opened by process, dtrace -n 'syscall::open*:entry { printf("%s %s",execname,copyinstr(arg0)); }' Observe the "copyinstr" used to copy the syscall's string argument into kernel space. [What happens without it? Why?] -----[ Pragmas ]----- Also note two important pragmas that change the default way of DTrace outputs: #pragma D option quiet suppresses DTrace's own default print actions #pragma D option flowindent supplies indentation that follows entry and exit from functions -----[ Examples ]----- Lots of examples found here: http://www.brendangregg.com/DTrace/dtrace_oneliners.txt Run them and interpret the results. Undertand the syntax involved. -----[ Special variables ]----- NOTE: DTrace tries to provide its action blocks with variables and structs that are most convenient to work with. Which variables will be made accessible in the action block depends on the probes that match the provider:module:probefunc:probename expression. But: - if a variable is not defined for some matching probe, the block won't compile; - always check the DTrace Guide for the tables of variables available for specific providers and probes: this can save a lot of time. # dtrace -n 'proc:::exec-success { printf("%s", curpsinfo->pr_psargs); }' dtrace: description 'proc:::exec-success ' matched 1 probe CPU ID FUNCTION:NAME 0 16132 exec_common:exec-success ls callout.d intr1.d proc1.d 0 16132 exec_common:exec-success hostname (output produced by "ls *.d" in another terminal window) Observe the curpsinfo variable pointing to a special "struct psinfo_t" filled with info about the "current" process (i.e., the process that caused the probe to fire). as described in "Table 25–1 proc Probes" at https://wikis.oracle.com/display/DTrace/proc+Provider Observe that this struct's pr_psargs member contains the string of arguments to ls after the Bash shell expanded them. ----- Proceed to dtrace-exercises.txt .