A simple illustration of how DTrace internals work. Consider this probe of the kernel's "worker" function for the getpid() syscall: dtrace -n 'fbt::getpid:entry {printf("%s[%d] kthread: %p proc: %p\n", execname, curthread->t_procp->p_pidp->pid_id, curthread, curthread->t_procp)}' This probe shows how to chase pointer chains from the kthread_t * pointer to the current thread's context to the process the thread belongs to, and then to the integer PID of that process. Consider the disassembly of the getpid function: root@openindiana:/home/sergey# mdb -k > getpid::dis getpid: pushq %rbp getpid+1: movq %rsp,%rbp getpid+4: subq $0x10,%rsp getpid+8: movq %gs:0x18,%rax getpid+0x11: movq 0x190(%rax),%r9 getpid+0x18: movq 0xb0(%r9),%r8 getpid+0x1f: movl 0x4(%r8),%eax getpid+0x23: movl %eax,-0x8(%rbp) getpid+0x26: testl $0x400,0xcc(%r9) getpid+0x31: jne +0x9 getpid+0x33: movl 0x34(%r9),%eax getpid+0x37: movl %eax,-0x4(%rbp) getpid+0x3a: jmp +0x2c getpid+0x3c: movq %gs:0x18,%rax getpid+0x45: movq 0x190(%rax),%r8 getpid+0x4c: movq 0x620(%r8),%r8 getpid+0x53: movq 0x150(%r8),%r8 getpid+0x5a: movq 0xb0(%r8),%r8 getpid+0x61: movl 0x4(%r8),%eax getpid+0x65: movl %eax,-0x4(%rbp) getpid+0x68: movq -0x8(%rbp),%rax getpid+0x6c: leave getpid+0x6d: ret Note the "mov* (), " instructions. They do the actual pointer chasing behind the C '->' operator. You can convince yourself that the right member of the struct is being referenced by checking the offsets in mdb: Recall that "movq %gs:0x18,%rax" fetches the current thread pointer into RAX (see thread.h for the chain of macros that implement curthread, http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/sys/thread.h#530 , then http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/intel/ia32/ml/i86_subr.s#2404 for the implementation of threadp()) So RAX gets the pointer to curthread. DTrace exposes this very same pointer. Then > ::print -ta kthread_t ! grep proc 190 struct proc *t_procp > ::print -ta proc_t ! grep pid b0 struct pid *p_pidp > ::print -ta 'struct pid' ! grep pid 4 pid_t pid_id -- exactly the sequence of hex offsets in the above disassembly. Once the DTrace probe is activated, however, the disassembly differs: root@openindiana:/home/sergey# mdb -k > getpid::dis getpid: int 0x3 getpid+1: movq %rsp,%rbp getpid+4: subq $0x10,%rsp getpid+8: movq %gs:0x18,%rax getpid+0x11: movq 0x190(%rax),%r9 getpid+0x18: movq 0xb0(%r9),%r8 getpid+0x1f: movl 0x4(%r8),%eax getpid+0x23: movl %eax,-0x8(%rbp) Int 0x3 transfers control to the DTrace interpreter, via a hook in the Interrupt Descriptor Table (IDT), which it invokes. The IDT is poined to by the special IDTR register (and so it found by the processor). The opcode 0xcc of int 0x3 is, in fact, the standard way of implementing a debugger breakpoint. When executed, it passes control to the INT3 slot of the IDT. The IDT is documented in Intel's manuals. See http://pdos.csail.mit.edu/6.097/lec/l8.html link and the summary. Hacker coverage of the IDT is more succinct: http://www.phrack.org/issues.html?issue=59&id=4 Q.: What happens to the stack pointer when INT3 hits? Find the answer in the above. Q.: Observe the change in the IDT caused by arming the probe. After DTrace is stopped (^C), the original instruction at the function preamble is restored. It was kept saved away in a buffer. ------------------------------------------------------------- Note that DTrace internal variables are quite sensitive to types. For example, the FBT provider's "return" probes can get the function's return value from args[1] (see https://wikis.oracle.com/display/DTrace/fbt+Provider or the Solaris Dynamic Tracing Guide), but the (int) cast in the probe below is essential: dtrace -n 'fbt::getpid:return {printf("%s[%d] %d\n", execname, curthread->t_procp->p_pidp->pid_id, (int)args[1])}' [try this probe without the (int) cast and see what happens] ------------------------------------------------------------ Read more about how DTrace creates its built-ins: http://blogs.oracle.com/mws/entry/dtrace_inlines_translators_and_file