Ktrace

ktrace is a kernel trace in an UBO system. You can invoke it from userland and watch all system call activity of a process with it. ktrace comes with kdump to inspect a trace file.

kdump
Often when a program gets traced the reader of the ktrace often gets confused about a pile of files being opened and mmap'ed. This is most likely the dynamic linker reading in libraries that are dependencies to the dynamically linked program.

Here is the difference between a dynamically linked and a statically linked hello world program:

francisco$ ls -l hello.c -rw-r--r-- 1 pjp  pjp  77 May 30 21:06 hello.c francisco$ cc -o hello hello.c francisco$ ktrace ./hello hello, world francisco$ kdump | wc -l 580 francisco$ cc -static -o hello hello.c francisco$ ktrace ./hello hello, world francisco$ kdump | wc -l 54

Here is the dump from the static program:

francisco$ kdump 26974 ktrace  RET   ktrace 0 26974 ktrace  CALL  execve(0xcfbd4b03,0xcfbd498c,0xcfbd4994) 26974 ktrace  NAMI  "./hello" 26974 hello   EMUL  "native" 26974 hello   RET   execve 0 26974 hello   CALL  __sysctl(1.37,0x3c003260,0xcfbd2338,0,0) 26974 hello   RET   __sysctl 0 26974 hello   CALL  __sysctl(6.7,0x3c0077f4,0xcfbd2308,0,0) 26974 hello   RET   __sysctl 0 26974 hello   CALL  mmap(0,0x1000,0x3,0x1002,0xffffffff,0,0,0) 26974 hello   RET   mmap -2146906112/0x8008d000 26974 hello   CALL  mprotect(0x8008d000,0x1000,0x1) 26974 hello   RET   mprotect 0 26974 hello   CALL  mprotect(0x8008d000,0x1000,0x3) 26974 hello   RET   mprotect 0 26974 hello   CALL  mprotect(0x8008d000,0x1000,0x1) 26974 hello   RET   mprotect 0 26974 hello   CALL  fstat(0x1,0xcfbd1f20) 26974 hello   RET   fstat 0 26974 hello   CALL  readlink(0x3c001c68,0xcfbd1f00,0x3f) 26974 hello   NAMI  "/etc/malloc.conf" 26974 hello   RET   readlink -1 errno 2 No such file or directory 26974 hello   CALL  issetugid 26974 hello   RET   issetugid 0 26974 hello   CALL  mmap(0,0x1000,0x3,0x1002,0xffffffff,0,0,0) 26974 hello   RET   mmap -2119225344/0x81af3000 26974 hello   CALL  mmap(0,0x1000,0x3,0x1002,0xffffffff,0,0,0) 26974 hello   RET   mmap 2081595392/0x7c12a000 26974 hello   CALL  mmap(0,0x10000,0x3,0x1002,0xffffffff,0,0,0) 26974 hello   RET   mmap -2023108608/0x8769d000 26974 hello   CALL  mmap(0,0x1000,0x3,0x1002,0xffffffff,0,0,0) 26974 hello   RET   mmap -2087710720/0x83901000 26974 hello   CALL  mprotect(0x8008d000,0x1000,0x3) 26974 hello   RET   mprotect 0 26974 hello   CALL  mprotect(0x8008d000,0x1000,0x1) 26974 hello   RET   mprotect 0 26974 hello   CALL  ioctl(0x1,TIOCGETA,0xcfbd1f60) 26974 hello   RET   ioctl 0 26974 hello   CALL  write(0x1,0x8769d000,0xd) 26974 hello   GIO   fd 1 wrote 13 bytes "hello, world      " 26974 hello   RET   write 13/0xd 26974 hello   CALL  mprotect(0x8008d000,0x1000,0x3) 26974 hello   RET   mprotect 0 26974 hello   CALL  mprotect(0x8008d000,0x1000,0x1) 26974 hello   RET   mprotect 0 26974 hello   CALL  mprotect(0x8008d000,0x1000,0x3) 26974 hello   RET   mprotect 0 26974 hello   CALL  mprotect(0x8008d000,0x1000,0x1) 26974 hello   RET   mprotect 0 26974 hello   CALL  munmap(0x8008d000,0x1000) 26974 hello   RET   munmap 0 26974 hello   CALL  exit(0)

Ktrace leaves behind the ktrace.out file for kdump to display, when not needed this can be deleted, also one can attach a ktrace to a process, to turn off ktracing on all processes one can type ktrace -C to stop tracing.

francisco$ ls -lc ktrace.out -rw--- 1 pjp  pjp  3010 May 30 21:07 ktrace.out francisco$ ktrace -C francisco$ rm ktrace.out