Psio is a open source tool for Solaris that runs ps and inserts an I/O column to reflect disk I/O on a per process basis. Extended forms of output can show I/O time, size and count of operations as a percent or a value, as well as a breakdown by filesystem. This is a performance tuning or troubleshooting tool.

Note: This page was last updated in 2005 and is no longer maintained. It is now part of my Crypt.

Psio was my first disk I/O by process tool. Since writing it I have written many more that use different techniques, each having their own pros and cons. They are all on this site in the Download section below. The following table compares features. This is listed from most to least recommended.

For troubleshooting I start with prusage as it doesn't require root access. Then I'll use iosnoop and iotop if it is Solaris 10, or psio for older Solaris.

* This is the pr_inblk and pr_oublk block counts from procfs. Originally they may well have tracked, say, 8 Kb blocks, however now they no longer take into account the event size - so they serve more as a disk event count. If you know generally how large the events should be, then the count can help you estimate the actual size. Sound vague? Even accurate size info is vague - you can't tell if it's random or sequential bytes... I/O Time turns out to be the most useful measurement.

Prusage - Any Solaris This version is written in Perl and reads the /proc data directly for usage statistics, such as I/O, USR/SYS times and context switches. This version is reliable and runs as any user, and so is a good place to start when troubleshooting process I/O. (see Further Info below for a similar program in C).

iosnoop - Solaris 10 a program that uses DTrace to snoop I/O events live, example here.

iotop - Solaris 10 a program that uses DTrace to print an I/O summary, example here.

Psio - Solaris 9 (and older) This version is written in Perl and works by running "prex -k" for a short period of time (default is 1 second) and enabling kernel tracing for disk I/O operations. Running prex on a kernel is not an everyday event, and this version is not designed as an everyday tool - more a demonstration program. WARNING: This version runs prex on the kernel to activate kernel tracing - use with caution. Best to run for short periods only, eg less than a minute.

Psio10 - Solaris 10 This version is written in Perl and runs DTrace to examine I/O events, which is only available in Solaris 10. This version is reliable and well featured, and will likely be the most developed version in the future.


Psio.se - SE Toolkit is a version written in the SE Toolkit language. This version is reliable and can be run for long periods of time; however it has less features and includes all types of I/O (disk, network, streams, ...) unlike the psio versions above which only target disk I/O. The SE Toolkit must be installed (see Further Info below).

listprusage - Any Solaris prints process resource usage statistics such as in blocks and out blocks by PID. It's written in C. here.

psio and psio10 have very similar output. In their most basic form,

# ./psio UID PID PPID %I/O STIME TTY TIME CMD brendan 25768 24654 1.7 23:26:22 pts/4 0:00 pine root 0 0 0.4 Mar 16 ? 0:16 sched root 5857 5856 0.3 Nov 08 ? 5:09 (squid) -s root 3 0 0.2 Mar 16 ? 18:44:10 fsflush brendan 6094 10068 0.1 Nov 11 pts/15 7:34 opera root 1 0 0.0 Mar 16 ? 0:10 /etc/init - root 2 0 0.0 Mar 16 ? 0:01 pageout [...]




while running grep on a large file,


# ./psio UID PID PPID %I/O STIME TTY TIME CMD brendan 13271 10093 65.4 23:20:16 pts/20 0:01 grep brendan contents root 0 0 0.0 Mar 16 ? 0:16 sched root 1 0 0.0 Mar 16 ? 0:10 /etc/init - root 2 0 0.0 Mar 16 ? 0:00 pageout [...]




the following prints I/O stats due to the "-x" option (time, size and count),


# ./psio -x UID PID %CPU %I/Ot %I/Os %I/Oc %MEM S TIME CMD brendan 25426 1.1 60.8 96.9 94.4 0.2 R 0:00 grep brendan contents brendan 25425 0.8 3.5 3.1 5.6 1.0 S 0:00 pine root 0 0.0 0.0 0.0 0.0 0.0 T 0:16 sched root 1 0.0 0.0 0.0 0.0 0.1 S 0:10 /etc/init - [...]




the "-n" switch produces a numerical output,


# ./psio -n 10 UID PID IOTIME IOSIZE IOCOUNT CMD root 0 2825 2260992 462 sched brendan 25489 368 843776 52 pine brendan 25491 121 90112 11 find /usr root 5857 5 1024 1 (squid) -s root 1 0 0 0 /etc/init - root 2 0 0 0 pageout root 3 0 0 0 fsflush [...]




the "-f" switch will print a break down of I/O per filesystem,


# ./psio -xf 10 UID PID %CPU %I/Ot %I/Os %I/Oc %MEM S TIME CMD brendan 25259 2.7 6.3 23.3 56.9 0.3 S 0:01 find /var " " " 6.3 23.3 56.9 /dev/dsk/c0t0d0s5, /var brendan 25257 0.6 2.1 73.5 28.6 1.0 S 0:00 pine " " " 2.1 73.5 28.6 /dev/dsk/c0t0d0s6, /export/home root 3 0.5 1.6 3.1 14.1 0.0 S 18:44:00 fsflush " " " 0.7 1.1 9.3 /dev/dsk/c0t0d0s3, /usr " " " 0.4 0.3 2.4 /dev/dsk/c0t0d0s1, / " " " 0.3 0.9 1.6 /dev/dsk/c0t0d0s5, /var " " " 0.2 0.9 0.8 /dev/dsk/c0t0d0s6, /export/home brendan 6094 0.0 0.0 0.1 0.4 5.5 S 7:34 opera " " " 0.0 0.1 0.4 /dev/dsk/c0t0d0s6, /export/home root 0 0.0 0.0 0.0 0.0 0.0 T 0:16 sched root 1 0.0 0.0 0.0 0.0 0.1 S 0:10 /etc/init - root 2 0.0 0.0 0.0 0.0 0.0 S 0:01 pageout [...]

See More Examples for analysis using output files.



iosnoop is able to snoop I/O events live, (more examples here)


# ./iosnoop UID PID D BLOCK SIZE COMM PATHNAME 100 15795 R 3808 8192 tar /usr/bin/eject 100 15795 R 35904 6144 tar /usr/bin/eject 100 15795 R 39828 6144 tar /usr/bin/env 100 15795 R 3872 8192 tar /usr/bin/expr 100 15795 R 21120 7168 tar /usr/bin/expr 100 15795 R 43680 6144 tar /usr/bin/false 100 15795 R 44176 6144 tar /usr/bin/fdetach 100 15795 R 3920 8192 tar /usr/bin/fdformat 100 15795 R 3936 8192 tar /usr/bin/fdformat 100 15795 R 4080 8192 tar /usr/bin/fdformat 100 15795 R 9680 3072 tar /usr/bin/fdformat 100 15795 R 4096 8192 tar /usr/bin/fgrep 100 15795 R 46896 6144 tar /usr/bin/fgrep 100 15795 R 4112 8192 tar /usr/bin/file 100 15795 R 4128 8192 tar /usr/bin/file 100 15795 R 4144 8192 tar /usr/bin/file [...]




prusage the default output prints a process listing with I/O as in blocks, out blocks and character I/O, sorted on total block I/O, and is printing the summary since boot data (historic),

$ ./prusage PID MINF MAJF INBLK OUBLK CHAR-kb COMM 3 0 0 58001 1685914 0 fsflush 7204 0 6236 10919 5396 2467294 (squid) 253 0 970 2425 5273 441901 /usr/sbin/syslogd 19368 0 5338 5499 2 930786 /usr/local/bin/Xvnc 2 0 0 0 3105 0 pageout 427 0 62 65 2406 74953 tail 388 0 827 1231 948 3707 /usr/lib/inet/in.dhcpd 19448 0 1709 2011 0 301024 dtwm [...]




the following limits the output to the top 5 (-t5), and prints an output every 3 secounds (the first is historic),


$ ./prusage -t5 3 PID MINF MAJF INBLK OUBLK CHAR-kb COMM 3 0 0 58001 1685943 0 fsflush 7204 0 6236 10919 5396 2467294 (squid) 253 0 970 2425 5273 441903 /usr/sbin/syslogd 19368 0 5338 5499 2 932108 /usr/local/bin/Xvnc 2 0 0 0 3105 0 pageout PID MINF MAJF INBLK OUBLK CHAR-kb COMM 7158 0 0 36 0 540 find 288 0 0 0 0 0 /usr/lib/utmpd 19657 0 0 0 0 0 -sh 2623 0 0 0 0 0 /usr/lib/power/powerd 7203 0 0 0 0 0 /opt/sfw/squid/bin/squid [...]




the following is printing a USR/SYS times report (-u), with the top 5 processes sorted on CPU usage every 2 seconds,


$ ./prusage -ut5 2 PID USR SYS WAIT LOCK TRAP COMM 19368 2474.42 1155.67 0.00 0.00 0.00 /usr/local/bin/Xvnc 438 1591.23 1884.77 0.00 0.00 0.00 mibiisa 19479 1864.98 1436.97 2759.56 0.00 0.03 sdtperfmeter 19480 1129.24 1163.68 4689.23 0.00 0.02 /usr/dt/bin/sdtperfmeter 19460 307.31 685.86 2954.27 0.00 0.01 /usr/dt/bin/dtterm PID USR SYS WAIT LOCK TRAP COMM 7231 0.12 0.60 0.70 0.00 0.00 find 6962 0.20 0.39 0.00 0.00 0.00 /usr/lib/ssh/sshd 19479 0.00 0.00 0.00 0.00 0.00 sdtperfmeter 19467 0.00 0.00 0.01 0.00 0.00 /usr/dt/bin/dtterm 19452 0.00 0.00 0.01 0.00 0.00 /usr/dt/bin/dtterm [...]




the following is printing the context switch report (-x), with the top 5 processes sorted on context swithes every 2 seconds,


$ ./prusage -xt5 2 PID USR SYS VCTX ICTX SYSC COMM 19368 2474.67 1156.29 84991843 3266777 1782490550 /usr/local/bin/Xvnc 15722 4.66 7.19 59544651 5073 101418323 /usr/dt/bin/dtterm 397 2.51 3.07 34422624 1141 38116238 /usr/apache/bin/httpd 15237 243.06 268.69 34301887 60734 563346823 /usr/local/bin/Xvnc 19479 1869.46 1440.45 30893642 9584 93519499 sdtperfmeter PID USR SYS VCTX ICTX SYSC COMM 6962 0.36 0.45 965 811 11581 /usr/lib/ssh/sshd 9223 0.22 0.84 564 1078 23574 find 9224 0.03 0.19 289 27 3374 tar 19368 0.00 0.00 41 2 897 /usr/local/bin/Xvnc 288 0.00 0.00 40 0 0 /usr/lib/utmpd [...]




the following prints the top 5 processes (-t5) from three reports (-iux), every 30 seconds, while clearing the screen (-c). The output refreshes similar to "top",


$ ./prusage -ciux -t5 30 PID MINF MAJF INBLK OUBLK CHAR-kb COMM 3 0 0 20 26 0 fsflush 7204 0 2 12 10 669 (squid) 7205 0 0 4 2 1 (unlinkd) 9233 0 0 0 2 81 vi 6729 0 0 0 0 0 /usr/lib/saf/ttymon PID USR SYS VCTX ICTX SYSC COMM 7204 0.14 0.13 506 26 3895 (squid) 19368 0.00 0.01 513 13 11305 /usr/local/bin/Xvnc 19452 0.01 0.02 339 1 586 /usr/dt/bin/dtterm 19460 0.01 0.01 340 0 586 /usr/dt/bin/dtterm 15722 0.00 0.00 340 0 586 /usr/dt/bin/dtterm PID USR SYS WAIT LOCK TRAP COMM 7204 0.14 0.13 0.00 0.00 0.00 (squid) 19479 0.02 0.01 0.05 0.00 0.00 sdtperfmeter 239 0.02 0.01 0.00 0.00 0.00 /usr/sbin/in.named 19452 0.01 0.02 0.06 0.00 0.00 /usr/dt/bin/dtterm 19480 0.01 0.01 0.02 0.00 0.00 /usr/dt/bin/sdtperfmeter




the following examines one process only (7204) every 2 seconds, 8 times. The first line is historic.


$ ./prusage -x -p7204 2 8 PID USR SYS VCTX ICTX SYSC COMM 7204 190.14 170.44 2649662 26074 12859733 (squid) PID USR SYS VCTX ICTX SYSC COMM 7204 0.00 0.00 10 0 23 (squid) PID USR SYS VCTX ICTX SYSC COMM 7204 0.01 0.00 15 0 100 (squid) PID USR SYS VCTX ICTX SYSC COMM 7204 0.01 0.02 47 1 355 (squid) PID USR SYS VCTX ICTX SYSC COMM 7204 0.01 0.03 55 4 514 (squid) PID USR SYS VCTX ICTX SYSC COMM 7204 0.02 0.00 40 4 350 (squid) PID USR SYS VCTX ICTX SYSC COMM 7204 0.01 0.00 74 0 564 (squid) PID USR SYS VCTX ICTX SYSC COMM 7204 0.00 0.00 4 0 4 (squid)

These have been tested on Solaris 9 and 10, and so far have been written for Solaris only.

Feature List

psio and psio10 can print,

* %I/O time per process (I/O service time, % of sample time)
* %I/O size per process (bytes transferred by process, % of total seen)
* %I/O count per process (number of I/O operations, % of total seen)
* I/O time per process (milliseconds)
* I/O size per process (bytes transferred)
* I/O count per process (number of operations)
* All of the above, per filesystem per process
* I/O time, size & count per I/O event

Further Info

For more performance tuning on Solaris see,

• Sun Performance Sun's website that links to performance papers and articles.
• SE Toolkit The SE Toolkit is a well developed suite of tools for the performance professional.
• Solaris Internals This is a website to support the "Solaris Internals" book by Richard McDougall & Jim Mauro.
• Books Recommended reading, Unix Internals.
• pio Process I/O and topio report on process blk usage, written in C by Yong Huang.

Back to Brendan Gregg's Homepage