Linux Performance

Observability:

static, benchmarking, tuning:

sar, perf-tools, bcc/BPF:

Images license: creative commons Attribution-ShareAlike 4.0.

This page links to various Linux performance material I've created, including the tools maps on the right. These show: Linux observability tools, Linux benchmarking tools, Linux tuning tools, and Linux sar. For more diagrams, see my slide decks below.

Tools

Documentation

Talks

In recommended viewing order:

Velocity 2015

At Velocity 2015, I gave a 90 minute tutorial on Linux performance tools, summarizing performance observability, benchmarking, tuning, static performance tuning, and tracing tools. I also covered performance methodology, and included some live demos. This should be useful for everyone working on Linux systems.

A video of the talk is on youtube (playlist; part 1, part 2) and the slides are on slideshare or as a PDF.

This was similar to my SCaLE11x and LinuxCon talks, however, with 90 minutes I was able to cover more tools and methodologies, making it the most complete tour of the topic I've done. I also posted about it on the Netflix Tech Blog.

SCaLE14x (2016)

At the Southern California Linux Expo (SCaLE 14x), I gave a talk on Broken Linux Performance Tools. This was a follow-on to my earlier Linux Performance Tools talk originally at SCaLE11x (and more recently at Velocity as a tutorial). This broken tools talk was a tour of common problems with Linux system tools, metrics, statistics, visualizations, measurement overhead, and benchmarks. It also includes advice on how to cope (the green "What You Can Do" slides).

The slides are on slideshare or as a PDF.

I'll post the video soon.

SCaLE13x (2015)

At (SCaLE 13x), I gave a talk on Linux Profiling at Netflix using perf_events (aka "perf"), covering CPU profiling and a tour of other features. This talk covered gotchas, such as fixing stack traces and symbols when profiling Java and Node.js.

A video of the talk is on youtube, and the slides are on slideshare or as a PDF.

In a post about this talk, I included the interactive CPU flame graph SVG I was demonstrating.

LISA 2014

At USENIX LISA 2014, I gave a talk on the new ftrace and perf_events tools I've been developing: the perf-tools collection on github.

A video of the talk is on youtube, and the slides are on slideshare or as a PDF.

The perf-tools collection, inspired by my earlier DTraceToolkit, provides advanced system performance analysis tools for Linux. Each tool has a man page and example file. They are unstable and unsupported, and they currently use shell scripting, hacks, and the ftrace and perf_events kernel tracing frameworks. They should be developed (and improved) as the Linux kernel acquires more tracing capabilities (eg, eBPF).

In a post about this talk, I included some more screenshots of these tools in action.

Performance @Scale (2016)

This talk introduced new Linux performance analysis capabilities in the Linux 4.x series, using BPF. This was the first session for the 2016 Performance @Scale conference at Facebook.

A video of the talk is on facebook, and the slides are on slideshare or as a PDF.

SCaLE12x (2014)

At SCaLE 12x, I gave the keynote on What Linux can learn from Solaris performance and vice-versa. This drew on my experience analyzing Linux vs SmartOS performance, and my work for the Systems Performance book. SmartOS is an OS based on the illumos kernel, which is fork of the OpenSolaris kernel.

A video of the talk is on youtube, and the slides are on slideshare and as a PDF.

I'd never seen a good talk comparing performance features of both, I suspect in part because it's hard to know them both in enough depth, and also hard to choose from the many differences which should be highlighted.

This presentation also contains ponies. Lots of ponies. These are the unofficial mascots for DTrace, perf_events, SystemTap, ktap, and LTTng, and were designed by the same person (Deirdré) who designed the original (and popular) DTrace ponycorn.

Resources

Other resources (not by me) I'd recommend for the topic of Linux performance:


Last updated: 17-Apr-2016
Copyright 2016 Brendan Gregg, all rights reserved