Saint's Log

Experimenting with perf on Linux

In the post on Experimenting with Async Profiler, I mentioned the basic (trial division) integer factorization app I wrote. I’ve been experimenting with perf to see what the system looks like when running this application. On Ubuntu, I started with this command:

perf record -F 97 -a -g -- sleep 10

Turns out perf isn’t installed by default.

WARNING: perf not found for kernel 5.19.0-41

  You may need to install the following packages for this specific kernel:
    linux-tools-5.19.0-41-generic
    linux-cloud-tools-5.19.0-41-generic

  You may also want to install one of the following packages to keep up to date:
    linux-tools-generic
    linux-cloud-tools-generic

Interestingly, running sudo apt install linux-tools-generic only picks up 5.17:

...
The following NEW packages will be installed:
  linux-tools-5.15.0-72 linux-tools-5.15.0-72-generic linux-tools-generic
...

which perf now shows /usr/bin/perf but even perf -v fails with the above warning so I have to run

sudo apt install linux-tools-5.19.0-41-generic

...
The following NEW packages will be installed:
  linux-hwe-5.19-tools-5.19.0-41 linux-tools-5.19.0-41-generic
...

Once that completes, perf can now run but perf version doesn’t display anything meaningful. Back to the original command:

perf record -F 97 -a -g -- sleep 10

This fails with an error about restricted access. Interesting reading but I just use sudo and carry on.

Error:
Access to performance monitoring and observability operations is limited.
Consider adjusting /proc/sys/kernel/perf_event_paranoid setting to open
access to performance monitoring and observability operations for processes
without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
More information can be found at 'Perf events and tool security' document:
https://www.kernel.org/doc/html/latest/admin-guide/perf-security.html
perf_event_paranoid setting is 4:
  -1: Allow use of (almost) all events by all users
      Ignore mlock limit after perf_event_mlock_kb without CAP_IPC_LOCK
>= 0: Disallow raw and ftrace function tracepoint access
>= 1: Disallow CPU event access
>= 2: Disallow kernel profiling
To make the adjusted perf_event_paranoid setting permanent preserve it
in /etc/sysctl.conf (e.g. kernel.perf_event_paranoid = <setting>)

Once the command completes, a perf.data file is created in the current directory. To generate a report, run this command. See the sample perf-report.txt file on GitHub.

perf report -n --stdio > perf-report.txt

To generate a flame graph, use Brendan Gregg’s scripts:

cd ~/repos
git clone https://github.com/brendangregg/FlameGraph

cd -

perf script --header > stacks.txt

~/repos/FlameGraph/stackcollapse-perf.pl < stacks.txt | ~/repos/FlameGraph/flamegraph.pl --hash > myflamegraph.svg