################################################################################
#                                                                              #
#                              fastcall-spma                                   #
#                                    -                                         #
#  A collection of helper scripts for reproducing fastcall benchmark results.  #
#                                                                              #
################################################################################

This repository contains several scripts that reproduce the results of the
fastcall benchmarks and plot the numbers as shown in publications.


Locations
=========

Beside the directories containing git submodules and the scripts for performing
the benchmarks, the repository contains a directory called 'results' for 
storing the plain (CSV) output of the benchmarks and a directory called 'plots'
for saving the final graphical output.


Files included
==============

install.sh     - Compiles and installs the components necessary for executing
                 the benchmarks.

load_kernel.sh - Configures a new kernel for use after next power cycle.

execute.sh     - Performs the benchmarks.


Setup
=====

Make sure that you have installed all necessary packages for building Google's
benchmarks library and the Linux kernel. You will also need Python 3 to run
the plotting script.

After cloning the repository, set up its submodules by executing

git submodule update --init --recursive

then, on some systems (but x86 only) you can run 

./install.sh 

from the root of this repository. Note that errors during the compilation
process of any component (e.g. due to missing packages or build tools named
differently on your distribution (e.g. cmake3 instead of cmake) may require 
you to execute the commands listed in 'install.sh' manually.

Take also care when configuring the fastcall and fccmp kernels that are needed
to run the benchmarks. 'install.sh' will configure them with 'make defconfig', 
which might not be sufficient to create a working kernel image for your 
machine. If in doubt, execute the installation commands manually, configuring 
the kernel with 'make localmodconfig'. Also take care that all required file 
system and networking drivers are built. For instance, when using a machine
from AWS, make sure that the drivers for Amazon's Elastic Network Adapter are 
enabled in the respective kernel configurations.


Running the Benchmarks
======================

In order to run the benchmarks for your machine, execute

./execute.sh [run-micro | run-cycle | run-misc]

from the command line. This will guide you through the benchmarks. Note that
in  order to complete all experiments, you will have to load different kernels
and thus reboot the machine multiple times. Follow the instructions printed
by the 'execute.sh' script. It will provide you with the correct command line
that you should enter in order to load the appropriate kernel for continuing
with the experiments. After rebooting the machine, continue with the command 
chosen from the options above. Repeat these steps, until 'execute.sh' exits
with a success message.

'execute.sh' provides the following benchmark suites:

run-micro - Run microbenchmarks for obtaining the latency for bare operations
            as well as memcpy examples for the vDSO, system call, ioctl, and
            fastcall mechanism. Results are computed in nanoseconds.

run-cycle - Same as above but counts CPU cycles.

run-misc  - Conducts auxiliary benchmarks regarding the performance of fastcall
            (de-) registration as well as forking.

Note that 'execute.sh' will only conduct the experiments for the CPU type
present on the local machine. In order to obtain the all numbers, you may have
to run the same procedure on different machines.

Also, the 'load_kernel.sh' script will not work for some distributions. It 
currently only supports a GRUB config which has an "Advanced Options" menu that
contains the non-default boot entries. 'load_kernel.sh' will ask you before
updating the bootloader config, so you can intervene if things go wrong. In
case that your bootloader config is not supported by 'load_kernel.sh', you will
have to update your bootloader manually, setting the right kernel to boot next,
as well as the appropriate kernel options for continuing with the benchmarks.

By default, 'execute.sh' will skip benchmark runs for which results are 
already present in the 'results' directory. If you wish to reset the numbers
collected for your local CPU type, run

./execute.sh reset


Plotting
========

In order to generate plots from the data available in 'results', run

python3 -m plotting

Hints for Running the Benchmarks on ARM
=======================================

Kernel Setup: Regardless of the CPU architecture, the fastcall kernel (subdir
              linux-fastcall) and the fccmp kernel (subdir linux-fccmp) must be
              built and installed.

              Furthermore, the fastcall mechanism is currently not supported
              for ARM kernels with mitigations. In order to build a working
              fastcall kernel on ARM, you thus have to configure it 
              accordingly: Enable the expert settings ("General Setup"), then
              disable KAISER which currently contradicts fastcalls on ARM
              ("General Setup"). Afterwards, check whether fastcalls are
              actually enabled ("Device Drivers").

lscpu support: On ARM, Linux may not print out human-readable CPU information
               by default. However, the scripts in this repository rely on
               this information to be present (e.g. like human-readable 
               vendor and model information in lscpu or /proc/cpuinfo).
               On ARM, you may need to install an alternative version of lscpu
               that supports decoding of CPU types and model names.

               For example, on AWS machines with Graviton CPUs, the following
               may work:

               git clone https://github.com/karelzak/util-linux
               cd util-linux
               ./autogen.sh
               ./configure --disable-all-programs --enable-lscpu \
                        --enable-libsmartcols
               make lscpu

               Install the resulting lscpu binary on your system.