source: doc/theses/mubeen_zulfiqar_MMath/performance.tex @ f6e6a55

\chapter{Performance}
\label{c:Performance}

\section{Machine Specification}

The performance experiments were run on three different multicore systems to determine if there is consistency across platforms:
\begin{itemize}
\item
{\bf Nasus} AMD EPYC 7662, 64-core socket $\times$ 2, 2.0 GHz, GCC version 9.3.0
\item
{\bf Algol} Huawei ARM TaiShan 2280 V2 Kunpeng 920, 24-core socket $\times$ 4, 2.6 GHz, GCC version 9.4.0
\end{itemize}


\section{Existing Memory Allocators}\label{sec:curAllocatorSec}
With dynamic allocation being an important feature of C, there are many stand-alone memory allocators that have been designed for different purposes. For this thesis, we chose 7 of the most popular and widely used memory allocators.

\subsection{dlmalloc}
dlmalloc (FIX ME: cite allocator with download link) is a thread-safe allocator that is single threaded and single heap. dlmalloc maintains free-lists of different sizes to store freed dynamic memory. (FIX ME: cite wasik)
\\
\\
{\bf Version:} 2.8.6\\
{\bf Configuration:} Compiled with pre-processor USE\_LOCKS.\\
{\bf Compilation command:}\\
cc -g3 -O3 -Wall -Wextra -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -fPIC -shared -DUSE\_LOCKS -o libdlmalloc.so malloc-2.8.6.c

\subsection{hoard}
Hoard (FIX ME: cite allocator) is a thread-safe allocator that is multi-threaded and using a heap layer framework. It has per-thread heaps that have thread-local free-lists, and a global shared heap. (FIX ME: cite wasik)
\\
\\
{\bf Version:} 3.13\\
{\bf Configuration:} Compiled with hoard's default configurations and Makefile.\\
{\bf Compilation command:}\\
make all

\subsection{jemalloc}
jemalloc (FIX ME: cite allocator) is a thread-safe allocator that uses multiple arenas. Each thread is assigned an arena. Each arena has chunks that contain contagious memory regions of same size. An arena has multiple chunks that contain regions of multiple sizes.
\\
\\
{\bf Version:} 5.2.1\\
{\bf Configuration:} Compiled with jemalloc's default configurations and Makefile.\\
{\bf Compilation command:}\\
./autogen.sh\\
./configure\\
make\\
make install

\subsection{pt3malloc}
pt3malloc (FIX ME: cite allocator) is a modification of dlmalloc. It is a thread-safe multi-threaded memory allocator that uses multiple heaps. pt3malloc heap has similar design to dlmalloc's heap.
\\
\\
{\bf Version:} 1.8\\
{\bf Configuration:} Compiled with pt3malloc's Makefile using option "linux-shared".\\
{\bf Compilation command:}\\
make linux-shared

\subsection{rpmalloc}
rpmalloc (FIX ME: cite allocator) is a thread-safe allocator that is multi-threaded and uses per-thread heap. Each heap has multiple size-classes and each size-class contains memory regions of the relevant size.
\\
\\
{\bf Version:} 1.4.1\\
{\bf Configuration:} Compiled with rpmalloc's default configurations and ninja build system.\\
{\bf Compilation command:}\\
python3 configure.py\\
ninja

\subsection{tbb malloc}
tbb malloc (FIX ME: cite allocator) is a thread-safe allocator that is multi-threaded and uses private heap for each thread. Each private-heap has multiple bins of different sizes. Each bin contains free regions of the same size.
\\
\\
{\bf Version:} intel tbb 2020 update 2, tbb\_interface\_version == 11102\\
{\bf Configuration:} Compiled with tbbmalloc's default configurations and Makefile.\\
{\bf Compilation command:}\\
make

\section{Experiment Environment}
We used our micro becnhmark suite (FIX ME: cite mbench) to evaluate these memory allocators \ref{sec:curAllocatorSec} and our own memory allocator uHeap \ref{sec:allocatorSec}.

\section{Results}
FIX ME: add experiment, knobs, graphs, description+analysis

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% CHURN
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Churn Benchmark}

Churn benchmark tested memory allocators for speed under intensive dynamic memory usage.

This experiment was run with following configurations:

-maxS            : 500

-minS            : 50

-stepS           : 50

-distroS         : fisher

-objN            : 100000

-cSpots          : 16

-threadN         : \{ 1, 2, 4, 8, 16 \} *

* Each allocator was tested for its performance across different number of threads. Experiment was repeated for each allocator for 1, 2, 4, 8, and 16 threads by setting the configuration -threadN.

Results are shown in figure \ref{fig:churn} for both algol and nasus.
X-axis shows number of threads. Each allocator's performance for each thread is shown in different colors.
Y-axis shows the total time experiment took to finish.

\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/churn} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/churn} }
\caption{Churn}
\label{fig:churn}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% THRASH
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Cache Thrash}

Thrash benchmark tested memory allocators for active false sharing.

This experiment was run with following configurations:

-cacheIt        : 1000

-cacheRep       : 1000000

-cacheObj       : 1

-threadN        : \{ 1, 2, 4, 8, 16 \} *

* Each allocator was tested for its performance across different number of threads. Experiment was repeated for each allocator for 1, 2, 4, 8, and 16 threads by setting the configuration -threadN.

Results are shown in figure \ref{fig:cacheThrash} for both algol and nasus.
X-axis shows number of threads. Each allocator's performance for each thread is shown in different colors.
Y-axis shows the total time experiment took to finish.

\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/cache-time-0-thrash} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/cache-time-0-thrash} }
\caption{Cache Thrash}
\label{fig:cacheThrash}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% SCRATCH
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Cache Scratch}

Scratch benchmark tested memory allocators for program induced allocator preserved passive false sharing.

This experiment was run with following configurations:

-cacheIt        : 1000

-cacheRep       : 1000000

-cacheObj       : 1

-threadN        : \{ 1, 2, 4, 8, 16 \} *

* Each allocator was tested for its performance across different number of threads. Experiment was repeated for each allocator for 1, 2, 4, 8, and 16 threads by setting the configuration -threadN.

Results are shown in figure \ref{fig:cacheScratch} for both algol and nasus.
X-axis shows number of threads. Each allocator's performance for each thread is shown in different colors.
Y-axis shows the total time experiment took to finish.

\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/cache-time-0-scratch} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/cache-time-0-scratch} }
\caption{Cache Scratch}
\label{fig:cacheScratch}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% SPEED
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Speed Benchmark}

Speed benchmark tested memory allocators for program induced allocator preserved passive false sharing.

This experiment was run with following configurations:

-threadN :  sets number of threads, K\\
-cSpots  :  sets number of spots for churn, M\\
-objN    :  sets number of objects per thread, N\\
-maxS    :  sets max object size\\
-minS    :  sets min object size\\
-stepS   :  sets object size increment\\
-distroS :  sets object size distribution

%speed-1-malloc-null.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-1-malloc-null}
\caption{speed-1-malloc-null}
\label{fig:speed-1-malloc-null}
\end{figure}

%speed-2-free-null.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-2-free-null}
\caption{speed-2-free-null}
\label{fig:speed-2-free-null}
\end{figure}

%speed-3-malloc.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-3-malloc}
\caption{speed-3-malloc}
\label{fig:speed-3-malloc}
\end{figure}

%speed-4-realloc.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-4-realloc}
\caption{speed-4-realloc}
\label{fig:speed-4-realloc}
\end{figure}

%speed-5-free.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-5-free}
\caption{speed-5-free}
\label{fig:speed-5-free}
\end{figure}

%speed-6-calloc.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-6-calloc}
\caption{speed-6-calloc}
\label{fig:speed-6-calloc}
\end{figure}

%speed-7-malloc-free.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-7-malloc-free}
\caption{speed-7-malloc-free}
\label{fig:speed-7-malloc-free}
\end{figure}

%speed-8-realloc-free.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-8-realloc-free}
\caption{speed-8-realloc-free}
\label{fig:speed-8-realloc-free}
\end{figure}

%speed-9-calloc-free.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-9-calloc-free}
\caption{speed-9-calloc-free}
\label{fig:speed-9-calloc-free}
\end{figure}

%speed-10-malloc-realloc.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-10-malloc-realloc}
\caption{speed-10-malloc-realloc}
\label{fig:speed-10-malloc-realloc}
\end{figure}

%speed-11-calloc-realloc.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-11-calloc-realloc}
\caption{speed-11-calloc-realloc}
\label{fig:speed-11-calloc-realloc}
\end{figure}

%speed-12-malloc-realloc-free.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-12-malloc-realloc-free}
\caption{speed-12-malloc-realloc-free}
\label{fig:speed-12-malloc-realloc-free}
\end{figure}

%speed-13-calloc-realloc-free.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-13-calloc-realloc-free}
\caption{speed-13-calloc-realloc-free}
\label{fig:speed-13-calloc-realloc-free}
\end{figure}

%speed-14-{m,c,re}alloc-free.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/speed-14-{m,c,re}alloc-free}
\caption{speed-14-{m,c,re}alloc-free}
\label{fig:speed-14-{m,c,re}alloc-free}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% MEMORY
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Memory Benchmark}

%mem-1-prod-1-cons-100-cfa.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-cfa}
\caption{mem-1-prod-1-cons-100-cfa}
\label{fig:mem-1-prod-1-cons-100-cfa}
\end{figure}

%mem-1-prod-1-cons-100-dl.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-dl}
\caption{mem-1-prod-1-cons-100-dl}
\label{fig:mem-1-prod-1-cons-100-dl}
\end{figure}

%mem-1-prod-1-cons-100-glc.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-glc}
\caption{mem-1-prod-1-cons-100-glc}
\label{fig:mem-1-prod-1-cons-100-glc}
\end{figure}

%mem-1-prod-1-cons-100-hrd.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-hrd}
\caption{mem-1-prod-1-cons-100-hrd}
\label{fig:mem-1-prod-1-cons-100-hrd}
\end{figure}

%mem-1-prod-1-cons-100-je.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-je}
\caption{mem-1-prod-1-cons-100-je}
\label{fig:mem-1-prod-1-cons-100-je}
\end{figure}

%mem-1-prod-1-cons-100-pt3.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-pt3}
\caption{mem-1-prod-1-cons-100-pt3}
\label{fig:mem-1-prod-1-cons-100-pt3}
\end{figure}

%mem-1-prod-1-cons-100-rp.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-rp}
\caption{mem-1-prod-1-cons-100-rp}
\label{fig:mem-1-prod-1-cons-100-rp}
\end{figure}

%mem-1-prod-1-cons-100-tbb.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-tbb}
\caption{mem-1-prod-1-cons-100-tbb}
\label{fig:mem-1-prod-1-cons-100-tbb}
\end{figure}

%mem-4-prod-4-cons-100-cfa.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-cfa}
\caption{mem-4-prod-4-cons-100-cfa}
\label{fig:mem-4-prod-4-cons-100-cfa}
\end{figure}

%mem-4-prod-4-cons-100-dl.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-dl}
\caption{mem-4-prod-4-cons-100-dl}
\label{fig:mem-4-prod-4-cons-100-dl}
\end{figure}

%mem-4-prod-4-cons-100-glc.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-glc}
\caption{mem-4-prod-4-cons-100-glc}
\label{fig:mem-4-prod-4-cons-100-glc}
\end{figure}

%mem-4-prod-4-cons-100-hrd.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-hrd}
\caption{mem-4-prod-4-cons-100-hrd}
\label{fig:mem-4-prod-4-cons-100-hrd}
\end{figure}

%mem-4-prod-4-cons-100-je.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-je}
\caption{mem-4-prod-4-cons-100-je}
\label{fig:mem-4-prod-4-cons-100-je}
\end{figure}

%mem-4-prod-4-cons-100-pt3.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-pt3}
\caption{mem-4-prod-4-cons-100-pt3}
\label{fig:mem-4-prod-4-cons-100-pt3}
\end{figure}

%mem-4-prod-4-cons-100-rp.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-rp}
\caption{mem-4-prod-4-cons-100-rp}
\label{fig:mem-4-prod-4-cons-100-rp}
\end{figure}

%mem-4-prod-4-cons-100-tbb.eps
\begin{figure}
\centering
\includegraphics[width=1\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-tbb}
\caption{mem-4-prod-4-cons-100-tbb}
\label{fig:mem-4-prod-4-cons-100-tbb}
\end{figure}