source: doc/theses/mubeen_zulfiqar_MMath/performance.tex @ c9136d9

\chapter{Performance}
\label{c:Performance}

This chapter uses the micro-benchmarks from \VRef[Chapter]{s:Benchmarks} to test a number of current memory allocators, including llheap.
The goal is to see if llheap is competitive with the current best memory allocators.


\section{Machine Specification}

The performance experiments were run on two different multi-core architectures (x86 and ARM) to determine if there is consistency across platforms:
\begin{itemize}
\item
\textbf{Nasus} AMD EPYC 7662, 64-core socket $\times$ 2, 2.0 GHz, GCC version 9.3.0
\item
\textbf{Algol} Huawei ARM TaiShan 2280 V2 Kunpeng 920, 24-core socket $\times$ 4, 2.6 GHz, GCC version 9.4.0
\end{itemize}


\section{Existing Memory Allocators}
\label{sec:curAllocatorSec}

With dynamic allocation being an important feature of C, there are many stand-alone memory allocators that have been designed for different purposes.
For this thesis, 7 of the most popular and widely used memory allocators were selected for comparison.

\subsection{glibc}
glibc~\cite{glibc} is the default gcc thread-safe allocator.
\\
\textbf{Version:} Ubuntu GLIBC 2.31-0ubuntu9.7 2.31\\
\textbf{Configuration:} Compiled by Ubuntu 20.04.\\
\textbf{Compilation command:} N/A

\subsection{dlmalloc}
dlmalloc~\cite{dlmalloc} is a thread-safe allocator that is single threaded and single heap.
It maintains free-lists of different sizes to store freed dynamic memory.
\\
\textbf{Version:} 2.8.6\\
\textbf{Configuration:} Compiled with preprocessor @USE_LOCKS@.\\
\textbf{Compilation command:} @gcc -g3 -O3 -Wall -Wextra -fno-builtin-malloc -fno-builtin-calloc@ @-fno-builtin-realloc -fno-builtin-free -fPIC -shared -DUSE_LOCKS -o libdlmalloc.so malloc-2.8.6.c@

\subsection{hoard}
Hoard~\cite{hoard} is a thread-safe allocator that is multi-threaded and using a heap layer framework. It has per-thread heaps that have thread-local free-lists, and a global shared heap.
\\
\textbf{Version:} 3.13\\
\textbf{Configuration:} Compiled with hoard's default configurations and @Makefile@.\\
\textbf{Compilation command:} @make all@

\subsection{jemalloc}
jemalloc~\cite{jemalloc} is a thread-safe allocator that uses multiple arenas. Each thread is assigned an arena. Each arena has chunks that contain contagious memory regions of same size. An arena has multiple chunks that contain regions of multiple sizes.
\\
\textbf{Version:} 5.2.1\\
\textbf{Configuration:} Compiled with jemalloc's default configurations and @Makefile@.\\
\textbf{Compilation command:} @autogen.sh; configure; make; make install@

\subsection{pt3malloc}
pt3malloc~\cite{pt3malloc} is a modification of dlmalloc.
It is a thread-safe multi-threaded memory allocator that uses multiple heaps. pt3malloc heap has similar design to dlmalloc's heap.
\\
\textbf{Version:} 1.8\\
\textbf{Configuration:} Compiled with pt3malloc's @Makefile@ using option ``linux-shared''.\\
\textbf{Compilation command:} @make linux-shared@

\subsection{rpmalloc}
rpmalloc~\cite{rpmalloc} is a thread-safe allocator that is multi-threaded and uses per-thread heap. Each heap has multiple size-classes and each size-class contains memory regions of the relevant size.
\\
\textbf{Version:} 1.4.1\\
\textbf{Configuration:} Compiled with rpmalloc's default configurations and ninja build system.\\
\textbf{Compilation command:} @python3 configure.py; ninja@

\subsection{tbb malloc}
tbb malloc~\cite{tbbmallocmail
} is a thread-safe allocator that is multi-threaded and uses private heap for each thread. Each private-heap has multiple bins of different sizes. Each bin contains free regions of the same size.
\\
\textbf{Version:} intel tbb 2020 update 2, tbb\_interface\_version == 11102\\
\textbf{Configuration:} Compiled with tbbmalloc's default configurations and @Makefile@.\\
\textbf{Compilation command:} @make@

% \section{Experiment Environment}
% We used our micro benchmark suite (FIX ME: cite mbench) to evaluate these memory allocators \ref{sec:curAllocatorSec} and our own memory allocator uHeap \ref{sec:allocatorSec}.

\section{Experiments}
% FIX ME: add experiment, knobs, graphs, description+analysis

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% CHURN
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Churn Micro-Benchmark}

Churn tests allocators for speed under intensive dynamic memory usage (see \VRef{s:ChurnBenchmark}).
This experiment was run with following configurations:
\begin{description}[itemsep=0pt,parsep=0pt]
\item[thread:]
1, 2, 4, 8, 16
\item[spots:]
16
\item[obj:]
100,000
\item[max:]
500
\item[min:]
50
\item[step:]
50
\item[distro:]
fisher
\end{description}

% -maxS          : 500
% -minS          : 50
% -stepS                 : 50
% -distroS       : fisher
% -objN          : 100000
% -cSpots                : 16
% -threadN       : 1, 2, 4, 8, 16

\VRef[Figure]{fig:churn} shows the results for algol and nasus.
The X-axis shows the number of threads.
Each allocator's performance for each thread is shown in different colors.
The Y-axis shows the total experiment time.

\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/churn} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/churn} }
\caption{Churn}
\label{fig:churn}
\end{figure}

All allocators did well in this micro-benchmark, except for dmalloc on the ARM, 


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% THRASH
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Cache Thrash}

Thrash tests memory allocators for active false sharing (see \VRef{sec:benchThrashSec}).
This experiment was run with following configurations:
\begin{description}[itemsep=0pt,parsep=0pt]
\item[thread:]
1, 2, 4, 8, 16
\item[iterations:]
1,000
\item[cacheRW:]
1,000,000
\item[size:]
1
\end{description}

% * Each allocator was tested for its performance across different number of threads.
% Experiment was repeated for each allocator for 1, 2, 4, 8, and 16 threads by setting the configuration -threadN.

Results are shown in figure \ref{fig:cacheThrash} for both algol and nasus.
X-axis shows number of threads. Each allocator's performance for each thread is shown in different colors.
Y-axis shows the total time experiment took to finish.

\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/cache-time-0-thrash} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/cache-time-0-thrash} }
\caption{Cache Thrash}
\label{fig:cacheThrash}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% SCRATCH
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Cache Scratch}
\label{s:CacheScratch}

Scratch tests memory allocators for program-induced allocator-preserved passive false-sharing.
This experiment was run with following configurations:
\begin{description}[itemsep=0pt,parsep=0pt]
\item[threads:]
1, 2, 4, 8, 16
\item[iterations:]
1,000
\item[cacheRW:]
1,000,000
\item[size:]
1
\end{description}

% * Each allocator was tested for its performance across different number of threads.
% Experiment was repeated for each allocator for 1, 2, 4, 8, and 16 threads by setting the configuration -threadN.

Results are shown in figure \ref{fig:cacheScratch} for both algol and nasus.
X-axis shows number of threads. Each allocator's performance for each thread is shown in different colors.
Y-axis shows the total time experiment took to finish.

\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/cache-time-0-scratch} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/cache-time-0-scratch} }
\caption{Cache Scratch}
\label{fig:cacheScratch}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% SPEED
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Speed Micro-Benchmark}

Speed testa memory allocators for runtime latency (see \VRef{s:SpeedMicroBenchmark}).
This experiment was run with following configurations:
\begin{description}[itemsep=0pt,parsep=0pt]
\item[max:]
500
\item[min:]
50
\item[step:]
50
\item[distro:]
fisher
\item[objects:]
1,000,000
\item[workers:]
1, 2, 4, 8, 16
\end{description}

% -maxS    :  500
% -minS    :  50
% -stepS   :  50
% -distroS :  fisher
% -objN    :  1000000
% -threadN    : \{ 1, 2, 4, 8, 16 \} *

%* Each allocator was tested for its performance across different number of threads.
%Experiment was repeated for each allocator for 1, 2, 4, 8, and 16 threads by setting the configuration -threadN.

Results for speed benchmark are shown in 12 figures, one figure for each chain of speed benchmark.
X-axis shows number of threads. Each allocator's performance for each thread is shown in different colors.
Y-axis shows the total time experiment took to finish. Less time an allocator takes to complete allocation, better it is so lower the better.

\begin{itemize}
\item Fig. \ref{fig:speed-3-malloc} shows results for chain: malloc
\item Fig. \ref{fig:speed-4-realloc} shows results for chain: realloc
\item Fig. \ref{fig:speed-5-free} shows results for chain: free
\item Fig. \ref{fig:speed-6-calloc} shows results for chain: calloc
\item Fig. \ref{fig:speed-7-malloc-free} shows results for chain: malloc-free
\item Fig. \ref{fig:speed-8-realloc-free} shows results for chain: realloc-free
\item Fig. \ref{fig:speed-9-calloc-free} shows results for chain: calloc-free
\item Fig. \ref{fig:speed-10-malloc-realloc} shows results for chain: malloc-realloc
\item Fig. \ref{fig:speed-11-calloc-realloc} shows results for chain: calloc-realloc
\item Fig. \ref{fig:speed-12-malloc-realloc-free} shows results for chain: malloc-realloc-free
\item Fig. \ref{fig:speed-13-calloc-realloc-free} shows results for chain: calloc-realloc-free
\item Fig. \ref{fig:speed-14-malloc-calloc-realloc-free} shows results for chain: malloc-realloc-free-calloc
\end{itemize}

%speed-3-malloc.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-3-malloc} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-3-malloc} }
\caption{Speed benchmark chain: malloc}
\label{fig:speed-3-malloc}
\end{figure}

%speed-4-realloc.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-4-realloc} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-4-realloc} }
\caption{Speed benchmark chain: realloc}
\label{fig:speed-4-realloc}
\end{figure}

%speed-5-free.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-5-free} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-5-free} }
\caption{Speed benchmark chain: free}
\label{fig:speed-5-free}
\end{figure}

%speed-6-calloc.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-6-calloc} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-6-calloc} }
\caption{Speed benchmark chain: calloc}
\label{fig:speed-6-calloc}
\end{figure}

%speed-7-malloc-free.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-7-malloc-free} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-7-malloc-free} }
\caption{Speed benchmark chain: malloc-free}
\label{fig:speed-7-malloc-free}
\end{figure}

%speed-8-realloc-free.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-8-realloc-free} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-8-realloc-free} }
\caption{Speed benchmark chain: realloc-free}
\label{fig:speed-8-realloc-free}
\end{figure}

%speed-9-calloc-free.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-9-calloc-free} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-9-calloc-free} }
\caption{Speed benchmark chain: calloc-free}
\label{fig:speed-9-calloc-free}
\end{figure}

%speed-10-malloc-realloc.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-10-malloc-realloc} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-10-malloc-realloc} }
\caption{Speed benchmark chain: malloc-realloc}
\label{fig:speed-10-malloc-realloc}
\end{figure}

%speed-11-calloc-realloc.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-11-calloc-realloc} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-11-calloc-realloc} }
\caption{Speed benchmark chain: calloc-realloc}
\label{fig:speed-11-calloc-realloc}
\end{figure}

%speed-12-malloc-realloc-free.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-12-malloc-realloc-free} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-12-malloc-realloc-free} }
\caption{Speed benchmark chain: malloc-realloc-free}
\label{fig:speed-12-malloc-realloc-free}
\end{figure}

%speed-13-calloc-realloc-free.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-13-calloc-realloc-free} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-13-calloc-realloc-free} }
\caption{Speed benchmark chain: calloc-realloc-free}
\label{fig:speed-13-calloc-realloc-free}
\end{figure}

%speed-14-{m,c,re}alloc-free.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/speed-14-m-c-re-alloc-free} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/speed-14-m-c-re-alloc-free} }
\caption{Speed benchmark chain: malloc-calloc-realloc-free}
\label{fig:speed-14-malloc-calloc-realloc-free}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% MEMORY
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Memory Micro-Benchmark}

This experiment is run with following two configurations for each allocator.
The difference between the two configurations is the number of producers and consumers.
Configuration 1 has one producer and one consumer, and configuration 2 has 4 producers where each producer has 4 consumers.

\noindent
Configuration 1:
\begin{description}[itemsep=0pt,parsep=0pt]
\item[producer (K):]
1
\item[consumer (M):]
1
\item[round:]
100,000
\item[max:]
500
\item[min:]
50
\item[step:]
50
\item[distro:]
fisher
\item[objects (N):]
100,000
\end{description}

% -threadA :  1
% -threadF :  1
% -maxS    :  500
% -minS    :  50
% -stepS   :  50
% -distroS :  fisher
% -objN    :  100000
% -consumeS:  100000

\noindent
Configuration 2:
\begin{description}[itemsep=0pt,parsep=0pt]
\item[producer (K):]
4
\item[consumer (M):]
4
\item[round:]
100,000
\item[max:]
500
\item[min:]
50
\item[step:]
50
\item[distro:]
fisher
\item[objects (N):]
100,000
\end{description}

% -threadA :  4
% -threadF :  4
% -maxS    :  500
% -minS    :  50
% -stepS   :  50
% -distroS :  fisher
% -objN    :  100000
% -consumeS:  100000

\begin{table}[b]
\centering
    \begin{tabular}{ |c|c|c| }
     \hline
    Memory Allocator & Configuration 1 Result & Configuration 2 Result\\
     \hline
    cfa & Fig. \ref{fig:mem-1-prod-1-cons-100-cfa} & Fig. \ref{fig:mem-4-prod-4-cons-100-cfa}\\
     \hline
    dl & Fig. \ref{fig:mem-1-prod-1-cons-100-dl} & Fig. \ref{fig:mem-4-prod-4-cons-100-dl}\\
     \hline
    glibc & Fig. \ref{fig:mem-1-prod-1-cons-100-glc} & Fig. \ref{fig:mem-4-prod-4-cons-100-glc}\\
     \hline
    hoard & Fig. \ref{fig:mem-1-prod-1-cons-100-hrd} & Fig. \ref{fig:mem-4-prod-4-cons-100-hrd}\\
     \hline
    je & Fig. \ref{fig:mem-1-prod-1-cons-100-je} & Fig. \ref{fig:mem-4-prod-4-cons-100-je}\\
     \hline
    pt3 & Fig. \ref{fig:mem-1-prod-1-cons-100-pt3} & Fig. \ref{fig:mem-4-prod-4-cons-100-pt3}\\
     \hline
    rp & Fig. \ref{fig:mem-1-prod-1-cons-100-rp} & Fig. \ref{fig:mem-4-prod-4-cons-100-rp}\\
     \hline
    tbb & Fig. \ref{fig:mem-1-prod-1-cons-100-tbb} & Fig. \ref{fig:mem-4-prod-4-cons-100-tbb}\\
     \hline
    \end{tabular}
\caption{Memory benchmark results}
\label{table:mem-benchmark-figs}
\end{table}

Results for memory benchmark are shown in 16 figures, two figures for each of the 8 allocators, one for each configuration.
Table \ref{table:mem-benchmark-figs} shows the list of figures that contain memory benchmark results.

Each figure has 2 graphs, one for each experiment environment.
Each graph has following 5 subgraphs that show program's memory usage and statistics throughout the program lifetime.

\begin{itemize}
\item \textit{\textbf{current\_req\_mem(B)}} shows the amount of dynamic memory requested and currently in-use of the benchmark.
\item \textit{\textbf{heap}}* shows the memory requested by the program (allocator) from the system that lies in the heap area.
\item \textit{\textbf{mmap\_so}}* shows the memory requested by the program (allocator) from the system that lies in the mmap area.
\item \textit{\textbf{mmap}}* shows the memory requested by the program (allocator or shared libraries) from the system that lies in the mmap area.
\item \textit{\textbf{total\_dynamic}} shows the total usage of dynamic memory by the benchmark program which is a sum of heap, mmap, and mmap\_so.
\end{itemize}

* These statistics are gathered by monitoring the \textit{/proc/self/maps} file of the process in linux system.

For each subgraph, x-axis shows the time during the program lifetime at which the data point was generated.
Y-axis shows the memory usage in bytes.

For the experiment, at a certain time in the program's life, the difference between the memory requested by the benchmark (\textit{current\_req\_mem(B)})
and the memory that the process has received from system (\textit{heap}, \textit{mmap}) should be minimum.
This difference is the memory overhead caused by the allocator and shows the level of fragmentation in the allocator.

%mem-1-prod-1-cons-100-cfa.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-1-prod-1-cons-100-cfa} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-cfa} }
\caption{Memory benchmark results with 1 producer for cfa memory allocator}
\label{fig:mem-1-prod-1-cons-100-cfa}
\end{figure}

%mem-1-prod-1-cons-100-dl.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-1-prod-1-cons-100-dl} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-dl} }
\caption{Memory benchmark results with 1 producer for dl memory allocator}
\label{fig:mem-1-prod-1-cons-100-dl}
\end{figure}

%mem-1-prod-1-cons-100-glc.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-1-prod-1-cons-100-glc} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-glc} }
\caption{Memory benchmark results with 1 producer for glibc memory allocator}
\label{fig:mem-1-prod-1-cons-100-glc}
\end{figure}

%mem-1-prod-1-cons-100-hrd.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-1-prod-1-cons-100-hrd} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-hrd} }
\caption{Memory benchmark results with 1 producer for hoard memory allocator}
\label{fig:mem-1-prod-1-cons-100-hrd}
\end{figure}

%mem-1-prod-1-cons-100-je.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-1-prod-1-cons-100-je} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-je} }
\caption{Memory benchmark results with 1 producer for je memory allocator}
\label{fig:mem-1-prod-1-cons-100-je}
\end{figure}

%mem-1-prod-1-cons-100-pt3.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-1-prod-1-cons-100-pt3} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-pt3} }
\caption{Memory benchmark results with 1 producer for pt3 memory allocator}
\label{fig:mem-1-prod-1-cons-100-pt3}
\end{figure}

%mem-1-prod-1-cons-100-rp.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-1-prod-1-cons-100-rp} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-rp} }
\caption{Memory benchmark results with 1 producer for rp memory allocator}
\label{fig:mem-1-prod-1-cons-100-rp}
\end{figure}

%mem-1-prod-1-cons-100-tbb.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-1-prod-1-cons-100-tbb} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-1-prod-1-cons-100-tbb} }
\caption{Memory benchmark results with 1 producer for tbb memory allocator}
\label{fig:mem-1-prod-1-cons-100-tbb}
\end{figure}

%mem-4-prod-4-cons-100-cfa.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-4-prod-4-cons-100-cfa} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-cfa} }
\caption{Memory benchmark results with 4 producers for cfa memory allocator}
\label{fig:mem-4-prod-4-cons-100-cfa}
\end{figure}

%mem-4-prod-4-cons-100-dl.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-4-prod-4-cons-100-dl} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-dl} }
\caption{Memory benchmark results with 4 producers for dl memory allocator}
\label{fig:mem-4-prod-4-cons-100-dl}
\end{figure}

%mem-4-prod-4-cons-100-glc.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-4-prod-4-cons-100-glc} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-glc} }
\caption{Memory benchmark results with 4 producers for glibc memory allocator}
\label{fig:mem-4-prod-4-cons-100-glc}
\end{figure}

%mem-4-prod-4-cons-100-hrd.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-4-prod-4-cons-100-hrd} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-hrd} }
\caption{Memory benchmark results with 4 producers for hoard memory allocator}
\label{fig:mem-4-prod-4-cons-100-hrd}
\end{figure}

%mem-4-prod-4-cons-100-je.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-4-prod-4-cons-100-je} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-je} }
\caption{Memory benchmark results with 4 producers for je memory allocator}
\label{fig:mem-4-prod-4-cons-100-je}
\end{figure}

%mem-4-prod-4-cons-100-pt3.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-4-prod-4-cons-100-pt3} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-pt3} }
\caption{Memory benchmark results with 4 producers for pt3 memory allocator}
\label{fig:mem-4-prod-4-cons-100-pt3}
\end{figure}

%mem-4-prod-4-cons-100-rp.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-4-prod-4-cons-100-rp} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-rp} }
\caption{Memory benchmark results with 4 producers for rp memory allocator}
\label{fig:mem-4-prod-4-cons-100-rp}
\end{figure}

%mem-4-prod-4-cons-100-tbb.eps
\begin{figure}
\centering
    \subfigure[Algol]{ \includegraphics[width=0.9\textwidth]{evaluations/algol-perf-eps/mem-4-prod-4-cons-100-tbb} }
    \subfigure[Nasus]{ \includegraphics[width=0.9\textwidth]{evaluations/nasus-perf-eps/mem-4-prod-4-cons-100-tbb} }
\caption{Memory benchmark results with 4 producers for tbb memory allocator}
\label{fig:mem-4-prod-4-cons-100-tbb}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% ANALYSIS
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%