Changeset e378c730 for doc/theses/thierry_delisle_PhD/thesis/text/eval_macro.tex

Timestamp:

Aug 13, 2022, 4:54:32 PM (2 years ago)

Author:

Thierry Delisle <tdelisle@…>

Branches:

ADT, ast-experimental, master, pthread-emulation

Children:

2ae6a99

Parents:

111d993

Message:

Fleshed out some more the evaluation sections, still waiting on some new data for churn nasus, locality nasus and memcached update

File:

• doc/theses/thierry_delisle_PhD/thesis/text/eval_macro.tex (modified) (5 diffs)

doc/theses/thierry_delisle_PhD/thesis/text/eval_macro.tex

@@ -72 +72 @@
 \begin{figure}
         \centering
-        \input{result.memcd.updt.qps.pstex_t}
-        \caption[Churn Benchmark : Throughput on Intel]{Churn Benchmark : Throughput on Intel\smallskip\newline Description}
-        \label{fig:memcd:updt:qps}
-\end{figure}
-
-\begin{figure}
-        \centering
-        \input{result.memcd.updt.lat.pstex_t}
-        \caption[Churn Benchmark : Throughput on Intel]{Churn Benchmark : Throughput on Intel\smallskip\newline Description}
-        \label{fig:memcd:updt:lat}
+        \subfloat[][Throughput]{
+                \input{result.memcd.forall.qps.pstex_t}
+        }
+
+        \subfloat[][Latency]{
+                \input{result.memcd.forall.lat.pstex_t}
+        }
+        \caption[forall Latency results at different update rates]{forall Latency results at different update rates\smallskip\newline Description}
+        \label{fig:memcd:updt:forall}
+\end{figure}
+
+\begin{figure}
+        \centering
+        \subfloat[][Throughput]{
+                \input{result.memcd.fibre.qps.pstex_t}
+        }
+
+        \subfloat[][Latency]{
+                \input{result.memcd.fibre.lat.pstex_t}
+        }
+        \caption[fibre Latency results at different update rates]{fibre Latency results at different update rates\smallskip\newline Description}
+        \label{fig:memcd:updt:fibre}
+\end{figure}
+
+\begin{figure}
+        \centering
+        \subfloat[][Throughput]{
+                \input{result.memcd.vanilla.qps.pstex_t}
+        }
+
+        \subfloat[][Latency]{
+                \input{result.memcd.vanilla.lat.pstex_t}
+        }
+        \caption[vanilla Latency results at different update rates]{vanilla Latency results at different update rates\smallskip\newline Description}
+        \label{fig:memcd:updt:vanilla}
 \end{figure}
 
@@ -89 +114 @@
 The memcached experiment has two aspects of the \io subsystem it does not exercise, accepting new connections and interacting with disks.
 On the other hand, static webservers, servers that offer static webpages, do stress disk \io since they serve files from disk\footnote{Dynamic webservers, which construct pages as they are sent, are not as interesting since the construction of the pages do not exercise the runtime in a meaningfully different way.}.
-The static webserver experiments will compare NGINX with a custom webserver developped for this experiment.
+The static webserver experiments will compare NGINX~\cit{nginx} with a custom webserver developped for this experiment.
 
 \subsection{\CFA webserver}
@@ -98 +123 @@
 
 Normally, webservers use @sendfile@\cite{MAN:sendfile} to send files over the socket.
-@io_uring@ does not support @sendfile@, it supports @splice@\cite{splice} instead, which is strictly more powerful.
+@io_uring@ does not support @sendfile@, it supports @splice@\cite{MAN:splice} instead, which is strictly more powerful.
 However, because of how linux implements file \io, see Subsection~\ref{ononblock}, @io_uring@'s implementation must delegate calls to splice to worker threads inside the kernel.
 As of Linux 5.13, @io_uring@ caps the numer of these worker threads to @RLIMIT_NPROC@ and therefore, when tens of thousands of splice requests are made, it can create tens of thousands of \glspl{kthrd}.
@@ -140 +165 @@
 
 \subsection{Throughput}
+To measure the throughput of both webservers, each server is loaded with over 30,000 files making over 4.5 Gigabytes in total.
+Each client runs httperf~\cit{httperf} which establishes a connection, does an http request for one or more files, closes the connection and repeats the process.
+The connections and requests are made according to a Zipfian distribution~\cite{zipf}.
+Throughput is measured by aggregating the results from httperf of all the clients.
 \begin{figure}
         \subfloat[][Throughput]{
@@ -153 +182 @@
         \label{fig:swbsrv}
 \end{figure}
-Figure~\ref{fig:swbsrv} shows the results comparing \CFA to nginx in terms of throughput.
-It demonstrate that the \CFA webserver described above is able to match the performance of nginx up-to and beyond the saturation point of the machine.
-Furthermore, Figure~\ref{fig:swbsrv:err} shows the rate of errors, a gross approximation of tail latency, where \CFA achives notably fewet errors once the machine reaches saturation.
+Figure~\ref{fig:swbsrv} shows the results comparing \CFA to NGINX in terms of throughput.
+These results are fairly straight forward.
+Both servers achieve the same throughput until around 57,500 requests per seconds.
+Since the clients are asking for the same files, the fact that the throughput matches exactly is expected as long as both servers are able to serve the desired rate.
+Once the saturation point is reached, both servers are still very close.
+NGINX achieves slightly better throughtput.
+However, Figure~\ref{fig:swbsrv:err} shows the rate of errors, a gross approximation of tail latency, where \CFA achives notably fewet errors once the machine reaches saturation.
+This suggest that \CFA is slightly more fair and NGINX may sloghtly sacrifice some fairness for improved throughtput.
+It demonstrate that the \CFA webserver described above is able to match the performance of NGINX up-to and beyond the saturation point of the machine.
+
+\subsection{Disk Operations}
+The throughput was made using a server with 25gb of memory, this was sufficient to hold the entire fileset in addition to all the code and data needed to run the webserver and the reste of the machine.
+Previous work like \cit{Cite Ashif's stuff} demonstrate that an interesting follow-up experiment is to rerun the same throughput experiment but allowing significantly less memory on the machine.
+If the machine is constrained enough, it will force the OS to evict files from the file cache and cause calls to @sendfile@ to have to read from disk.
+However, what these low memory experiments demonstrate is how the memory footprint of the webserver affects the performance.
+However, since what I am to evaluate in this thesis is the runtime of \CFA, I diceded to forgo experiments on low memory server.
+The implementation of the webserver itself is simply too impactful to be an interesting evaluation of the underlying runtime.