source: doc/theses/andrew_beach_MMath/performance.tex @ ea593a3

\chapter{Performance}
\label{c:performance}

\textbf{Just because of the stage of testing there are design notes for
the tests as well as commentary on them.}

Performance has been of secondary importance for most of this project.
Instead, the focus has been to get the features working. The only performance
requirements is to ensure the tests for correctness run in a reasonable
amount of time.

\section{Test Set-Up}
Tests will be run on \CFA, C++ and Java.

C++ is the most comparable language because both it and \CFA use the same
framework, libunwind.
In fact, the comparison is almost entirely a quality of implementation
comparison. \CFA's EHM has had significantly less time to be optimized and
does not generate its own assembly. It does have a slight advantage in that
there are some features it does not handle.

Java is another very popular language with similar termination semantics.
It is implemented in a very different environment, a virtual machine with
garbage collection.
It also implements the finally clause on try blocks allowing for a direct
feature-to-feature comparison.

All tests are run inside a main loop which will perform the test
repeatedly. This is to avoids start-up or tear-down time from
affecting the timing results.
A consequence of this is that tests cannot terminate the program,
which does limit how tests can be implemented.
There are catch-alls to keep unhandled
exceptions from terminating tests.

The exceptions used in these tests will always be a exception based off of
the base exception. This requirement minimizes performance differences based
on the object model.
Catch-alls are done by catching the root exception type (not using \Cpp's
\code{C++}{catch(...)}).

Tests run in Java were not warmed because exception code paths should not be
hot.

\section{Tests}
The following tests were selected to test the performance of different
components of the exception system.
The should provide a guide as to where the EHM's costs can be found.

Tests are run in \CFA, \Cpp and Java.
Not every test is run in every language, if the feature under test is missing
the test is skipped. These cases will be noted.
In addition to the termination tests for every language,
\CFA has a second set of tests that test resumption. These are the same
except that the raise statements and handler clauses are replaced with the
resumption variants.

\paragraph{Raise and Handle}
The first group of tests involve setting up
So there is three layers to the test. The first is set up and a loop, which
configures the test and then runs it repeatedly to reduce the impact of
start-up and shutdown on the results.
Each iteration of the main loop
\begin{itemize}
\item Empty Function:
The repeating function is empty except for the necessary control code.
\item Destructor:
The repeating function creates an object with a destructor before calling
itself.
(Java is skipped as it does not destructors.)
\item Finally:
The repeating function calls itself inside a try block with a finally clause
attached.
(\Cpp is skipped as it does not have finally clauses.)
\item Other Handler:
The repeating function calls itself inside a try block with a handler that
will not match the raised exception. (But is of the same kind of handler.)
\end{itemize}

\paragraph{Cross Try Statement}
The next group measures the cost of a try statement when no exceptions are
raised. The test is set-up, then there is a loop to reduce the impact of
start-up and shutdown on the results.
In each iteration, a try statement is executed. Entering and leaving a loop
is all the test wants to do.
\begin{itemize}
\item Handler:
The try statement has a handler (of the matching kind).
\item Finally:
The try statement has a finally clause.
\end{itemize}

\paragraph{Conditional Matching}
This group of tests checks the cost of conditional matching.
Only \CFA implements the language level conditional match,
the other languages must mimic with an ``unconditional" match (it still
checks the exception's type) and conditional re-raise.
\begin{itemize}
\item Catch All:
The condition is always true. (Always matches or never re-raises.)
\item Catch None:
The condition is always false. (Never matches or always re-raises.)
\end{itemize}

%\section{Cost in Size}
%Using exceptions also has a cost in the size of the executable.
%Although it is sometimes ignored
%
%There is a size cost to defining a personality function but the later problem
%is the LSDA which will be generated for every function.
%
%(I haven't actually figured out how to compare this, probably using something
%related to -fexceptions.)

% Some languages I left out:
% Python: Its a scripting language, different
% uC++: Not well known and should the same results as C++, except for
%   resumption which should be the same.

%\section{Resumption Comparison}
\todo{Can we find a good language to compare resumptions in.}