source: doc/theses/andrew_beach_MMath/performance.tex @ b6749fd

\chapter{Performance}
\label{c:performance}

\textbf{Just because of the stage of testing there are design notes for
the tests as well as commentary on them.}

Performance has been of secondary importance for most of this project.
Instead, the goal has been to get the features working.
The only performance
requirements is to ensure the exception tests for correctness ran in a reasonable
amount of time.
Much of the implementation is still reasonable and could be used for similar prototypes.
Hence,
the work still has some use.
To get a rough idea about the \CFA implementation, tests are run on \CFA, C++ and Java, which have similar termination-handling exceptions.
Tests are also run on \CFA and uC++, which has similar resumption-handling exceptions.

\section{Termination Comparison}
C++ is the most comparable language because both it and \CFA use the same
framework, libunwind.
In fact, the comparison is almost entirely a quality of implementation
comparison. \CFA's EHM has had significantly less time to be optimized and
does not generate its own assembly. It does have a slight advantage in that
there are some features it does not handle.

The Java comparison is an opportunity to compare a managed memory model with unmanaged,
to see if there are any effects related to the exception model.

\subsection{Test Set-Up}
All tests are run inside a main loop that performs the test
repeatedly. This design avoids start-up or tear-down time from
affecting the timing results.
A consequence is that tests cannot terminate the program, which does limit
how tests can be implemented. There are catch-alls to keep unhandled
exceptions from terminating tests.

The exceptions used in this test are always a new exception based off of
the base exception. This requirement minimizes performance differences based
on the object model.
Catch-alls are done by catching the root exception type (not using \Cpp's
\code{C++}{catch(...)}).

Tests run in Java were not warmed because exception code paths should not be
hot.

\subsection{Tests}
The following tests capture the most important aspects of exception handling and should provide
a reasonable guide to programmers of where EHM costs occur.

\paragraph{Raise/Handle}
What is the basic cost to raise and handle an exception?

There are a number of factors that can effect this.
For \CFA this includes
the type of raise,

Main loop, pass through a catch-all, call through some empty helper functions
to put frames on the stack then raise and exception.
\todo{Raise/Handle (or a similar test) could also test how much it costs to
search over things, not sure if that is a useful test.}

\paragraph{Unwinding}
Isolating the unwinding of the stack as much as possible.

This has the same set-up as the raise/handle test except the intermediate
stack frames contain either an object declaration with a destructor or a
@try@ statement with no handlers except and a @finally@ clause.

\paragraph{Enter/Leave}
What is the cost of entering and leaving a try block, even if no exception
is thrown?

The test is a simple matter of entering
and leaving a try statement.

The only tunables here are which clauses are attached to the try block:
termination handlers, resumption handlers and finally clauses.

\paragraph{Re-throw and Conditional-Catch}
How expensive it is to run a non-exception type check for a handler?

In this case different languages approach this problem differently, either
through a re-throw or a conditional-catch.
Where \CFA uses its condition, other languages must unconditionally
catch the exception then re-throw if the condition if the condition is false.

The set up is as follows: main loop, a catch-all exception handler,
a conditional catch and then the raise.

% We could do a Cforall test without the catch all and a new default handler
% that does a catch all.
As a point of comparison, one of the raise/handle tests (which one?) has
same layout but never catches anything.

The main tunable in this test is how often the conditional-catch matches.

%\section{Cost in Size}
%Using exceptions also has a cost in the size of the executable.
%Although it is sometimes ignored
%
%There is a size cost to defining a personality function but the later problem
%is the LSDA which will be generated for every function.
%
%(I haven't actually figured out how to compare this, probably using something
%related to -fexceptions.)


\section{Resumption Comparison}
% Some languages I left out:
% Python: Its a scripting language, different
% uC++: Not well known and should the same results as C++, except for
%   resumption which should be the same.
\todo{Can we find a good language to compare resumptions in.}