Context Navigation

-                      r2ead704
+                      r9698690
 \chapter{Performance}
 \label{c:performance}
-\textbf{Just because of the stage of testing there are design notes for
-the tests as well as commentary on them.}
 Performance has been of secondary importance for most of this project.
 …
 \section{Test Set-Up}
+Tests will be run on \CFA, C++ and Java.
+Tests will be run in \CFA, C++, Java and Python.
+In addition there are two sets of tests for \CFA,
+one for termination exceptions and once with resumption exceptions.
 C++ is the most comparable language because both it and \CFA use the same
 …
 comparison. \CFA's EHM has had significantly less time to be optimized and
 does not generate its own assembly. It does have a slight advantage in that
+there are some features it does not handle.
+there are some features it does not handle, through utility functions,
+but otherwise \Cpp has a significant advantage.
 Java is another very popular language with similar termination semantics.
 …
 It also implements the finally clause on try blocks allowing for a direct
 feature-to-feature comparison.
+As with \Cpp, Java's implementation is more mature, has more optimizations
+and more extra features.
+Python was used as a point of comparison because of the \CFA EHM's
+current performance goals, which is not be prohibitively slow while the
+features are designed and examined. Python has similar performance goals for
+creating quick scripts and its wide use suggests it has achieved those goals.
+Unfortunately there are no notable modern programming languages with
+resumption exceptions. Even the older programming languages with resumptions
+seem to be notable only for having resumptions.
+So instead resumptions are compared to a less similar but much more familiar
+feature, termination exceptions.
 All tests are run inside a main loop which will perform the test
 repeatedly. This is to avoids start-up or tear-down time from
 affecting the timing results.
+A consequence of this is that tests cannot terminate the program,
+which does limit how tests can be implemented.
+There are catch-alls to keep unhandled
+exceptions from terminating tests.
+Most test were run 1 000 000 (a million) times.
+The Java versions of the test also run this loop an extra 1000 times before
+beginning to time the results to ``warm-up" the JVM.
+Timing is done internally, with time measured immediately before and
+immediately after the test loop. The difference is calculated and printed.
+The loop structure and internal timing means it is impossible to test
+unhandled exceptions in \Cpp and Java as that would cause the process to
+terminate.
+Luckily, performance on the ``give-up and kill the process" path is not
+critical.
 The exceptions used in these tests will always be a exception based off of
 the base exception. This requirement minimizes performance differences based
+on the object model.
+Catch-alls are done by catching the root exception type (not using \Cpp's
+\code{C++}{catch(...)}).
+on the object model used to repersent the exception.
+Tests run in Java were not warmed because exception code paths should not be
+hot.
+All tests were designed to be as minimal as possible while still preventing
+exessive optimizations.
+For example, empty inline assembly blocks are used in \CFA and \Cpp to
+prevent excessive optimizations while adding no actual work.
+% We don't use catch-alls but if we did:
+% Catch-alls are done by catching the root exception type (not using \Cpp's
+% \code{C++}{catch(...)}).
 \section{Tests}
 …
 components of the exception system.
 The should provide a guide as to where the EHM's costs can be found.
-Tests are run in \CFA, \Cpp and Java.
-Not every test is run in every language, if the feature under test is missing
-the test is skipped. These cases will be noted.
-In addition to the termination tests for every language,
-\CFA has a second set of tests that test resumption. These are the same
-except that the raise statements and handler clauses are replaced with the
-resumption variants.
 \paragraph{Raise and Handle}
 …
 start-up and shutdown on the results.
 Each iteration of the main loop
 \begin{itemize}
+\begin{itemize}[nosep]
 \item Empty Function:
 The repeating function is empty except for the necessary control code.
 …
 The repeating function creates an object with a destructor before calling
 itself.
-(Java is skipped as it does not destructors.)
 \item Finally:
 The repeating function calls itself inside a try block with a finally clause
 attached.
-(\Cpp is skipped as it does not have finally clauses.)
 \item Other Handler:
 The repeating function calls itself inside a try block with a handler that
 …
 In each iteration, a try statement is executed. Entering and leaving a loop
 is all the test wants to do.
 \begin{itemize}
+\begin{itemize}[nosep]
 \item Handler:
 The try statement has a handler (of the matching kind).
 …
 Only \CFA implements the language level conditional match,
 the other languages must mimic with an ``unconditional" match (it still
+checks the exception's type) and conditional re-raise.
+\begin{itemize}
+\item Catch All:
+checks the exception's type) and conditional re-raise if it was not supposed
+to handle that exception.
+\begin{itemize}[nosep]
+\item Match All:
 The condition is always true. (Always matches or never re-raises.)
 \item Catch None:
+\item Match None:
 The condition is always false. (Never matches or always re-raises.)
 \end{itemize}
 …
 %related to -fexceptions.)
+% Some languages I left out:
+% Python: Its a scripting language, different
+% uC++: Not well known and should the same results as C++, except for
+%   resumption which should be the same.
+\section{Results}
+Each test is was run five times, the best and worst result were discarded and
+the remaining values were averaged.
+%\section{Resumption Comparison}
+\todo{Can we find a good language to compare resumptions in.}
+In cases where a feature is not supported by a language the test is skipped
+for that language. Similarly, if a test is does not change between resumption
+and termination in \CFA, then only one test is written and the result
+was put into the termination column.
+\begin{tabular}{|l|c c c c c|}
+\hline
+              & \CFA (Terminate) & \CFA (Resume) & \Cpp & Java & Python \\
+\hline
+Raise Empty   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
+Raise D'tor   & 0.0 & 0.0 & 0.0 & N/A & N/A \\
+Raise Finally & 0.0 & 0.0 & N/A & 0.0 & 0.0 \\
+Raise Other   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
+Cross Handler & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
+Cross Finally & 0.0 & N/A & N/A & 0.0 & 0.0 \\
+Match All     & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
+Match None    & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
+\hline
+\end{tabular}

Note: See TracChangeset for help on using the changeset viewer.

Context Navigation

Changeset 9698690

Legend:

doc/theses/andrew_beach_MMath/performance.tex

Download in other formats: