Jul 21, 2021, 2:55:01 PM (2 years ago)
Andrew Beach <ajbeach@…>
ADT, ast-experimental, enum, forall-pointer-decay, jacob/cs343-translation, master, new-ast-unique-expr, pthread-emulation, qualifiedEnum

Andrew MMath: A bit more work on the performance chapter before I stick in the results.

1 edited


  • doc/theses/andrew_beach_MMath/performance.tex

    r2ead704 r9698690  
    4 \textbf{Just because of the stage of testing there are design notes for
    5 the tests as well as commentary on them.}
    74Performance has been of secondary importance for most of this project.
    129\section{Test Set-Up}
    13 Tests will be run on \CFA, C++ and Java.
     10Tests will be run in \CFA, C++, Java and Python.
     11In addition there are two sets of tests for \CFA,
     12one for termination exceptions and once with resumption exceptions.
    1514C++ is the most comparable language because both it and \CFA use the same
    1817comparison. \CFA's EHM has had significantly less time to be optimized and
    1918does not generate its own assembly. It does have a slight advantage in that
    20 there are some features it does not handle.
     19there are some features it does not handle, through utility functions,
     20but otherwise \Cpp has a significant advantage.
    2222Java is another very popular language with similar termination semantics.
    2525It also implements the finally clause on try blocks allowing for a direct
    2626feature-to-feature comparison.
     27As with \Cpp, Java's implementation is more mature, has more optimizations
     28and more extra features.
     30Python was used as a point of comparison because of the \CFA EHM's
     31current performance goals, which is not be prohibitively slow while the
     32features are designed and examined. Python has similar performance goals for
     33creating quick scripts and its wide use suggests it has achieved those goals.
     35Unfortunately there are no notable modern programming languages with
     36resumption exceptions. Even the older programming languages with resumptions
     37seem to be notable only for having resumptions.
     38So instead resumptions are compared to a less similar but much more familiar
     39feature, termination exceptions.
    2841All tests are run inside a main loop which will perform the test
    2942repeatedly. This is to avoids start-up or tear-down time from
    3043affecting the timing results.
    31 A consequence of this is that tests cannot terminate the program,
    32 which does limit how tests can be implemented.
    33 There are catch-alls to keep unhandled
    34 exceptions from terminating tests.
     44Most test were run 1 000 000 (a million) times.
     45The Java versions of the test also run this loop an extra 1000 times before
     46beginning to time the results to ``warm-up" the JVM.
     48Timing is done internally, with time measured immediately before and
     49immediately after the test loop. The difference is calculated and printed.
     51The loop structure and internal timing means it is impossible to test
     52unhandled exceptions in \Cpp and Java as that would cause the process to
     54Luckily, performance on the ``give-up and kill the process" path is not
    3657The exceptions used in these tests will always be a exception based off of
    3758the base exception. This requirement minimizes performance differences based
    38 on the object model.
    39 Catch-alls are done by catching the root exception type (not using \Cpp's
    40 \code{C++}{catch(...)}).
     59on the object model used to repersent the exception.
    42 Tests run in Java were not warmed because exception code paths should not be
    43 hot.
     61All tests were designed to be as minimal as possible while still preventing
     62exessive optimizations.
     63For example, empty inline assembly blocks are used in \CFA and \Cpp to
     64prevent excessive optimizations while adding no actual work.
     66% We don't use catch-alls but if we did:
     67% Catch-alls are done by catching the root exception type (not using \Cpp's
     68% \code{C++}{catch(...)}).
    4772components of the exception system.
    4873The should provide a guide as to where the EHM's costs can be found.
    50 Tests are run in \CFA, \Cpp and Java.
    51 Not every test is run in every language, if the feature under test is missing
    52 the test is skipped. These cases will be noted.
    53 In addition to the termination tests for every language,
    54 \CFA has a second set of tests that test resumption. These are the same
    55 except that the raise statements and handler clauses are replaced with the
    56 resumption variants.
    5875\paragraph{Raise and Handle}
    6279start-up and shutdown on the results.
    6380Each iteration of the main loop
    64 \begin{itemize}
    6582\item Empty Function:
    6683The repeating function is empty except for the necessary control code.
    6885The repeating function creates an object with a destructor before calling
    70 (Java is skipped as it does not destructors.)
    7187\item Finally:
    7288The repeating function calls itself inside a try block with a finally clause
    74 (\Cpp is skipped as it does not have finally clauses.)
    7590\item Other Handler:
    7691The repeating function calls itself inside a try block with a handler that
    8499In each iteration, a try statement is executed. Entering and leaving a loop
    85100is all the test wants to do.
    86 \begin{itemize}
    87102\item Handler:
    88103The try statement has a handler (of the matching kind).
    95110Only \CFA implements the language level conditional match,
    96111the other languages must mimic with an ``unconditional" match (it still
    97 checks the exception's type) and conditional re-raise.
    98 \begin{itemize}
    99 \item Catch All:
     112checks the exception's type) and conditional re-raise if it was not supposed
     113to handle that exception.
     115\item Match All:
    100116The condition is always true. (Always matches or never re-raises.)
    101 \item Catch None:
     117\item Match None:
    102118The condition is always false. (Never matches or always re-raises.)
    113129%related to -fexceptions.)
    115 % Some languages I left out:
    116 % Python: Its a scripting language, different
    117 % uC++: Not well known and should the same results as C++, except for
    118 %   resumption which should be the same.
     132Each test is was run five times, the best and worst result were discarded and
     133the remaining values were averaged.
    120 %\section{Resumption Comparison}
    121 \todo{Can we find a good language to compare resumptions in.}
     135In cases where a feature is not supported by a language the test is skipped
     136for that language. Similarly, if a test is does not change between resumption
     137and termination in \CFA, then only one test is written and the result
     138was put into the termination column.
     140\begin{tabular}{|l|c c c c c|}
     142              & \CFA (Terminate) & \CFA (Resume) & \Cpp & Java & Python \\
     144Raise Empty   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     145Raise D'tor   & 0.0 & 0.0 & 0.0 & N/A & N/A \\
     146Raise Finally & 0.0 & 0.0 & N/A & 0.0 & 0.0 \\
     147Raise Other   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     148Cross Handler & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     149Cross Finally & 0.0 & N/A & N/A & 0.0 & 0.0 \\
     150Match All     & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     151Match None    & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
Note: See TracChangeset for help on using the changeset viewer.