source: doc/theses/andrew_beach_MMath/performance.tex @ ea593a3

Last change on this file since ea593a3 was ea593a3, checked in by Andrew Beach <ajbeach@…>, 2 years ago

First draft of all the exception benchmarks. There is an issue with the Cforall linking.

  • Property mode set to 100644
File size: 4.8 KB
4\textbf{Just because of the stage of testing there are design notes for
5the tests as well as commentary on them.}
7Performance has been of secondary importance for most of this project.
8Instead, the focus has been to get the features working. The only performance
9requirements is to ensure the tests for correctness run in a reasonable
10amount of time.
12\section{Test Set-Up}
13Tests will be run on \CFA, C++ and Java.
15C++ is the most comparable language because both it and \CFA use the same
16framework, libunwind.
17In fact, the comparison is almost entirely a quality of implementation
18comparison. \CFA's EHM has had significantly less time to be optimized and
19does not generate its own assembly. It does have a slight advantage in that
20there are some features it does not handle.
22Java is another very popular language with similar termination semantics.
23It is implemented in a very different environment, a virtual machine with
24garbage collection.
25It also implements the finally clause on try blocks allowing for a direct
26feature-to-feature comparison.
28All tests are run inside a main loop which will perform the test
29repeatedly. This is to avoids start-up or tear-down time from
30affecting the timing results.
31A consequence of this is that tests cannot terminate the program,
32which does limit how tests can be implemented.
33There are catch-alls to keep unhandled
34exceptions from terminating tests.
36The exceptions used in these tests will always be a exception based off of
37the base exception. This requirement minimizes performance differences based
38on the object model.
39Catch-alls are done by catching the root exception type (not using \Cpp's
42Tests run in Java were not warmed because exception code paths should not be
46The following tests were selected to test the performance of different
47components of the exception system.
48The should provide a guide as to where the EHM's costs can be found.
50Tests are run in \CFA, \Cpp and Java.
51Not every test is run in every language, if the feature under test is missing
52the test is skipped. These cases will be noted.
53In addition to the termination tests for every language,
54\CFA has a second set of tests that test resumption. These are the same
55except that the raise statements and handler clauses are replaced with the
56resumption variants.
58\paragraph{Raise and Handle}
59The first group of tests involve setting up
60So there is three layers to the test. The first is set up and a loop, which
61configures the test and then runs it repeatedly to reduce the impact of
62start-up and shutdown on the results.
63Each iteration of the main loop
65\item Empty Function:
66The repeating function is empty except for the necessary control code.
67\item Destructor:
68The repeating function creates an object with a destructor before calling
70(Java is skipped as it does not destructors.)
71\item Finally:
72The repeating function calls itself inside a try block with a finally clause
74(\Cpp is skipped as it does not have finally clauses.)
75\item Other Handler:
76The repeating function calls itself inside a try block with a handler that
77will not match the raised exception. (But is of the same kind of handler.)
80\paragraph{Cross Try Statement}
81The next group measures the cost of a try statement when no exceptions are
82raised. The test is set-up, then there is a loop to reduce the impact of
83start-up and shutdown on the results.
84In each iteration, a try statement is executed. Entering and leaving a loop
85is all the test wants to do.
87\item Handler:
88The try statement has a handler (of the matching kind).
89\item Finally:
90The try statement has a finally clause.
93\paragraph{Conditional Matching}
94This group of tests checks the cost of conditional matching.
95Only \CFA implements the language level conditional match,
96the other languages must mimic with an ``unconditional" match (it still
97checks the exception's type) and conditional re-raise.
99\item Catch All:
100The condition is always true. (Always matches or never re-raises.)
101\item Catch None:
102The condition is always false. (Never matches or always re-raises.)
105%\section{Cost in Size}
106%Using exceptions also has a cost in the size of the executable.
107%Although it is sometimes ignored
109%There is a size cost to defining a personality function but the later problem
110%is the LSDA which will be generated for every function.
112%(I haven't actually figured out how to compare this, probably using something
113%related to -fexceptions.)
115% Some languages I left out:
116% Python: Its a scripting language, different
117% uC++: Not well known and should the same results as C++, except for
118%   resumption which should be the same.
120%\section{Resumption Comparison}
121\todo{Can we find a good language to compare resumptions in.}
Note: See TracBrowser for help on using the repository browser.