Changeset d2cdd4f


Ignore:
Timestamp:
Aug 5, 2021, 2:22:05 PM (4 years ago)
Author:
Thierry Delisle <tdelisle@…>
Branches:
ADT, ast-experimental, enum, forall-pointer-decay, jacob/cs343-translation, master, new-ast-unique-expr, pthread-emulation, qualifiedEnum
Children:
ebf3989
Parents:
0a061c0 (diff), b2525d9 (diff)
Note: this is a merge changeset, the changes displayed below correspond to the merge itself.
Use the (diff) links above to see all the changes relative to each parent.
Message:

Merge branch 'master' of plg.uwaterloo.ca:software/cfa/cfa-cc

Files:
2 edited

Legend:

Unmodified
Added
Removed

  • benchmark/Makefile.am

    r0a061c0 rd2cdd4f  
    7474        echo "int main() { return 0; }" > ${@}
    7575
    76 .SILENT:                # do not print recipe
    7776.NOTPARALLEL:
    7877.PHONY: jenkins cleancsv
    … …  
    116115
    117116%.run : %$(EXEEXT) ${REPEAT}
    118         rm -f .result.log
    119         echo "------------------------------------------------------"
    120         echo $<
    121         ${REPEAT} ${repeats} -- ./a.out\
     117        @rm -f .result.log
     118        @echo "------------------------------------------------------"
     119        @echo $<
     120        @${REPEAT} ${repeats} -- ./a.out\
    122121                $(if ${$(subst -,_,$(basename $@))_DURATION},\
    123122                        ${$(subst -,_,$(basename $@))_DURATION},\
    124123                        ${$(firstword $(subst -, ,$(basename $@)))_DURATION}) | tee -a .result.log
    125         ${STATS} .result.log
    126         echo "------------------------------------------------------"
    127         rm -f a.out .result.log *.class
     124        @${STATS} .result.log
     125        @echo "------------------------------------------------------"
     126        @rm -f a.out .result.log *.class
    128127
    129128#       ${REPEAT} ${repeats} -- /usr/bin/time -f "%Uu %Ss %Er %Mkb" ./a.out
    … …  
    150149jenkins$(EXEEXT): cleancsv
    151150@DOifskipcompile@
    152         +make compile.csv
    153         -+make compile.diff.csv
     151        @+make --silent compile.csv
     152        @-+make --silent compile.diff.csv
    154153@DOendif@
    155         +make ctxswitch.csv
    156         -+make ctxswitch.diff.csv
    157         +make mutex.csv
    158         -+make mutex.diff.csv
    159         +make schedint.csv
    160         -+make schedint.diff.csv
     154        @+make --silent ctxswitch.csv
     155        @-+make --silent ctxswitch.diff.csv
     156        @+make --silent mutex.csv
     157        @-+make --silent mutex.diff.csv
     158        @+make --silent schedint.csv
     159        @-+make --silent schedint.diff.csv
    161160@DOifskipcompile@
    162161        cat compile.csv

  • doc/theses/andrew_beach_MMath/performance.tex

    r0a061c0 rd2cdd4f  
    4242repeatedly. This is to avoids start-up or tear-down time from
    4343affecting the timing results.
    44 Most test were run 1 000 000 (a million) times.
     44Tests ran their main loop a million times.
    4545The Java versions of the test also run this loop an extra 1000 times before
    4646beginning to time the results to ``warm-up" the JVM.
    … …  
    130130
    131131\section{Results}
    132 Each test is was run five times, the best and worst result were discarded and
    133 the remaining values were averaged.
     132Each test was run eleven times. The top three and bottom three results were
     133discarded and the remaining five values are averaged.
    134134
    135135In cases where a feature is not supported by a language the test is skipped
    … …  
    138138was put into the termination column.
    139139
     140% Raw Data:
     141% run-algol-a.sat
     142% ---------------
     143% Raise Empty   &  82687046678 &  291616256 &   3252824847 & 15422937623 & 14736271114 \\
     144% Raise D'tor   & 219933199603 &  297897792 & 223602799362 &         N/A &         N/A \\
     145% Raise Finally & 219703078448 &  298391745 &          N/A &         ... & 18923060958 \\
     146% Raise Other   & 296744104920 & 2854342084 & 112981255103 & 15475924808 & 21293137454 \\
     147% Cross Handler &      9256648 &   13518430 &       769328 &     3486252 &    31790804 \\
     148% Cross Finally &       769319 &        N/A &          N/A &     2272831 &    37491962 \\
     149% Match All     &   3654278402 &   47518560 &   3218907794 &  1296748192 &   624071886 \\
     150% Match None    &   4788861754 &   58418952 &   9458936430 &  1318065020 &   625200906 \\
     151%
     152% run-algol-thr-c
     153% ---------------
     154% Raise Empty   &   3757606400 &   36472972 &   3257803337 & 15439375452 & 14717808642 \\
     155% Raise D'tor   &  64546302019 &  102148375 & 223648121635 &         N/A &         N/A \\
     156% Raise Finally &  64671359172 &  103285005 &          N/A & 15442729458 & 18927008844 \\
     157% Raise Other   & 294143497130 & 2630130385 & 112969055576 & 15448220154 & 21279953424 \\
     158% Cross Handler &      9646462 &   11955668 &       769328 &     3453707 &    31864074 \\
     159% Cross Finally &       773412 &        N/A &          N/A &     2253825 &    37266476 \\
     160% Match All     &   3719462155 &   43294042 &   3223004977 &  1286054154 &   623887874 \\
     161% Match None    &   4971630929 &   55311709 &   9481225467 &  1310251289 &   623752624 \\
    140162\begin{tabular}{|l|c c c c c|}
    141163\hline
    … …  
    152174\hline
    153175\end{tabular}
     176
     177% run-plg7a-a.sat
     178% ---------------
     179% Raise Empty   &  57169011329 &  296612564 &   2788557155 & 17511466039 & 23324548496 \\
     180% Raise D'tor   & 150599858014 &  318443709 & 149651693682 &         N/A &         N/A \\
     181% Raise Finally & 148223145000 &  373325807 &          N/A &         ... & 29074552998 \\
     182% Raise Other   & 189463708732 & 3017109322 &  85819281694 & 17584295487 & 32602686679 \\
     183% Cross Handler &      8001654 &   13584858 &      1555995 &     6626775 &    41927358 \\
     184% Cross Finally &      1002473 &        N/A &          N/A &     4554344 &    51114381 \\
     185% Match All     &   3162460860 &   37315018 &   2649464591 &  1523205769 &   742374509 \\
     186% Match None    &   4054773797 &   47052659 &   7759229131 &  1555373654 &   744656403 \\
     187%
     188% run-plg7a-thr-a
     189% ---------------
     190% Raise Empty   &   3604235388 &   29829965 &   2786931833 & 17576506385 & 23352975105 \\
     191% Raise D'tor   &  46552380948 &  178709605 & 149834207219 &         N/A &         N/A \\
     192% Raise Finally &  46265157775 &  177906320 &          N/A & 17493045092 & 29170962959 \\
     193% Raise Other   & 195659245764 & 2376968982 &  86070431924 & 17552979675 & 32501882918 \\
     194% Cross Handler &    397031776 &   12503552 &      1451225 &     6658628 &    42304965 \\
     195% Cross Finally &      1136746 &        N/A &          N/A &     4468799 &    46155817 \\
     196% Match All     &   3189512499 &   39124453 &   2667795989 &  1525889031 &   733785613 \\
     197% Match None    &   4094675477 &   48749857 &   7850618572 &  1566713577 &   733478963 \\
     198
     199% PLG7A (in seconds)
     200\begin{tabular}{|l|c c c c c|}
     201\hline
     202              & \CFA (Terminate) & \CFA (Resume) & \Cpp & Java & Python \\
     203\hline
     204% Raise Empty   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     205% Raise D'tor   & 0.0 & 0.0 & 0.0 & N/A & N/A \\
     206% Raise Finally & 0.0 & 0.0 & N/A & 0.0 & 0.0 \\
     207% Raise Other   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     208% Cross Handler & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     209% Cross Finally & 0.0 & N/A & N/A & 0.0 & 0.0 \\
     210% Match All     & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     211% Match None    & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     212Raise Empty   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     213Raise D'tor   & 0.0 & 0.0 & 0.0 & N/A & N/A \\
     214Raise Finally & 0.0 & 0.0 & N/A & 0.0 & 0.0 \\
     215Raise Other   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     216Cross Handler & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     217Cross Finally & 0.0 & N/A & N/A & 0.0 & 0.0 \\
     218Match All     & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     219Match None    & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
     220\hline
     221\end{tabular}
     222
     223One result that is not directly related to \CFA but is important to keep in
     224mind is that in exceptions the standard intuitions about which languages
     225should go faster often do not hold. There are cases where Python out-preforms
     226\Cpp and Java. The most likely explination is that, since exceptions are
     227rarely considered to be the common case, the more optimized langages have
     228optimized at their expence. In addition languages with high level           
     229repersentations have a much easier time scanning the stack as there is less
     230to decode.
     231
     232This means that while \CFA does not actually keep up with Python in every
     233case it is no worse than roughly half the speed of \Cpp. This is good
     234enough for the prototyping purposes of the project.
     235
     236One difference not shown is that optimizations in \CFA is very fragile.
     237The \CFA compiler uses gcc as part of its complation process and the version
     238of gcc could change the speed of some of the benchmarks by 10 times or more.
     239Similar changes to g++ for the \Cpp benchmarks had no significant changes.
     240Because of the connection between gcc and g++; this suggests it is not the
     241optimizations that are changing but how the optimizer is detecting if the
     242optimizations can be applied. So the optimizations are always applied in
     243g++, but only newer versions of gcc can detect that they can be applied in
     244the more complex \CFA code.
     245
     246Resumption exception handling is also incredibly fast. Often an order of
     247magnitude or two better than the best termination speed.
     248There is a simple explination for this; traversing a linked list is much   
     249faster than examining and unwinding the stack. When resumption does not do as
     250well its when more try statements are used per raise. Updating the interal
     251linked list is not very expencive but it does add up.
     252
     253The relative speed of the Match All and Match None tests (within each
     254language) can also show the effectiveness conditional matching as compared
     255to catch and rethrow.
     256\begin{itemize}[nosep]
     257\item
     258Java and Python get similar values in both tests.
     259Between the interperated code, a higher level repersentation of the call
     260stack and exception reuse it it is possible the cost for a second
     261throw can be folded into the first.
     262% Is this due to optimization?
     263\item
     264Both types of \CFA are slighly slower if there is not a match.
     265For termination this likely comes from unwinding a bit more stack through
     266libunwind instead of executing the code normally.
     267For resumption there is extra work in traversing more of the list and running
     268more checks for a matching exceptions.
     269% Resumption is a bit high for that but this is my best theory.
     270\item
     271Then there is \Cpp, which takes 2--3 times longer to catch and rethrow vs.
     272just the catch. This is very high, but it does have to repeat the same
     273process of unwinding the stack and may have to parse the LSDA of the function
     274with the catch and rethrow twice, once before the catch and once after the
     275rethrow.
     276% I spent a long time thinking of what could push it over twice, this is all
     277% I have to explain it.
     278\end{itemize}
     279The difference in relative performance does show that there are savings to
     280be made by performing the check without catching the exception.
Note: See TracChangeset for help on using the changeset viewer.