Ignore:
Timestamp:
Aug 9, 2021, 4:11:10 PM (3 years ago)
Author:
Andrew Beach <ajbeach@…>
Branches:
ADT, ast-experimental, enum, forall-pointer-decay, jacob/cs343-translation, master, new-ast-unique-expr, pthread-emulation, qualifiedEnum
Children:
50b29d9, f42a6b8
Parents:
9373b6a
Message:

Andrew MMath: Had some bad performance numbers, updated the description as well.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • doc/theses/andrew_beach_MMath/performance.tex

    r9373b6a r5438e41  
    160160% Match All     &   3719462155 &   43294042 &   3223004977 &  1286054154 &   623887874 \\
    161161% Match None    &   4971630929 &   55311709 &   9481225467 &  1310251289 &   623752624 \\
     162%
     163% run-algol-04-a
     164% --------------
     165% Raise Empty   & 0.0 & 0.0 &  3250260945 & 0.0 & 0.0 \\
     166% Raise D'tor   & 0.0 & 0.0 & 29017675113 & N/A & N/A \\
     167% Raise Finally & 0.0 & 0.0 &         N/A & 0.0 & 0.0 \\
     168% Raise Other   & 0.0 & 0.0 & 24411823773 & 0.0 & 0.0 \\
     169% Cross Handler & 0.0 & 0.0 &      769334 & 0.0 & 0.0 \\
     170% Cross Finally & 0.0 & N/A &         N/A & 0.0 & 0.0 \\
     171% Match All     & 0.0 & 0.0 &  3254283504 & 0.0 & 0.0 \\
     172% Match None    & 0.0 & 0.0 &  9476060146 & 0.0 & 0.0 \\
     173
    162174\begin{tabular}{|l|c c c c c|}
    163175\hline
     
    196208% Match All     &   3189512499 &   39124453 &   2667795989 &  1525889031 &   733785613 \\
    197209% Match None    &   4094675477 &   48749857 &   7850618572 &  1566713577 &   733478963 \\
     210%
     211% run-plg7a-04-a
     212% --------------
     213% 0.0 are unfilled.
     214% Raise Empty   & 0.0 & 0.0 &  2770781479 & 0.0 & 0.0 \\
     215% Raise D'tor   & 0.0 & 0.0 & 23530084907 & N/A & N/A \\
     216% Raise Finally & 0.0 & 0.0 &         N/A & 0.0 & 0.0 \\
     217% Raise Other   & 0.0 & 0.0 & 23816827982 & 0.0 & 0.0 \\
     218% Cross Handler & 0.0 & 0.0 &     1422188 & 0.0 & 0.0 \\
     219% Cross Finally & 0.0 & N/A &         N/A & 0.0 & 0.0 \\
     220% Match All     & 0.0 & 0.0 &  2671989778 & 0.0 & 0.0 \\
     221% Match None    & 0.0 & 0.0 &  7829059869 & 0.0 & 0.0 \\
    198222
    199223% PLG7A (in seconds)
     
    202226              & \CFA (Terminate) & \CFA (Resume) & \Cpp & Java & Python \\
    203227\hline
    204 % Raise Empty   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    205 % Raise D'tor   & 0.0 & 0.0 & 0.0 & N/A & N/A \\
    206 % Raise Finally & 0.0 & 0.0 & N/A & 0.0 & 0.0 \\
    207 % Raise Other   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    208 % Cross Handler & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    209 % Cross Finally & 0.0 & N/A & N/A & 0.0 & 0.0 \\
    210 % Match All     & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    211 % Match None    & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    212228Raise Empty   & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    213229Raise D'tor   & 0.0 & 0.0 & 0.0 & N/A & N/A \\
     
    231247
    232248This means that while \CFA does not actually keep up with Python in every
    233 case it is no worse than roughly half the speed of \Cpp. This is good
     249case it is usually no worse than roughly half the speed of \Cpp. This is good
    234250enough for the prototyping purposes of the project.
    235251
    236 One difference not shown is that optimizations in \CFA is very fragile.
    237 The \CFA compiler uses gcc as part of its complation process and the version
    238 of gcc could change the speed of some of the benchmarks by 10 times or more.
    239 Similar changes to g++ for the \Cpp benchmarks had no significant changes.
    240 Because of the connection between gcc and g++; this suggests it is not the
    241 optimizations that are changing but how the optimizer is detecting if the
    242 optimizations can be applied. So the optimizations are always applied in
    243 g++, but only newer versions of gcc can detect that they can be applied in
    244 the more complex \CFA code.
     252The test case where \CFA falls short is Raise Other, the case where the
     253stack is unwound including a bunch of non-matching handlers.
     254This slowdown seems to come from missing optimizations,
     255the results above came from gcc/g++ 10 (gcc as \CFA backend or g++ for \Cpp)
     256but the results change if they are run in gcc/g++ 9 instead.
     257Importantly, there is a huge slowdown in \Cpp's results bringing that brings
     258\CFA's performace back in that roughly half speed area. However many other
     259\CFA benchmarks increase their run-time by a similar amount falling far
     260behind their \Cpp counter-parts.
     261
     262This suggests that the performance issue in Raise Other is just an
     263optimization not being applied. Later versions of gcc may be able to
     264optimize this case further, at least down to the half of \Cpp mark.
     265A \CFA compiler that directly produced assembly could do even better as it
     266would not have to work across some of \CFA's current abstractions, like
     267the try terminate function.
    245268
    246269Resumption exception handling is also incredibly fast. Often an order of
Note: See TracChangeset for help on using the changeset viewer.