source: doc/theses/mike_brooks_MMath/benchmarks/list/peter-summary.txt

Last change on this file was 2b0e754, checked in by Michael Brooks <mlbrooks@…>, 2 months ago

Continue disabling harness features, achieving best-case times that match Peter's, on the three tests with new CSVs here.

The fully-disabled harness probably still fools the optimizer because it uses the Buhr-Dice "pass function" trick in place of the disabled observation and iterators array. That these times are now often beating Peter's requires further investigation.

Make STL run with (cpp-sltref) and without (cpp-stlrefAlloGlib) llheap.
Include peter-*.csv from runs on Swift. (Thesis still builds with original numbers from labpc.)
Add data post-processing for sorting and aggregation, leveraging thesis's plots' framework.

Add makefile-saved setups for matching Peter.
Add ability to disable "observation" (use of volatiles) and iterators array (support for remelem).
Add integrated clean-build-run targets for varying the "disable" switches.
Strengthen disable-interleaving mode to avoid the extra loops on remove.

Still to entertain some of this stripping being unnecessary for achieving near Peter-match.
Still to compare costs of the relevant stripped features, and recommend harness variation(s) to for use in thesis.
  • Property mode set to 100644
File size: 18.7 KB
Line 
1#size mean std min max ntrials ignore ignore
2
3-------------- results-peter-linear.csv ----------------
4"cfa-mandHead"
54 3.340529666666667 0.008761095384330123 3.33278 3.350036 3 10.021589 3.338773000000001
68 4.505142666666667 0.01860111115856651 4.490174 4.525967 3 13.515428 4.499287000000001
716 4.294048666666666 0.22011393951618174 4.041588 4.445732 3 12.882145999999999 4.394825999999998
832 2.5542206666666667 0.006641025322443421 2.546974 2.560016 3 7.662662 2.5556720000000004
964 2.501508 0.003747215366108648 2.499122 2.505827 3 7.504524 2.499575
10128 2.692693333333333 0.0015901529276560872 2.691421 2.694476 3 8.07808 2.692183
11256 2.525259 0.0009889089948016709 2.524388 2.526334 3 7.575777 2.5250550000000005
12512 2.4427886666666665 0.0004827632269894966 2.442255 2.443195 3 7.328365999999999 2.442916
131024 2.478586 0.06422890664646261 2.441373 2.552751 3 7.435758 2.441633999999999
142048 2.5016516666666666 0.020388664726590938 2.489061 2.525175 3 7.504955 2.4907189999999995
154096 2.552877666666667 0.013081143731850474 2.538889 2.564807 3 7.658633 2.554937
168192 2.574939666666667 0.015278738342328485 2.559341 2.589877 3 7.724819 2.5756010000000003
1716384 2.6037713333333334 0.01923444577660948 2.584864 2.623317 3 7.811314 2.6031329999999997
1832768 2.6426916666666664 0.005081165647106241 2.6396 2.648556 3 7.928075 2.639919
1965536 2.623124666666667 0.00870073297678601 2.615657 2.632679 3 7.8693740000000005 2.621038
20131072 2.625349 0.005596630325472773 2.618943 2.62929 3 7.876047 2.627814
21262144 2.6255913333333334 0.0020117617486505534 2.623352 2.627246 3 7.876774 2.6261760000000005
22524288 2.6356289999999998 0.005655197962936275 2.630649 2.641777 3 7.906886999999999 2.6344609999999995
231048576 2.6578273333333335 0.001038507262051355 2.657162 2.659024 3 7.973482000000001 2.657296000000001
242097152 2.6627096666666668 0.00045676069591577825 2.662307 2.663206 3 7.988129000000001 2.6626160000000003
254194304 2.6638876666666667 0.0025938576162412874 2.661239 2.666423 3 7.991663 2.664001
268388608 2.663504666666667 0.0004014789326144097 2.663044 2.66378 3 7.990514 2.66369
2716777216 2.665862666666667 0.0012121589554732152 2.664776 2.66717 3 7.997588 2.6656420000000005
28
29
30
31"cpp-stlref"
324 16.276131333333336 0.5494111256111703 15.849802 16.896156 3 48.828394 16.082436000000005
338 16.028297 0.688629419780333 15.54007 16.815951 3 48.084891 15.72887
3416 16.370842666666665 0.31107334721948404 16.11476 16.717019 3 49.112528 16.280748999999993
3532 16.089928666666665 0.4089804973618348 15.628542 16.407839 3 48.269785999999996 16.233405
3664 15.862842666666666 0.3678171071950493 15.597278 16.282672 3 47.588528 15.708577999999996
37128 15.657848333333334 0.07779775566385744 15.608806 15.747551 3 46.973545 15.617188
38256 15.951337333333333 0.05863847400242824 15.896764 16.013334 3 47.854012 15.943914
39512 15.743774333333334 0.09084123824196441 15.661153 15.841053 3 47.231323 15.729117000000004
401024 16.101161 0.5699947883191555 15.7544 16.759012 3 48.303483 15.790070999999998
412048 16.154151666666667 0.6814711520250398 15.647362 16.928868 3 48.462455 15.886224999999996
424096 15.753359000000001 0.1007028054673754 15.671371 15.865764 3 47.260077 15.722942000000002
438192 15.594199333333334 0.012105043838554857 15.584895 15.607885 3 46.782598 15.589818000000001
4416384 15.607441333333334 0.010023201201878135 15.596045 15.614888 3 46.822324 15.611391000000001
4532768 15.657772666666666 0.011978340132645015 15.650821 15.671604 3 46.973318 15.650892999999998
4665536 16.302417666666667 0.3955774443422339 16.027306 16.755753 3 48.907253 16.124194000000003
47131072 16.484600666666665 0.497479630407451 16.195867 17.059038 3 49.453801999999996 16.198896999999995
48262144 16.15355 0.11938578762566385 16.039748 16.277829 3 48.46065 16.143072999999998
49524288 16.720857 0.4053741552306945 16.252952 16.966074 3 50.162571 16.943545
501048576 17.611560666666666 0.06862991368443752 17.535111 17.667859 3 52.834682 17.631712
512097152 17.568843333333334 0.5502932928233218 16.939517 17.959559 3 52.70653 17.807454000000007
524194304 18.17669 0.26978990868636926 17.876003 18.397582 3 54.53007 18.256484999999998
538388608 18.706492666666666 0.2003463984311512 18.480551 18.862492 3 56.119478 18.776435000000003
5416777216 19.716534333333332 0.13481877965006636 19.615948 19.869725 3 59.149603 19.663929999999997
55
56
57
58"cpp-stlrefAllocGlib"
594 19.224453333333333 0.44544137306084813 18.819225 19.701404 3 57.67336 19.152731
608 20.083212666666668 0.3074090156653381 19.818401 20.420329 3 60.249638000000004 20.010908000000004
6116 19.940246 0.21710925006318896 19.697487 20.115824 3 59.820738 20.007426999999996
6232 20.20882333333333 0.16411592010019274 20.096762 20.397201 3 60.62647 20.132507
6364 20.112860333333334 0.26804319986213776 19.956445 20.422364 3 60.338581000000005 19.959772
64128 21.207455666666664 0.6801519098365703 20.482704 21.831858 3 63.622367 21.307805
65256 20.468528333333335 0.17266237070170748 20.279176 20.617256 3 61.405585 20.509153
66512 20.840860666666668 0.7295546086965193 20.401091 21.683001 3 62.522582 20.438489999999998
671024 20.861529666666666 0.7630444147868014 20.334591 21.736545 3 62.584588999999994 20.51345299999999
682048 20.47649733333333 0.05366503078666292 20.421794 20.52906 3 61.429491999999996 20.478637999999997
694096 20.483762 0.027744203304474305 20.455471 20.510925 3 61.451285999999996 20.484889999999993
708192 20.460311 0.015628581925433916 20.44675 20.477403 3 61.380933 20.45678
7116384 20.467579 0.01575372251247119 20.453471 20.484578 3 61.402737 20.464688000000002
7232768 21.103698333333334 0.06190962175892705 21.049569 21.171202 3 63.311095 21.090324000000003
7365536 21.22763433333333 0.1595218077766575 21.12116 21.411043 3 63.682902999999996 21.150699999999993
74131072 21.656289666666666 0.2818656622618887 21.400088 21.958226 3 64.968869 21.610555
75262144 21.395606333333333 0.37086180086028425 20.993423 21.724075 3 64.186819 21.469321
76524288 21.980743333333333 0.26432120025516814 21.77704 22.279431 3 65.94223 21.885758999999997
771048576 22.90828433333333 0.28435203453172697 22.591719 23.142038 3 68.724853 22.991096
782097152 23.702545666666666 0.5537059649103804 23.145829 24.253191 3 71.107637 23.708616999999997
794194304 25.14661866666667 0.1268405781693439 25.02952 25.281355 3 75.439856 25.128981
808388608 25.77380033333333 0.07199834949728631 25.721839 25.855984 3 77.321401 25.743577999999992
8116777216 27.38273433333333 0.15407887582771798 27.214544 27.517072 3 82.148203 27.416586999999993
82
83
84
85-------------- results-peter-random2.csv ----------------
86"cfa-mandHead"
874 3.8446580000000004 0.04442244751249069 3.811747 3.895187 3 11.533974 3.82704
888 4.875527666666667 0.010263203025046633 4.866984 4.886912 3 14.626583 4.872687
8916 4.160045666666666 0.23566209183984865 3.955342 4.417668 3 12.480137 4.107126999999999
9032 2.599097 0.008299745538268044 2.589913 2.606061 3 7.7972909999999995 2.6013169999999994
9164 2.583352 0.15829297938000925 2.479641 2.765552 3 7.750056 2.504863
92128 2.594545333333333 0.010607974468923463 2.587485 2.606744 3 7.7836359999999996 2.5894069999999996
93256 2.5687273333333334 0.05500095680379863 2.534304 2.63216 3 7.706182 2.5397179999999997
94512 2.4483833333333336 0.013572043852468742 2.432845 2.457919 3 7.34515 2.454386
951024 2.4539416666666667 0.006324427431264631 2.450027 2.461238 3 7.361825 2.4505599999999994
962048 3.9189399999999996 0.1666979599065326 3.807611 4.110592 3 11.75682 3.838617
974096 4.991256333333333 0.03926631001677352 4.96377 5.036228 3 14.973769 4.973771
988192 5.652936 0.1771160933088803 5.449181 5.770079 3 16.958808 5.739548000000002
9916384 6.6726719999999995 0.18459916363840873 6.532732 6.881888 3 20.018016 6.603396
10032768 8.741499666666668 0.14506381014689163 8.633212 8.906318 3 26.224499 8.684969
10165536 13.195417666666666 0.12780091720067346 13.073379 13.328291 3 39.586253 13.184583
102131072 17.858402333333334 0.05435147556721232 17.808503 17.916316 3 53.575207 17.850388
103262144 21.103452333333333 0.0814247657002044 21.048639 21.197015 3 63.310357 21.064703
104524288 28.313810999999998 0.8406181384380201 27.35316 28.914557 3 84.94143299999999 28.673715999999988
1051048576 71.38568766666667 0.6014280748271877 70.824693 72.020698 3 214.157063 71.311672
1062097152 106.33540133333334 0.789487415632025 105.711664 107.223034 3 319.006204 106.07150600000004
1074194304 123.86176333333333 0.8874223864599892 123.047113 124.807398 3 371.58529 123.73077899999998
1088388608 135.366453 0.21509593732100057 135.150711 135.580897 3 406.099359 135.367751
10916777216 143.476079 0.20722226799018506 143.293858 143.701494 3 430.42823699999997 143.43288499999997
110
111
112
113"cpp-stlref"
1144 15.915370666666668 0.07833789890808569 15.84974 16.002096 3 47.746112000000004 15.894276000000001
1158 16.032191 0.13848856093916237 15.88888 16.165292 3 48.09657300000001 16.042401000000005
11616 16.196648999999997 0.1607113816722393 16.037612 16.358984 3 48.589946999999995 16.193351
11732 16.109795000000002 0.12627428116208128 15.987786 16.239941 3 48.329385 16.101658
11864 16.047202000000002 0.13967066195518624 15.935047 16.203648 3 48.141606 16.002910999999997
119128 16.143997666666667 0.26743045952982564 15.920017 16.44009 3 48.431993 16.071885999999996
120256 16.22089766666667 0.061915584842051924 16.17775 16.29184 3 48.662693000000004 16.193103
121512 16.030787 0.08936869791487559 15.937505 16.115646 3 48.092361000000004 16.03921
1221024 16.067847999999998 0.055631530870542435 16.024254 16.130505 3 48.203543999999994 16.048784999999995
1232048 15.961977666666668 0.034348125979933204 15.922576 15.985606 3 47.885933 15.977751000000001
1244096 15.975152333333332 0.06497286587748251 15.90946 16.039381 3 47.925456999999994 15.976616
1258192 16.005367 0.1504008899042818 15.917287 16.179029 3 48.016101 15.919784999999997
12616384 16.042678333333335 0.18560050480624657 15.919384 16.256136 3 48.128035000000004 15.952515000000002
12732768 15.970481999999999 0.02251440820896761 15.948655 15.993626 3 47.911446 15.969164999999995
12865536 15.963838 0.004959809371336667 15.958378 15.968065 3 47.891514 15.965071000000002
129131072 16.159582999999998 0.11400060573523164 16.079157 16.290045 3 48.47874899999999 16.109547
130262144 16.882718333333333 0.5000908546717612 16.307476 17.214078 3 50.648155 17.126601
131524288 16.3538 0.3247682727253382 15.980275 16.569437 3 49.0614 16.511688
1321048576 16.870553 0.7176977237723418 16.249377 17.656215 3 50.611659 16.706067000000008
1332097152 17.572880333333334 0.3976848388389145 17.31286 18.030679 3 52.718641 17.375102
1344194304 18.10557633333333 0.029533561186103237 18.072766 18.130034 3 54.316728999999995 18.113928999999995
1358388608 18.839737 0.2315488997145098 18.598002 19.059539 3 56.519211 18.861669999999997
13616777216 19.789655999999997 0.16025939614574827 19.604915 19.891308 3 59.368967999999995 19.872745
137
138
139
140"cpp-stlrefAllocGlib"
1414 18.941975333333332 0.10773994834940963 18.852994 19.061763 3 56.825925999999995 18.911169
1428 19.87940433333333 0.3963261118628617 19.56545 20.324737 3 59.63821299999999 19.748025999999996
14316 20.238258 0.4378155019651545 19.941289 20.741058 3 60.714774 20.032426999999995
14432 20.308837666666665 0.16467547529712334 20.120426 20.425265 3 60.926513 20.380822000000006
14564 20.614343333333334 0.3254977106913869 20.239342 20.823736 3 61.84303 20.779951999999998
146128 20.571918999999998 0.22682495038906028 20.436141 20.833774 3 61.715756999999996 20.445842
147256 20.781418333333335 0.13218721043026063 20.684039 20.931899 3 62.344255000000004 20.728317000000004
148512 21.126021333333334 0.631068886358956 20.706277 21.851751 3 63.378064 20.820036
1491024 21.028246666666664 0.37246455816672414 20.744808 21.450103 3 63.08474 20.889829
1502048 20.78677 0.11080288855891728 20.691626 20.908423 3 62.36031 20.760261
1514096 20.806661000000002 0.09642768155980845 20.749695 20.917996 3 62.419983 20.752292
1528192 21.070319333333334 0.028216666215791643 21.044583 21.100491 3 63.210958000000005 21.065884
15316384 21.095253666666668 0.006388735581735984 21.091035 21.102604 3 63.285761 21.092122000000003
15432768 22.163546666666665 0.42766503472967493 21.713697 22.564891 3 66.49064 22.212052000000003
15565536 21.965538999999996 0.2921802096788901 21.780763 22.302391 3 65.89661699999999 21.81346299999999
156131072 21.74838733333333 0.308388176722671 21.414177 22.021939 3 65.245162 21.809046
157262144 22.046801 0.3753479197597875 21.651819 22.398817 3 66.14040299999999 22.089766999999988
158524288 22.286398333333334 0.17156444356081899 22.110954 22.4538 3 66.859195 22.294441
1591048576 23.110943333333335 0.43967250848140693 22.604161 23.390615 3 69.33283 23.338053999999996
1602097152 24.137377666666666 0.06622887568374777 24.070422 24.202855 3 72.412133 24.138855999999997
1614194304 25.533254666666664 0.049896312692758044 25.484809 25.584485 3 76.599764 25.530469999999994
1628388608 25.989402 0.11912766696280118 25.873406 26.111432 3 77.968206 25.98336799999999
16316777216 26.881743 0.15939301674477419 26.699781 26.996672 3 80.645229 26.948776
164
165
166
167-------------- results-peter-removeHere.csv ----------------
168"cfa-mandHead"
1694 2.36922 0.0022503015353502495 2.366623 2.370593 3 7.10766 2.3704440000000004
1708 2.186883 0.00045484832636825224 2.186565 2.187404 3 6.560649 2.18668
17116 2.1378903333333334 0.0004371685411067379 2.137403 2.138248 3 6.413671 2.13802
17232 3.0958493333333337 1.5491722333770166 2.200472 4.884679 3 9.287548000000001 2.2023970000000013
17364 2.2463683333333333 0.0003403444333808968 2.246136 2.246759 3 6.739105 2.2462100000000005
174128 3.0993243333333336 1.330225066353184 2.32831 4.635332 3 9.297973 2.3343310000000006
175256 2.3696503333333334 0.005681040779059159 2.365336 2.376087 3 7.108951 2.367528
176512 2.8005343333333332 0.4225188505727684 2.360931 3.203595 3 8.401603 2.837077
1771024 2.459085333333333 0.1162703367344111 2.373026 2.591357 3 7.377255999999999 2.412873
1782048 2.498574 0.10671955121719744 2.408528 2.616452 3 7.495722000000001 2.470742
1794096 2.542479333333333 0.08124841365425742 2.451918 2.60898 3 7.627438 2.56654
1808192 2.537698333333333 0.029191205770460086 2.504454 2.559141 3 7.6130949999999995 2.5494999999999997
18116384 2.529886 0.027836545116087803 2.497754 2.546676 3 7.589658 2.5452279999999994
18232768 2.5621786666666666 0.00766751076512673 2.55374 2.568718 3 7.686536 2.5640780000000007
18365536 2.6233 0.008139090305433493 2.613981 2.629014 3 7.8698999999999995 2.6269049999999994
184131072 2.6416036666666667 0.0252856532906177 2.616369 2.66694 3 7.924811 2.6415019999999996
185262144 2.618085 0.002215498138117177 2.616613 2.620633 3 7.854255 2.617009
186524288 2.672014666666667 0.001754454996097903 2.670033 2.67337 3 8.016044 2.672641000000001
1871048576 2.7498653333333336 0.004245263988650465 2.745776 2.754251 3 8.249596 2.749569
1882097152 2.7790606666666666 0.005893248368542756 2.77558 2.785865 3 8.337182 2.775737000000001
1894194304 2.7797783333333332 0.003060768748751403 2.776809 2.782923 3 8.339335 2.7796030000000003
1908388608 2.782700333333333 0.002322469016657506 2.780519 2.785142 3 8.348101 2.78244
19116777216 2.785602666666667 0.00142696753058127 2.784009 2.786762 3 8.356808000000001 2.7860370000000008
192
193
194
195"cpp-stlref"
1964 16.52649033333333 0.24657610874602873 16.246723 16.712172 3 49.579471 16.620575999999996
1978 16.838661333333334 0.7715724984188681 16.006542 17.530415 3 50.515984 16.979027000000006
19816 16.436042999999998 0.3763050186843123 16.048996 16.800597 3 49.308128999999994 16.45853599999999
19932 16.274640333333334 0.2665678774952708 16.116769 16.582412 3 48.823921 16.124739999999992
20064 17.383321666666667 0.8572756472292523 16.836122 18.37131 3 52.149965 16.942533000000005
201128 16.13021433333333 0.1425726994004569 15.992777 16.27742 3 48.390643 16.120445999999994
202256 16.724054 0.7101011196660665 16.126905 17.509253 3 50.172162 16.536004
203512 16.38449233333333 0.28172649243962494 16.068592 16.609711 3 49.153476999999995 16.475174
2041024 16.487834333333335 0.33633567416248683 16.146726 16.819185 3 49.463503 16.497592
2052048 16.339519999999997 0.049047848892689 16.285704 16.381711 3 49.018559999999994 16.351145
2064096 16.145166333333332 0.17450450871348167 15.943667 16.246497 3 48.435499 16.245335
2078192 16.239072 0.0048785203699480235 16.233781 16.243392 3 48.717216 16.240043
20816384 16.581089000000002 0.3596252538448892 16.16583 16.789262 3 49.743267 16.788175000000003
20932768 16.250555000000002 0.049435486960279405 16.19396 16.285304 3 48.751665 16.272401
21065536 16.60103666666667 0.4055201871970547 16.357564 17.069165 3 49.803110000000004 16.376381000000006
211131072 16.708181333333332 0.31643033856021446 16.52455 17.073562 3 50.124544 16.526431999999996
212262144 16.834255666666667 0.3487601370373809 16.436937 17.08981 3 50.502767 16.97602
213524288 17.313745 0.5946245421785086 16.644876 17.782489 3 51.941235000000006 17.513870000000008
2141048576 17.414923666666667 0.2226544666121333 17.25476 17.669177 3 52.244771 17.320833999999994
2152097152 17.626780333333333 0.28852538194469907 17.395531 17.950106 3 52.880341 17.534704
2164194304 18.242337333333335 0.1611989752583221 18.118954 18.424725 3 54.727012 18.183333
2178388608 19.071105333333335 0.29437500586213566 18.741661 19.308329 3 57.213316000000006 19.163326000000005
21816777216 19.736657333333333 0.29974515276870406 19.547069 20.082229 3 59.209972 19.580674
219
220
221
222"cpp-stlrefAllocGlib"
2234 19.075187666666668 0.13807014138594031 18.932756 19.208438 3 57.225563 19.084368999999995
2248 20.35843 0.3067780864061182 20.095395 20.695428 3 61.075289999999995 20.284467
22516 20.2228 0.16661633644093907 20.078245 20.405027 3 60.6684 20.185127999999995
22632 20.699953333333337 0.27850907696937505 20.523689 21.021035 3 62.09986000000001 20.555136
22764 20.69399 0.4304234483726482 20.406522 21.188845 3 62.08197 20.486603000000002
228128 20.690834666666664 0.09756554623602186 20.603125 20.795921 3 62.072503999999995 20.673457999999997
229256 21.443757333333334 1.3057143943046392 20.643359 22.950488 3 64.331272 20.737424999999995
230512 21.176359666666666 0.6098610469970474 20.674502 21.855114 3 63.529078999999996 20.99946299999999
2311024 21.258606666666665 0.8317208463555151 20.769132 22.218935 3 63.775819999999996 20.787753
2322048 20.955873999999998 0.19034410857181847 20.80585 21.169992 3 62.867622 20.891779999999997
2334096 20.843175666666667 0.021817964211478143 20.820346 20.863817 3 62.529527 20.845364
2348192 21.028745333333333 0.24298482576558836 20.864014 21.307807 3 63.086236 20.914415
23516384 21.094754 0.16951372091662584 20.922581 21.261477 3 63.284262 21.100203999999998
23632768 22.09043966666667 0.5149106007379675 21.72799 22.679838 3 66.271319 21.863491000000007
23765536 22.378137 0.5302717402643651 21.912406 22.955254 3 67.134411 22.266750999999996
238131072 21.370668 0.3722365565013726 21.137112 21.799934 3 64.112004 21.174957999999997
239262144 21.831169333333335 0.1833485149144469 21.636358 22.00036 3 65.493508 21.856790000000004
240524288 22.334094000000004 0.38576379609419126 22.083422 22.778313 3 67.00228200000001 22.14054700000001
2411048576 22.772765333333336 0.5403520615805341 22.373587 23.387653 3 68.318296 22.557056000000003
2422097152 24.477956000000002 0.13257567351893884 24.325468 24.565899 3 73.433868 24.542501
2434194304 25.636546666666664 0.11416840116833182 25.517562 25.745196 3 76.90964 25.646881999999998
2448388608 26.297145999999998 0.15534560009540102 26.128939 26.435213 3 78.891438 26.32728599999999
24516777216 27.60076533333333 0.11077747047271427 27.498203 27.718246 3 82.802296 27.585846999999994
246
247
248
Note: See TracBrowser for help on using the repository browser.