Index: doc/theses/mike_brooks_MMath/list.tex =================================================================== --- doc/theses/mike_brooks_MMath/list.tex (revision d1ccc57d1942a4fa9108a4a0eda75497665de223) +++ doc/theses/mike_brooks_MMath/list.tex (revision e2e927eec770a0ff2d35ee0a0ec54765732411c3) @@ -993,5 +993,5 @@ \VRef[Figure]{fig:plot-list-zoomin} has the pair machines, each in a column. It also has a pair of operating scenarios, each in a row. -The experiment runs twelve operating scenarios; the ones chosen for their variety happen to be items I and VIII from the listing of \MLB{TODO: cross reference}. +The experiment runs twelve operating scenarios; the ones chosen for their variety happen to be items I and VIII from the listing of \VRef[Figure]{fig:plot-list-mchn-szz}. Note that LQ-list does not support queue operations, so this framework is absent from Operation VIII. @@ -1010,9 +1010,12 @@ So these zones are used for basing comparison. +\MLB{Peter, caution beyond here. I am reconsidering if this first-dismiss-physical approach to comparison is simplest.} + \begin{figure} \centering \includegraphics{plot-list-mchn-szz.pdf} - \caption{Histogram of operation durations, decomposed by physical factos. - Measurements are included from only the sizes in the ``small'' and ``medium'' stable zones.} + \caption{Histogram of operation durations, decomposed by physical factors. + Measurements are included from only the sizes in the ``small'' and ``medium'' stable zones. + This breakdown divides the entire population of test results into four mutually disjoint constituents.} \label{fig:plot-list-mchn-szz} \end{figure} @@ -1025,9 +1028,20 @@ From the perspective of assessing winning/losing frameworks, these physical effects are noise. So, subsequent analysis conditions on the phisical effects. -That is, it presents results relative to the mean of the physical quadrant (\VRef[fig]{fig:plot-list-mchn-szz} histogram) to which it belogs. +That is, it supposes you are put into an unknown physical situation (that is one of the four being tested), then presents all the ways your outcome could change as a result of non-physical factors, assuming that the physical situation is kept constant. +It does do by presenting results relative to the mean of the physical quadrant (\VRef[fig]{fig:plot-list-mchn-szz} histogram) to which it belogs. With this adjustment, absolute duration values (in nonsecods) are lost. In return, the physical quadrants are re-combined, enabling assessment of the non-physical factors. -\MLB{Peter, stop here. Graph and discussion of these non-physical factos is coming.} +\begin{figure} + \centering + \includegraphics{plot-list-op-fx.pdf} + \caption{Histogram of operation durations, decomposed by non-physical factors. + The twelve-way by-operation breakdown, and the four-way by-framework breakdown, each divides the entire population of test results into its mutually disjoint constituents.} + \label{fig:plot-list-op-fx} +\end{figure} + +\VRef[Figure]{fig:plot-list-op-fx} gives this non-physical decomposition. + +\MLB{Peter, stop here. Discussion of these non-physical factos is coming.} \begin{comment} @@ -1038,68 +1052,8 @@ On the server, \CFA and \uCpp lists are can be fast by up to 100\%. Overall, LQ-tailq does the best at short lengths but loses out above a dozen elements. - -\VRef[Figure]{f:RelativeTime} shows the percentage difference by treating @tailq@ as the control benchmark, and showing the other list results relative to it. -This change does not affect the who-wins statements, it just removes the ``sweet spot'' bend that the earlier discussion dismissed as incidental. -Runs faster than @tailq@'s are below the zero and slower runs are above; -@tailq@'s mean is always zero by definition, but its error bars, representing a single scenario's re-run stability, are still meaningful. -With this bend straightened out, aggregating across lengths is feasible. \end{comment} -\begin{figure} -\centering - \setlength{\tabcolsep}{0pt} - \begin{tabular}{p{0.75in}p{2.75in}p{3in}} - & - \subfloat[Supersets, AMD]{\label{f:Superset-swift} - \hspace*{-0.75in} - \includegraphics{plot-list-cmp-exout-swift.pdf} - } % subfigure - & - \subfloat[Supersets, Intel]{\label{f:Superset-java} - \includegraphics{plot-list-cmp-exout-java.pdf} - } % subfigure - \\ - & - \subfloat[1st Level Slice, AMD]{\label{f:1stLevelSlice-swift} - \hspace*{-0.75in} - \includegraphics{plot-list-cmp-survey-swift.pdf} - } % subfigure - & - \subfloat[1st Level Slice, Intel]{\label{f:1stLevelSlice-java} - \includegraphics{plot-list-cmp-survey-java.pdf} - } % subfigure - \end{tabular} - \caption{Operation duration ranges across operational scenarios. (a) has the supersets of the running example operation. (b) has the first-level slices of the full space of operations.} - \label{fig:plot-list-cmp-overall} -\end{figure} - -\VRef[Figure]{fig:plot-list-cmp-overall} introduces alternative views of the data. -Part \ref*{f:Superset-swift}--\ref*{f:Superset-java}'s first column summarizes all the data of \VRef{f:RelativeTime}. -Its x-axis label, ``stack/insfirst/allhead,'' names the concrete scenario that has been discussed until now. -Moving across the columns, the next three each stretch to include more scenarios on each of the operation dimensions, one at a time. -The second column considers the scenarios $\{\mathrm{stack}\} \times \{\mathrm{insfirst}\} \times \{\mathrm{allhead}, \mathrm{inselem}, \mathrm{remelem}\}$, -while the third stretches polarity and the fourth stretches accessor. -Then next three columns each stretch two scenario dimensions and the last column stretches all three. -The \CFA bar in the last column is summarizing 840 test-program runs: 14 list lengths, 2 movements, 2 polarities, 3 accessors and 5 repetitions. - -In the earlier plots of a single scenario broken down by length, each data point, with its error bars, represents just 5 repetitions. -With a couple exceptions, this reproducibility error was small. -Now, for a \CFA bar, summarizing 70 (first column) to 840 (last column) runs, a bar's height is dominated by the different behaviours of the scenarios and list lengths that it summarizes. -Accordingly, the first column's bars are short and last one's are tall. -A box represents the inner 68\% of the durations, while its lines extend to cover 95\%. -The symbol on the bar is the mean duration. - -The chosen benchmark of LQ-@tailq@ is not shown in this format because it would be trivial here. -With iter-scenario differences dominating the bar size, and @tailq@'s mean performance defined to be zero in all scenarios, a @tailq@ bar on this plot would only show @tailq@'s re-run stability, which is of no comparison value. - -The LQ-@list@ implementation does not support all scenarios, only stack movement with insert-first polarity. -So, its 1, 3, 4 and 7\textsuperscript{th} bars all summarize the same set of points (those with accessor constrained to all-head), as do its 2, 5, 6 and 8\textsuperscript{th} (those with accessor unconstrained). - -Rather than exploring from one scenario out, \VRef{fig:plot-list-cmp-overall}-(b) gives a more systematic breakdown of the entire experimental space. -Other than the last grand-total column, each breakdown column shows one value from one operation dimension. - -LQ-@list@'s partial scenario coverage gives missing bars where it does not support the operation. -And, again, it gives repetition where all data points occur in several columns' intersection, such as stack/*/* and */insfirst/*. - + +\MLB{Here is the conclusion of the earlier analysis. It basically stands, but needs refreshing.} In the grand total, and in all halves by movement or polarity, \CFA and uC++ are equivalent, while LQ-@list@ beats them slightly. Splitting on accessor, \CFA has a poor result on element removal, LQ-@list@ has a great result on the other accessors, and uC++ is unaffected. Index: doc/theses/mike_brooks_MMath/plots/ListCommon.py =================================================================== --- doc/theses/mike_brooks_MMath/plots/ListCommon.py (revision d1ccc57d1942a4fa9108a4a0eda75497665de223) +++ doc/theses/mike_brooks_MMath/plots/ListCommon.py (revision e2e927eec770a0ff2d35ee0a0ec54765732411c3) @@ -488,5 +488,6 @@ histo.insert(y_lo_col_loc + 1, "y_hi", histo["y_lo"].apply(topValOfBucketBotVal)) - header = str.join(', ', dkey) + dkey_str = list( map( str, dkey ) ) + header = str.join(', ', dkey_str) print(f'"{header}"') text = histo.to_csv(header=False, index=False, sep='\t') Index: doc/theses/mike_brooks_MMath/plots/list-op-fx.d =================================================================== --- doc/theses/mike_brooks_MMath/plots/list-op-fx.d (revision e2e927eec770a0ff2d35ee0a0ec54765732411c3) +++ doc/theses/mike_brooks_MMath/plots/list-op-fx.d (revision e2e927eec770a0ff2d35ee0a0ec54765732411c3) @@ -0,0 +1,2 @@ +plots/list-op-fx.gp.INPUTS: build/plot-list-op-fx.dat | build +plots/list-op-fx.py.INPUTS: benchmarks/list/results-swift-general.csv Index: doc/theses/mike_brooks_MMath/plots/list-op-fx.gp =================================================================== --- doc/theses/mike_brooks_MMath/plots/list-op-fx.gp (revision e2e927eec770a0ff2d35ee0a0ec54765732411c3) +++ doc/theses/mike_brooks_MMath/plots/list-op-fx.gp (revision e2e927eec770a0ff2d35ee0a0ec54765732411c3) @@ -0,0 +1,98 @@ +set terminal pdfcairo color enhanced size 6.5in,3.0in font "Times,17" + +set size 1.0, 1.0 # scale of plot area inside terminal +set origin 0.0, 0.0 # bottom-left corner + +INDIR="build" +OUTDIR="build" + +set macros +set output OUTDIR."/plot-list-op-fx.pdf" + +set lmargin 10 +set bmargin 5 + +set grid +set nokey +set logscale y 2 +set yrange [0.4:3.0]; +set ytics ( \ + "+40%%" 1.666666667, \ + "+30%%" 1.428571429, \ + "+20%%" 1.25, \ + "+10%%" 1.111111111, \ + "0" 1, \ + "-20%%" 0.833333333, \ + "-40%%" 0.714285714, \ + "-60%%" 0.625, \ + "-80%%" 0.555555556, \ + "-100%%" 0.5, \ + "-130%%" 0.434782609, \ + "-160%%" 0.384615385, \ + "-200%%" 0.333333333 \ +) +set xrange [-0.5:17.5]; +set xlabel "Operation; Prevalence Framework; Prevalence" offset 7,-1 +set xtics ( \ + "I" 0, \ + "II" 1, \ + "III" 2, \ + "IV" 3, \ + "V" 4, \ + "VI" 5, \ + "VII" 6, \ + "VIII" 7, \ + "IX" 8, \ + "X" 9, \ + "XI" 10, \ + "XII" 11, \ +\ + "cfa" 13, \ + "upp" 14, \ + "lq\ntailq" 15, \ + "lq\nlist" 16, \ +) + +set ylabel "Duration (relative)" offset -1.0,0 +set errorbars 2.0 +set pointintervalbox 0 + +barHtScale_op = 0.1 +barHtScale_fx = 0.05 + + +plot INDIR."/plot-list-op-fx.dat" \ + i 6 using (0):(0):( 0):( 0 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 7 using (0):(0):( 1):( 1 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 8 using (0):(0):( 2):( 2 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 9 using (0):(0):( 3):( 3 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 10 using (0):(0):( 4):( 4 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 11 using (0):(0):( 5):( 5 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 0 using (0):(0):( 6):( 6 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 1 using (0):(0):( 7):( 7 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 2 using (0):(0):( 8):( 8 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 3 using (0):(0):( 9):( 9 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 4 using (0):(0):(10):(10 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 5 using (0):(0):(11):(11 + barHtScale_op * $3):1:2 title columnheader(1) with boxxyerror fc rgb "black" fs transparent solid 0.15 noborder, \ + '' i 6 using ( 0 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 7 using ( 1 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 8 using ( 2 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 9 using ( 3 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 10 using ( 4 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 11 using ( 5 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 0 using ( 6 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 1 using ( 7 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 2 using ( 8 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 3 using ( 9 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 4 using (10 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ + '' i 5 using (11 + barHtScale_op * $3):2 notitle with steps lc rgb "black" lw 0.3, \ +\ + '' i 12 using (0):(0):(1 + 12):(1 + 12 + barHtScale_fx * $3):1:2 title columnheader(1) with boxxyerror fc rgb "blue" fs transparent solid 0.15 noborder, \ + '' i 15 using (0):(0):(1 + 13):(1 + 13 + barHtScale_fx * $3):1:2 title columnheader(1) with boxxyerror fc rgb "dark-orange" fs transparent solid 0.15 noborder, \ + '' i 14 using (0):(0):(1 + 14):(1 + 14 + barHtScale_fx * $3):1:2 title columnheader(1) with boxxyerror fc rgb "magenta" fs transparent solid 0.15 noborder, \ + '' i 13 using (0):(0):(1 + 15):(1 + 15 + 3.5 * barHtScale_fx * $3):1:2 title columnheader(1) with boxxyerror fc rgb "purple" fs transparent solid 0.35 noborder, \ + '' i 12 using (1 + 12 + barHtScale_fx * $3):2 notitle with steps lc rgb "blue" lw 0.3, \ + '' i 15 using (1 + 13 + barHtScale_fx * $3):2 notitle with steps lc rgb "dark-orange" lw 0.3, \ + '' i 14 using (1 + 14 + barHtScale_fx * $3):2 notitle with steps lc rgb "magenta" lw 0.3, \ + '' i 13 using (1 + 15 + 3.5 * barHtScale_fx * $3):2 notitle with steps lc rgb "purple" lw 0.3 + Index: doc/theses/mike_brooks_MMath/plots/list-op-fx.py =================================================================== --- doc/theses/mike_brooks_MMath/plots/list-op-fx.py (revision e2e927eec770a0ff2d35ee0a0ec54765732411c3) +++ doc/theses/mike_brooks_MMath/plots/list-op-fx.py (revision e2e927eec770a0ff2d35ee0a0ec54765732411c3) @@ -0,0 +1,29 @@ +import pandas as pd +import numpy as np +import os +import sys + +sys.path.insert(0, os.path.dirname(__file__)) +from ListCommon import * + +conditionOnBase = ['machine', 'NumNodes', 'SizeZone'] # physicals +drillOn1 = ['movement', 'polarity', 'accessor'] +drillOn2 = ['fx'] + +assert( set( explanations ) - + set( ['InterleaveFrac'] ) # unused and always zero + == + set(conditionOnBase ) | + set(drillOn1) | set(drillOn2) ) + +printHistos( + tgtMovement = 'all', + tgtPolarity = 'all', + tgtAccessor = 'all', + drillOn=drillOn1 ) + +printHistos( + tgtMovement = 'all', + tgtPolarity = 'all', + tgtAccessor = 'all', + drillOn=drillOn2 )