Ignore:
Timestamp:
Oct 6, 2020, 12:07:09 PM (4 years ago)
Author:
Peter A. Buhr <pabuhr@…>
Branches:
ADT, arm-eh, ast-experimental, enum, forall-pointer-decay, jacob/cs343-translation, master, new-ast-unique-expr, pthread-emulation, qualifiedEnum
Children:
25744d2, 9e18677
Parents:
c6391e6
Message:

reformat for submission

File:
1 edited

Legend:

Unmodified
Added
Removed

  • doc/papers/concurrency/response3

    rc6391e6 raa5777c  
    1     I would like you address the comments of Reviewer 2, particularly with
    2     regard to the description of the adaptation Java harness to deal with
    3     warmup. I would expect to see a convincing argument that the computation
    4     has reached a steady state.
     1        I would like you address the comments of Reviewer 2, particularly with regard to the description of the adaptation Java harness to deal with warmup. I would expect to see a convincing argument that the computation has reached a steady state.
    52
    6 We understand referee2 and your concern about the JIT experiments, which is why
    7 we verified our experiments with two experts in JIT development for both Java
    8 and Node.js before submitting the paper. We also read the supplied papers, but
    9 most of the information is not applicable to our work for the following
    10 reasons.
     3We understand referee2 and your concern about the JIT experiments, which is why we verified our experiments with two experts in JIT development for both Java and Node.js before submitting the paper. We also read the supplied papers, but most of the information is not applicable to our work for the following reasons.
    114
    12 1. SPEC benchmarks are medium to large. In contrast, our benchmarks are 5-15
    13    lines in length for each programming language (see code for the Cforall
    14    tests in the paper). Hence, there is no significant computations, complex
    15    control flow, or use of memory.  They test one specific language features
    16    (context switch, mutex call, etc.) in isolation over and over again. These
    17    language features are fixed (e.g., acquiring and releasing a lock is a fixed
    18    cost). Therefore, unless the feature can be removed there is nothing to
    19    optimize at runtime. But these features cannot be removed without changing
    20    the meaning of the benchmark. If the feature is removed, the timing result
    21    would be 0. In fact, it was difficult to prevent the JIT from completely
    22    eliding some benchmarks because there are no side-effects.
    23    
    24 2. All of our benchmark results correlate across programming languages with and
    25    without JIT, indicating the JIT has completed any runtime optimizations
    26    (added this sentence to Section 8.1). Any large differences are explained by
    27    how a language implements a feature not by how the compiler/JIT precesses
    28    that feature.  Section 8.1 discusses these points in detail.
     51. SPEC benchmarks are medium to large. In contrast, our benchmarks are 5-15 lines in length for each programming language (see code for the Cforall tests in the paper). Hence, there is no significant computations, complex control flow, or use of memory. They test one specific language features (context switch, mutex call, etc.) in isolation over and over again. These language features are fixed (e.g., acquiring and releasing a lock is a fixed cost). Therefore, unless the feature can be removed there is nothing to optimize at runtime. But these features cannot be removed without changing the meaning of the benchmark. If the feature is removed, the timing result would be 0. In fact, it was difficult to prevent the JIT from completely eliding some benchmarks because there are no side-effects.
    296
    30 3. We also added a sentence about running all JIT-base programming language
    31    experiments for 30 minutes and there was no statistical difference,
    32    med/avg/std correlated with the short-run experiments, which seems a
    33    convincing argument that the benchmark has reached a steady state. If the
    34    JIT takes longer than 30 minutes to achieve its optimization goals, it is
    35    unlikely to be useful.
     72. All of our benchmark results correlate across programming languages with and without JIT, indicating the JIT has completed any runtime optimizations (added this sentence to Section 8.1). Any large differences are explained by how a language implements a feature not by how the compiler/JIT precesses that feature. Section 8.1 discusses these points in detail.
    368
    37 4. The purpose of the performance section is not to draw conclusions about
    38    improvements. It is to contrast program-language implementation approaches.
    39    Section 8.1 talks about ramifications of certain design and implementation
    40    decisions with respect to overall performance. The only conclusion we draw
    41    about performance is:
     93. We also added a sentence about running all JIT-base programming language experiments for 30 minutes and there was no statistical difference, med/avg/std correlated with the short-run experiments, which seems a convincing argument that the benchmark has reached a steady state. If the JIT takes longer than 30 minutes to achieve its optimization goals, it is unlikely to be useful.
    4210
    43      Performance comparisons with other concurrent systems and languages show
    44      the Cforall approach is competitive across all basic operations, which
    45      translates directly into good performance in well-written applications
    46      with advanced control-flow.
     114. The purpose of the performance section is not to draw conclusions about improvements. It is to contrast program-language implementation approaches. Section 8.1 talks about ramifications of certain design and implementation decisions with respect to overall performance. The only conclusion we draw about performance is:
     12
     13   Performance comparisons with other concurrent systems and languages show the Cforall approach is competitive across all basic operations, which translates directly into good performance in well-written applications with advanced control-flow.
    4714
    4815
    49    I would also like you to provide the values for N for each benchmark run.
     16       I would also like you to provide the values for N for each benchmark run.
    5017
    5118Done.
    … …  
    5623   * don't start sentences with "However"
    5724
    58 However, there are numerous grammar sites on the web indicating "however" (a
    59 conjunction) at the start of a sentence is acceptable, e.g.:
     25However, there are numerous grammar sites on the web indicating "however" (a conjunction) at the start of a sentence is acceptable, e.g.:
    6026
    61  https://www.merriam-webster.com/words-at-play/can-you-start-a-sentence-with-however
    62  This is a stylistic choice, more than anything else, as we have a
    63  considerable body of evidence of writers using however to begin sentences,
    64  frequently with the meaning of "nevertheless."
     27https://www.merriam-webster.com/words-at-play/can-you-start-a-sentence-with-however This is a stylistic choice, more than anything else, as we have a considerable body of evidence of writers using however to begin sentences, frequently with the meaning of "nevertheless."
Note: See TracChangeset for help on using the changeset viewer.