- Timestamp:
- Oct 6, 2020, 12:07:09 PM (4 years ago)
- Branches:
- ADT, arm-eh, ast-experimental, enum, forall-pointer-decay, jacob/cs343-translation, master, new-ast-unique-expr, pthread-emulation, qualifiedEnum
- Children:
- 25744d2, 9e18677
- Parents:
- c6391e6
- File:
-
- 1 edited
-
doc/papers/concurrency/response3 (modified) (2 diffs)
Legend:
- Unmodified
- Added
- Removed
-
doc/papers/concurrency/response3
rc6391e6 raa5777c 1 I would like you address the comments of Reviewer 2, particularly with 2 regard to the description of the adaptation Java harness to deal with 3 warmup. I would expect to see a convincing argument that the computation 4 has reached a steady state. 1 I would like you address the comments of Reviewer 2, particularly with regard to the description of the adaptation Java harness to deal with warmup. I would expect to see a convincing argument that the computation has reached a steady state. 5 2 6 We understand referee2 and your concern about the JIT experiments, which is why 7 we verified our experiments with two experts in JIT development for both Java 8 and Node.js before submitting the paper. We also read the supplied papers, but 9 most of the information is not applicable to our work for the following 10 reasons. 3 We understand referee2 and your concern about the JIT experiments, which is why we verified our experiments with two experts in JIT development for both Java and Node.js before submitting the paper. We also read the supplied papers, but most of the information is not applicable to our work for the following reasons. 11 4 12 1. SPEC benchmarks are medium to large. In contrast, our benchmarks are 5-15 13 lines in length for each programming language (see code for the Cforall 14 tests in the paper). Hence, there is no significant computations, complex 15 control flow, or use of memory. They test one specific language features 16 (context switch, mutex call, etc.) in isolation over and over again. These 17 language features are fixed (e.g., acquiring and releasing a lock is a fixed 18 cost). Therefore, unless the feature can be removed there is nothing to 19 optimize at runtime. But these features cannot be removed without changing 20 the meaning of the benchmark. If the feature is removed, the timing result 21 would be 0. In fact, it was difficult to prevent the JIT from completely 22 eliding some benchmarks because there are no side-effects. 23 24 2. All of our benchmark results correlate across programming languages with and 25 without JIT, indicating the JIT has completed any runtime optimizations 26 (added this sentence to Section 8.1). Any large differences are explained by 27 how a language implements a feature not by how the compiler/JIT precesses 28 that feature. Section 8.1 discusses these points in detail. 5 1. SPEC benchmarks are medium to large. In contrast, our benchmarks are 5-15 lines in length for each programming language (see code for the Cforall tests in the paper). Hence, there is no significant computations, complex control flow, or use of memory. They test one specific language features (context switch, mutex call, etc.) in isolation over and over again. These language features are fixed (e.g., acquiring and releasing a lock is a fixed cost). Therefore, unless the feature can be removed there is nothing to optimize at runtime. But these features cannot be removed without changing the meaning of the benchmark. If the feature is removed, the timing result would be 0. In fact, it was difficult to prevent the JIT from completely eliding some benchmarks because there are no side-effects. 29 6 30 3. We also added a sentence about running all JIT-base programming language 31 experiments for 30 minutes and there was no statistical difference, 32 med/avg/std correlated with the short-run experiments, which seems a 33 convincing argument that the benchmark has reached a steady state. If the 34 JIT takes longer than 30 minutes to achieve its optimization goals, it is 35 unlikely to be useful. 7 2. All of our benchmark results correlate across programming languages with and without JIT, indicating the JIT has completed any runtime optimizations (added this sentence to Section 8.1). Any large differences are explained by how a language implements a feature not by how the compiler/JIT precesses that feature. Section 8.1 discusses these points in detail. 36 8 37 4. The purpose of the performance section is not to draw conclusions about 38 improvements. It is to contrast program-language implementation approaches. 39 Section 8.1 talks about ramifications of certain design and implementation 40 decisions with respect to overall performance. The only conclusion we draw 41 about performance is: 9 3. We also added a sentence about running all JIT-base programming language experiments for 30 minutes and there was no statistical difference, med/avg/std correlated with the short-run experiments, which seems a convincing argument that the benchmark has reached a steady state. If the JIT takes longer than 30 minutes to achieve its optimization goals, it is unlikely to be useful. 42 10 43 Performance comparisons with other concurrent systems and languages show 44 the Cforall approach is competitive across all basic operations, which 45 translates directly into good performance in well-written applications 46 with advanced control-flow. 11 4. The purpose of the performance section is not to draw conclusions about improvements. It is to contrast program-language implementation approaches. Section 8.1 talks about ramifications of certain design and implementation decisions with respect to overall performance. The only conclusion we draw about performance is: 12 13 Performance comparisons with other concurrent systems and languages show the Cforall approach is competitive across all basic operations, which translates directly into good performance in well-written applications with advanced control-flow. 47 14 48 15 49 I would also like you to provide the values for N for each benchmark run.16 I would also like you to provide the values for N for each benchmark run. 50 17 51 18 Done. … … 56 23 * don't start sentences with "However" 57 24 58 However, there are numerous grammar sites on the web indicating "however" (a 59 conjunction) at the start of a sentence is acceptable, e.g.: 25 However, there are numerous grammar sites on the web indicating "however" (a conjunction) at the start of a sentence is acceptable, e.g.: 60 26 61 https://www.merriam-webster.com/words-at-play/can-you-start-a-sentence-with-however 62 This is a stylistic choice, more than anything else, as we have a 63 considerable body of evidence of writers using however to begin sentences, 64 frequently with the meaning of "nevertheless." 27 https://www.merriam-webster.com/words-at-play/can-you-start-a-sentence-with-however This is a stylistic choice, more than anything else, as we have a considerable body of evidence of writers using however to begin sentences, frequently with the meaning of "nevertheless."
Note: See TracChangeset
for help on using the changeset viewer.
