source: doc/papers/general/response @ 38f6e66

Last change on this file since 38f6e66 was 3d60c08, checked in by Peter A. Buhr <pabuhr@…>, 6 years ago

complete referee changes

  • Property mode set to 100644
File size: 17.6 KB
Line 
1Date: Thu, 19 Apr 2018 17:01:14 -0400 (EDT)
2From: "Software: Practice and Experience" <onbehalfof@manuscriptcentral.com>
3Reply-To: judithbishop@outlook.com
4To: a3moss@uwaterloo.ca, rschlunt@uwaterloo.ca, pabuhr@uwaterloo.ca
5Subject: Software: Practice and Experience - Decision on Manuscript ID SPE-18-0065
6
719-Apr-2018
8
9Dear Dr Buhr,
10
11Many thanks for submitting SPE-18-0065 entitled "Cforall : Adding Modern Programming
12Language Features to C" to Software: Practice and Experience. The paper has now
13been reviewed and the comments of the referee(s) are included at the bottom of
14this letter.
15
16I am delighted to inform you that the referee(s) have recommended publication,
17but also suggest some minor revisions to your manuscript.  Therefore, I invite
18you to respond to the referee(s)' comments and revise your manuscript. All of
19the referees' comments are important, but I would like you to pay special
20attention to the following general points:
21
221. If there is any more evaluation that a stack, please add it.
232. How is the compiler implemented?
243. What features are actually implemented?
254. The article lacks some related work. such as Haskell, ML etc.
265. Most of the content in Section 10 RELATED WORK appears to belong to Section 1 INTRODUCTION as a Subsection or as a new Section after Section 1.
276. Many references are not properly formatted
287. A statement about any presence or absence of conflicts of interest with Huawei should be explicitly added.
29
30The paper is long by SPE standards (33 pages). We have a maximum of 40
31pages. Please do not extend the paper beyond 35 pages. If necessary, find ways
32to cut the examples or text. If you have an accompanying website for the system
33where some examples are stored, please mention it.
34
35You have 42 days from the date of this email to submit your revision. If you
36are unable to complete the revision within this time, please contact me to
37request a short extension.
38
39You can upload your revised manuscript and submit it through your Author
40Center. Log into https://mc.manuscriptcentral.com/spe and enter your Author
41Center, where you will find your manuscript title listed under "Manuscripts
42with Decisions".
43
44When submitting your revised manuscript, you will be able to respond to the
45comments made by the referee(s) in the space provided.  You can use this space
46to document any changes you make to the original manuscript.
47
48If you feel that your paper could benefit from English language polishing, you
49may wish to consider having your paper professionally edited for English
50language by a service such as Wiley's at
51http://wileyeditingservices.com. Please note that while this service will
52greatly improve the readability of your paper, it does not guarantee acceptance
53of your paper by the journal.
54
55Once again, thank you for submitting your manuscript to Software: Practice and Experience. I look forward to receiving your revision.
56
57Sincerely,
58
59Dr Judith Bishop
60Editor, Software: Practice and Experience
61judithbishop@outlook.com
62
63
64We have attempted to response to all the issues raised by the Editor and referee's comments. For two
65of the issues, we have "pushed back", with an explanation. Specifically, moving the related work
66forward, and moving text from Section 9 into the captions of Table2 and Figure 10.  Our reasons for
67not making these changes are address below. Finally, as pointed out below, there are a couple of
68issues with the Wiley LaTeX macros that we worked around as best as possible.
69
70   The paper is long by SPE standards (33 pages). We have a maximum of 40 pages. Please do not
71   extend the paper beyond 35 pages. If necessary, find ways to cut the examples or text. If you
72   have an accompanying website for the system where some examples are stored, please mention it.
73
74The paper is 35 pages using the supplied Wiley LaTeX macros.
75
76
77Referee(s)' Comments to Author:
78
79Reviewing: 1
80
81   Most of the content in Section 10 RELATED WORK appears to belong to Section 1 INTRODUCTION as a
82   Subsection or as a new Section after Section 1. (Please also see #4.1 below.) Remaining
83   discussion that cannot be moved earlier can become a DISCUSSION Section or a Subsection within
84   the last Section of the paper.
85
86Sometimes it is appropriate to put related work at the start of a paper and sometimes at the
87end. For this paper, it seems appropriate to put the related work at the end of the paper. The
88purpose of the related work in this paper is two fold: to introduce prior work and to contrast it
89with Cforall.  Only at the end of the paper does the reader have sufficient knowledge about Cforall
90to make detailed contrasts with other programming languages possible. If the related work is moved
91to the end of the introduction, the reader knows nothing about Cforall so talking about other
92programming languages in isolation makes little sense, especially non-C-related languages, like
93Java, Go, Rust, Haskell. We see no easy way to separate the related work into a general discussion
94at the start and a specific discussion at the end. We explicitly attempt to deal with the reader's
95anticipation at the end of the introduction:
96
97 Finally, it is impossible to describe a programming language without usages before definitions.
98 Therefore, syntax and semantics appear before explanations; hence, patience is necessary until
99 details are presented.
100
101
102   2.1 More information should be moved from the text and added to Figure 10 and Table 2 so that
103   readers can understand the comparison quickly. Imagine a reader read the summary and jump
104   directly to these two display elements. Questions would be raised about the binary size and pop
105   pair result of Cforall and it would take time to find answers in the text.
106
107This suggestion is an alternative writing style. The experiment is complex enough that it is
108unlikely a reader could jump to the table/graph and understand the experiment without putting a
109substantive amount of the text from Section 9 into the table and figure, which the reader then has
110to read anyway.  In fact, we prefer a writing style where the reader does not have to look at the
111table/figure to understand the experiment and the results, i.e., the table/figure are only there to
112complement the discussion.
113
114
115   2.2 The pronunciation of ("C-for-all") should be provided in the summary (page 1 line 22) so that
116   people not having an access to the full-text can see it.
117
118Done.
119
120
121   2.3 Error comment in the code should be written with the same capitalization and it will be
122   helpful if you say specifically compilation error or runtime error. (Please see attached
123   annotated manuscript.)
124
125Fixed. All errors in the paper are compilation errors because they are related to the type system.
126
127
128   2.4 It is possible to provide a bit more information in Appendix A e.g. how many lines/bytes of
129   code and some details about software/hardware can be added/moved here. The aim is to provide
130   sufficient information for readers to reproduce the results and to appreciate the context of the
131   comparison.
132
133Table 2 indicates the source-code size in lines of code. The third paragraph of Section 9 gives
134precise details of the software/hardware used in the experiments.
135
136   3. Practical information about the work
137
138   There are three separate pieces of information on pages 2 ("All features discussed in this paper
139   are working, unless otherwise stated as under construction."),
140
141This sentence is replace with:
142
143 All languages features discussed in this paper are working, except some advanced exception-handling
144 features.
145
146and Section 5.4 Exception Handling states:
147
148 The following framework for Cforall exception handling is in place, excluding some runtime
149 type-information and virtual functions.
150
151
152   page 4 ("Under construction is a mechanism to distribute...")
153
154The feature on page 4 is now complete.
155
156
157   and page 33 ("There is ongoing work on a wide range ... ")
158
159This sentence is replace to indicate the ongoing work is future work.
160
161 While all examples in the paper compile and run, a public beta-release of Cforall will take 6-8
162 months to reduce compilation time, provide better debugging, and add a few more libraries.  There
163 is also new work on a number of Cforall features, including arrays with size, runtime
164 type-information, virtual functions, user-defined conversions, and modules.
165
166
167   My recommendation is to move them to an appendix so that the length is preserved.
168
169There is nothing to move into an appendix, except 3 sentences. We do not intend to discuss these
170items in this paper.
171
172
173   3.1 Any under construction work (only small part of page 4) should not be mingled into the main
174   part of the manuscript.
175
176See above.
177
178
179   3.2 Instructions on how to access/use the working functionality of Cforall should be given.
180
181We will indicate release of Cforall in a public location, when we believe the code base is
182acceptable. In the interim, we have made public all the experimental code from section 9, and there
183is a reference in the paper to access this code. We can make a private beta-copy of Cforall
184available to the SP&E editor for distribution to the referees so they can verify our claims.
185
186   3.3 Planned work should be given a specific time of completion/release not just "8-12 months".
187
188Software development is not a rigorous engineering discipline. Given our small research
189development-team and the size of the project, we cannot give a specific time for completion of
190anything associated with the project. Having said that, we have reduced our expected time for
191Cforall release to 6-8 months as work is progressing well.
192
193
194   4.1 The impression after reading Section 1 INTRODUCTION is that the referencing is poor. It is
195   not until Section 10 RELATED WORK where majority of the prior literature are discussed. Please
196   consider moving the content and improve citations - at least cite all main variations of C
197   languages.
198
199See point 1.
200
201
202   4.2 I also would like to see citations at these specific places: Page 2 after Phil Karlton, page
203   22 after const hell problem.
204
205The Phil-Karlton quote is an urban legend without a specific academic citation:
206
207  https://skeptics.stackexchange.com/questions/19836/has-phil-karlton-ever-said-there-are-only-two-hard-things-in-computer-science
208
209The term "const hell" is replaced with "const poisoning" with a citation.
210
211
212   5.1 Footnotes and citations will need to have different schemes - number and perhaps letter.
213
214Switched to letters. SP&E uses symbol footnotes but this macros fails with their macros:
215
216 \renewcommand*{\thefootnote}{\fnsymbol{footnote}}
217
218
219   5.2 Many references are not properly formatted e.g. date is incomplete, extra/missing white
220   spaces, extra dots, use of page number or section number as part of superscript ref
221   number. Please refer to attached document.
222
223Agreed. The bibtex BST macros are at fault. I have fixed some issues but I cannot fix them all as my
224BST macro-knowledge is limited.
225
226
227   5.3 Typos:
228   - Page 3 "eager" should be "earlier"
229
230Fixed.
231
232   - Page 4 "vals" should be "arr"
233
234Actually it is "vals", and the example is changed so it is clear why.
235
236   - Page 21 "than" should be "then"
237
238Fixed.
239
240
241   6. Conflict of interest
242   I see that the work is partially supported by Huawei. Perhaps statement about any presence or
243   absence of conflicts of interest should be explicitly added. Please get a clear direction on this
244   from the editor of the journal.
245
246The paper now states the project is open-source, hence there is no conflict of interest with the
247funding received from Huawei.
248
249
250Reviewing: 2
251
252Comments to the Author
253
254   Overloading requires the compiler to mangle a function's signature into its name in the object
255   file.  I'm pretty sure that this will complicate the build process of mixed Cforall/C projects.
256
257There is no complexity with building Cforall/C programs, and there is an existence proof because C++
258has name mangling for overloading and has no problem interacting with C.
259
260
261   I found the evaluation underwhelming.  There were only ~200 LoC ported from C to Cforall.  This
262   is too less to encounter potential caveats Cforall's type system might impose.
263
264We have clarified that the evaluation is not for the type system, but rather the underlying
265implementation approach for the parametric polymorphism. Section 9 now starts:
266
267 Cforall adds parametric polymorphism to C.  A runtime evaluation is performed to compare the cost
268 of alternative styles of polymorphism.  The goal is to compare just the underlying mechanism for
269 implementing different kinds of polymorphism.
270
271and ends with:
272
273 We conjecture these results scale across most generic data-types as the underlying polymorphic
274 implement is constant.
275 
276
277   Also, how is the compiler implemented?  I guess, Cforall is a source-to-source compiler (from
278   Cforall to C).  But this is left in the dark.  What features are actually implemented?
279
280The following paragraph has been added to the introduction to address this comment:
281
282 All languages features discussed in this paper are working, except some advanced exception-handling
283 features.  Not discussed in this paper are the integrated concurrency-constructs and user-level
284 threading-library~\cite{Delisle18}.  Cforall is an open-source project implemented as an
285 source-to-source translator from Cforall to the gcc-dialect of C~\cite{GCCExtensions}, allowing it
286 to leverage the portability and code optimizations provided by gcc, meeting goals (1)--(3).
287 Ultimately, a compiler is necessary for advanced features and optimal performance.  The Cforall
288 translator is 200+ files and 46,000+ lines of code written in C/C++.  Starting with a translator
289 versus a compiler makes it easier and faster to generate and debug C object-code rather than
290 intermediate, assembler or machine code.  The translator design is based on the visitor pattern,
291 allowing multiple passes over the abstract code-tree, which works well for incrementally adding new
292 feature through additional visitor passes.  At the heart of the translator is the type resolver,
293 which handles the polymorphic routine/type overload-resolution.  The Cforall runtime system is 100+
294 files and 11,000+ lines of code, written in Cforall.  Currently, the Cforall runtime is the largest
295 user of Cforall providing a vehicle to test the language features and implementation.  The Cforall
296 tests are 290+ files and 27,000+ lines of code.  The tests illustrate syntactic and semantic
297 features in Cforall, plus a growing number of runtime benchmarks.  The tests check for correctness
298 and are used for daily regression testing of commits (3800+).
299
300
301   Furthermore, the article lacks some related work.  Many proposed features are present in
302   functional languages such as Haskell, ML etc.  In particular, the dealing of parametric
303   polymorphism reminds me of Haskell.
304
305The following paragraph has been added at the start of Section 10.1:
306
307 ML~\cite{ML} was the first language to support parametric polymorphism.  Like Cforall, it supports
308 universal type parameters, but not the use of assertions and traits to constrain type arguments.
309 Haskell~\cite{Haskell10} combines ML-style polymorphism, polymorphic data types, and type inference
310 with the notion of type classes, collections of overloadable methods that correspond in intent to
311 traits in Cforall.  Unlike Cforall, Haskell requires an explicit association between types and
312 their classes that specifies the implementation of operations.  These associations determine the
313 functions that are assertion arguments for particular combinations of class and type, in contrast
314 to Cforall where the assertion arguments are selected at function call sites based upon the set of
315 operations in scope at that point.  Haskell also severely restricts the use of overloading: an
316 overloaded name can only be associated with a single class, and methods with overloaded names can
317 only be defined as part of instance declarations.
318
319
320   Cforall's approach to tuples is also quite similar to many functional languages.
321
322At the end of Section 10.2, we state:
323
324 Tuples are a fundamental abstraction in most functional programming languages, such as Standard ML,
325 Haskell}, and Scala, which decompose tuples using pattern matching.
326
327
328From: Judith Bishop <judithbishop@outlook.com>
329To: "Peter A. Buhr" <pabuhr@uwaterloo.ca>
330Subject: RE: Software: Practice and Experience - Decision on Manuscript ID
331 SPE-18-0065
332Date: Tue, 24 Apr 2018 16:45:51 +0000
333Accept-Language: em-US
334
335Hi Peter
336
337Great to hear from you. I am also glad your paper got through, as it is in the
338mainline of the SPE scope.
339
340It is important to mention that the software is open source. People really
341value that. In the acknowledgements, you can refer to Huawei for funding. It is
342quite normal to have industrial funding, and in fact it is a plus.
343
344I think that sorts out the comment from the referee.
345
346Looking forward to your revised submission.
347
348Kind regards
349
350Judith Bishop
351Extraordinary Professor, Computer Science
352Stellenbosch University, South Africa
353082 301 5220 / 021 671 5133
354judithbishop@outlook.com     LinkedIn
355
356-----Original Message-----
357From: Peter A. Buhr <pabuhr@uwaterloo.ca>
358Sent: Tuesday, April 24, 2018 6:25 PM
359To: judithbishop@outlook.com
360Cc: a3moss@uwaterloo.ca; rschlunt@uwaterloo.ca
361Subject: Re: Software: Practice and Experience - Decision on Manuscript ID SPE-18-0065
362
363Hi Judy! Hope all is well.
364
365We are over-the-moon to get our paper accepted at SP&E, and we are actively
366working on your and the referee's comments.
367
368One comment where we need assistance is:
369
370  7. A statement about any presence or absence of conflicts of interest with
371     Huawei should be explicitly added.
372
373We forgotten to mention in the paper that our project is open-source. So Huawei
374was funding an open-source project. In fact, the Huawei funding ends soon, so
375there will be no direct affiliation in a couple of months, although there are a
376few people at Huawei who remain very interested in the project.
377
378So does stating that the Cforall project is an open-source project deal with
379the issue of conflict of interest?
Note: See TracBrowser for help on using the repository browser.