source: doc/papers/general/response @ 65a7050

Last change on this file since 65a7050 was 3d60c08, checked in by Peter A. Buhr <pabuhr@…>, 6 years ago

complete referee changes

  • Property mode set to 100644
File size: 17.6 KB
1Date: Thu, 19 Apr 2018 17:01:14 -0400 (EDT)
2From: "Software: Practice and Experience" <>
5Subject: Software: Practice and Experience - Decision on Manuscript ID SPE-18-0065
9Dear Dr Buhr,
11Many thanks for submitting SPE-18-0065 entitled "Cforall : Adding Modern Programming
12Language Features to C" to Software: Practice and Experience. The paper has now
13been reviewed and the comments of the referee(s) are included at the bottom of
14this letter.
16I am delighted to inform you that the referee(s) have recommended publication,
17but also suggest some minor revisions to your manuscript.  Therefore, I invite
18you to respond to the referee(s)' comments and revise your manuscript. All of
19the referees' comments are important, but I would like you to pay special
20attention to the following general points:
221. If there is any more evaluation that a stack, please add it.
232. How is the compiler implemented?
243. What features are actually implemented?
254. The article lacks some related work. such as Haskell, ML etc.
265. Most of the content in Section 10 RELATED WORK appears to belong to Section 1 INTRODUCTION as a Subsection or as a new Section after Section 1.
276. Many references are not properly formatted
287. A statement about any presence or absence of conflicts of interest with Huawei should be explicitly added.
30The paper is long by SPE standards (33 pages). We have a maximum of 40
31pages. Please do not extend the paper beyond 35 pages. If necessary, find ways
32to cut the examples or text. If you have an accompanying website for the system
33where some examples are stored, please mention it.
35You have 42 days from the date of this email to submit your revision. If you
36are unable to complete the revision within this time, please contact me to
37request a short extension.
39You can upload your revised manuscript and submit it through your Author
40Center. Log into and enter your Author
41Center, where you will find your manuscript title listed under "Manuscripts
42with Decisions".
44When submitting your revised manuscript, you will be able to respond to the
45comments made by the referee(s) in the space provided.  You can use this space
46to document any changes you make to the original manuscript.
48If you feel that your paper could benefit from English language polishing, you
49may wish to consider having your paper professionally edited for English
50language by a service such as Wiley's at
51 Please note that while this service will
52greatly improve the readability of your paper, it does not guarantee acceptance
53of your paper by the journal.
55Once again, thank you for submitting your manuscript to Software: Practice and Experience. I look forward to receiving your revision.
59Dr Judith Bishop
60Editor, Software: Practice and Experience
64We have attempted to response to all the issues raised by the Editor and referee's comments. For two
65of the issues, we have "pushed back", with an explanation. Specifically, moving the related work
66forward, and moving text from Section 9 into the captions of Table2 and Figure 10.  Our reasons for
67not making these changes are address below. Finally, as pointed out below, there are a couple of
68issues with the Wiley LaTeX macros that we worked around as best as possible.
70   The paper is long by SPE standards (33 pages). We have a maximum of 40 pages. Please do not
71   extend the paper beyond 35 pages. If necessary, find ways to cut the examples or text. If you
72   have an accompanying website for the system where some examples are stored, please mention it.
74The paper is 35 pages using the supplied Wiley LaTeX macros.
77Referee(s)' Comments to Author:
79Reviewing: 1
81   Most of the content in Section 10 RELATED WORK appears to belong to Section 1 INTRODUCTION as a
82   Subsection or as a new Section after Section 1. (Please also see #4.1 below.) Remaining
83   discussion that cannot be moved earlier can become a DISCUSSION Section or a Subsection within
84   the last Section of the paper.
86Sometimes it is appropriate to put related work at the start of a paper and sometimes at the
87end. For this paper, it seems appropriate to put the related work at the end of the paper. The
88purpose of the related work in this paper is two fold: to introduce prior work and to contrast it
89with Cforall.  Only at the end of the paper does the reader have sufficient knowledge about Cforall
90to make detailed contrasts with other programming languages possible. If the related work is moved
91to the end of the introduction, the reader knows nothing about Cforall so talking about other
92programming languages in isolation makes little sense, especially non-C-related languages, like
93Java, Go, Rust, Haskell. We see no easy way to separate the related work into a general discussion
94at the start and a specific discussion at the end. We explicitly attempt to deal with the reader's
95anticipation at the end of the introduction:
97 Finally, it is impossible to describe a programming language without usages before definitions.
98 Therefore, syntax and semantics appear before explanations; hence, patience is necessary until
99 details are presented.
102   2.1 More information should be moved from the text and added to Figure 10 and Table 2 so that
103   readers can understand the comparison quickly. Imagine a reader read the summary and jump
104   directly to these two display elements. Questions would be raised about the binary size and pop
105   pair result of Cforall and it would take time to find answers in the text.
107This suggestion is an alternative writing style. The experiment is complex enough that it is
108unlikely a reader could jump to the table/graph and understand the experiment without putting a
109substantive amount of the text from Section 9 into the table and figure, which the reader then has
110to read anyway.  In fact, we prefer a writing style where the reader does not have to look at the
111table/figure to understand the experiment and the results, i.e., the table/figure are only there to
112complement the discussion.
115   2.2 The pronunciation of ("C-for-all") should be provided in the summary (page 1 line 22) so that
116   people not having an access to the full-text can see it.
121   2.3 Error comment in the code should be written with the same capitalization and it will be
122   helpful if you say specifically compilation error or runtime error. (Please see attached
123   annotated manuscript.)
125Fixed. All errors in the paper are compilation errors because they are related to the type system.
128   2.4 It is possible to provide a bit more information in Appendix A e.g. how many lines/bytes of
129   code and some details about software/hardware can be added/moved here. The aim is to provide
130   sufficient information for readers to reproduce the results and to appreciate the context of the
131   comparison.
133Table 2 indicates the source-code size in lines of code. The third paragraph of Section 9 gives
134precise details of the software/hardware used in the experiments.
136   3. Practical information about the work
138   There are three separate pieces of information on pages 2 ("All features discussed in this paper
139   are working, unless otherwise stated as under construction."),
141This sentence is replace with:
143 All languages features discussed in this paper are working, except some advanced exception-handling
144 features.
146and Section 5.4 Exception Handling states:
148 The following framework for Cforall exception handling is in place, excluding some runtime
149 type-information and virtual functions.
152   page 4 ("Under construction is a mechanism to distribute...")
154The feature on page 4 is now complete.
157   and page 33 ("There is ongoing work on a wide range ... ")
159This sentence is replace to indicate the ongoing work is future work.
161 While all examples in the paper compile and run, a public beta-release of Cforall will take 6-8
162 months to reduce compilation time, provide better debugging, and add a few more libraries.  There
163 is also new work on a number of Cforall features, including arrays with size, runtime
164 type-information, virtual functions, user-defined conversions, and modules.
167   My recommendation is to move them to an appendix so that the length is preserved.
169There is nothing to move into an appendix, except 3 sentences. We do not intend to discuss these
170items in this paper.
173   3.1 Any under construction work (only small part of page 4) should not be mingled into the main
174   part of the manuscript.
176See above.
179   3.2 Instructions on how to access/use the working functionality of Cforall should be given.
181We will indicate release of Cforall in a public location, when we believe the code base is
182acceptable. In the interim, we have made public all the experimental code from section 9, and there
183is a reference in the paper to access this code. We can make a private beta-copy of Cforall
184available to the SP&E editor for distribution to the referees so they can verify our claims.
186   3.3 Planned work should be given a specific time of completion/release not just "8-12 months".
188Software development is not a rigorous engineering discipline. Given our small research
189development-team and the size of the project, we cannot give a specific time for completion of
190anything associated with the project. Having said that, we have reduced our expected time for
191Cforall release to 6-8 months as work is progressing well.
194   4.1 The impression after reading Section 1 INTRODUCTION is that the referencing is poor. It is
195   not until Section 10 RELATED WORK where majority of the prior literature are discussed. Please
196   consider moving the content and improve citations - at least cite all main variations of C
197   languages.
199See point 1.
202   4.2 I also would like to see citations at these specific places: Page 2 after Phil Karlton, page
203   22 after const hell problem.
205The Phil-Karlton quote is an urban legend without a specific academic citation:
209The term "const hell" is replaced with "const poisoning" with a citation.
212   5.1 Footnotes and citations will need to have different schemes - number and perhaps letter.
214Switched to letters. SP&E uses symbol footnotes but this macros fails with their macros:
216 \renewcommand*{\thefootnote}{\fnsymbol{footnote}}
219   5.2 Many references are not properly formatted e.g. date is incomplete, extra/missing white
220   spaces, extra dots, use of page number or section number as part of superscript ref
221   number. Please refer to attached document.
223Agreed. The bibtex BST macros are at fault. I have fixed some issues but I cannot fix them all as my
224BST macro-knowledge is limited.
227   5.3 Typos:
228   - Page 3 "eager" should be "earlier"
232   - Page 4 "vals" should be "arr"
234Actually it is "vals", and the example is changed so it is clear why.
236   - Page 21 "than" should be "then"
241   6. Conflict of interest
242   I see that the work is partially supported by Huawei. Perhaps statement about any presence or
243   absence of conflicts of interest should be explicitly added. Please get a clear direction on this
244   from the editor of the journal.
246The paper now states the project is open-source, hence there is no conflict of interest with the
247funding received from Huawei.
250Reviewing: 2
252Comments to the Author
254   Overloading requires the compiler to mangle a function's signature into its name in the object
255   file.  I'm pretty sure that this will complicate the build process of mixed Cforall/C projects.
257There is no complexity with building Cforall/C programs, and there is an existence proof because C++
258has name mangling for overloading and has no problem interacting with C.
261   I found the evaluation underwhelming.  There were only ~200 LoC ported from C to Cforall.  This
262   is too less to encounter potential caveats Cforall's type system might impose.
264We have clarified that the evaluation is not for the type system, but rather the underlying
265implementation approach for the parametric polymorphism. Section 9 now starts:
267 Cforall adds parametric polymorphism to C.  A runtime evaluation is performed to compare the cost
268 of alternative styles of polymorphism.  The goal is to compare just the underlying mechanism for
269 implementing different kinds of polymorphism.
271and ends with:
273 We conjecture these results scale across most generic data-types as the underlying polymorphic
274 implement is constant.
277   Also, how is the compiler implemented?  I guess, Cforall is a source-to-source compiler (from
278   Cforall to C).  But this is left in the dark.  What features are actually implemented?
280The following paragraph has been added to the introduction to address this comment:
282 All languages features discussed in this paper are working, except some advanced exception-handling
283 features.  Not discussed in this paper are the integrated concurrency-constructs and user-level
284 threading-library~\cite{Delisle18}.  Cforall is an open-source project implemented as an
285 source-to-source translator from Cforall to the gcc-dialect of C~\cite{GCCExtensions}, allowing it
286 to leverage the portability and code optimizations provided by gcc, meeting goals (1)--(3).
287 Ultimately, a compiler is necessary for advanced features and optimal performance.  The Cforall
288 translator is 200+ files and 46,000+ lines of code written in C/C++.  Starting with a translator
289 versus a compiler makes it easier and faster to generate and debug C object-code rather than
290 intermediate, assembler or machine code.  The translator design is based on the visitor pattern,
291 allowing multiple passes over the abstract code-tree, which works well for incrementally adding new
292 feature through additional visitor passes.  At the heart of the translator is the type resolver,
293 which handles the polymorphic routine/type overload-resolution.  The Cforall runtime system is 100+
294 files and 11,000+ lines of code, written in Cforall.  Currently, the Cforall runtime is the largest
295 user of Cforall providing a vehicle to test the language features and implementation.  The Cforall
296 tests are 290+ files and 27,000+ lines of code.  The tests illustrate syntactic and semantic
297 features in Cforall, plus a growing number of runtime benchmarks.  The tests check for correctness
298 and are used for daily regression testing of commits (3800+).
301   Furthermore, the article lacks some related work.  Many proposed features are present in
302   functional languages such as Haskell, ML etc.  In particular, the dealing of parametric
303   polymorphism reminds me of Haskell.
305The following paragraph has been added at the start of Section 10.1:
307 ML~\cite{ML} was the first language to support parametric polymorphism.  Like Cforall, it supports
308 universal type parameters, but not the use of assertions and traits to constrain type arguments.
309 Haskell~\cite{Haskell10} combines ML-style polymorphism, polymorphic data types, and type inference
310 with the notion of type classes, collections of overloadable methods that correspond in intent to
311 traits in Cforall.  Unlike Cforall, Haskell requires an explicit association between types and
312 their classes that specifies the implementation of operations.  These associations determine the
313 functions that are assertion arguments for particular combinations of class and type, in contrast
314 to Cforall where the assertion arguments are selected at function call sites based upon the set of
315 operations in scope at that point.  Haskell also severely restricts the use of overloading: an
316 overloaded name can only be associated with a single class, and methods with overloaded names can
317 only be defined as part of instance declarations.
320   Cforall's approach to tuples is also quite similar to many functional languages.
322At the end of Section 10.2, we state:
324 Tuples are a fundamental abstraction in most functional programming languages, such as Standard ML,
325 Haskell}, and Scala, which decompose tuples using pattern matching.
328From: Judith Bishop <>
329To: "Peter A. Buhr" <>
330Subject: RE: Software: Practice and Experience - Decision on Manuscript ID
331 SPE-18-0065
332Date: Tue, 24 Apr 2018 16:45:51 +0000
333Accept-Language: em-US
335Hi Peter
337Great to hear from you. I am also glad your paper got through, as it is in the
338mainline of the SPE scope.
340It is important to mention that the software is open source. People really
341value that. In the acknowledgements, you can refer to Huawei for funding. It is
342quite normal to have industrial funding, and in fact it is a plus.
344I think that sorts out the comment from the referee.
346Looking forward to your revised submission.
348Kind regards
350Judith Bishop
351Extraordinary Professor, Computer Science
352Stellenbosch University, South Africa
353082 301 5220 / 021 671 5133     LinkedIn
356-----Original Message-----
357From: Peter A. Buhr <>
358Sent: Tuesday, April 24, 2018 6:25 PM
361Subject: Re: Software: Practice and Experience - Decision on Manuscript ID SPE-18-0065
363Hi Judy! Hope all is well.
365We are over-the-moon to get our paper accepted at SP&E, and we are actively
366working on your and the referee's comments.
368One comment where we need assistance is:
370  7. A statement about any presence or absence of conflicts of interest with
371     Huawei should be explicitly added.
373We forgotten to mention in the paper that our project is open-source. So Huawei
374was funding an open-source project. In fact, the Huawei funding ends soon, so
375there will be no direct affiliation in a couple of months, although there are a
376few people at Huawei who remain very interested in the project.
378So does stating that the Cforall project is an open-source project deal with
379the issue of conflict of interest?
Note: See TracBrowser for help on using the repository browser.