source: doc/papers/OOPSLA17/refereeReport.txt @ e98c7ab

Last change on this file since e98c7ab was f4e3419d, checked in by Peter A. Buhr <pabuhr@…>, 6 years ago

restructure paper documents

  • Property mode set to 100644
File size: 12.3 KB
2                           OOPSLA'17 Review #20A
4 Paper #20: Generic and Tuple Types with Efficient Dynamic Layout in C∀
7                      Overall merit: C. Weak paper, though I will not fight
8                                        strongly against it
9                         Confidence: X. I am an expert in this area
11                         ===== Paper summary =====
13This presents an extension of the C programming language that tries to preserve the character of the existing language, while adding tuples and generics. Unlike C++ templates, generics preserve separate compilation. Types are represented at runtime, if needed, by size and alignment values, along with pointers to the code for any needed operators. A microbenchmark performance comparison is provided.
15                      ===== Comments for author =====
17This is an interesting extension to C, that may be of interest to some C programmers. It generally seems to be fairly well engineered, and mostly respects C's design goals.
19Unfortunately, there have been enough proposals for extended C dialects that this sort of design is tough to sell. And I don't think the evaluation really went far enough to make that case.
21The ideas in the paper don't appear to be fundamentally new. The idea of passing types as runtime objects has certainly been explored before. An additional ancient reference is
23There seems to be a new idea of minimally describing types using alignment and size attributes instead of (?) pointers to assignment operators and the like. But this scheme is not very well described. Notably, it is not clear how, say, a struct with atomic field or bit-fields would be described.
25I wasn't quite clear on the extent to which operator overloading is supported. The MAX example appears to me like it would be quite controversial among C programmers.
27It is not obvious that type inference here always converges. An outline of the algorithm would be useful.
29Above all, this needs experience results from a more complete implementation.
33Relying on TIOBE here seems a bit dubious. Since it counts web pages, and C isn't exactly new and hot, it may actually understate your case.
35The print example seems a little simplistic, since it's not clear how it handles formatting.
37"does not using the return type"
39              ===== Questions for authors’ response =====
41How are atomics, volatile, and bit-fields in structs handled?
44                           OOPSLA'17 Review #20B
46 Paper #20: Generic and Tuple Types with Efficient Dynamic Layout in C∀
49                      Overall merit: D. Reject
50                         Confidence: X. I am an expert in this area
52                         ===== Paper summary =====
54The authors present an extension to C, adding universal polymorphism and tuples. These features are described in prose. There is an implementation, though this is not described in depth in the paper. There is a benchmark evaluation.
56                      ===== Comments for author =====
58The paper is well-written and the concepts explained well. It is nice to see work in the low-level/C space - I believe that it is an area that has not been well-served by the OOPSLA community. My concerns with the paper are that the contribution is rather small and the concepts are not well-evaluated; specifically this is a language design paper and there is no attempt to evaluate the actual language design.
60While it is reasonable to describe only a couple of features in a paper, I would then expect a detailed description of the implementation and/or a formalism with proven safety properties and a thorough evaluation of the design. For a paper which only describes the design of a language the bar is higher than two features - for example, a description of a 'large' language such as D or Rust, even then I would expect a stronger evaluation.
62## On the design of C-forall
64There are some interesting points in the design of generics, notably the otype/dtype distinction. The design seems reasonable and follows what I would expect from other languages. The design for tuples is more unusual - the usual design of simple anonymous records with anonymous fields is extended with a mix of 'spread'ing, variadics, and implicit conversions. Importantly, the authors neither justify nor evaluate this departure - that is a severe omission for this paper. Furthermore, the only in-depth description of the implementation in the paper concerns tuples, and it seems to me that this is only interesting because of the unusual design - further reason for justifying it.
66## Evaluation
68The paper evaluates the implementation of C-forall with (effectively) a single micro-benchmark. That benchmark seems to show that C-forall performs worse than C++ on every measure, but this is not really discussed.
70A better performance evaluation would consist of multiple tests, both micro-benchmarks and realistic code and would test C-forall compared to alternatives (D, Rust, Go, etc.) not just C/C++.
72However, performance is not the really interesting thing to test here. The authors propose a new language and while performance is an important consideration for systems languages, it is far from the most important. I would like to see the usability of the language tested with user studies of different kinds (various levels of skill-level and coding scenarios). The authors could also use case studies or programming idioms to compare programming in C-forall vs the alternatives (again, comparing with D, Rust, etc. is more interesting to me than C).
74Finally, in designing C-forall, the authors make several assumptions about why C programmers use C. These should be backed up either with evaluation or citation. Statements in the paper certainly do not reflect my experience discussing language design with C programmers, and I would like to see them verified.
77## Related work
79The related work section is broad and gives good descriptions of other languages. However, the comparisons between languages focus more on the high-level goals of the language. It would be more interesting to focus on the details of the languages - the comparisons between Cyclone, C++, Java, and C-forall generics are good, I would like to see more of this with D and Rust, which are the more modern alternatives to C-forall (for example, Rust's notion of Sized and ?Sized types seems similar to otypes/dtypes).
81The related work is really missing any discussion of why the C-forall design choices are better than other languages. To clarify, I mean the specific design of generics and tuples, c.f., the suitability of the language in general because of garbage collection or learning difficulties.
84                           OOPSLA'17 Review #20C
86 Paper #20: Generic and Tuple Types with Efficient Dynamic Layout in C∀
89                      Overall merit: D. Reject
90                         Confidence: Z. I am not an expert; my evaluation
91                                        is that of an informed outsider
93                         ===== Paper summary =====
95The paper presents two language features of "Cforall": generics and tuples.
97                      ===== Comments for author =====
99The authors really need to talk about C++ as early as possible IMHO. That's the first thing that came to mind when reading the abstract: how is this different from C++?
101Comparison with C++:
102The main difference with C++ seems to be that Cforall favors separate compilation at the expense of runtime overhead while C++ systematically avoids any runtime overhead (at the expense of slow compilation times). C++ approach makes more sense IMHO. While it's true that people where using C for almost everything 30 years ago, that is just not true anymore. Most people writing C today are doing system programming, otherwise there would be using a higher level programming language (C#, Java etc ...).
103Now, when doing system programming, one needs very fine grain control over the resources: memory layout, etc ...
104It is pretty clear to me that the people writing that kind of code will favor generics that do not cost any overhead at runtime, otherwise they would be writing Java in the first place.
105The authors need to better justify the runtime overhead, or give escape hatches for those who don't want to pay that cost at runtime.
106They very often go back to the benefit of separate compilation, but that's not enough IMHO. Here is a proposal: why not have 2 modes, one called debug mode, used while developing the code, that would compile generics with a runtime overhead. Another, called production, that would unfold the world like C++ does?
108About Tuples:
109The section about tuples is too long. I would have spent more time explaining generics.
112"This installation base"
113Unclear what you mean by that.
115"Prior projects ... but failed ..."
116Hummm ... What about C++.
118"... object-oriented or functional programming with garbage collection ..."
119You are really mixing apples and oranges here. Many C programmers have nothing agains object-oriented features, not even functional programming (C++ 11 adds
120a bunch of features proving my point), but it's clear that most of them feel very strongly against automated garbage collection.
122"In many cases, C++ is often ..."
123This sentence feels like it is coming out of nowhere.
125"... the polymorphic runtime-cost ..."
126Is there any way to avoid that overhead? It's true it will make the compiler faster, but there are cases where the user might not want to pay for
127the overhead at runtime. Is there a way to force the compiler to specialize the code?
129"... to write a type-safe Cforall wrapper malloc based ..."
130That cannot be true in general. Malloc produces a pointer (of any type), given an integer (the size).
131It looks like Cforall is assuming that the integer is the result of a call to sizeof (a good practice in C).
132However, if that's the case, it should be explained.
134"... allows variable overloading ..."
135How are conflict resolved? In other words, what happens when two variables could be used?
137"... reuses the generated structure declarations where appropriate."
138This is too vague.
140"... have multiple outcomes, some exceptional."
141Humm, I would say these two things are distinct. Let's just way that this way of presenting things is strange, I woulds ay that a function can either
142return one or multiple values or throw an exception. Not that some of the values returned are "exceptional".
144"The type-resolver ..."
145What's that? Type-checker? Type-inference?
147"... applies C conversions."
148Noooo! That's exactly what leads to very subtle bugs. Is there any way to stop those conversions from happening?
150"The minimal cost ..."
151In what regard? Runtime cost? How does the "resolver" know how expensive the conversions are?
153"z = 10 // mass assignments"
154That stuff is completely unreadable. Why not introduce a new operator?
156"... roughly equivalent time ..."
157Well, C++ looks faster to me.
159"... is restricted because the resolution does not using ..."
160Did you mean, does not use?
162"... D and go are garbage collected ..."
163Yes, but in D, the use of the GC is optional.
165"... while respecting the talent and skill of C programmers."
166Are you implying that other approaches are not?
168"On the surface, the project may appear as a rehash of similar mechanisms in C++."
171"... integration with C and its programmers ..."
172Bold claim. What makes you think you are integrated with programmers? Number of users?
174"... inline annotation at polymorphic function call sites to create a template-specialization ..."
175This should have been mentioned sooner. Plus conflating inlining and specialization is unfortunate.
176Does "inline" also inline the function? Or does it only specialize the code?
177If it also inline, that's a very unfortunate design. I might want to specialize the code, but without inlining ...
178How do I specialize a recursive function?
Note: See TracBrowser for help on using the repository browser.