Index: doc/proposals/concurrency/Makefile =================================================================== --- doc/proposals/concurrency/Makefile (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ doc/proposals/concurrency/Makefile (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -9,4 +9,6 @@ SOURCES = ${addsuffix .tex, \ concurrency \ +style \ +glossary \ } Index: doc/proposals/concurrency/concurrency.tex =================================================================== --- doc/proposals/concurrency/concurrency.tex (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ doc/proposals/concurrency/concurrency.tex (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -14,10 +14,10 @@ % Latex packages used in the document. -\usepackage[T1]{fontenc} % allow Latin1 (extended ASCII) characters +\usepackage[T1]{fontenc} % allow Latin1 (extended ASCII) characters \usepackage{textcomp} \usepackage[latin1]{inputenc} \usepackage{fullpage,times,comment} \usepackage{epic,eepic} -\usepackage{upquote} % switch curled `'" to straight +\usepackage{upquote} % switch curled `'" to straight \usepackage{calc} \usepackage{xspace} @@ -25,15 +25,15 @@ \usepackage{tabularx} \usepackage[acronym]{glossaries} -\usepackage{varioref} % extended references +\usepackage{varioref} % extended references \usepackage{inconsolata} -\usepackage{listings} % format program code -\usepackage[flushmargin]{footmisc} % support label/reference in footnote -\usepackage{latexsym} % \Box glyph -\usepackage{mathptmx} % better math font with "times" +\usepackage{listings} % format program code +\usepackage[flushmargin]{footmisc} % support label/reference in footnote +\usepackage{latexsym} % \Box glyph +\usepackage{mathptmx} % better math font with "times" \usepackage[usenames]{color} \usepackage[pagewise]{lineno} \usepackage{fancyhdr} \renewcommand{\linenumberfont}{\scriptsize\sffamily} -\input{common} % bespoke macros used in the document +\input{style} % bespoke macros used in the document \usepackage[dvips,plainpages=false,pdfpagelabels,pdfpagemode=UseNone,colorlinks=true,pagebackref=true,linkcolor=blue,citecolor=blue,urlcolor=blue,pagebackref=true,breaklinks=true]{hyperref} \usepackage{breakurl} @@ -44,5 +44,5 @@ \renewcommand{\UrlFont}{\small\sf} -\setlength{\topmargin}{-0.45in} % move running title into header +\setlength{\topmargin}{-0.45in} % move running title into header \setlength{\headsep}{0.25in} @@ -86,5 +86,5 @@ \title{Concurrency in \CFA} \author{Thierry Delisle \\ -Dept. of Computer Science, University of Waterloo, \\ Waterloo, Ontario, Canada +School of Computer Science, University of Waterloo, \\ Waterloo, Ontario, Canada } @@ -100,8 +100,7 @@ \section{Introduction} -This proposal provides a minimal core concurrency API that is both simple, efficient and can be reused to build higher-level features. The simplest possible core is a thread and a lock but this low-level approach is hard to master. An easier approach for users is to support higher-level construct as the basis of the concurrency in \CFA. -Indeed, for highly productive parallel programming high-level approaches are much more popular\cite{HPP:Study}. Examples are task based parallelism, message passing, implicit threading. - -There are actually two problems that need to be solved in the design of the concurrency for a language. Which concurrency tools are available to the users and which parallelism tools are available. While these two concepts are often seen together, they are in fact distinct concepts that require different sorts of tools\cite{Buhr05a}. Concurrency tools need to handle mutual exclusion and synchronization while parallelism tools are more about performance, cost and resource utilization. +This proposal provides a minimal core concurrency API that is both simple, efficient and can be reused to build higher-level features. The simplest possible concurrency core is a thread and a lock but this low-level approach is hard to master. An easier approach for users is to support higher-level constructs as the basis of the concurrency in \CFA. Indeed, for highly productive parallel programming, high-level approaches are much more popular~\cite{HPP:Study}. Examples are task based, message passing and implicit threading. + +There are actually two problems that need to be solved in the design of the concurrency for a programming language: which concurrency tools are available to the users and which parallelism tools are available. While these two concepts are often seen together, they are in fact distinct concepts that require different sorts of tools~\cite{Buhr05a}. Concurrency tools need to handle mutual exclusion and synchronization, while parallelism tools are more about performance, cost and resource utilization. % ##### ####### # # ##### # # ###### ###### ####### # # ##### # # @@ -114,14 +113,5 @@ \section{Concurrency} -% Several tool can be used to solve concurrency challenges. Since these challenges always appear with the use of mutable shared state, some languages and libraries simply disallow mutable shared-state (Erlang\cite{Erlang}, Haskell\cite{Haskell}, Akka (Scala)\cite{Akka}). In these paradigms, interaction among concurrent objects rely on message passing or other paradigms that often closely relate to networking concepts. However, in imperative or OO languages, these approaches entail a clear distinction between concurrent and non-concurrent paradigms (i.e. message passing versus routine call). Which in turns mean that programmers need to learn two sets of designs patterns in order to be effective. Approaches based on shared memory are more closely related to non-concurrent paradigms since they often rely on non-concurrent constructs like routine calls and objects. At a lower level these can be implemented as locks and atomic operations. However, for productivity reasons it is desireable to have a higher-level construct to be the core concurrency paradigm\cite{HPP:Study}. This project proposes Monitors\cite{Hoare74} as the core concurrency construct. -% \\ - -Several tool can be used to solve concurrency challenges. Since these challenges always appear with the use of mutable shared state, some languages and libraries simply disallow mutable shared-state (Erlang\cite{Erlang}, Haskell\cite{Haskell}, Akka (Scala)\cite{Akka}). In these paradigms, interaction among concurrent objects rely on message passing\cite{Thoth,Harmony,V-Kernel} or other paradigms that often closely relate to networking concepts. However, in imperative or OO languages, these approaches entail a clear distinction between concurrent and non-concurrent paradigms (i.e. message passing versus routine call). Which in turns mean that programmers need to learn two sets of designs patterns in order to be effective. Approaches based on shared memory are more closely related to non-concurrent paradigms since they often rely on non-concurrent constructs like routine calls and objects. At a lower level these can be implemented as locks and atomic operations. Many such mechanisms have been proposed, including semaphores~\cite{Dijkstra68b} and path expressions~\cite{Campbell74}. However, for productivity reasons it is desireable to have a higher-level construct to be the core concurrency paradigm\cite{HPP:Study}. One of the most natural, elegant, and efficient mechanisms for synchronization and communication, especially for shared memory systems, is the \emph{monitor}. - -Monitors were first proposed by Brinch Hansen~\cite{Hansen73} and later described and extended by C.A.R.~Hoare~\cite{Hoare74}. -Many programming languages---e.g., Concurrent Pascal~\cite{ConcurrentPascal}, Mesa~\cite{Mesa}, Modula~\cite{Modula-2}, Turing~\cite{Turing:old}, Modula-3~\cite{Modula-3}, NeWS~\cite{NeWS}, Emerald~\cite{Emerald}, \uC~\cite{Buhr92a} and Java~\cite{Java}---provide monitors as explicit language constructs. In addition, operating-system kernels and device drivers have a monitor-like structure, although they often use lower-level primitives such as semaphores or locks to simulate monitors. For these reasons, this project proposes Monitors as the core concurrency construct. -\\ - -Finally, an approach that is worth mentionning because it is gaining in popularity is transactionnal memory\cite{Dice10}. However, the performance and feature set is currently too restrictive to be possible to add such a paradigm to a language like C or \CC\cit, which is why it was rejected as the core paradigm for concurrency in \CFA. +Several tool can be used to solve concurrency challenges. Since these challenges always appear with the use of mutable shared-state, some languages and libraries simply disallow mutable shared-state (Erlang~\cite{Erlang}, Haskell~\cite{Haskell}, Akka (Scala)~\cite{Akka}). In these paradigms, interaction among concurrent objects relies on message passing~\cite{Thoth,Harmony,V-Kernel} or other paradigms that closely relate to networking concepts (channels\cit for example). However, in languages that use routine calls as their core abstraction mechanism, these approaches force a clear distinction between concurrent and non-concurrent paradigms (i.e., message passing versus routine call). Which in turn means that, in order to be effective, programmers need to learn two sets of designs patterns. This distinction can be hidden away in library code, but effective use of the librairy still has to take both paradigms into account. Approaches based on shared memory are more closely related to non-concurrent paradigms since they often rely on basic constructs like routine calls and objects. At a lower level these can be implemented as locks and atomic operations. Many such mechanisms have been proposed, including semaphores~\cite{Dijkstra68b} and path expressions~\cite{Campbell74}. However, for productivity reasons it is desireable to have a higher-level construct be the core concurrency paradigm~\cite{HPP:Study}. An approach that is worth mentionning because it is gaining in popularity is transactionnal memory~\cite{Dice10}[Check citation]. While this approach is even pursued by system languages like \CC\cit, the performance and feature set is currently too restrictive to add such a paradigm to a language like C or \CC\cit, which is why it was rejected as the core paradigm for concurrency in \CFA. One of the most natural, elegant, and efficient mechanisms for synchronization and communication, especially for shared memory systems, is the \emph{monitor}. Monitors were first proposed by Brinch Hansen~\cite{Hansen73} and later described and extended by C.A.R.~Hoare~\cite{Hoare74}. Many programming languages---e.g., Concurrent Pascal~\cite{ConcurrentPascal}, Mesa~\cite{Mesa}, Modula~\cite{Modula-2}, Turing~\cite{Turing:old}, Modula-3~\cite{Modula-3}, NeWS~\cite{NeWS}, Emerald~\cite{Emerald}, \uC~\cite{Buhr92a} and Java~\cite{Java}---provide monitors as explicit language constructs. In addition, operating-system kernels and device drivers have a monitor-like structure, although they often use lower-level primitives such as semaphores or locks to simulate monitors. For these reasons, this project proposes monitors as the core concurrency construct. % # # ####### # # ### ####### ####### ###### ##### @@ -134,5 +124,5 @@ \subsection{Monitors} -A monitor is a set of routines that ensure mutual exclusion when accessing shared state. This concept is generally associated with Object-Oriented Languages like Java\cite{Java} or \uC\cite{uC++book} but does not strictly require OOP semantics. The only requirements is the ability to declare a handle to a shared object and a set of routines that act on it : +A monitor is a set of routines that ensure mutual exclusion when accessing shared state. This concept is generally associated with Object-Oriented Languages like Java~\cite{Java} or \uC~\cite{uC++book} but does not strictly require OOP semantics. The only requirements is the ability to declare a handle to a shared object and a set of routines that act on it : \begin{lstlisting} typedef /*some monitor type*/ monitor; @@ -154,27 +144,23 @@ \subsubsection{Call semantics} \label{call} -The above example of monitors already displays some of their intrinsic caracteristics. Indeed, it is necessary to use pass-by-reference over pass-by-value for monitor routines. This semantics is important because at their core, monitors are implicit mutual exclusion objects (locks), and these objects cannot be copied. Therefore, monitors are implicitly non-copyable. -\\ - -Another aspect to consider is when a monitor acquires its mutual exclusion. Indeed, a monitor may need to be passed through multiple helper routines that do not acquire the monitor mutual exclusion on entry. Examples of this can be both generic helper routines (\code{swap}, \code{sort}, etc.) or specific helper routines like the following example : - -\begin{lstlisting} - mutex struct counter_t { /*...*/ }; - - void ?{}(counter_t & nomutex this); - int ++?(counter_t & mutex this); - void ?{}(Int * this, counter_t & mutex cnt); -\end{lstlisting} -*semantics of the declaration of \code{mutex struct counter_t} are discussed in details in section \ref{data} -\\ - -This example is of a monitor implementing an atomic counter. Here, the constructor uses the \code{nomutex} keyword to signify that it does not acquire the coroutine mutual exclusion when constructing. This is because object not yet constructed should never be shared and therefore do not require mutual exclusion. The prefix increment operator -uses \code{mutex} to protect the incrementing process from race conditions. Finally, we have a conversion operator from \code{counter_t} to \code{Int}. This conversion may or may not require the \code{mutex} key word depending whether or not reading an \code{Int} is an atomic operation or not. -\\ - -Having both \code{mutex} and \code{nomutex} keywords could be argued to be redundant based on the meaning of a routine having neither of these keywords. If there were a meaning to routine \code{void foo(counter_t & this)} then one could argue that it should be to default to the safest option : \code{mutex}. On the other hand, the option of having routine \code{void foo(counter_t & this)} mean \code{nomutex} is unsafe by default and may easily cause subtle errors. It can be argued that this is the more "normal" behavior, \code{nomutex} effectively stating explicitly that "this routine has nothing special". An other alternative is to make one of these keywords mandatory, which would provide the same semantics but without the ambiguity of supporting routine \code{void foo(counter_t & this)}. Mandatory keywords would also have the added benefice of being more clearly self-documented but at the cost of extra typing. In the end, which solution should be picked is still up for debate. For the reminder of this proposal, the explicit approach will be used for the sake of clarity. -\\ - -Regardless of which keyword is kept, it is important to establish when mutex/nomutex may be used depending on type parameters. +The above monitor example displays some of the intrinsic characteristics. Indeed, it is necessary to use pass-by-reference over pass-by-value for monitor routines. This semantics is important because at their core, monitors are implicit mutual-exclusion objects (locks), and these objects cannot be copied. Therefore, monitors are implicitly non-copyable. + +Another aspect to consider is when a monitor acquires its mutual exclusion. For example, a monitor may need to be passed through multiple helper routines that do not acquire the monitor mutual-exclusion on entry. Pass through can be both generic helper routines (\code{swap}, \code{sort}, etc.) or specific helper routines like the following to implement an atomic counter : + +\begin{lstlisting} + mutex struct counter_t { /*...see section §\ref{data}§...*/ }; + + void ?{}(counter_t & nomutex this); //constructor + size_t ++?(counter_t & mutex this); //increment + + //need for mutex is platform dependent here + void ?{}(size_t * this, counter_t & mutex cnt); //conversion +\end{lstlisting} + +Here, the constructor(\code{?\{\}}) uses the \code{nomutex} keyword to signify that it does not acquire the monitor mutual exclusion when constructing. This semantics is because an object not yet constructed should never be shared and therefore does not require mutual exclusion. The prefix increment operator uses \code{mutex} to protect the incrementing process from race conditions. Finally, there is a conversion operator from \code{counter_t} to \code{size_t}. This conversion may or may not require the \code{mutex} key word depending on whether or not reading an \code{size_t} is an atomic operation or not. + +Having both \code{mutex} and \code{nomutex} keywords could be argued to be redundant based on the meaning of a routine having neither of these keywords. For example, given a routine without wualifiers \code{void foo(counter_t & this)} then one could argue that it should default to the safest option \code{mutex}. On the other hand, the option of having routine \code{void foo(counter_t & this)} mean \code{nomutex} is unsafe by default and may easily cause subtle errors. It can be argued that \code{nomutex} is the more "normal" behaviour, the \code{nomutex} keyword effectively stating explicitly that "this routine has nothing special". Another alternative is to make having exactly one of these keywords mandatory, which would provide the same semantics but without the ambiguity of supporting routine \code{void foo(counter_t & this)}. Mandatory keywords would also have the added benefice of being self-documented but at the cost of extra typing. In the end, which solution should be picked is still up for debate. For the reminder of this proposal, the explicit approach is used for clarity. + +The next semantic decision is to establish when mutex/nomutex may be used as a type qualifier. Consider the following declarations: \begin{lstlisting} int f1(monitor & mutex m); @@ -184,6 +170,5 @@ int f5(graph(monitor*) & mutex m); \end{lstlisting} - -The problem is to indentify which object(s) should be acquired. Furthermore we also need to acquire each objects only once. In case of simple routines like \code{f1} and \code{f2} it is easy to identify an exhaustive list of objects to acquire on entering. Adding indirections (\code{f3}) still allows the compiler and programmer to indentify which object will be acquired. However, adding in arrays (\code{f4}) makes it much harder. Array lengths aren't necessarily known in C and even then making sure we only acquire objects once becomes also none trivial. This can be extended to absurd limits like \code{f5} which uses a custom graph of monitors. To keep everyone as sane as possible\cite{Chicken}, this projects imposes the requirement that a routine may only acquire one monitor per parameter and it must be the type of the parameter (ignoring potential qualifiers and indirections). +The problem is to indentify which object(s) should be acquired. Furthermore, each object needs to be acquired only once. In the case of simple routines like \code{f1} and \code{f2} it is easy to identify an exhaustive list of objects to acquire on entry. Adding indirections (\code{f3}) still allows the compiler and programmer to indentify which object is acquired. However, adding in arrays (\code{f4}) makes it much harder. Array lengths are not necessarily known in C and even then making sure we only acquire objects once becomes also none trivial. This can be extended to absurd limits like \code{f5}, which uses a graph of monitors. To keep everyone as sane as possible~\cite{Chicken}, this projects imposes the requirement that a routine may only acquire one monitor per parameter and it must be the type of the parameter (ignoring potential qualifiers and indirections). Also note that while routine \code{f3} can be supported, meaning that monitor \code{**m} is be acquired, passing an array to this routine would be type safe and yet result in undefined behavior because only the first element of the array is acquired. However, this ambiguity is part of the C type system with respects to arrays. For this reason, it would also be reasonnable to disallow mutex in the context where arrays may be passed. % ###### # ####### # @@ -196,5 +181,5 @@ \subsubsection{Data semantics} \label{data} -Once the call semantics are established, the next step is to establish data semantics. Indeed, until now a monitor is used simply as a generic handle but in most cases monitors contian shared data. This data should be intrinsic to the monitor declaration to prevent any accidental use of data without its appripriate protection. For example here is a more fleshed-out version of the counter showed in \ref{call}: +Once the call semantics are established, the next step is to establish data semantics. Indeed, until now a monitor is used simply as a generic handle but in most cases monitors contian shared data. This data should be intrinsic to the monitor declaration to prevent any accidental use of data without its appropriate protection. For example, here is a complete version of the counter showed in section \ref{call}: \begin{lstlisting} mutex struct counter_t { @@ -207,34 +192,31 @@ int ++?(counter_t & mutex this) { - return ++this->value; - } - + return ++this.value; + } + + //need for mutex is platform dependent here void ?{}(int * this, counter_t & mutex cnt) { *this = (int)cnt; } \end{lstlisting} -\begin{tabular}{ c c } -Thread 1 & Thread 2 \\ -\begin{lstlisting} - void f(counter_t & mutex c) { - for(;;) { - sout | (int)c | endl; - } - } -\end{lstlisting} &\begin{lstlisting} - void g(counter_t & mutex c) { - for(;;) { - ++c; - } - } - + +This simple counter is used as follows: +\begin{center} +\begin{tabular}{c @{\hskip 0.35in} c @{\hskip 0.35in} c} +\begin{lstlisting} + //shared counter + counter_t cnt; + + //multiple threads access counter + thread 1 : cnt++; + thread 2 : cnt++; + thread 3 : cnt++; + ... + thread N : cnt++; \end{lstlisting} \end{tabular} -\\ - - -This simple counter offers an example of monitor usage. Notice how the counter is used without any explicit synchronisation and yet supports thread-safe semantics for both reading and writting. \\ - -These simple mutual exclusion semantics also naturally expand to multi-monitor calls. +\end{center} + +Notice how the counter is used without any explicit synchronisation and yet supports thread-safe semantics for both reading and writting. Unlike object-oriented monitors, where calling a mutex member \emph{implicitly} acquires mutual-exclusion, \CFA uses an explicit mechanism to acquire mutual-exclusion. A consequence of this approach is that it extends to multi-monitor calls. \begin{lstlisting} int f(MonitorA & mutex a, MonitorB & mutex b); @@ -244,25 +226,30 @@ f(a,b); \end{lstlisting} - -This code acquires both locks before entering the critical section. In practice, writing multi-locking routines that can not lead to deadlocks can be very tricky. Having language level support for such feature is therefore a significant asset for \CFA. However, this does have significant repercussions relating to scheduling (see \ref{insched} and \ref{extsched}). Furthermore, the ability to acquire multiple monitors at the same time does incur a significant pitfall even without looking into scheduling. For example : -\begin{lstlisting} - void foo(A & mutex a, B & mutex a) { - //... - } - - void bar(A & mutex a, B & nomutex a) - //... - foo(a, b); - //... - } - - void baz(A & nomutex a, B & mutex a) - //... - foo(a, b); - //... - } -\end{lstlisting} - -Recursive mutex routine calls are allowed in \CFA but if not done carefully it can lead to nested monitor call problems\cite{Lister77}. These problems which are a specific implementation of the lock acquiring order problem. In the example above, the user uses implicit ordering in the case of function \code{bar} but explicit ordering in the case of \code{baz}. This subtle mistake can mean that calling these two functions concurrently will lead to deadlocks, depending on the implicit ordering matching the explicit ordering. As shown on several occasion\cit, there isn't really any solutions to this problem, users simply need to be carefull when acquiring multiple monitors at the same time. +This code acquires both locks before entering the critical section, called \emph{\gls{group-acquire}}. In practice, writing multi-locking routines that do not lead to deadlocks is tricky. Having language support for such a feature is therefore a significant asset for \CFA. In the case presented above, \CFA guarantees that the order of aquisition is consistent across calls to routines using the same monitors as arguments. However, since \CFA monitors use multi-acquisition locks, users can effectively force the acquiring order. For example, notice which routines use \code{mutex}/\code{nomutex} and how this affects aquiring order : +\begin{lstlisting} + void foo(A & mutex a, B & mutex b) { //acquire a & b + //... + } + + void bar(A & mutex a, B & nomutex b) { //acquire a + //... + foo(a, b); //acquire b + //... + } + + void baz(A & nomutex a, B & mutex b) { //acquire b + //... + foo(a, b); //acquire a + //... + } +\end{lstlisting} + +The multi-acquisition monitor lock allows a monitor lock to be acquired by both \code{bar} or \code{baz} and acquired again in \code{foo}. In the calls to \code{bar} and \code{baz} the monitors are acquired in opposite order. such use leads to nested monitor call problems~\cite{Lister77}, which is a specific implementation of the lock acquiring order problem. In the example above, the user uses implicit ordering in the case of function \code{foo} but explicit ordering in the case of \code{bar} and \code{baz}. This subtle mistake means that calling these routines concurrently may lead to deadlock and is therefore undefined behavior. As shown on several occasion\cit, solving this problem requires : +\begin{enumerate} + \item Dynamically tracking of the monitor-call order. + \item Implement rollback semantics. +\end{enumerate} + +While the first requirement is already a significant constraint on the system, implementing a general rollback semantics in a C-like language is prohibitively complex \cit. In \CFA, users simply need to be carefull when acquiring multiple monitors at the same time. % ###### ####### ####### # ### # ##### @@ -283,7 +270,74 @@ \subsubsection{Implementation Details: Interaction with polymorphism} -At first glance, interaction between monitors and \CFA's concept of polymorphism seem complexe to support. However, it can be reasoned that entry-point locking can solve most of the issues that could be present with polymorphism. - -First of all, interaction between \code{otype} polymorphism and monitors is impossible since monitors do not support copying. Therefore the main question is how to support \code{dtype} polymorphism. We must remember that monitors' main purpose is to ensure mutual exclusion when accessing shared data. This implies that mutual exclusion is only required for routines that do in fact access shared data. However, since \code{dtype} polymorphism always handle incomplete types (by definition) no \code{dtype} polymorphic routine can access shared data since the data would require knowledge about the type. Therefore the only concern when combining \code{dtype} polymorphism and monitors is to protect access to routines. With callsite-locking, this would require significant amount of work since any \code{dtype} routine could have to obtain some lock before calling a routine. However, with entry-point-locking calling a monitor routine becomes exactly the same as calling it from anywhere else. +At first glance, interaction between monitors and \CFA's concept of polymorphism seems complex to support. However, it is shown that entry-point locking can solve most of the issues. + +Before looking into complex control flow, it is important to present the difference between the two acquiring options : \gls{callsite-locking} and \gls{entry-point-locking}, i.e. acquiring the monitors before making a mutex call or as the first instruction of the mutex call. For example: + +\begin{center} +\begin{tabular}{|c|c|c|} +Code & \gls{callsite-locking} & \gls{entry-point-locking} \\ +\CFA & pseudo-code & pseudo-code \\ +\hline +\begin{lstlisting} +void foo(monitor & mutex a) { + + + + //Do Work + //... + +} + +void main() { + monitor a; + + + + foo(a); + +} +\end{lstlisting} &\begin{lstlisting} +foo(& a) { + + + + //Do Work + //... + +} + +main() { + monitor a; + //calling routine + //handles concurrency + acquire(a); + foo(a); + release(a); +} +\end{lstlisting} &\begin{lstlisting} +foo(& a) { + //called routine + //handles concurrency + acquire(a); + //Do Work + //... + release(a); +} + +main() { + monitor a; + + + + foo(a); + +} +\end{lstlisting} +\end{tabular} +\end{center} + +First of all, interaction between \code{otype} polymorphism and monitors is impossible since monitors do not support copying. Therefore, the main question is how to support \code{dtype} polymorphism. Since a monitor's main purpose is to ensure mutual exclusion when accessing shared data, this implies that mutual exclusion is only required for routines that do in fact access shared data. However, since \code{dtype} polymorphism always handles incomplete types (by definition), no \code{dtype} polymorphic routine can access shared data since the data requires knowledge about the type. Therefore, the only concern when combining \code{dtype} polymorphism and monitors is to protect access to routines. \Gls{callsite-locking} would require a significant amount of work, since any \code{dtype} routine may have to obtain some lock before calling a routine, depending on whether or not the type passed is a monitor. However, with \gls{entry-point-locking} calling a monitor routine becomes exactly the same as calling it from anywhere else. + + % ### # # ####### ##### ##### # # ####### ###### @@ -296,5 +350,5 @@ \subsection{Internal scheduling} \label{insched} -Monitors should also be able to schedule what threads access it as a mean of synchronization. Internal scheduling is one of the simple examples of such a feature. It allows users to declare condition variables and wait for them to be signaled. Here is a simple example of such a technique : +Monitors also need to schedule waiting threads internally as a mean of synchronization. Internal scheduling is one of the simple examples of such a feature. It allows users to declare condition variables and have threads wait and signaled from them. Here is a simple example of such a technique : \begin{lstlisting} @@ -314,6 +368,5 @@ \end{lstlisting} -Here routine \code{foo} waits on the \code{signal} from \code{bar} before making further progress, effectively ensuring a basic ordering. This semantic can easily be extended to multi-monitor calls by offering the same guarantee. - +Note that in \CFA, \code{condition} have no particular need to be stored inside a monitor, beyond any software engineering reasons. Here routine \code{foo} waits for the \code{signal} from \code{bar} before making further progress, effectively ensuring a basic ordering. This semantic can easily be extended to multi-monitor calls by offering the same guarantee. \begin{center} \begin{tabular}{ c @{\hskip 0.65in} c } @@ -321,5 +374,5 @@ \begin{lstlisting} void foo(monitor & mutex a, - monitor & mutex b) { + monitor & mutex b) { //... wait(a.e); @@ -330,5 +383,5 @@ \end{lstlisting} &\begin{lstlisting} void bar(monitor & mutex a, - monitor & mutex b) { + monitor & mutex b) { signal(a.e); } @@ -340,6 +393,5 @@ \end{tabular} \end{center} - -A direct extension of the single monitor semantics would be to release all locks when waiting and transferring ownership of all locks when signalling. However, for the purpose of synchronization it may be usefull to only release some of the locks but keep others. On the technical side, partially releasing lock is feasible but from the user perspective a choice must be made for the syntax of this feature. It is possible to do without any extra syntax by relying on order of acquisition (Note that here the use of helper routines is irrelevant, only routines the acquire mutual exclusion have an impact on internal scheduling): +A direct extension of the single monitor semantics is to release all locks when waiting and transferring ownership of all locks when signalling. However, for the purpose of synchronization it may be usefull to only release some of the locks but keep others. It is possible to support internal scheduling and \gls{group-acquire} without any extra syntax by relying on order of acquisition. Here is an example of the different contexts in which internal scheduling can be used. (Note that here the use of helper routines is irrelevant, only routines acquire mutual exclusion have an impact on internal scheduling): \begin{center} @@ -350,7 +402,9 @@ condition e; +//acquire a & b void foo(monitor & mutex a, - monitor & mutex b) { - wait(e); + monitor & mutex b) { + + wait(e); //release a & b } @@ -364,12 +418,14 @@ condition e; +//acquire a void bar(monitor & mutex a, - monitor & nomutex b) { + monitor & nomutex b) { foo(a,b); } +//acquire a & b void foo(monitor & mutex a, - monitor & mutex b) { - wait(e); + monitor & mutex b) { + wait(e); //release a & b } @@ -378,12 +434,14 @@ condition e; +//acquire a void bar(monitor & mutex a, - monitor & nomutex b) { - foo(a,b); -} - + monitor & nomutex b) { + baz(a,b); +} + +//acquire b void baz(monitor & nomutex a, - monitor & mutex b) { - wait(e); + monitor & mutex b) { + wait(e); //release b } @@ -393,147 +451,109 @@ \end{center} -This can be interpreted in two different ways : -\begin{flushleft} -\begin{enumerate} - \item \code{wait} atomically releases the monitors acquired by the inner-most routine, \underline{ignoring} nested calls. - \item \code{wait} atomically releases the monitors acquired by the inner-most routine, \underline{considering} nested calls. -\end{enumerate} -\end{flushleft} -While the difference between these two is subtle, it has a significant impact. In the first case it means that the calls to \code{foo} would behave the same in Context 1 and 2. This semantic would also mean that the call to \code{wait} in routine \code{baz} would only release \code{monitor b}. While this may seem intuitive with these examples, it does have one significant implication, it creates a strong distinction between acquiring multiple monitors in sequence and acquiring the same monitors simulatenously, i.e. : +Context 1 is the simplest way of acquiring more than one monitor (\gls{group-acquire}), using a routine with multiple parameters having the \code{mutex} keyword. Context 2 also uses \gls{group-acquire} as well in routine \code{foo}. However, the routine is called by routine \code{bar}, which only acquires monitor \code{a}. Since monitors can be acquired multiple times this does not cause a deadlock by itself but it does force the acquiring order to \code{a} then \code{b}. Context 3 also forces the acquiring order to be \code{a} then \code{b} but does not use \gls{group-acquire}. The previous example tries to illustrate the semantics that must be established to support releasing monitors in a \code{wait} statement. In all cases, the behavior of the wait statment is to release all the locks that were acquired my the inner-most monitor call. That is \code{a & b} in context 1 and 2 and \code{b} only in context 3. Here are a few other examples of this behavior. + \begin{center} -\begin{tabular}{c @{\hskip 0.35in} c @{\hskip 0.35in} c} -\begin{lstlisting} -enterMonitor(a); -enterMonitor(b); -// do stuff -leaveMonitor(b); -leaveMonitor(a); -\end{lstlisting} & != &\begin{lstlisting} -enterMonitor(a); -enterMonitor(a, b); -// do stuff -leaveMonitor(a, b); -leaveMonitor(a); +\begin{tabular}{|c|c|c|} +\begin{lstlisting} +condition e; + +//acquire b +void foo(monitor & nomutex a, + monitor & mutex b) { + bar(a,b); +} + +//acquire a +void bar(monitor & mutex a, + monitor & nomutex b) { + + wait(e); //release a + //keep b +} + +foo(a, b); +\end{lstlisting} &\begin{lstlisting} +condition e; + +//acquire a & b +void foo(monitor & mutex a, + monitor & mutex b) { + bar(a,b); +} + +//acquire b +void bar(monitor & mutex a, + monitor & nomutex b) { + + wait(e); //release b + //keep a +} + +foo(a, b); +\end{lstlisting} &\begin{lstlisting} +condition e; + +//acquire a & b +void foo(monitor & mutex a, + monitor & mutex b) { + bar(a,b); +} + +//acquire none +void bar(monitor & nomutex a, + monitor & nomutex b) { + + wait(e); //release a & b + //keep none +} + +foo(a, b); \end{lstlisting} \end{tabular} \end{center} - -This is not intuitive because even if both methods display the same monitors state both inside and outside the critical section respectively, the behavior is different. Furthermore, the actual acquiring order will be exaclty the same since acquiring a monitor from inside its mutual exclusion is a no-op. This means that even if the data and the actual control flow are the same using both methods, the behavior of the \code{wait} will be different. The alternative is option 2, that is releasing acquired monitors, \underline{considering} nesting. This solves the issue of having the two acquiring method differ at the cost of making routine \code{foo} behave differently depending on from which context it is called (Context 1 or 2). Indeed in Context 2, routine \code{foo} actually behaves like routine \code{baz} rather than having the same behavior than in Context 1. The fact that both implicit approaches can be unintuitive depending on the perspective may be a sign that the explicit approach is superior. For this reason this \CFA does not support implicit monitor releasing and uses explicit semantics. -\\ - -The following examples shows three alternatives of explicit wait semantics : -\\ - +Note the right-most example is actually a trick pulled on the reader. Monitor state information is stored in thread local storage rather then in the routine context, which means that helper routines and other \code{nomutex} routines are invisible to the runtime system in regards to concurrency. This means that in the right-most example, the routine parameters are completly unnecessary. However, calling this routine from outside a valid monitor context is undefined. + +These semantics imply that in order to release of subset of the monitors currently held, users must write (and name) a routine that only acquires the desired subset and simply calls wait. While users can use this method, \CFA offers the \code{wait_release}\footnote{Not sure if an overload of \code{wait} would work...} which will release only the specified monitors. In the center previous examples, the code in the center uses the \code{bar} routine to only release monitor \code{b}. Using the \code{wait_release} helper, this can be rewritten without having the name two routines : \begin{center} -\begin{tabular}{|c|c|c|} -Case 1 & Case 2 & Case 3 \\ -Branding on construction & Explicit release list & Explicit ignore list \\ -\hline -\begin{lstlisting} -void foo(monitor & mutex a, - monitor & mutex b, - condition & c) -{ - // Releases monitors - // branded in ctor - wait(c); -} - -monitor a; -monitor b; -condition1 c1 = {a}; -condition2 c2 = {a, b}; - -//Will release only a -foo(a,b,c1); - -//Will release a and b -foo(a,b,c2); +\begin{tabular}{ c c c } +\begin{lstlisting} + condition e; + + //acquire a & b + void foo(monitor & mutex a, + monitor & mutex b) { + bar(a,b); + } + + //acquire b + void bar(monitor & mutex a, + monitor & nomutex b) { + + wait(e); //release b + //keep a + } + + foo(a, b); \end{lstlisting} &\begin{lstlisting} -void foo(monitor & mutex a, - monitor & mutex b, - condition & c) -{ - // Releases monitor a - // Holds monitor b - waitRelease(c, [a]); -} - -monitor a; -monitor b; -condition c; - - - -foo(a,b,c); - - - + => \end{lstlisting} &\begin{lstlisting} -void foo(monitor & mutex a, - monitor & mutex b, - condition & c) -{ - // Releases monitor a - // Holds monitor b - waitHold(c, [b]); -} - -monitor a; -monitor b; -condition c; - - - -foo(a,b,c); - - - + condition e; + + //acquire a & b + void foo(monitor & mutex a, + monitor & mutex b) { + wait_release(e,b); //release b + //keep a + } + + foo(a, b); \end{lstlisting} \end{tabular} \end{center} -(Note : Case 2 and 3 use tuple semantics to pass a variable length list of elements.) -\\ - -All these cases have their pros and cons. Case 1 is more distinct because it means programmers need to be carefull about where the condition is initialized as well as where it is used. On the other hand, it is very clear and explicitly states which monitor is released and which monitor stays acquired. This is similar to Case 2, which releases only the monitors explictly listed. However, in Case 2, calling the \code{wait} routine instead of the \code{waitRelease} routine releases all the acquired monitor. The Case 3 is an improvement on that since it releases all the monitors except those specified. The result is that the \code{wait} routine can be written as follows : -\begin{lstlisting} -void wait(condition & cond) { - waitHold(cond, []); -} -\end{lstlisting} -This alternative offers nice and consistent behavior between \code{wait} and \code{waitHold}. However, one large pitfall is that mutual exclusion can now be violated by calls to library code. Indeed, even if the following example seems benign there is one significant problem : -\begin{lstlisting} -monitor global; - -extern void doStuff(); //uses global - -void foo(monitor & mutex m) { - //... - doStuff(); //warning can release monitor m - //... -} - -foo(global); -\end{lstlisting} - -Indeed, if Case 2 or 3 are chosen it any code can violate the mutual exclusion of the calling code by issuing calls to \code{wait} or \code{waitHold} in a nested monitor context. Case 2 can be salvaged by removing the \code{wait} routine from the API but Case 3 cannot prevent users from calling \code{waitHold(someCondition, [])}. For this reason the syntax proposed in Case 3 is rejected. Note that the syntax proposed in case 1 and 2 are not exclusive. Indeed, by supporting two types of condition both cases can be supported : -\begin{lstlisting} -struct condition { /*...*/ }; - -// Second argument is a variable length tuple. -void wait(condition & cond, [...] monitorsToRelease); -void signal(condition & cond); - -struct conditionN { /*...*/ }; - -void ?{}(conditionN* this, /*list of N monitors to release*/); -void wait(conditionN & cond); -void signal(conditionN & cond); -\end{lstlisting} - -Regardless of the option chosen for wait semantics, signal must be symmetrical. In all cases, signal only needs a single parameter, the condition variable that needs to be signalled. But \code{signal} needs to be called from the same monitor(s) that call to \code{wait}. Otherwise, mutual exclusion cannot be properly transferred back to the waiting monitor. - -Finally, an additionnal semantic which can be very usefull is the \code{signalBlock} routine. This routine behaves like signal for all of the semantics discussed above, but with the subtelty that mutual exclusion is transferred to the waiting task immediately rather than wating for the end of the critical section. + +Regardless of the context in which the \code{wait} statement is used, \code{signal} must be called holding the same set of monitors. In all cases, signal only needs a single parameter, the condition variable that needs to be signalled. But \code{signal} needs to be called from the same monitor(s) that call to \code{wait}. Otherwise, mutual exclusion cannot be properly transferred back to the waiting monitor. + +Finally, an additional semantic which can be very usefull is the \code{signal_block} routine. This routine behaves like signal for all of the semantics discussed above, but with the subtelty that mutual exclusion is transferred to the waiting task immediately rather than wating for the end of the critical section. \\ @@ -545,7 +565,7 @@ % # # # # ### # # # # # # # # # % ####### # # # ### ##### ##### # # ####### ###### - +\newpage \subsection{External scheduling} \label{extsched} -As one might expect, the alternative to Internal scheduling is to use External scheduling instead. This method is somewhat more robust to deadlocks since one of the threads keeps a relatively tight control on scheduling. Indeed, as the following examples will demonstrate, external scheduling allows users to wait for events from other threads without the concern of unrelated events occuring. External scheduling can generally be done either in terms of control flow (ex: \uC) or in terms of data (ex: Go). Of course, both of these paradigms have their own strenghts and weaknesses but for this project control flow semantics where chosen to stay consistent with the rest of the languages semantics. Two challenges specific to \CFA arise when trying to add external scheduling with loose object definitions and multi-monitor routines. The following example shows what a simple use \code{accept} versus \code{wait}/\code{signal} and its advantages. +An alternative to internal scheduling is to use external scheduling instead. This method is more constrained and explicit which may help users tone down the undeterministic nature of concurrency. Indeed, as the following examples demonstrates, external scheduling allows users to wait for events from other threads without the concern of unrelated events occuring. External scheduling can generally be done either in terms of control flow (ex: \uC) or in terms of data (ex: Go). Of course, both of these paradigms have their own strenghts and weaknesses but for this project control flow semantics where chosen to stay consistent with the rest of the languages semantics. Two challenges specific to \CFA arise when trying to add external scheduling with loose object definitions and multi-monitor routines. The following example shows a simple use \code{accept} versus \code{wait}/\code{signal} and its advantages. \begin{center} @@ -565,5 +585,5 @@ public: - void f(); + void f() { /*...*/ } void g() { _Accept(f); } private: @@ -573,5 +593,5 @@ \end{center} -In the case of internal scheduling, the call to \code{wait} only guarantees that \code{g} was the last routine to access the monitor. This intails that the routine \code{f} may have acquired mutual exclusion several times while routine \code{h} was waiting. On the other hand, external scheduling guarantees that while routine \code{h} was waiting, no routine other than \code{g} could acquire the monitor. +In the case of internal scheduling, the call to \code{wait} only guarantees that \code{g} is the last routine to access the monitor. This intails that the routine \code{f} may have acquired mutual exclusion several times while routine \code{h} was waiting. On the other hand, external scheduling guarantees that while routine \code{h} was waiting, no routine other than \code{g} could acquire the monitor. \\ @@ -756,21 +776,23 @@ % # # # # # # # ####### ####### ####### ####### ### ##### # # \section{Parallelism} -Historically, computer performance was about processor speeds and instructions count. However, with heat dissipation being an ever growing challenge, parallelism has become the new source of greatest performance \cite{Sutter05, Sutter05b}. In this decade, it is not longer reasonnable to create high-performance application without caring about parallelism. Indeed, parallelism is an important aspect of performance and more specifically throughput and hardware utilization. The lowest level approach of parallelism is to use \glspl{kthread}. However since these have significant costs and limitations \glspl{kthread} are now mostly used as an implementation tool rather than a user oriented one. There are several alternatives to solve these issues which all have strengths and weaknesses. +Historically, computer performance was about processor speeds and instructions count. However, with heat dissipation being a direct consequence of speed increase, parallelism has become the new source for increased performance~\cite{Sutter05, Sutter05b}. In this decade, it is not longer reasonnable to create a high-performance application without caring about parallelism. Indeed, parallelism is an important aspect of performance and more specifically throughput and hardware utilization. The lowest-level approach of parallelism is to use \glspl{kthread} in combination with semantics like \code{fork}, \code{join}, etc. However, since these have significant costs and limitations, \glspl{kthread} are now mostly used as an implementation tool rather than a user oriented one. There are several alternatives to solve these issues that all have strengths and weaknesses. While there are many variations of the presented paradigms, most of these variations do not actually change the guarantees or the semantics, they simply move costs in order to achieve better performance for certain workloads. \subsection{User-level threads} -A direct improvement on the \gls{kthread} approach is to use \glspl{uthread}. These threads offer most of the same features that the operating system already provide but can be used on a much larger scale. This is the most powerfull solution as it allows all the features of multi-threading while removing several of the more expensives costs of using kernel threads. The down side is that almost none of the low-level threading complexities are hidden, users still have to think about data races, deadlocks and synchronization issues. This can be somewhat alleviated by a concurrency toolkit with strong garantees but the parallelism toolkit offers very little to reduce complexity in itself. - -Examples of languages that support are Java\cite{Java}, Haskell\cite{Haskell} and \uC\cite{uC++book}. +A direct improvement on the \gls{kthread} approach is to use \glspl{uthread}. These threads offer most of the same features that the operating system already provide but can be used on a much larger scale. This approach is the most powerfull solution as it allows all the features of multi-threading, while removing several of the more expensives costs of using kernel threads. The down side is that almost none of the low-level threading problems are hidden, users still have to think about data races, deadlocks and synchronization issues. These issues can be somewhat alleviated by a concurrency toolkit with strong garantees but the parallelism toolkit offers very little to reduce complexity in itself. + +Examples of languages that support \glspl{uthread} are Erlang~\cite{Erlang} and \uC~\cite{uC++book}. + +\subsubsection{Fibers : user-level threads without preemption} +A popular varient of \glspl{uthread} is what is often reffered to as \glspl{fiber}. However, \glspl{fiber} do not present meaningful semantical differences with \glspl{uthread}. Advocates of \glspl{fiber} list their high performance and ease of implementation as majors strenghts of \glspl{fiber} but the performance difference between \glspl{uthread} and \glspl{fiber} is controversial and the ease of implementation, while true, is a weak argument in the context of language design. Therefore this proposal largely ignore fibers. + +An example of a language that uses fibers is Go~\cite{Go} \subsection{Jobs and thread pools} -The approach on the opposite end of the spectrum is to base parallelism on \glspl{job}. Indeed, \glspl{job} offer limited flexibility but at the benefit of a simpler user interface. In \gls{job} based systems users express parallelism as units of work and the dependency graph (either explicit or implicit) that tie them together. This means users need not to worry about concurrency but significantly limits the interaction that can occur between different jobs. Indeed, any \gls{job} that blocks also blocks the underlying \gls{kthread}, this effectively mean the CPU utilization, and therefore throughput, will suffer noticeably. -The golden standard of this implementation is Intel's TBB library\cite{TBB}. - -\subsection{Fibers : user-level threads without preemption} -Finally, in the middle of the flexibility versus complexity spectrum lay \glspl{fiber} which offer \glspl{uthread} without the complexity of preemption. This means users don't have to worry about other \glspl{fiber} suddenly executing between two instructions which signficantly reduces complexity. However, any call to IO or other concurrency primitives can lead to context switches. Furthermore, users can also block \glspl{fiber} in the middle of their execution without blocking a full processor core. This means users still have to worry about mutual exclusion, deadlocks and race conditions in their code, raising the complexity significantly. -An example of a language that uses fibers is Go\cite{Go} +The approach on the opposite end of the spectrum is to base parallelism on \glspl{pool}. Indeed, \glspl{pool} offer limited flexibility but at the benefit of a simpler user interface. In \gls{pool} based systems, users express parallelism as units of work and a dependency graph (either explicit or implicit) that tie them together. This approach means users need not worry about concurrency but significantly limits the interaction that can occur among jobs. Indeed, any \gls{job} that blocks also blocks the underlying worker, which effectively means the CPU utilization, and therefore throughput, suffers noticeably. It can be argued that a solution to this problem is to use more workers than available cores. However, unless the number of jobs and the number of workers are comparable, having a significant amount of blocked jobs always results in idles cores. + +The gold standard of this implementation is Intel's TBB library~\cite{TBB}. \subsection{Paradigm performance} -While the choice between the three paradigms listed above may have significant performance implication, it is difficult to pin the performance implications of chosing a model at the language level. Indeed, in many situations one of these paradigms will show better performance but it all strongly depends on the usage. Having mostly indepent units of work to execute almost guarantess that the \gls{job} based system will have the best performance. However, add interactions between jobs and the processor utilisation might suffer. User-level threads may allow maximum ressource utilisation but context switches will be more expansive and it is also harder for users to get perfect tunning. As with every example, fibers sit somewhat in the middle of the spectrum. Furthermore, if the units of uninterrupted work are large enough the paradigm choice will be largely amorticised by the actual work done. +While the choice between the three paradigms listed above may have significant performance implication, it is difficult to pindown the performance implications of chosing a model at the language level. Indeed, in many situations one of these paradigms may show better performance but it all strongly depends on the workload. Having a large amount of mostly independent units of work to execute almost guarantess that the \gls{pool} based system has the best performance thanks to the lower memory overhead. However, interactions between jobs can easily exacerbate contention. User-level threads allow fine-grain context switching, which results in better resource utilisation, but context switches will be more expansive and the extra control means users need to tweak more variables to get the desired performance. Furthermore, if the units of uninterrupted work are large enough the paradigm choice is largely amorticised by the actual work done. % ##### ####### # ####### ###### ###### @@ -783,20 +805,5 @@ \section{\CFA 's Thread Building Blocks} -As a system level language, \CFA should offer both performance and flexibilty as its primary goals, simplicity and user-friendliness being a secondary concern. Therefore, the core of parallelism in \CFA should prioritize power and efficiency. With this said, it is possible to deconstruct the three paradigms details aboved in order to get simple building blocks. Here is a table showing the core caracteristics of the mentionned paradigms : -\begin{center} -\begin{tabular}[t]{| r | c | c |} -\cline{2-3} -\multicolumn{1}{ c| }{} & Has a stack & Preemptive \\ -\hline -\Glspl{job} & X & X \\ -\hline -\Glspl{fiber} & \checkmark & X \\ -\hline -\Glspl{uthread} & \checkmark & \checkmark \\ -\hline -\end{tabular} -\end{center} - -As shown in section \ref{cfaparadigms} these different blocks being available in \CFA it is trivial to reproduce any of these paradigm. +As a system-level language, \CFA should offer both performance and flexibilty as its primary goals, simplicity and user-friendliness being a secondary concern. Therefore, the core of parallelism in \CFA should prioritize power and efficiency. With this said, deconstructing popular paradigms in order to get simple building blocks yields \glspl{uthread} as the core parallelism block. \Glspl{pool} and other parallelism paradigms can then be built on top of the underlying threading model. % ####### # # ###### ####### # ###### ##### @@ -809,5 +816,5 @@ \subsection{Thread Interface} -The basic building blocks of \CFA are \glspl{cfathread}. By default these are implemented as \glspl{uthread} and as such offer a flexible and lightweight threading interface (lightweight comparatievely to \glspl{kthread}). A thread can be declared using a struct declaration prefix with the \code{thread} as follows : +The basic building blocks of \CFA are \glspl{cfathread}. By default these are implemented as \glspl{uthread}, and as such, offer a flexible and lightweight threading interface (lightweight compared to \glspl{kthread}). A thread can be declared using a struct declaration with prefix \code{thread} as follows : \begin{lstlisting} @@ -815,14 +822,14 @@ \end{lstlisting} -Obviously, for this thread implementation to be usefull it must run some user code. Several other threading interfaces use some function pointer representation as the interface of threads (for example : \Csharp \cite{Csharp} and Scala \cite{Scala}). However, we consider that statically tying a \code{main} routine to a thread superseeds this approach. Since the \code{main} routine is definetely a special routine in \CFA, we can reuse the existing syntax for declaring routines with unordinary name, i.e. operator overloading. As such the \code{main} routine of a thread can be defined as such : +Obviously, for this thread implementation to be usefull it must run some user code. Several other threading interfaces use a function-pointer representation as the interface of threads (for example : \Csharp~\cite{Csharp} and Scala~\cite{Scala}). However, this proposal considers that statically tying a \code{main} routine to a thread superseeds this approach. Since the \code{main} routine is already a special routine in \CFA (where the program begins), the existing syntax for declaring routines names with special semantics can be extended, i.e. operator overloading. As such the \code{main} routine of a thread can be defined as : \begin{lstlisting} thread struct foo {}; - void ?main(thread foo* this) { - /*... Some useful code ...*/ - } -\end{lstlisting} - -With these semantics it is trivial to write a thread type that takes a function pointer as parameter and executes it on its stack asynchronously : + void ?main(foo* this) { + sout | "Hello World!" | endl; + } +\end{lstlisting} + +In this example, threads of type \code{foo} will start there execution in the \code{void ?main(foo*)} routine which in this case prints \code{"Hello World!"}. While this proposoal encourages this approach which is enforces strongly type programming. Users may prefer to use the routine based thread semantics for the sake of simplicity. With these semantics it is trivial to write a thread type that takes a function pointer as parameter and executes it on its stack asynchronously : \begin{lstlisting} typedef void (*voidFunc)(void); @@ -833,26 +840,24 @@ //ctor - void ?{}(thread FuncRunner* this, voidFunc inFunc) { + void ?{}(FuncRunner* this, voidFunc inFunc) { func = inFunc; } //main - void ?main(thread FuncRunner* this) { + void ?main(FuncRunner* this) { this->func(); } \end{lstlisting} -% In this example \code{func} is a function pointer stored in \acrfull{tls}, which is \CFA is both easy to use and completly typesafe. - -Of course for threads to be useful, it must be possible to start and stop threads and wait for them to complete execution. While using an \acrshort{api} such as \code{fork} and \code{join} is relatively common in the literature, such an interface is not needed. Indeed, the simplest approach is to use \acrshort{raii} principles and have threads \code{fork} once the constructor has completed and \code{join} before the destructor runs. -\begin{lstlisting} -thread struct FuncRunner; //FuncRunner declared above - -void world() { +Of course for threads to be useful, it must be possible to start and stop threads and wait for them to complete execution. While using an \acrshort{api} such as \code{fork} and \code{join} is relatively common in the literature, such an interface is unnecessary. Indeed, the simplest approach is to use \acrshort{raii} principles and have threads \code{fork} once the constructor has completed and \code{join} before the destructor runs. +\begin{lstlisting} +thread struct World; //FuncRunner declared above + +void ?main(thread World* this) { sout | "World!" | endl; } void main() { - FuncRunner run = {world}; + World w; //Thread run forks here @@ -863,7 +868,6 @@ } \end{lstlisting} -This semantic has several advantages over explicit semantics : typesafety is guaranteed, any thread will always be started and stopped exaclty once and users can't make any progamming errors. Furthermore it naturally follows the memory allocation semantics which means users don't need to learn multiple semantics. - -These semantics also naturally scale to multiple threads meaning basic synchronisation is very simple : +This semantic has several advantages over explicit semantics : typesafety is guaranteed, a thread is always started and stopped exaclty once and users cannot make any progamming errors. However, one of the apparent drawbacks of this system is that threads now always form a lattice, that is they are always destroyed in opposite order of construction. While this seems like a significant limitation, existing \CFA semantics can solve this problem. Indeed, by using dynamic allocation to create threads will naturally let threads outlive the scope in which the thread was created much like dynamically allocating memory will let objects outlive the scope in which thy were created : + \begin{lstlisting} thread struct MyThread { @@ -872,8 +876,45 @@ //ctor - void ?{}(thread MyThread* this) {} + void ?{}(MyThread* this, + bool is_special = false) { + //... + } //main - void ?main(thread MyThread* this) { + void ?main(MyThread* this) { + //... + } + + void foo() { + MyThread* special_thread; + { + MyThread thrds = {false}; + //Start a thread at the beginning of the scope + + DoStuff(); + + //create a other thread that will outlive the thread in this scope + special_thread = new MyThread{true}; + + //Wait for the thread to finish + } + DoMoreStuff(); + + //Now wait for the special + } +\end{lstlisting} + +Another advantage of this semantic is that it naturally scale to multiple threads meaning basic synchronisation is very simple : + +\begin{lstlisting} + thread struct MyThread { + //... + }; + + //ctor + void ?{}(MyThread* this) {} + + //main + void ?main(MyThread* this) { //... } @@ -889,6 +930,64 @@ \end{lstlisting} +\subsection{Coroutines : A stepping stone}\label{coroutine} +While the main focus of this proposal is concurrency and paralellism, it is important to adress coroutines which are actually a significant underlying aspect of the concurrency system. Indeed, while having nothing todo with parallelism and arguably very little to do with concurrency, coroutines need to deal with context-switchs and and other context management operations. Therefore, this proposal includes coroutines both as an intermediate step for the implementation of threads and a first class feature of \CFA. + +The core API of coroutines revolve around two features : independent stacks and suspedn/resume. Much like threads the syntax for declaring a coroutine is declaring a type and a main routine for it to start : +\begin{lstlisting} + coroutine struct MyCoroutine { + //... + }; + + //ctor + void ?{}(MyCoroutine* this) { + + } + + //main + void ?main(MyCoroutine* this) { + sout | "Hello World!" | endl; + } +\end{lstlisting} + +One a coroutine is created, users can context switch to it using \code{suspend} and come back using \code{resume}. Here is an example of a solution to the fibonnaci problem using coroutines : +\begin{lstlisting} + coroutine struct Fibonacci { + int fn; // used for communication + }; + + void ?main(Fibonacci* this) { + int fn1, fn2; // retained between resumes + this->fn = 0; + fn1 = this->fn; + suspend(this); // return to last resume + + this->fn = 1; + fn2 = fn1; + fn1 = this->fn; + suspend(this); // return to last resume + + for ( ;; ) { + this->fn = fn1 + fn2; + fn2 = fn1; + fn1 = this->fn; + suspend(this); // return to last resume + } + } + + int next(Fibonacci& this) { + resume(&this); // transfer to last suspend + return this.fn; + } + + void main() { + Fibonacci f1, f2; + for ( int i = 1; i <= 10; i += 1 ) { + sout | next(f1) | '§\verb+ +§' | next(f2) | endl; + } + } +\end{lstlisting} + \newpage -\large{\textbf{WORK IN PROGRESS}} +\bf{WORK IN PROGRESS} \subsection{The \CFA Kernel : Processors, Clusters and Threads}\label{kernel} Index: doc/proposals/concurrency/glossary.tex =================================================================== --- doc/proposals/concurrency/glossary.tex (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ doc/proposals/concurrency/glossary.tex (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -1,3 +1,23 @@ \makeglossaries + +\longnewglossaryentry{callsite-locking} +{name={callsite-locking}} +{ +Locking done by the calling routine. With this technique, a routine calling a monitor routine will aquire the monitor \emph{before} making the call to the actuall routine. +} + +\longnewglossaryentry{entry-point-locking} +{name={entry-point-locking}} +{ +Locking done by the called routine. With this technique, a monitor routine called by another routine will aquire the monitor \emph{after} entering the routine body but prior to any other code. +} + +\longnewglossaryentry{group-acquire} +{name={bulked acquiring}} +{ +Implicitly acquiring several monitors when entering a monitor. +} + + \longnewglossaryentry{uthread} {name={user-level thread}} @@ -30,4 +50,12 @@ \textit{Synonyms : Tasks.} +} + +\longnewglossaryentry{pool} +{name={thread-pool}} +{ +Group of homogeneuous threads that loop executing units of works after another. + +\textit{Synonyms : } } Index: doc/proposals/concurrency/style.tex =================================================================== --- doc/proposals/concurrency/style.tex (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) +++ doc/proposals/concurrency/style.tex (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -0,0 +1,9 @@ +\input{common} % bespoke macros used in the document + +\lstset{ +morekeywords=[2]{nomutex,mutex,thread,wait,wait_release,signal,signal_block,accept,monitor,suspend,resume,coroutine}, +keywordstyle=[2]\color{blue}, % second set of keywords for concurency +basicstyle=\linespread{0.9}\tt\small, % reduce line spacing and use typewriter font +stringstyle=\sf\color{Mahogany}, % use sanserif font +commentstyle=\itshape\color{OliveGreen}, % green and italic comments +}% Index: doc/proposals/concurrency/version =================================================================== --- doc/proposals/concurrency/version (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ doc/proposals/concurrency/version (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -1,1 +1,1 @@ -0.4.99 +0.7.48 Index: doc/proposals/virtual.txt =================================================================== --- doc/proposals/virtual.txt (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) +++ doc/proposals/virtual.txt (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -0,0 +1,163 @@ +Proposal for virtual functionality + +Imagine the following code : + +trait drawable(otype T) { + void draw(T* ); +}; + +struct text { + char* text; +}; + +void draw(text*); + +struct line{ + vec2 start; + vec2 end; +}; + +void draw(line*); + +While all the members of this simple UI support drawing creating a UI that easily +supports both these UI requires some tedious boiler-plate code : + +enum type_t { text, line }; + +struct widget { + type_t type; + union { + text t; + line l; + }; +}; + +void draw(widget* w) { + switch(w->type) { + case text : draw(&w->text); break; + case line : draw(&w->line); break; + default : handle_error(); break; + } +} + +While this code will work as indented, adding any new widgets or any new widget behaviors +requires changing existing code to add the desired functionality. To ease this maintenance +effort required CFA introduces the concept of dynamic types, in a manner similar to C++. + +A simple usage of dynamic type with the previous example would look like : + +drawable* objects[10]; +fill_objects(objects); + +while(running) { + for(drawable* object : objects) { + draw(object); + } +} + +However, this is not currently do-able in the current CFA and furthermore is not +possible to implement statically. Therefore we need to add a new feature to handle +having dynamic types like this (That is types that are found dynamically not types +that change dynamically). + +C++ uses inheritance and virtual functions to find the +desired type dynamically. CFA takes inspiration from this solution. + +What we really want to do is express the fact that calling draw() on a object +should find the dynamic type of the parameter before calling the routine, much like the +hand written example given above. We can express this by adding the virtual keyword on +the parameter of the constraints on our trait: + +trait drawable(otype T) { + void draw(virtual T* ); +}; + +This expresses the idea that drawable is similar to an abstract base class in C++ and +also gives meaning to trying to take a pointer of drawable. That is anything that can +be cast to a drawable pointer has the necessary information to call the draw routine on +that type. Before that drawable was only a abstract type while now it also points to a +piece of storage which specify which behavior the object will have at run time. + +This storage needs to be allocate somewhere. C++ just adds an invisible pointer at +the beginning of the struct but we can do something more explicit for users, actually +have a visible special field : + +struct text { + char* text; + vtable drawable; +}; + +struct line{ + vtable drawable; + vec2 start; + vec2 end; +}; + +With these semantics, adding a "vtable drawable" means that text pointers and line pointers are now +convertible to drawable pointers. This conversion will not necessarily be a type only change however, indeed, +the drawable pointer will point to the field "vtable drawable" not the head of the struct. However, since all +the types are known at compile time, converting pointers becomes a simple offset operations. + +The vtable field contains a pointer to a vtable which contains all the information needed for the caller +to find the function pointer of the desired behavior. + +One of the limitations of this design is that it does not support double dispatching, which +concretely means traits cannot have routines with more than one virtual parameter. This design +would have many ambiguities if it did support multiple virtual parameter. A futher limitation is +that traits over more than one type cannot have vtables meaningfully defined for them, as the +particular vtable to use would be a function of the other type(s) the trait is defined over. + +It is worth noting that the function pointers in these vtables are bound at object construction, rather than +function call-site, as in Cforall's existing polymorphic functions. As such, it is possible that two objects +with the same static type would have a different vtable (consider what happens if draw(line*) is overridden +between the definitions of two line objects). Given that the virtual drawable* erases static types though, +this should not be confusing in practice. A more distressing possibility is that of creating an object that +outlives the scope of one of the functions in its vtable. This is certainly a possible bug, but it is of a +type that C programmers are familiar with, and should be able to avoid by the usual methods. + +Extensibility. + +One of the obvious critics of this implementation is that it lacks extensibility for classes +that cannot be modified (ex: Linux C headers). However this solution can be extended to +allow more extensibility by adding "Fat pointers". + +Indeed, users could already "solve" this issue by writing their own fat pointers as such: + +trait MyContext(otype T) { + void* get_stack(virtual T*) +}; + +void* get_stack(ucontext_t *context); + +struct fat_ucontext_t { + vtable MyContext; + ucontext_t *context; +} + +//Tedious forwarding routine +void* get_stack(fat_ucontext_t *ptr) { + return get_stack(ptr->context); +} + +However, users would have to write all the virtual methods they want to override and make +them all simply forward to the existing method that takes the corresponding POCO(Plain Old C Object). + +The alternative we propose is to use language level fat pointers : + +trait MyContext(otype T) { + void* get_stack(virtual T*) +}; + +void* get_stack(ucontext_t *context); + +//The type vptr(ucontext_t) all +vptr(ucontext_t) context; + +These behave exactly as the previous example but all the forwarding routines are automatically generated. + +Bikeshedding. + +It may be desirable to add fewer new keywords than discussed in this proposal; it is possible that "virtual" +could replace both "vtable" and "vptr" above with unambiguous contextual meaning. However, for purposes of +clarity in the design discussion it is beneficial to keep the keywords for separate concepts distinct. + Index: doc/working/.gitignore =================================================================== --- doc/working/.gitignore (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) +++ doc/working/.gitignore (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -0,0 +1,4 @@ +.declarative_resolver.tex.swp +declarative_resolver.aux +declarative_resolver.log +declarative_resolver.pdf Index: doc/working/declarative_resolver.tex =================================================================== --- doc/working/declarative_resolver.tex (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) +++ doc/working/declarative_resolver.tex (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -0,0 +1,172 @@ +\documentclass{article} + +\usepackage{amsmath} +\usepackage{amssymb} + +\usepackage{listings} +\lstset{ + basicstyle=\ttfamily, + mathescape +} + +\newcommand{\TODO}{\textbf{TODO:}~} +\newcommand{\NOTE}{\textit{NOTE:}~} + +\newcommand{\Z}{\mathbb{Z}} +\newcommand{\Znn}{\Z^{\oplus}} + +\newcommand{\conv}[2]{#1 \rightarrow #2} +\newcommand{\C}[1]{\mathtt{#1}} + +\newcommand{\ls}[1]{\left[ #1 \right]} +\newcommand{\rng}[2]{\left\{#1, \cdots #2\right\}} +\title{Declarative Description of Expression Resolution Problem} +\author{Aaron Moss} + +\begin{document} +\maketitle + +\section{Inputs} +\begin{itemize} +\item A set of types $T$. +\item A set of conversions $C \subset \{ \conv{from}{to} : from, to \in T \}$. + \begin{itemize} + \item $C$ is a directed acyclic graph (DAG). + \item \TODO There should be two of these, to separate the safe and unsafe conversions. + \end{itemize} +\item A set of names $N$ +\item A set of declarations $F$. Each declaration $f \in F$ has the following properties: + \begin{itemize} + \item A name $f.name \in N$, not guaranteed to be unqiue in $F$. + \item A return type $f.type \in T$ + \item A number of parameters $f.n \in \Znn$. + \item A list of parameter types $params = \ls{f_1, \cdots f_{f.n}}$, where each $f_i \in T$. + \begin{itemize} + \item \TODO This should be a list of elements from $T$ to account for tuples and void-returning functions. + \end{itemize} + \item \TODO This model needs to account for polymorphic functions. + \end{itemize} +\item A tree of expressions $E$, rooted at an expression $root$. Each expression $e \in E$ has the following properties: + \begin{itemize} + \item A name $e.name \in N$, not guaranteed to be unique in $E$ + \item A number of arguments $e.n \in \Znn$ + \item A list of arguments $args = \ls{e_1, \cdots e_{e.n}}$, where each $e_i \in E$; these arguments $e_i$ are considered the children of $e$ in the tree. + \end{itemize} +\end{itemize} + +\section{Problem} +An interpretation $x \in I$ has the following properties: +\begin{itemize} +\item An interpreted expression $x.expr \in E$. +\item A base declaration $x.decl \in F$. +\item A type $x.type \in T$ +\item A cost $x.cost \in \Znn$. + \begin{itemize} + \item \TODO Make this cost a tuple containing unsafe and polymorphic conversion costs later. + \end{itemize} +\item A number of sub-interpretations $x.n \in \Znn$. +\item A list of sub-interpretations $subs = \ls{x_1, \cdots x_{x.n}}$, where each $x_i \in I$. +\end{itemize} + +Starting from $I = \{\}$, iteratively generate interpretations according to the following rules until a fixed point is reached: +\begin{itemize} +\item \textbf{Generate all interpretations, given subexpression interpretations.} \\ + $\forall e \in E, f \in F$ such that $e.name = f.name$ and $e.n = f.n$, let $n = e.n$. \\ + If $\forall i \in \rng{1}{n}, \exists x_i \in I$ such that $x_i.expr = e_i \land x_i.type = f_i$, \\ + For each combination of $x_i$, generate a new interpretation $x$ as follows: + \begin{itemize} + \item $x.expr = e$. + \item $x.decl = f$. + \item $x.type = f.type$. + \item $x.cost = \sum_{i \in \rng{1}{n}} x_i.cost$. + \item $x.n = n$. + \end{itemize} + +\item \textbf{Generate conversions.} \\ + $\forall x \in I, \forall t \in T, \exists (x.type, t) \in C$, \\ + generate a new interpretation $x'$ as follows: + \begin{itemize} + \item $x'.type = t$. + \item $x'.cost = x.cost + 1$. + \item All other properties of $x'$ are identical to those of $x$. + \end{itemize} +\end{itemize} + + +Once $I$ has been completely generated, let $I' = { x \in I : x.expr = root }$. +\begin{itemize} +\item If $I' = \{\}$, report failure (no valid interpretation). +\item If there exists a unqiue $x^* \in I'$ such that $x^*.cost$ is minimal in $I'$, report $x^*$ (success). +\item Otherwise report failure (ambiguous). +\end{itemize} + +\section{Example} + +Here is a worked example for the following C code block: +\begin{lstlisting} +int x; // $x$ +void* x; // $x'$ + +long f(int, void*); // $f$ +void* f(void*, int); // $f'$ +void* f(void*, long); // $f''$ + +f( f( x, x ), x ); // $root:$f( $\gamma:$f( $\alpha:$x, $\beta:$x ), $\delta:$x ) +\end{lstlisting} + +Using the following subset of the C type system, this example includes the following set of declarations and expressions\footnote{$n$ can be inferred from the length of the appropriate list in the elements of $F$, $E$, and $I$, and has been ommitted for brevity.}: +\begin{align*} + T = \{ &\C{int}, \C{long}, \C{void*} \} \\ + C = \{ &\conv{\C{int}}{\C{long}} \} \\ + N = \{ &\C{x}, \C{f} \} \\ + F = \{ &x = \{ name: \C{x}, type: \C{int}, params: \ls{} \}, \\ + &x' = \{ name: \C{x}, type: \C{void*}, params: \ls{} \}, \\ + &f = \{ name: \C{f}, type: \C{long}, params: \ls{\C{int}, \C{void*}} \}, \\ + &f' = \{ name: \C{f}, type: \C{void*}, params: \ls{\C{void*}, \C{int}} \}, \\ + &f'' = \{ name: \C{f}, type: \C{void*}, params: \ls{\C{void*}, \C{long}} \} \} \\ + E = \{ &\alpha = \{ name: \C{x}, args: \ls{} \}, \\ + &\beta = \{ name: \C{x}, args: \ls{} \}, \\ + &\gamma = \{ name: \C{f}, args: \ls{\alpha, \beta} \}, \\ + &\delta = \{ name: \C{x}, args: \ls{} \}, \\ + &root = \{ name: \C{f}, args: \ls{\gamma, \delta} \} \} +\end{align*} + +Given these initial facts, the initial interpretations for the leaf expressions $\alpha$, $\beta$ \& $\delta$ can be generated from the subexpression rule: +\begin{align} + \{ &expr: \alpha, decl: x, type: \C{int}, cost: 0, subs: \ls{} \} \\ + \{ &expr: \alpha, decl: x', type: \C{void*}, cost: 0, subs: \ls{} \} \\ + \{ &expr: \beta, decl: x, type: \C{int}, cost: 0, subs: \ls{} \} \\ + \{ &expr: \beta, decl: x', type: \C{void*}, cost: 0, subs: \ls{} \} \\ + \{ &expr: \delta, decl: x, type: \C{int}, cost: 0, subs: \ls{} \} \\ + \{ &expr: \delta, decl: x', type: \C{void*}, cost: 0, subs: \ls{} \} +\end{align} + +These new interpretations allow generation of further interpretations by the conversion rule and the $\conv{\C{int}}{\C{long}}$ conversion: +\begin{align} +\{ &expr: \alpha, decl: x, type: \C{long}, cost: 1, subs: \ls{} \} \\ +\{ &expr: \beta, decl: x, type: \C{long}, cost: 1, subs: \ls{} \} \\ +\{ &expr: \delta, decl: x, type: \C{long}, cost: 1, subs: \ls{} \} +\end{align} + +Applying the subexpression rule again to this set of interpretations, we can generate interpretations for $\gamma$ [$\C{f( x, x )}$]: +\begin{align} +\{ &expr: \gamma, decl: f, type: \C{long}, cost: 0, subs: \ls{ (1), (4) } \} \\ +\{ &expr: \gamma, decl: f', type: \C{void*}, cost: 0, subs: \ls{ (2), (3) } \} \\ +\{ &expr: \gamma, decl: f'', type: \C{void*}, cost: 1, subs: \ls{ (2), (8) } \} +\end{align} + +Since all of the new interpretations have types for which no conversions are applicable ($\C{void*}$ and $\C{long}$), the conversion rule generates no new interpretations. +If $\C{f(x, x)}$ was the root expression, the set of candidate interpretations $I'$ would equal $\{ (10), (11), (12) \}$. Since both $(10)$ and $(11)$ have cost $0$, there is no unique minimal-cost element of this set, and the resolver would report failure due to this ambiguity. + +However, having generated all the interpretations of $\C{f( x, x )}$, the subexpression rule can now be applied again to generate interpretations of the $root$ expression: +\begin{align} +\{ &expr: root, decl: f', type: \C{void*}, cost: 0, subs: \ls{ (11), (5) } \} \\ +\{ &expr: root, decl: f'', type: \C{void*}, cost: 1, subs: \ls{ (11), (9) } \} \\ +\{ &expr: root, decl: f', type: \C{void*}, cost: 1, subs: \ls{ (12), (5) } \} \\ +\{ &expr: root, decl: f'', type: \C{void*}, cost: 2, subs: \ls{ (12), (9) } \} +\end{align} + +Since again none of these new interpretations are of types with conversions defined, the conversion rule cannot be applied again; since the root expression has been resolved, no further applications of the subexpression rule are applicable either, therefore a fixed point has been reached and we have found the complete set of interpretations. If this fixed point had been reached before finding any valid interpretations of $root$ (e.g.~as would have happened if $f$ was the only declaration of $\C{f}$ in the program), the algorithm would have reported a failure with no valid interpretations. + +At the termination of this process, the set $I'$ of valid root interpretations is $\{ (13), (14), (15), (16)\}$; since $(13)$ has the unique minimal cost, it is the accepted interpretation of the root expression, and in this case the source $\C{f( f( x, x ), x )}$ is interpreted as $f'( f'( x', x ), x )$. +\end{document} Index: doc/working/resolver_design.md =================================================================== --- doc/working/resolver_design.md (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ doc/working/resolver_design.md (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -1382,4 +1382,12 @@ hypothesis needs to be empirically validated. +Another approach would be to abandon expression-tree ordering for +subexpression matching, and order by "most constrained symbol"; symbols would +be more constrained if there were fewer matching declarations, fewer +subexpressions yet to resolve, or possibly fewer possible types the expression +could resolve to. Ordering the expressions in a priority-queue by this metric +would not necessarily produce a top-down or a bottom-up order, but would add +opportunities for pruning based on memoized upper and lower bounds. + Both Baker and Cormack explicitly generate all possible interpretations of a given expression; thinking of the set of interpretations of an expression as a Index: src/CodeGen/CodeGenerator.cc =================================================================== --- src/CodeGen/CodeGenerator.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/CodeGen/CodeGenerator.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -307,10 +307,12 @@ } else { // no address-of operator, so must be a pointer - add dereference + // NOTE: if the assertion starts to trigger, check that the application expr isn't being shared. + // Since its arguments are modified here, this assertion most commonly triggers when the application + // is visited multiple times. UntypedExpr * newExpr = new UntypedExpr( new NameExpr( "*?" ) ); newExpr->get_args().push_back( *arg ); - assert( (*arg)->get_results().size() == 1 ); - Type * type = InitTweak::getPointerBase( (*arg)->get_results().front() ); - assert( type ); - newExpr->get_results().push_back( type->clone() ); + Type * type = InitTweak::getPointerBase( (*arg)->get_result() ); + assertf( type, "First argument to a derefence must be a pointer. Ensure that expressions are not being shared." ); + newExpr->set_result( type->clone() ); *arg = newExpr; } // if @@ -527,10 +529,10 @@ extension( castExpr ); output << "("; - if ( castExpr->get_results().empty() ) { + if ( castExpr->get_result()->isVoid() ) { output << "(void)" ; - } else if ( ! castExpr->get_results().front()->get_isLvalue() ) { + } else if ( ! castExpr->get_result()->get_isLvalue() ) { // at least one result type of cast, but not an lvalue output << "("; - output << genType( castExpr->get_results().front(), "" ); + output << genType( castExpr->get_result(), "" ); output << ")"; } else { @@ -640,5 +642,5 @@ } - void CodeGenerator::visit( TupleExpr * tupleExpr ) {} + void CodeGenerator::visit( TupleExpr * tupleExpr ) { assert( false ); } void CodeGenerator::visit( TypeExpr * typeExpr ) {} @@ -654,4 +656,39 @@ asmExpr->get_operand()->accept( *this ); output << " )"; + } + + void CodeGenerator::visit( CompoundLiteralExpr *compLitExpr ) { + assert( compLitExpr->get_type() && dynamic_cast< ListInit * > ( compLitExpr->get_initializer() ) ); + output << "(" << genType( compLitExpr->get_type(), "" ) << ")"; + compLitExpr->get_initializer()->accept( *this ); + } + + void CodeGenerator::visit( StmtExpr * stmtExpr ) { + std::list< Statement * > & stmts = stmtExpr->get_statements()->get_kids(); + output << "({" << std::endl; + cur_indent += CodeGenerator::tabsize; + unsigned int numStmts = stmts.size(); + unsigned int i = 0; + for ( Statement * stmt : stmts ) { + output << indent << printLabels( stmt->get_labels() ); + if ( i+1 == numStmts ) { + // last statement in a statement expression needs to be handled specially - + // cannot cast to void, otherwise the expression statement has no value + if ( ExprStmt * exprStmt = dynamic_cast< ExprStmt * >( stmt ) ) { + exprStmt->get_expr()->accept( *this ); + output << ";" << endl; + ++i; + break; + } + } + stmt->accept( *this ); + output << endl; + if ( wantSpacing( stmt ) ) { + output << endl; + } // if + ++i; + } + cur_indent -= CodeGenerator::tabsize; + output << indent << "})"; } Index: src/CodeGen/CodeGenerator.h =================================================================== --- src/CodeGen/CodeGenerator.h (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/CodeGen/CodeGenerator.h (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -70,7 +70,9 @@ virtual void visit( ConditionalExpr *conditionalExpr ); virtual void visit( CommaExpr *commaExpr ); + virtual void visit( CompoundLiteralExpr *compLitExpr ); virtual void visit( TupleExpr *tupleExpr ); virtual void visit( TypeExpr *typeExpr ); virtual void visit( AsmExpr * ); + virtual void visit( StmtExpr * ); //*** Statements Index: src/CodeGen/GenType.cc =================================================================== --- src/CodeGen/GenType.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/CodeGen/GenType.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -227,7 +227,4 @@ typeString = "_Atomic " + typeString; } // if - if ( type->get_isAttribute() ) { - typeString = "__attribute(( )) " + typeString; - } // if } } // namespace CodeGen Index: src/Common/utility.h =================================================================== --- src/Common/utility.h (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/Common/utility.h (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -148,4 +148,5 @@ } +// replace element of list with all elements of another list template< typename T > void replace( std::list< T > &org, typename std::list< T >::iterator pos, std::list< T > &with ) { @@ -158,4 +159,11 @@ return; +} + +// replace range of a list with a single element +template< typename T > +void replace( std::list< T > &org, typename std::list< T >::iterator begin, typename std::list< T >::iterator end, const T & with ) { + org.insert( begin, with ); + org.erase( begin, end ); } Index: c/ControlStruct/LabelTypeChecker.cc =================================================================== --- src/ControlStruct/LabelTypeChecker.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ (revision ) @@ -1,86 +1,0 @@ -// -// Cforall Version 1.0.0 Copyright (C) 2015 University of Waterloo -// -// The contents of this file are covered under the licence agreement in the -// file "LICENCE" distributed with Cforall. -// -// LabelTypeChecker.cc -- -// -// Author : Rodolfo G. Esteves -// Created On : Mon May 18 07:44:20 2015 -// Last Modified By : Rob Schluntz -// Last Modified On : Wed Jun 24 16:24:48 2015 -// Update Count : 3 -// - -#include -#include - -#include "SynTree/Type.h" -#include "SynTree/Statement.h" -#include "SynTree/Expression.h" -#include "SynTree/Declaration.h" - -#include "LabelTypeChecker.h" - -namespace ControlStruct { - void LabelTypeChecker::visit(UntypedExpr *untypedExpr) { - assert( untypedExpr != 0 ); - NameExpr *fname; - if ( ((fname = dynamic_cast(untypedExpr->get_function())) != 0) - && fname->get_name() == std::string("&&") ) - std::cerr << "Taking the label of an address." << std::endl; - else { - acceptAll( untypedExpr->get_results(), *this ); - acceptAll( untypedExpr->get_args(), *this ); - } // if - return; - } - - void LabelTypeChecker::visit(CompoundStmt *compoundStmt) { - index.enterScope(); - acceptAll( compoundStmt->get_kids(), *this ); - index.leaveScope(); - } - - void LabelTypeChecker::visit(DeclStmt *declStmt) { - declStmt->accept( index ); - - //ObjectDecl *odecl = 0; - // if ( ( odecl = dynamic_cast(declStmt->get_decl()) ) != 0 ) { - return; - } - - void LabelTypeChecker::visit(BranchStmt *branchStmt) { - if ( branchStmt->get_type() != BranchStmt::Goto ) return; - Expression *target; - if ( (target = branchStmt->get_computedTarget()) == 0 ) return; - - NameExpr *name; - if ( (name = dynamic_cast(target)) == 0 ) - return; // Not a name expression - - std::list< DeclarationWithType * > interps; - index.lookupId(name->get_name(), interps); - if ( interps.size() != 1) - // in case of multiple declarations - throw SemanticError("Illegal label expression: " + name->get_name() ); - - PointerType *ptr; - if ( (ptr = dynamic_cast(interps.front()->get_type())) != 0 ) - if ( dynamic_cast(ptr->get_base()) != 0 ) - return; - else - throw SemanticError("Wrong type of parameter for computed goto"); - else - throw SemanticError("Wrong type of parameter for computed goto"); - - return; - } -} // namespace ControlStruct - -// Local Variables: // -// tab-width: 4 // -// mode: c++ // -// compile-command: "make install" // -// End: // Index: c/ControlStruct/LabelTypeChecker.h =================================================================== --- src/ControlStruct/LabelTypeChecker.h (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ (revision ) @@ -1,45 +1,0 @@ -// -// Cforall Version 1.0.0 Copyright (C) 2015 University of Waterloo -// -// The contents of this file are covered under the licence agreement in the -// file "LICENCE" distributed with Cforall. -// -// LabelTypeChecker.h -- -// -// Author : Rodolfo G. Esteves -// Created On : Mon May 18 07:44:20 2015 -// Last Modified By : Peter A. Buhr -// Last Modified On : Mon Jan 25 21:22:30 2016 -// Update Count : 4 -// - -#ifndef LABEL_TYPE_H -#define LABEL_TYPE_H - -#include "SynTree/Visitor.h" -#include "SymTab/Indexer.h" -#include "SynTree/Statement.h" - -#include "Common/utility.h" - -namespace ControlStruct { - class LabelTypeChecker : public Visitor { - public: - //LabelTypeChecker() { - - virtual void visit( CompoundStmt *compoundStmt ); - virtual void visit( DeclStmt *declStmt ); - virtual void visit( BranchStmt *branchStmt ); - virtual void visit( UntypedExpr *untypedExpr ); - private: - SymTab::Indexer index; - }; -} // namespace ControlStruct - -#endif // LABEL_TYPE_H - -// Local Variables: // -// tab-width: 4 // -// mode: c++ // -// compile-command: "make install" // -// End: // Index: src/ControlStruct/Mutate.cc =================================================================== --- src/ControlStruct/Mutate.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/ControlStruct/Mutate.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -23,5 +23,4 @@ #include "MLEMutator.h" #include "ForExprMutator.h" -#include "LabelTypeChecker.h" //#include "ExceptMutator.h" @@ -41,10 +40,8 @@ //ExceptMutator exc; - // LabelTypeChecker lbl; mutateAll( translationUnit, formut ); acceptAll( translationUnit, lfix ); //mutateAll( translationUnit, exc ); - //acceptAll( translationUnit, lbl ); } } // namespace CodeGen Index: src/ControlStruct/module.mk =================================================================== --- src/ControlStruct/module.mk (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/ControlStruct/module.mk (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -6,5 +6,5 @@ ## file "LICENCE" distributed with Cforall. ## -## module.mk -- +## module.mk -- ## ## Author : Richard C. Bilson @@ -19,5 +19,4 @@ ControlStruct/MLEMutator.cc \ ControlStruct/Mutate.cc \ - ControlStruct/ForExprMutator.cc \ - ControlStruct/LabelTypeChecker.cc + ControlStruct/ForExprMutator.cc Index: src/GenPoly/Box.cc =================================================================== --- src/GenPoly/Box.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/GenPoly/Box.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -113,5 +113,5 @@ void addInferredParams( ApplicationExpr *appExpr, FunctionType *functionType, std::list< Expression *>::iterator &arg, const TyVarMap &tyVars ); /// Stores assignment operators from assertion list in local map of assignment operations - void findTypeOps( const std::list< TypeDecl *> &forall ); + void findTypeOps( const Type::ForallList &forall ); void passAdapters( ApplicationExpr *appExpr, FunctionType *functionType, const TyVarMap &exprTyVars ); FunctionDecl *makeAdapter( FunctionType *adaptee, FunctionType *realType, const std::string &mangleName, const TyVarMap &tyVars ); @@ -619,8 +619,8 @@ } - void Pass1::findTypeOps( const std::list< TypeDecl *> &forall ) { + void Pass1::findTypeOps( const Type::ForallList &forall ) { // what if a nested function uses an assignment operator? // assignOps.clear(); - for ( std::list< TypeDecl *>::const_iterator i = forall.begin(); i != forall.end(); ++i ) { + for ( Type::ForallList::const_iterator i = forall.begin(); i != forall.end(); ++i ) { for ( std::list< DeclarationWithType *>::const_iterator assert = (*i)->get_assertions().begin(); assert != (*i)->get_assertions().end(); ++assert ) { std::string typeName; @@ -687,5 +687,5 @@ std::list< DeclarationWithType *> ¶mList = functionType->get_parameters(); std::list< FunctionType *> functions; - for ( std::list< TypeDecl *>::iterator tyVar = functionType->get_forall().begin(); tyVar != functionType->get_forall().end(); ++tyVar ) { + for ( Type::ForallList::iterator tyVar = functionType->get_forall().begin(); tyVar != functionType->get_forall().end(); ++tyVar ) { for ( std::list< DeclarationWithType *>::iterator assert = (*tyVar)->get_assertions().begin(); assert != (*tyVar)->get_assertions().end(); ++assert ) { findFunction( (*assert)->get_type(), functions, scopeTyVars, needsAdapter ); @@ -789,6 +789,6 @@ // add size/align for generic types to parameter list - if ( appExpr->get_function()->get_results().empty() ) return; - FunctionType *funcType = getFunctionType( appExpr->get_function()->get_results().front() ); + if ( ! appExpr->get_function()->has_result() ) return; + FunctionType *funcType = getFunctionType( appExpr->get_function()->get_result() ); assert( funcType ); @@ -806,6 +806,6 @@ for ( ; fnParm != funcType->get_parameters().end() && fnArg != appExpr->get_args().end(); ++fnParm, ++fnArg ) { VariableExpr *fnArgBase = getBaseVar( *fnArg ); - if ( ! fnArgBase || fnArgBase->get_results().empty() ) continue; - passArgTypeVars( appExpr, (*fnParm)->get_type(), fnArgBase->get_results().front(), arg, exprTyVars, seenTypes ); + if ( ! fnArgBase ) continue; // xxx - previously had check for non-empty fnArgBase results + passArgTypeVars( appExpr, (*fnParm)->get_type(), fnArgBase->get_result(), arg, exprTyVars, seenTypes ); } } @@ -897,5 +897,5 @@ Type * adapteeType = new PointerType( Type::Qualifiers(), new FunctionType( Type::Qualifiers(), true ) ); appExpr->get_args().push_front( new CastExpr( appExpr->get_function(), adapteeType ) ); - appExpr->set_function( new NameExpr( adapterName ) ); + appExpr->set_function( new NameExpr( adapterName ) ); // xxx - result is never set on NameExpr return ret; @@ -903,10 +903,10 @@ void Pass1::boxParam( Type *param, Expression *&arg, const TyVarMap &exprTyVars ) { - assert( ! arg->get_results().empty() ); + assert( arg->has_result() ); if ( isPolyType( param, exprTyVars ) ) { - if ( isPolyType( arg->get_results().front() ) ) { + if ( isPolyType( arg->get_result() ) ) { // if the argument's type is polymorphic, we don't need to box again! return; - } else if ( arg->get_results().front()->get_isLvalue() ) { + } else if ( arg->get_result()->get_isLvalue() ) { // VariableExpr and MemberExpr are lvalues; need to check this isn't coming from the second arg of a comma expression though (not an lvalue) // xxx - need to test that this code is still reachable @@ -953,5 +953,5 @@ void Pass1::addInferredParams( ApplicationExpr *appExpr, FunctionType *functionType, std::list< Expression *>::iterator &arg, const TyVarMap &tyVars ) { std::list< Expression *>::iterator cur = arg; - for ( std::list< TypeDecl *>::iterator tyVar = functionType->get_forall().begin(); tyVar != functionType->get_forall().end(); ++tyVar ) { + for ( Type::ForallList::iterator tyVar = functionType->get_forall().begin(); tyVar != functionType->get_forall().end(); ++tyVar ) { for ( std::list< DeclarationWithType *>::iterator assert = (*tyVar)->get_assertions().begin(); assert != (*tyVar)->get_assertions().end(); ++assert ) { InferredParams::const_iterator inferParam = appExpr->get_inferParams().find( (*assert)->get_uniqueId() ); @@ -994,5 +994,5 @@ UntypedExpr *deref = new UntypedExpr( new NameExpr( "*?" ) ); deref->get_args().push_back( new CastExpr( new VariableExpr( param ), new PointerType( Type::Qualifiers(), arg->get_type()->clone() ) ) ); - deref->get_results().push_back( arg->get_type()->clone() ); + deref->set_result( arg->get_type()->clone() ); return deref; } // if @@ -1020,7 +1020,7 @@ Statement *bodyStmt; - std::list< TypeDecl *>::iterator tyArg = realType->get_forall().begin(); - std::list< TypeDecl *>::iterator tyParam = adapterType->get_forall().begin(); - std::list< TypeDecl *>::iterator realTyParam = adaptee->get_forall().begin(); + Type::ForallList::iterator tyArg = realType->get_forall().begin(); + Type::ForallList::iterator tyParam = adapterType->get_forall().begin(); + Type::ForallList::iterator realTyParam = adaptee->get_forall().begin(); for ( ; tyParam != adapterType->get_forall().end(); ++tyArg, ++tyParam, ++realTyParam ) { assert( tyArg != realType->get_forall().end() ); @@ -1071,5 +1071,5 @@ std::list< DeclarationWithType *> ¶mList = functionType->get_parameters(); std::list< FunctionType *> functions; - for ( std::list< TypeDecl *>::iterator tyVar = functionType->get_forall().begin(); tyVar != functionType->get_forall().end(); ++tyVar ) { + for ( Type::ForallList::iterator tyVar = functionType->get_forall().begin(); tyVar != functionType->get_forall().end(); ++tyVar ) { for ( std::list< DeclarationWithType *>::iterator assert = (*tyVar)->get_assertions().begin(); assert != (*tyVar)->get_assertions().end(); ++assert ) { findFunction( (*assert)->get_type(), functions, exprTyVars, needsAdapter ); @@ -1131,5 +1131,5 @@ } // if addAssign->get_args().push_back( new NameExpr( sizeofName( mangleType( polyType ) ) ) ); - addAssign->get_results().front() = appExpr->get_results().front()->clone(); + addAssign->set_result( appExpr->get_result()->clone() ); if ( appExpr->get_env() ) { addAssign->set_env( appExpr->get_env() ); @@ -1145,8 +1145,8 @@ if ( varExpr->get_var()->get_linkage() == LinkageSpec::Intrinsic ) { if ( varExpr->get_var()->get_name() == "?[?]" ) { - assert( ! appExpr->get_results().empty() ); + assert( appExpr->has_result() ); assert( appExpr->get_args().size() == 2 ); - Type *baseType1 = isPolyPtr( appExpr->get_args().front()->get_results().front(), scopeTyVars, env ); - Type *baseType2 = isPolyPtr( appExpr->get_args().back()->get_results().front(), scopeTyVars, env ); + Type *baseType1 = isPolyPtr( appExpr->get_args().front()->get_result(), scopeTyVars, env ); + Type *baseType2 = isPolyPtr( appExpr->get_args().back()->get_result(), scopeTyVars, env ); assert( ! baseType1 || ! baseType2 ); // the arguments cannot both be polymorphic pointers UntypedExpr *ret = 0; @@ -1168,5 +1168,5 @@ } // if if ( baseType1 || baseType2 ) { - ret->get_results().push_front( appExpr->get_results().front()->clone() ); + ret->set_result( appExpr->get_result()->clone() ); if ( appExpr->get_env() ) { ret->set_env( appExpr->get_env() ); @@ -1178,10 +1178,10 @@ } // if } else if ( varExpr->get_var()->get_name() == "*?" ) { - assert( ! appExpr->get_results().empty() ); + assert( appExpr->has_result() ); assert( ! appExpr->get_args().empty() ); - if ( isPolyType( appExpr->get_results().front(), scopeTyVars, env ) ) { + if ( isPolyType( appExpr->get_result(), scopeTyVars, env ) ) { Expression *ret = appExpr->get_args().front(); - delete ret->get_results().front(); - ret->get_results().front() = appExpr->get_results().front()->clone(); + delete ret->get_result(); + ret->set_result( appExpr->get_result()->clone() ); if ( appExpr->get_env() ) { ret->set_env( appExpr->get_env() ); @@ -1193,8 +1193,8 @@ } // if } else if ( varExpr->get_var()->get_name() == "?++" || varExpr->get_var()->get_name() == "?--" ) { - assert( ! appExpr->get_results().empty() ); + assert( appExpr->has_result() ); assert( appExpr->get_args().size() == 1 ); - if ( Type *baseType = isPolyPtr( appExpr->get_results().front(), scopeTyVars, env ) ) { - Type *tempType = appExpr->get_results().front()->clone(); + if ( Type *baseType = isPolyPtr( appExpr->get_result(), scopeTyVars, env ) ) { + Type *tempType = appExpr->get_result()->clone(); if ( env ) { env->apply( tempType ); @@ -1213,19 +1213,19 @@ } // if } else if ( varExpr->get_var()->get_name() == "++?" || varExpr->get_var()->get_name() == "--?" ) { - assert( ! appExpr->get_results().empty() ); + assert( appExpr->has_result() ); assert( appExpr->get_args().size() == 1 ); - if ( Type *baseType = isPolyPtr( appExpr->get_results().front(), scopeTyVars, env ) ) { + if ( Type *baseType = isPolyPtr( appExpr->get_result(), scopeTyVars, env ) ) { return makeIncrDecrExpr( appExpr, baseType, varExpr->get_var()->get_name() == "++?" ); } // if } else if ( varExpr->get_var()->get_name() == "?+?" || varExpr->get_var()->get_name() == "?-?" ) { - assert( ! appExpr->get_results().empty() ); + assert( appExpr->has_result() ); assert( appExpr->get_args().size() == 2 ); - Type *baseType1 = isPolyPtr( appExpr->get_args().front()->get_results().front(), scopeTyVars, env ); - Type *baseType2 = isPolyPtr( appExpr->get_args().back()->get_results().front(), scopeTyVars, env ); + Type *baseType1 = isPolyPtr( appExpr->get_args().front()->get_result(), scopeTyVars, env ); + Type *baseType2 = isPolyPtr( appExpr->get_args().back()->get_result(), scopeTyVars, env ); if ( baseType1 && baseType2 ) { UntypedExpr *divide = new UntypedExpr( new NameExpr( "?/?" ) ); divide->get_args().push_back( appExpr ); divide->get_args().push_back( new SizeofExpr( baseType1->clone() ) ); - divide->get_results().push_front( appExpr->get_results().front()->clone() ); + divide->set_result( appExpr->get_result()->clone() ); if ( appExpr->get_env() ) { divide->set_env( appExpr->get_env() ); @@ -1245,7 +1245,7 @@ } // if } else if ( varExpr->get_var()->get_name() == "?+=?" || varExpr->get_var()->get_name() == "?-=?" ) { - assert( ! appExpr->get_results().empty() ); + assert( appExpr->has_result() ); assert( appExpr->get_args().size() == 2 ); - Type *baseType = isPolyPtr( appExpr->get_results().front(), scopeTyVars, env ); + Type *baseType = isPolyPtr( appExpr->get_result(), scopeTyVars, env ); if ( baseType ) { UntypedExpr *multiply = new UntypedExpr( new NameExpr( "?*?" ) ); @@ -1273,9 +1273,7 @@ useRetval = oldUseRetval; - assert( ! appExpr->get_function()->get_results().empty() ); - PointerType *pointer = dynamic_cast< PointerType *>( appExpr->get_function()->get_results().front() ); - assert( pointer ); - FunctionType *function = dynamic_cast< FunctionType *>( pointer->get_base() ); - assert( function ); + assert( appExpr->get_function()->has_result() ); + PointerType *pointer = safe_dynamic_cast< PointerType *>( appExpr->get_function()->get_result() ); + FunctionType *function = safe_dynamic_cast< FunctionType *>( pointer->get_base() ); if ( Expression *newExpr = handleIntrinsics( appExpr ) ) { @@ -1315,5 +1313,5 @@ Expression *Pass1::mutate( UntypedExpr *expr ) { - if ( ! expr->get_results().empty() && isPolyType( expr->get_results().front(), scopeTyVars, env ) ) { + if ( expr->has_result() && isPolyType( expr->get_result(), scopeTyVars, env ) ) { if ( NameExpr *name = dynamic_cast< NameExpr *>( expr->get_function() ) ) { if ( name->get_name() == "*?" ) { @@ -1329,17 +1327,15 @@ Expression *Pass1::mutate( AddressExpr *addrExpr ) { - assert( ! addrExpr->get_arg()->get_results().empty() ); + assert( addrExpr->get_arg()->has_result() && ! addrExpr->get_arg()->get_result()->isVoid() ); bool needs = false; if ( UntypedExpr *expr = dynamic_cast< UntypedExpr *>( addrExpr->get_arg() ) ) { - if ( ! expr->get_results().empty() && isPolyType( expr->get_results().front(), scopeTyVars, env ) ) { + if ( expr->has_result() && isPolyType( expr->get_result(), scopeTyVars, env ) ) { if ( NameExpr *name = dynamic_cast< NameExpr *>( expr->get_function() ) ) { if ( name->get_name() == "*?" ) { if ( ApplicationExpr * appExpr = dynamic_cast< ApplicationExpr * >( expr->get_args().front() ) ) { - assert( ! appExpr->get_function()->get_results().empty() ); - PointerType *pointer = dynamic_cast< PointerType *>( appExpr->get_function()->get_results().front() ); - assert( pointer ); - FunctionType *function = dynamic_cast< FunctionType *>( pointer->get_base() ); - assert( function ); + assert( appExpr->get_function()->has_result() ); + PointerType *pointer = safe_dynamic_cast< PointerType *>( appExpr->get_function()->get_result() ); + FunctionType *function = safe_dynamic_cast< FunctionType *>( pointer->get_base() ); needs = needsAdapter( function, scopeTyVars ); } // if @@ -1350,10 +1346,10 @@ // isPolyType check needs to happen before mutating addrExpr arg, so pull it forward // out of the if condition. - bool polytype = isPolyType( addrExpr->get_arg()->get_results().front(), scopeTyVars, env ); + bool polytype = isPolyType( addrExpr->get_arg()->get_result(), scopeTyVars, env ); addrExpr->set_arg( mutateExpression( addrExpr->get_arg() ) ); if ( polytype || needs ) { Expression *ret = addrExpr->get_arg(); - delete ret->get_results().front(); - ret->get_results().front() = addrExpr->get_results().front()->clone(); + delete ret->get_result(); + ret->set_result( addrExpr->get_result()->clone() ); addrExpr->set_arg( 0 ); delete addrExpr; @@ -1393,5 +1389,5 @@ Statement * Pass1::mutate( ReturnStmt *returnStmt ) { if ( retval && returnStmt->get_expr() ) { - assert( ! returnStmt->get_expr()->get_results().empty() ); + assert( returnStmt->get_expr()->has_result() && ! returnStmt->get_expr()->get_result()->isVoid() ); // ***** Code Removal ***** After introducing a temporary variable for all return expressions, the following code appears superfluous. // if ( returnStmt->get_expr()->get_results().front()->get_isLvalue() ) { @@ -1427,7 +1423,7 @@ // find each of its needed secondary assignment operators std::list< Expression* > &tyParams = refType->get_parameters(); - std::list< TypeDecl* > &forallParams = functionDecl->get_type()->get_forall(); + Type::ForallList &forallParams = functionDecl->get_type()->get_forall(); std::list< Expression* >::const_iterator tyIt = tyParams.begin(); - std::list< TypeDecl* >::const_iterator forallIt = forallParams.begin(); + Type::ForallList::const_iterator forallIt = forallParams.begin(); for ( ; tyIt != tyParams.end() && forallIt != forallParams.end(); ++tyIt, ++forallIt ) { // Add appropriate mapping to assignment expression environment @@ -1473,5 +1469,5 @@ // replace return statement with appropriate assignment to out parameter Expression *retParm = new NameExpr( retval->get_name() ); - retParm->get_results().push_back( new PointerType( Type::Qualifiers(), retval->get_type()->clone() ) ); + retParm->set_result( new PointerType( Type::Qualifiers(), retval->get_type()->clone() ) ); assignExpr->get_args().push_back( retParm ); assignExpr->get_args().push_back( returnStmt->get_expr() ); @@ -1603,5 +1599,5 @@ ObjectDecl newPtr( "", DeclarationNode::NoStorageClass, LinkageSpec::C, 0, new PointerType( Type::Qualifiers(), new BasicType( Type::Qualifiers(), BasicType::LongUnsignedInt ) ), 0 ); - for ( std::list< TypeDecl *>::const_iterator tyParm = funcType->get_forall().begin(); tyParm != funcType->get_forall().end(); ++tyParm ) { + for ( Type::ForallList::const_iterator tyParm = funcType->get_forall().begin(); tyParm != funcType->get_forall().end(); ++tyParm ) { ObjectDecl *sizeParm, *alignParm; // add all size and alignment parameters to parameter list Index: src/GenPoly/CopyParams.cc =================================================================== --- src/GenPoly/CopyParams.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/GenPoly/CopyParams.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -54,5 +54,5 @@ std::map< std::string, DeclarationWithType* > assignOps; // assume the assignment operator is the first assert param after any "type" parameter - for ( std::list< TypeDecl* >::const_iterator tyVar = funcDecl->get_functionType()->get_forall().begin(); tyVar != funcDecl->get_functionType()->get_forall().end(); ++tyVar ) { + for ( Type::ForallList::const_iterator tyVar = funcDecl->get_functionType()->get_forall().begin(); tyVar != funcDecl->get_functionType()->get_forall().end(); ++tyVar ) { if ( (*tyVar)->get_kind() == TypeDecl::Any ) { assert( !(*tyVar)->get_assertions().empty() ); Index: src/GenPoly/FindFunction.cc =================================================================== --- src/GenPoly/FindFunction.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/GenPoly/FindFunction.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -29,5 +29,5 @@ virtual Type *mutate( PointerType *pointerType ); private: - void handleForall( const std::list< TypeDecl* > &forall ); + void handleForall( const Type::ForallList &forall ); std::list< FunctionType* > &functions; @@ -51,6 +51,6 @@ } - void FindFunction::handleForall( const std::list< TypeDecl* > &forall ) { - for ( std::list< TypeDecl* >::const_iterator i = forall.begin(); i != forall.end(); ++i ) { + void FindFunction::handleForall( const Type::ForallList &forall ) { + for ( Type::ForallList::const_iterator i = forall.begin(); i != forall.end(); ++i ) { TyVarMap::iterator var = tyVars.find( (*i)->get_name() ); if ( var != tyVars.end() ) { Index: src/GenPoly/GenPoly.cc =================================================================== --- src/GenPoly/GenPoly.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/GenPoly/GenPoly.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -110,5 +110,5 @@ ReferenceToType *isDynRet( FunctionType *function, const TyVarMap &forallTypes ) { if ( function->get_returnVals().empty() ) return 0; - + return (ReferenceToType*)isDynType( function->get_returnVals().front()->get_type(), forallTypes ); } @@ -127,5 +127,5 @@ // } // if if ( isDynRet( adaptee, tyVars ) ) return true; - + for ( std::list< DeclarationWithType* >::const_iterator innerArg = adaptee->get_parameters().begin(); innerArg != adaptee->get_parameters().end(); ++innerArg ) { // if ( isPolyType( (*innerArg)->get_type(), tyVars ) ) { @@ -228,5 +228,5 @@ void makeTyVarMap( Type *type, TyVarMap &tyVarMap ) { - for ( std::list< TypeDecl* >::const_iterator tyVar = type->get_forall().begin(); tyVar != type->get_forall().end(); ++tyVar ) { + for ( Type::ForallList::const_iterator tyVar = type->get_forall().begin(); tyVar != type->get_forall().end(); ++tyVar ) { assert( *tyVar ); tyVarMap[ (*tyVar)->get_name() ] = (*tyVar)->get_kind(); Index: src/GenPoly/Lvalue.cc =================================================================== --- src/GenPoly/Lvalue.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/GenPoly/Lvalue.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -5,5 +5,5 @@ // file "LICENCE" distributed with Cforall. // -// Lvalue.cc -- +// Lvalue.cc -- // // Author : Richard C. Bilson @@ -41,5 +41,5 @@ public: Pass1(); - + virtual Expression *mutate( ApplicationExpr *appExpr ); virtual Statement *mutate( ReturnStmt *appExpr ); @@ -55,4 +55,13 @@ private: }; + + /// GCC-like Generalized Lvalues (which have since been removed from GCC) + /// https://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc/Lvalues.html#Lvalues + /// Replaces &(a,b) with (a, &b), &(a ? b : c) with (a ? &b : &c) + class GeneralizedLvalue : public Mutator { + typedef Mutator Parent; + + virtual Expression * mutate( AddressExpr * addressExpr ); + }; } // namespace @@ -60,6 +69,8 @@ Pass1 p1; Pass2 p2; + GeneralizedLvalue genLval; mutateAll( translationUnit, p1 ); acceptAll( translationUnit, p2 ); + mutateAll( translationUnit, genLval ); } @@ -99,16 +110,12 @@ appExpr->get_function()->acceptMutator( *this ); mutateAll( appExpr->get_args(), *this ); - - assert( ! appExpr->get_function()->get_results().empty() ); - PointerType *pointer = dynamic_cast< PointerType* >( appExpr->get_function()->get_results().front() ); - assert( pointer ); - FunctionType *function = dynamic_cast< FunctionType* >( pointer->get_base() ); - assert( function ); + PointerType *pointer = safe_dynamic_cast< PointerType* >( appExpr->get_function()->get_result() ); + FunctionType *function = safe_dynamic_cast< FunctionType* >( pointer->get_base() ); Type *funType = isLvalueRet( function ); if ( funType && ! isIntrinsicApp( appExpr ) ) { Expression *expr = appExpr; - Type *appType = appExpr->get_results().front(); + Type *appType = appExpr->get_result(); if ( isPolyType( funType ) && ! isPolyType( appType ) ) { // make sure cast for polymorphic type is inside dereference @@ -116,6 +123,6 @@ } UntypedExpr *deref = new UntypedExpr( new NameExpr( "*?" ) ); - deref->get_results().push_back( appType->clone() ); - appExpr->get_results().front() = new PointerType( Type::Qualifiers(), appType ); + deref->set_result( appType->clone() ); + appExpr->set_result( new PointerType( Type::Qualifiers(), appType ) ); deref->get_args().push_back( expr ); return deref; @@ -127,6 +134,5 @@ Statement * Pass1::mutate(ReturnStmt *retStmt) { if ( retval && retStmt->get_expr() ) { - assert( ! retStmt->get_expr()->get_results().empty() ); - if ( retStmt->get_expr()->get_results().front()->get_isLvalue() ) { + if ( retStmt->get_expr()->get_result()->get_isLvalue() ) { // ***** Code Removal ***** because casts may be stripped already @@ -155,6 +161,23 @@ retParm->set_type( new PointerType( Type::Qualifiers(), retParm->get_type() ) ); } // if - + Visitor::visit( funType ); + } + + Expression * GeneralizedLvalue::mutate( AddressExpr * addrExpr ) { + addrExpr = safe_dynamic_cast< AddressExpr * >( Parent::mutate( addrExpr ) ); + if ( CommaExpr * commaExpr = dynamic_cast< CommaExpr * >( addrExpr->get_arg() ) ) { + Expression * arg1 = commaExpr->get_arg1()->clone(); + Expression * arg2 = commaExpr->get_arg2()->clone(); + delete addrExpr; + return new CommaExpr( arg1, new AddressExpr( arg2 ) ); + } else if ( ConditionalExpr * condExpr = dynamic_cast< ConditionalExpr * >( addrExpr->get_arg() ) ) { + Expression * arg1 = condExpr->get_arg1()->clone(); + Expression * arg2 = condExpr->get_arg2()->clone(); + Expression * arg3 = condExpr->get_arg3()->clone(); + delete addrExpr; + return new ConditionalExpr( arg1, new AddressExpr( arg2 ), new AddressExpr( arg3 ) ); + } + return addrExpr; } } // namespace Index: src/GenPoly/Specialize.cc =================================================================== --- src/GenPoly/Specialize.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/GenPoly/Specialize.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -148,6 +148,6 @@ Expression * Specialize::doSpecialization( Type *formalType, Expression *actual, InferredParams *inferParams ) { - assert( ! actual->get_results().empty() ); // using front, should have this assert - if ( needsSpecialization( formalType, actual->get_results().front(), env ) ) { + assertf( actual->has_result(), "attempting to specialize an untyped expression" ); + if ( needsSpecialization( formalType, actual->get_result(), env ) ) { FunctionType *funType; if ( ( funType = getFunctionType( formalType ) ) ) { @@ -172,6 +172,6 @@ void Specialize::handleExplicitParams( ApplicationExpr *appExpr ) { // create thunks for the explicit parameters - assert( ! appExpr->get_function()->get_results().empty() ); - FunctionType *function = getFunctionType( appExpr->get_function()->get_results().front() ); + assert( appExpr->get_function()->has_result() ); + FunctionType *function = getFunctionType( appExpr->get_function()->get_result() ); assert( function ); std::list< DeclarationWithType* >::iterator formal; @@ -201,6 +201,6 @@ Expression * Specialize::mutate( AddressExpr *addrExpr ) { addrExpr->get_arg()->acceptMutator( *this ); - assert( ! addrExpr->get_results().empty() ); - addrExpr->set_arg( doSpecialization( addrExpr->get_results().front(), addrExpr->get_arg() ) ); + assert( addrExpr->has_result() ); + addrExpr->set_arg( doSpecialization( addrExpr->get_result(), addrExpr->get_arg() ) ); return addrExpr; } @@ -208,9 +208,9 @@ Expression * Specialize::mutate( CastExpr *castExpr ) { castExpr->get_arg()->acceptMutator( *this ); - if ( castExpr->get_results().empty() ) { + if ( castExpr->get_result()->isVoid() ) { // can't specialize if we don't have a return value return castExpr; } - Expression *specialized = doSpecialization( castExpr->get_results().front(), castExpr->get_arg() ); + Expression *specialized = doSpecialization( castExpr->get_result(), castExpr->get_arg() ); if ( specialized != castExpr->get_arg() ) { // assume here that the specialization incorporates the cast Index: src/InitTweak/FixInit.cc =================================================================== --- src/InitTweak/FixInit.cc (revision 3a2128f5aaa77a03c76ff6198907b91256d8c603) +++ src/InitTweak/FixInit.cc (revision 1f44196dc72fd060b6183fe2bdc10f7b30f955d6) @@ -18,4 +18,6 @@ #include #include +#include +#include #include "InitTweak.h" #include "FixInit.h" @@ -35,10 +37,10 @@ #include "GenPoly/DeclMutator.h" #include "SynTree/AddStmtVisitor.h" -#include "CodeGen/GenType.h" // for warnings - -bool ctordtorp = false; -bool ctorp = false; -bool cpctorp = false; -bool dtorp = false; +#include "CodeGen/GenType.h" // for warning/error messages + +bool ctordtorp = false; // print all debug +bool ctorp = false; // print ctor debug +bool cpctorp = false; // print copy ctor debug +bool dtorp = false; // print dtor debug #define PRINT( text ) if ( ctordtorp ) { text } #define CP_CTOR_PRINT( text ) if ( ctordtorp || cpctorp ) { text } @@ -47,7 +49,4 @@ namespace InitTweak { namespace { - const std::list