Context Navigation

-                      r999c700
+                      r6d43cc57
 External scheduling allows users to wait for events from other threads without concern of unrelated events occurring.
 The mechnaism can be done in terms of control flow, \eg Ada @accept@ or \uC @_Accept@, or in terms of data, \eg Go channels.
+Of course, both of these paradigms have their own strengths and weaknesses, but for this project, control-flow semantics was chosen to stay consistent with the rest of the languages semantics.
+Two challenges specific to \CFA arise when trying to add external scheduling with loose object definitions and multiple-monitor routines.
+The previous example shows a simple use @_Accept@ versus @wait@/@signal@ and its advantages.
+Note that while other languages often use @accept@/@select@ as the core external scheduling keyword, \CFA uses @waitfor@ to prevent name collisions with existing socket \textbf{api}s.
+While both mechanisms have strengths and weaknesses, this project uses a control-flow mechanism to stay consistent with other language semantics.
+Two challenges specific to \CFA for external scheduling are loose object-definitions (see Section~\ref{s:LooseObjectDefinitions}) and multiple-monitor routines (see Section~\ref{s:Multi-MonitorScheduling}).
 For internal scheduling, non-blocking signalling (as in the producer/consumer example) is used when the signaller is providing the cooperation for a waiting thread;
 …
 \end{cfa}
 must have acquired monitor locks that are greater than or equal to the number of locks for the waiting thread signalled from the condition queue.
-{\color{red}In general, the signaller does not know the order of waiting threads, so in general, it must acquire the maximum number of mutex locks for the worst-case waiting thread.}
 Similarly, for @waitfor( rtn )@, the default semantics is to atomically block the acceptor and release all acquired mutex types in the parameter list, \ie @waitfor( rtn, m1, m2 )@.
 …
 The extra challenge is that this dependency graph is effectively post-mortem, but the runtime system needs to be able to build and solve these graphs as the dependencies unfold.
 Resolving dependency graphs being a complex and expensive endeavour, this solution is not the preferred one.
-\subsubsection{Partial Signalling} \label{partial-sig}
 \end{comment}
 …
 \subsection{Loose Object Definitions}
+In \uC, a monitor class declaration includes an exhaustive list of monitor operations.
+Since \CFA is not object oriented, monitors become both more difficult to implement and less clear for a user:
+\begin{cfa}
+monitor A {};
+void f(A & mutex a);
+void g(A & mutex a) {
+        waitfor(f); // Obvious which f() to wait for
+}
+void f(A & mutex a, int); // New different F added in scope
+void h(A & mutex a) {
+        waitfor(f); // Less obvious which f() to wait for
+}
+\end{cfa}
+Furthermore, external scheduling is an example where implementation constraints become visible from the interface.
+Here is the cfa-code for the entering phase of a monitor:
+\begin{center}
+\begin{tabular}{l}
+\begin{cfa}
+        if monitor is free
+                enter
+        elif already own the monitor
+                continue
+        elif monitor accepts me
+                enter
+        else
+                block
+\end{cfa}
+\end{tabular}
+\end{center}
+\label{s:LooseObjectDefinitions}
+In an object-oriented programming-language, a class includes an exhaustive list of operations.
+However, new members can be added via static inheritance or dynaic members, \eg JavaScript~\cite{JavaScript}.
+Similarly, monitor routines can be added at any time in \CFA, making it less clear for programmers and more difficult to implement.
+\begin{cfa}
+monitor M {};
+void `f`( M & mutex m );
+void g( M & mutex m ) { waitfor( `f` ); }       $\C{// clear which f}$
+void `f`( M & mutex m, int );                           $\C{// different f}$
+void h( M & mutex m ) { waitfor( `f` ); }       $\C{// unclear which f}$
+\end{cfa}
+Hence, the cfa-code for the entering a monitor looks like:
+\begin{cfa}
+if ( $\textrm{\textit{monitor is free}}$ ) $\LstCommentStyle{// \color{red}enter}$
+else if ( $\textrm{\textit{already own monitor}}$ ) $\LstCommentStyle{// \color{red}continue}$
+else if ( $\textrm{\textit{monitor accepts me}}$ ) $\LstCommentStyle{// \color{red}enter}$
+else $\LstCommentStyle{// \color{red}block}$
+\end{cfa}
 For the first two conditions, it is easy to implement a check that can evaluate the condition in a few instructions.
 However, a fast check for @monitor accepts me@ is much harder to implement depending on the constraints put on the monitors.
 Indeed, monitors are often expressed as an entry queue and some acceptor queue as in Figure~\ref{fig:ClassicalMonitor}.
+However, a fast check for \emph{monitor accepts me} is much harder to implement depending on the constraints put on the monitors.
+Figure~\ref{fig:ClassicalMonitor} shows monitors are often expressed as an entry (calling) queue, some acceptor queues, and an urgent stack/queue.
 \begin{figure}
 \centering
 \subfloat[Classical Monitor] {
+\subfloat[Classical monitor] {
 \label{fig:ClassicalMonitor}
 {\resizebox{0.45\textwidth}{!}{\input{monitor}}}
+{\resizebox{0.45\textwidth}{!}{\input{monitor.pstex_t}}}
 }% subfloat
 \qquad
 \subfloat[bulk acquire Monitor] {
+\quad
+\subfloat[Bulk acquire monitor] {
 \label{fig:BulkMonitor}
 {\resizebox{0.45\textwidth}{!}{\input{ext_monitor}}}
+{\resizebox{0.45\textwidth}{!}{\input{ext_monitor.pstex_t}}}
 }% subfloat
+\caption{External Scheduling Monitor}
+\caption{Monitor Implementation}
+\label{f:MonitorImplementation}
 \end{figure}
+There are other alternatives to these pictures, but in the case of the left picture, implementing a fast accept check is relatively easy.
+Restricted to a fixed number of mutex members, N, the accept check reduces to updating a bitmask when the acceptor queue changes, a check that executes in a single instruction even with a fairly large number (\eg 128) of mutex members.
+This approach requires a unique dense ordering of routines with an upper-bound and that ordering must be consistent across translation units.
+For OO languages these constraints are common, since objects only offer adding member routines consistently across translation units via inheritance.
+However, in \CFA users can extend objects with mutex routines that are only visible in certain translation unit.
+This means that establishing a program-wide dense-ordering among mutex routines can only be done in the program linking phase, and still could have issues when using dynamically shared objects.
+The alternative is to alter the implementation as in Figure~\ref{fig:BulkMonitor}.
+Here, the mutex routine called is associated with a thread on the entry queue while a list of acceptable routines is kept separate.
+Generating a mask dynamically means that the storage for the mask information can vary between calls to @waitfor@, allowing for more flexibility and extensions.
+Storing an array of accepted routine pointers replaces the single instruction bitmask comparison with dereferencing a pointer followed by a linear search.
+Furthermore, supporting nested external scheduling (\eg listing \ref{f:nest-ext}) may now require additional searches for the @waitfor@ statement to check if a routine is already queued.
+For a fixed (small) number of mutex routines (\eg 128), the accept check reduces to a bitmask of allowed callers, which can be checked with a single instruction.
+This approach requires a unique dense ordering of routines with a small upper-bound and the ordering must be consistent across translation units.
+For object-oriented languages these constraints are common, but \CFA mutex routines can be added in any scope and are only visible in certain translation unit, precluding program-wide dense-ordering among mutex routines.
+Figure~\ref{fig:BulkMonitor} shows the \CFA monitor implementation.
+The mutex routine called is associated with each thread on the entry queue, while a list of acceptable routines is kept separately.
+The accepted list is a variable-sized array of accepted routine pointers, so the single instruction bitmask comparison is replaced by dereferencing a pointer followed by a linear search.
+\begin{comment}
 \begin{figure}
 \begin{cfa}[caption={Example of nested external scheduling},label={f:nest-ext}]
 …
 In the end, the most flexible approach has been chosen since it allows users to write programs that would otherwise be  hard to write.
 This decision is based on the assumption that writing fast but inflexible locks is closer to a solved problem than writing locks that are as flexible as external scheduling in \CFA.
+% ======================================================================
+% ======================================================================
+\end{comment}
 \subsection{Multi-Monitor Scheduling}
+% ======================================================================
+% ======================================================================
+\label{s:Multi-MonitorScheduling}
 External scheduling, like internal scheduling, becomes significantly more complex when introducing multi-monitor syntax.
 Even in the simplest possible case, some new semantics needs to be established:
+Even in the simplest possible case, new semantics needs to be established:
 \begin{cfa}
 monitor M {};
+void f(M & mutex a);
+void g(M & mutex b, M & mutex c) {
+        waitfor(f); // two monitors M => unknown which to pass to f(M & mutex)
+}
+\end{cfa}
+The obvious solution is to specify the correct monitor as follows:
+void f( M & mutex m1 );
+void g( M & mutex m1, M & mutex m2 ) {
+        waitfor( f );                                                   $\C{// pass m1 or m2 to f?}$
+}
+\end{cfa}
+The solution is for the programmer to disambiguate:
+\begin{cfa}
+        waitfor( f, m2 );                                               $\C{// wait for call to f with argument m2}$
+\end{cfa}
+Routine @g@ has acquired both locks, so when routine @f@ is called, the lock for monitor @m2@ is passed from @g@ to @f@ (while @g@ still holds lock @m1@).
+This behaviour can be extended to the multi-monitor @waitfor@ statement.
 \begin{cfa}
 monitor M {};
+void f(M & mutex a);
+void g(M & mutex a, M & mutex b) {
+        // wait for call to f with argument b
+        waitfor(f, b);
+}
+\end{cfa}
+This syntax is unambiguous.
+Both locks are acquired and kept by @g@.
+When routine @f@ is called, the lock for monitor @b@ is temporarily transferred from @g@ to @f@ (while @g@ still holds lock @a@).
+This behaviour can be extended to the multi-monitor @waitfor@ statement as follows.
+\begin{cfa}
+monitor M {};
+void f(M & mutex a, M & mutex b);
+void g(M & mutex a, M & mutex b) {
+        // wait for call to f with arguments a and b
+        waitfor(f, a, b);
+}
+\end{cfa}
+Note that the set of monitors passed to the @waitfor@ statement must be entirely contained in the set of monitors already acquired in the routine. @waitfor@ used in any other context is undefined behaviour.
+void f( M & mutex m1, M & mutex m2 );
+void g( M & mutex m1, M & mutex m2 ) {
+        waitfor( f, m1, m2 );                                   $\C{// wait for call to f with arguments m1 and m2}$
+}
+\end{cfa}
+Again, the set of monitors passed to the @waitfor@ statement must be entirely contained in the set of monitors already acquired by accepting routine.
 An important behaviour to note is when a set of monitors only match partially:
 \begin{cfa}
 mutex struct A {};
 mutex struct B {};
+void g(A & mutex a, B & mutex b) {
+        waitfor(f, a, b);
+}
+void g( A & mutex m1, B & mutex m2 ) {
+        waitfor( f, m1, m2 );
+}
 A a1, a2;
 B b;
 void foo() {
+        g(a1, b); // block on accept
+}
+        g( a1, b ); // block on accept
+}
 void bar() {
         f(a2, b); // fulfill cooperation
+        f( a2, b ); // fulfill cooperation
+}
 \end{cfa}
 …
 It is also important to note that in the case of external scheduling the order of parameters is irrelevant; @waitfor(f,a,b)@ and @waitfor(f,b,a)@ are indistinguishable waiting condition.
+% ======================================================================
+% ======================================================================
 \subsection{\protect\lstinline|waitfor| Semantics}
-% ======================================================================
-% ======================================================================
 Syntactically, the @waitfor@ statement takes a routine identifier and a set of monitors.
 …
 \end{figure}
+% ======================================================================
+% ======================================================================
 \subsection{Waiting For The Destructor}
+% ======================================================================
+% ======================================================================
 An interesting use for the @waitfor@ statement is destructor semantics.
 Indeed, the @waitfor@ statement can accept any @mutex@ routine, which includes the destructor (see section \ref{data}).
 …
-% ######     #    ######     #    #       #       ####### #       ###  #####  #     #
-% #     #   # #   #     #   # #   #       #       #       #        #  #     # ##   ##
-% #     #  #   #  #     #  #   #  #       #       #       #        #  #       # # # #
-% ######  #     # ######  #     # #       #       #####   #        #   #####  #  #  #
-% #       ####### #   #   ####### #       #       #       #        #        # #     #
-% #       #     # #    #  #     # #       #       #       #        #  #     # #     #
-% #       #     # #     # #     # ####### ####### ####### ####### ###  #####  #     #
 \section{Parallelism}
 Historically, computer performance was about processor speeds and instruction counts.
 However, with heat dissipation being a direct consequence of speed increase, parallelism has become the new source for increased performance~\cite{Sutter05, Sutter05b}.
 …
 While there are many variations of the presented paradigms, most of these variations do not actually change the guarantees or the semantics, they simply move costs in order to achieve better performance for certain workloads.
 \section{Paradigms}
 \subsection{User-Level Threads}
 A direct improvement on the \textbf{kthread} approach is to use \textbf{uthread}.
 These threads offer most of the same features that the operating system already provides but can be used on a much larger scale.
 …
 Examples of languages that support \textbf{uthread} are Erlang~\cite{Erlang} and \uC~\cite{uC++book}.
 \subsection{Fibers : User-Level Threads Without Preemption} \label{fibers}
 A popular variant of \textbf{uthread} is what is often referred to as \textbf{fiber}.
 However, \textbf{fiber} do not present meaningful semantic differences with \textbf{uthread}.
 …
 An example of a language that uses fibers is Go~\cite{Go}
 \subsection{Jobs and Thread Pools}
 An approach on the opposite end of the spectrum is to base parallelism on \textbf{pool}.
 Indeed, \textbf{pool} offer limited flexibility but at the benefit of a simpler user interface.
 …
 The gold standard of this implementation is Intel's TBB library~\cite{TBB}.
 \subsection{Paradigm Performance}
 While the choice between the three paradigms listed above may have significant performance implications, it is difficult to pin down the performance implications of choosing a model at the language level.
 Indeed, in many situations one of these paradigms may show better performance but it all strongly depends on the workload.
 …
 Finally, if the units of uninterrupted work are large, enough the paradigm choice is largely amortized by the actual work done.
 \section{The \protect\CFA\ Kernel : Processors, Clusters and Threads}\label{kernel}
 A \textbf{cfacluster} is a group of \textbf{kthread} executed in isolation. \textbf{uthread} are scheduled on the \textbf{kthread} of a given \textbf{cfacluster}, allowing organization between \textbf{uthread} and \textbf{kthread}.
 It is important that \textbf{kthread} belonging to a same \textbf{cfacluster} have homogeneous settings, otherwise migrating a \textbf{uthread} from one \textbf{kthread} to the other can cause issues.
 …
 Currently \CFA only supports one \textbf{cfacluster}, the initial one.
 \subsection{Future Work: Machine Setup}\label{machine}
 While this was not done in the context of this paper, another important aspect of clusters is affinity.
 While many common desktop and laptop PCs have homogeneous CPUs, other devices often have more heterogeneous setups.
 …
 OS support for CPU affinity is now common~\cite{affinityLinux, affinityWindows, affinityFreebsd, affinityNetbsd, affinityMacosx}, which means it is both possible and desirable for \CFA to offer an abstraction mechanism for portable CPU affinity.
 \subsection{Paradigms}\label{cfaparadigms}
 Given these building blocks, it is possible to reproduce all three of the popular paradigms.
 Indeed, \textbf{uthread} is the default paradigm in \CFA.
 …
 \section{Behind the Scenes}
 There are several challenges specific to \CFA when implementing concurrency.
 These challenges are a direct result of bulk acquire and loose object definitions.
 …
 Note that since the major contributions of this paper are extending monitor semantics to bulk acquire and loose object definitions, any challenges that are not resulting of these characteristics of \CFA are considered as solved problems and therefore not discussed.
+% ======================================================================
+% ======================================================================
 \section{Mutex Routines}
-% ======================================================================
-% ======================================================================
 The first step towards the monitor implementation is simple @mutex@ routines.
 …
 \end{figure}
 \subsection{Details: Interaction with polymorphism}
 Depending on the choice of semantics for when monitor locks are acquired, interaction between monitors and \CFA's concept of polymorphism can be more complex to support.
 However, it is shown that entry-point locking solves most of the issues.
 …
 Furthermore, entry-point locking requires less code generation since any useful routine is called multiple times but there is only one entry point for many call sites.
+% ======================================================================
+% ======================================================================
 \section{Threading} \label{impl:thread}
-% ======================================================================
-% ======================================================================
 Figure \ref{fig:system1} shows a high-level picture if the \CFA runtime system in regards to concurrency.
 …
 \end{figure}
 \subsection{Processors}
 Parallelism in \CFA is built around using processors to specify how much parallelism is desired. \CFA processors are object wrappers around kernel threads, specifically @pthread@s in the current implementation of \CFA.
 Indeed, any parallelism must go through operating-system libraries.
 …
 Processors internally use coroutines to take advantage of the existing context-switching semantics.
 \subsection{Stack Management}
 One of the challenges of this system is to reduce the footprint as much as possible.
 Specifically, all @pthread@s created also have a stack created with them, which should be used as much as possible.
 …
 In order to respect C user expectations, the stack of the initial kernel thread, the main stack of the program, is used by the main user thread rather than the main processor, which can grow very large.
 \subsection{Context Switching}
 As mentioned in section \ref{coroutine}, coroutines are a stepping stone for implementing threading, because they share the same mechanism for context-switching between different stacks.
 To improve performance and simplicity, context-switching is implemented using the following assumption: all context-switches happen inside a specific routine call.
 …
 This option is not currently present in \CFA, but the changes required to add it are strictly additive.
 \subsection{Preemption} \label{preemption}
 Finally, an important aspect for any complete threading system is preemption.
 As mentioned in section \ref{basics}, preemption introduces an extra degree of uncertainty, which enables users to have multiple threads interleave transparently, rather than having to cooperate among threads for proper scheduling and CPU distribution.
 …
 Indeed, @sigwait@ can differentiate signals sent from @pthread_sigqueue@ from signals sent from alarms or the kernel.
 \subsection{Scheduler}
 Finally, an aspect that was not mentioned yet is the scheduling algorithm.
 …
 Further discussion on scheduling is present in section \ref{futur:sched}.
+% ======================================================================
+% ======================================================================
 \section{Internal Scheduling} \label{impl:intsched}
+% ======================================================================
+% ======================================================================
 The following figure is the traditional illustration of a monitor (repeated from page~\pageref{fig:ClassicalMonitor} for convenience):
 \begin{figure}
 \begin{center}
 {\resizebox{0.4\textwidth}{!}{\input{monitor}}}
+{\resizebox{0.4\textwidth}{!}{\input{monitor.pstex_t}}}
 \end{center}
 \caption{Traditional illustration of a monitor}

Note: See TracChangeset for help on using the changeset viewer.

Context Navigation

Changeset 6d43cc57 for doc/papers/concurrency/Paper.tex

Legend:

doc/papers/concurrency/Paper.tex

Download in other formats: