# source:doc/proposals/concurrency/text/internals.tex@64b272a

aaron-thesisarm-ehcleanup-dtorsdeferred_resndemanglerjacob/cs343-translationjenkins-sandboxnew-astnew-ast-unique-exprnew-envno_listpersistent-indexerresolv-newwith_gc
Last change on this file since 64b272a was 64b272a, checked in by Thierry Delisle <tdelisle@…>, 4 years ago

Prereview commit

• Property mode set to 100644
File size: 23.1 KB
Line
1
2\chapter{Behind the scene}
3There are several challenges specific to \CFA when implementing concurrency. These challenges are direct results of \gls{bulk-acq} and loose object definitions. These two constraints are to root cause of most design decisions in the implementation. Furthermore, to avoid the head-aches of dynamically allocating memory in a concurrent environment, the internal-scheduling design is (almost) entirely free of mallocs and other dynamic memory allocation scheme. This is to avoid the chicken and egg problem \cite{Chicken} of having a memory allocator that relies on the threading system and a threading system that relies on the runtime. This extra goal, means that memory management is a constant concern in the design of the system.
4
5The main memory concern for concurrency is queues. All blocking operations are made by parking threads onto queues. These queues need to be intrinsic\cit to avoid the need memory allocation. This entails that all the fields needed to keep track of all needed information. Since many conconcurrency operations can use an unbound amount of memory (depending on \gls{bulk-acq}) statically defining information in the intrusive fields of threads is insufficient. The only variable sized container that does not require memory allocation is the callstack, which is heavily used in the implementation of internal scheduling. Particularly the GCC extension variable length arrays which is used extensively.
6
7Since stack allocation is based around scope, the first step of the implementation is to identify the scopes that are available to store the information, and which of these can have a variable length. The threads and the condition both allow a fixed amount of memory to be stored, while mutex-routines and the actual blocking call allow for an unbound amount (though the later is preferable in terms of performance).
8
9Note that since the major contributions of this thesis are extending monitor semantics to \gls{bulk-acq} and loose object definitions, any challenges that are not resulting of these characteristiques of \CFA are consired as problems which have already been solved and therefore will not be discussed further.
10
11% ======================================================================
12% ======================================================================
13\section{Mutex routines}
14% ======================================================================
15% ======================================================================
16
17The first step towards the monitor implementation is simple mutex-routines using monitors. In the single monitor case, this is done using the entry/exit procedure highlighted in listing \ref{lst:entry1}. This entry/exit procedure doesn't actually have to be extended to support multiple monitors, indeed it is sufficient to enter/leave monitors one-by-one as long as the order is correct to prevent deadlocks\cit. In \CFA, ordering of monitor relies on memory ordering, this is sufficient because all objects are guaranteed to have distinct non-overlaping memory layouts and mutual-exclusion for a monitor is only defined for its lifetime, meaning that destroying a monitor while it is acquired is undefined behavior. When a mutex call is made, the concerned monitors are agregated into an variable-length pointer array and sorted based on pointer values. This array is concerved during the entire duration of the mutual-exclusion and it's ordering reused extensively.
18\begin{figure}
19\begin{multicols}{2}
20Entry
21\begin{pseudo}
22if monitor is free
23        enter
25        continue
26else
27        block
28increment recursions
29\end{pseudo}
30\columnbreak
31Exit
32\begin{pseudo}
33decrement recursion
34if recursion == 0
35        if entry queue not empty
37\end{pseudo}
38\end{multicols}
39\caption{Initial entry and exit routine for monitors}
40\label{lst:entry1}
41\end{figure}
42
43\subsection{ Details: Interaction with polymorphism}
44Depending on the choice of semantics for when monitor locks are acquired, interaction between monitors and \CFA's concept of polymorphism can be more complex to support. However, it is shown that entry-point locking solves most of the issues.
45
46First of all, interaction between \code{otype} polymorphism and monitors is impossible since monitors do not support copying. Therefore, the main question is how to support \code{dtype} polymorphism. It is important to present the difference between the two acquiring options : callsite and entry-point locking, i.e. acquiring the monitors before making a mutex routine call or as the first operation of the mutex routine-call. For example:
47\begin{figure}[H]
48\begin{center}
49\begin{tabular}{|c|c|c|}
50Mutex & \gls{callsite-locking} & \gls{entry-point-locking} \\
51call & pseudo-code & pseudo-code \\
52\hline
53\begin{cfacode}[tabsize=3]
54void foo(monitor& mutex a){
55
56        //Do Work
57        //...
58
59}
60
61void main() {
62        monitor a;
63
64        foo(a);
65
66}
67\end{cfacode} & \begin{pseudo}[tabsize=3]
68foo(& a) {
69
70        //Do Work
71        //...
72
73}
74
75main() {
76        monitor a;
77        acquire(a);
78        foo(a);
79        release(a);
80}
81\end{pseudo} & \begin{pseudo}[tabsize=3]
82foo(& a) {
83        acquire(a);
84        //Do Work
85        //...
86        release(a);
87}
88
89main() {
90        monitor a;
91
92        foo(a);
93
94}
95\end{pseudo}
96\end{tabular}
97\end{center}
98\caption{Callsite vs entry-point locking for mutex calls}
99\label{fig:locking-site}
100\end{figure}
101
102Note the \code{mutex} keyword relies on the type system, which means that in cases where a generic monitor routine is actually desired, writing a mutex routine is possible with the proper trait, for example:
103\begin{cfacode}
104//Incorrect: T is not a monitor
105forall(dtype T)
106void foo(T * mutex t);
107
108//Correct: this function only works on monitors (any monitor)
109forall(dtype T | is_monitor(T))
110void bar(T * mutex t));
111\end{cfacode}
112
113Both entry-point and callsite locking are valid implementations. The current \CFA implementations uses entry-point locking because it seems to require less work if done using \gls{raii}, effectively transferring the burden of implementation to object construction/destruction. The same could be said of callsite locking, the difference being that the later does not necessarily have an existing scope that matches exactly the scope of the mutual exclusion, i.e.: the function body.
114
115% ======================================================================
116% ======================================================================
118% ======================================================================
119% ======================================================================
120
121Figure \ref{fig:system1} shows a high-level picture if the \CFA runtime system in regards to concurrency.
122
123\begin{figure}
124\begin{center}
125{\resizebox{\textwidth}{!}{\input{system.pstex_t}}}
126\end{center}
127\caption{Overview of the entire system}
128\label{fig:system1}
129\end{figure}
130
131\subsection{Context Switching}
132As mentionned in section \ref{coroutine}, coroutines are a stepping stone for implementing threading. This is because they share the same mechanism for context-switching between different stacks. To improve performance and simplicity, context-switching is implemented using the following assumption: all context-switches happen inside a specific function call. This assumptions means that the basic recipe for context-switch is only to copy all callee-saved registers unto the stack and then switch the stack registers with the ones of the target coroutine/thread. Note that instruction pointer can be left untouched since the context-switch always inside the same function. In the case of coroutines, that is the entire story. Threads however do not simply context-switch between each other directly. The context-switch to processors which is where the scheduling happens. This method is called a 2-step context-switch and has the advantage of having a clear distinction between user code and the "kernel" where scheduling and other system operation happen. Obiously, this has the cost of doubling the context-switch cost from because threads must context-switch to an intermediate stack. However, the performance of the 2-step context-switch is still superior to a \code{pthread_yield}(see section \ref{results}). additionally, for users in need for optimal performance, it is important to note that having a 2-step context-switch as the default does not prevent \CFA from offering a 1-step context-switch to use manually (or as part of monitors). This option is not currently present in \CFA but the changes required to add it are strictly additive.
133
134\subsection{Processors}
135Parallelism in \CFA are built around using processors to specify how much parallelism is desired. \CFA processors are object wrappers around kernel threads, specifically pthreads in the current implementation of \CFA. Indeed, any parallelism must go through operatiing system librairies. However, \gls{cfathread} are still the main source of concurrency, processors are simply the underlying source of parallelism. Indeed, processor kernel threads simply fetch a user-level thread from the scheduler and run, they are effectively executers for user-threads. The main benefit of this approach is that it offers a well defined boundary between kernel code and user-code, for example kernel thread quiescing, scheduling and interrupt handling. Processors internally use coroutines to take advantage of the existing context-switching semantics.
136
137\subsection{Stack management}
138One of the challenges of this system is to reduce the footprint as much as possible. Specifically, all pthreads created also have a stack created with them, which should be used as much as possible. Normally, coroutines also create there own stack to run on, however, in the case of the coroutines used for processors, these coroutines run directly on the kernel thread stack, effectively stealing the processor stack. The exception to this rule is the Main Processor, i.e. the initial kernel thread that is given to any program. In order to respect user expectations, the stack of the initial kernel thread, the main stack of the program, is used by the main user thread rather than the main processor.
139
140\subsection{Preemption}
141Finally, an important aspect for any complete threading system is preemption. As mentionned in chapter \ref{basics}, preemption introduces an extra degree of unceretainty, which enables users to have multiple threads interleave transparrently between eachother, rather than having to cooperate between thread for proper scheduling and CPU distribution. Indeed, preemption is desireable because it adds a degree of isolation between tasks. In a fully cooperative system, any thread that runs into a long loop can starve other threads, while in a preemptive system starvation can still occur but it does not rely on every thread having to yield or block on a regular basis, which reduces significantly programmer burden. Obviously, preemption is not optimal for every workload, however any preemptive system can become a cooperative system by making the time-slices extremely large. Which is why \CFA uses a preemptive threading system.
142
143Preemption in \CFA is based on kernel timers which are used to run a discreet event simulation. Every processor keeps track of the current time and registers an expiration time with the preemption system. When the preemption system receives a change in preemption it sorts these expiration times in a list and sets a kernel timer for the closest one, effectiveling stepping between preemption events on each signals sent by the timer. These timers use the linux signal {\tt SIGALRM}, which is delivered to the process. This is important because when delivering signals to a process, the kernel documentation states that the signal can be delivered to any kernel thread for which the signal isn't block i.e. :
144\begin{quote}
145A process-directed signal may be delivered to any one of the threads that does not currently have the signal blocked. If more than one of the threads has the signal unblocked, then the kernel chooses an arbitrary thread to which to deliver the signal.
146SIGNAL(7) - Linux Programmer's Manual
147\end{quote}
148For the sake of simplicity and in order to prevent the case of having two threads receiving alarms simultaneously, \CFA programs block the {\tt SIGALRM} signal on every thread except one. Now because of how involontary context-switches are handled, the kernel thread handling {\tt SIGALRM} cannot also be a processor thread.
149
151
152\subsection{Scheduler} \footnote{ I'm not sure what to write here, is this section even needed. }
153Finally, an aspect that was not mentionned yet is the scheduling algorithm. Currently, the \CFA scheduler uses a single ready queue for all processors. Will this is not the highest performance algorithm, it has the significant advantage of being robust to heterogenous workloads. This is a very simple scheduling approach but is sufficient to for the context of this thesis.
154
155What to do here?
156
157However, when
158As will be mentionned \ref{futur:sched} it needs to be updated when clusters will be
159
160clusters
161
162
163
164Among the most pressing updates to the \CFA
165uses single queue
166in future should move to multiple queues with workstealing
167general purpouse means robust > fast
168worksharing can higher standard deviation in performance
169
170
171% ======================================================================
172% ======================================================================
173\section{Internal scheduling} \label{impl:intsched}
174% ======================================================================
175% ======================================================================
176To ease the understanding of monitors, like many other concepts, they are generelly represented graphically. While non-scheduled monitors are simple enough for a graphical representation to be useful, internal scheduling is complex enough to justify a visual representation. The following figure is the traditionnal illustration of a monitor :
177
178\begin{center}
179{\resizebox{0.4\textwidth}{!}{\input{monitor}}}
180\end{center}
181
182This picture has several components, the two most important being the entry-queue and the AS-stack. The entry-queue is a (almost) FIFO list where threads waiting to enter are parked, while the AS-stack is a FILO list used for threads that have been signaled or otherwise marked as running next. For \CFA, the previous picture does not have support for blocking multiple monitors on a single condition. To support \gls{bulk-acq} two changes to this picture are required. First, it doesn't make sense to tie the condition to a single monitor since blocking two monitors as one would require arbitrarily picking a monitor to hold the condition. Secondly, the object waiting on the conditions and AS-stack cannot simply contain the waiting thread since a single thread can potentially wait on multiple monitors. As mentionned in section \ref{intsched}, the handling in multiple monitors is done by partially passing, which entails that each concerned monitor needs to have a node object. However, for waiting on the condition, since all threads need to wait together, a single object needs to be queued in the condition. Moving out the condition and updating the node types yields :
183
184\begin{center}
185{\resizebox{0.8\textwidth}{!}{\input{int_monitor}}}
186\end{center}
187
188This picture and the proper entry and leave algorithms is the fundamental implementation of internal scheduling (see listing \ref{lst:entry2}).
189
190\begin{figure}[b]
191\begin{multicols}{2}
192Entry
193\begin{pseudo}
194if monitor is free
195        enter
197        continue
198else
199        block
200increment recursion
201
202\end{pseudo}
203\columnbreak
204Exit
205\begin{pseudo}
206decrement recursion
207if recursion == 0
208        if signal_stack not empty
212
213        if entry queue not empty
215\end{pseudo}
216\end{multicols}
217\caption{Entry and exit routine for monitors with internal scheduling}
218\label{lst:entry2}
219\end{figure}
220
221Some important things to notice about the exit routine. The solution discussed in \ref{intsched} can be seen in the exit routine of listing \ref{lst:entry2}. Basically, the solution boils down to having a seperate data structure for the condition queue and the AS-stack, and unconditionally transferring ownership of the monitors but only unblocking the thread when the last monitor has transferred ownership. This solution is deadlock safe as well as preventing any potential barging.
222
223The data structure used for the AS-stack are reused extensively for external scheduling, but in the case of internal scheduling, the data is allocated using variable-length arrays on the callstack of the \code{wait} and \code{signal_block} routines.
224
225% ======================================================================
226% ======================================================================
227\section{External scheduling}
228% ======================================================================
229% ======================================================================
230Similarly to internal scheduling, external scheduling for multiple monitors relies on the idea that entry-queues are no longer specific to a single monitor, as mentionned in section \ref{extsched}. This means that some kind of entry-queues must be used that is aware of both monitors and which holds threads that are currently waiting to enter the critical section. This challenge is solved for internal scheduling by having the entry-queues in conditions no longer be tied to a monitor, effectively allowing conditions to be moved outside of monitors. However, in the case of external scheduling, acceptable routines must be aware of the entry queues, which means they must be stored inside at least one of the monitors that will be acquired. This in turn adds the requirement that a systematic algorithm of disambiguating which monitor holds the relevant queue regardless of user ordering. The proposed algorithm is to fall back on monitor lock ordering and specify that the monitor that is acquired first is the one with the relevant entry queue. This assumes that the lock acquiring order is static for the lifetime of all concerned objects but that is a reasonable constraint.
231
232This algorithm choice has two consequences, the entry queue of the highest priority monitor is no longer a true FIFO queue and the queue of the lowest priority monitor is both required and probably unused. The queue can no longer be a FIFO queue because instead of simply containing the waiting threads in order of arrival, they also contain a set of monitors. Therefore, another thread whos set contains the same highest priority monitor but different lower priority monitors may arrive first but enter the critical section after a thread with the correct pairing. Secondly, since it is not known at compile time which monitor will be the lowest priority monitor, every monitor needs to have the correct queues even though it is probable that some queues will go unused for the entire duration of the program, for example if a monitor is only used in a pair.
233
234Therefore, the following modifications need to be made to support external scheduling :
235\begin{itemize}
236        \item The threads waiting on the entry-queue need to keep track of which routine is trying to enter, and using which set of monitors. The \code{mutex} routine already has all the required information on it's stack so the thread only needs to keep a pointer to that information.
237        \item The monitors need to keep a mask of acceptable routines. This mask contains for each acceptable routine, a routine pointer and an array of monitors to go with it. It also needs storage to keep track of which routine was accepted. Since this information is not specific to any monitor, the monitors actually contain a pointer to an integer on the stack of the waiting thread. Note that the complete mask can be pushed to any owned monitors, regardless of \code{when} statements, the \code{waitfor} statement is used in a context where the thread already has full ownership of (at least) every concerned monitor and therefore monitors will refuse all calls no matter what.
238        \item The entry/exit routine need to be updated as shown in listing \ref{lst:entry3}.
239\end{itemize}
240
241Finally, to support the ordering inversion of destructors, the code generation needs to be modified to use a special entry routine. This routine is needed because of the storage requirements of the call order inversion. Indeed, when waiting for the destructors, storage is need for the waiting context and the lifetime of said storage needs to outlive the waiting operation it is needed for. For regular \code{waitfor} statements, the callstack of the routine itself matches this requirement but it is no longer the case when waiting for the destructor since it is pushed on to the AS-stack for later. The waitfor semantics can then be adjusted correspondingly, as seen in listing \ref{lst:entry-dtor}
242
243\begin{figure}
244\begin{multicols}{2}
245Entry
246\begin{pseudo}
247if monitor is free
248        enter
250        continue
252        push waiter to AS-stack
253        continue
254else
255        block
256increment recursion
257\end{pseudo}
258\columnbreak
259Exit
260\begin{pseudo}
261decrement recursion
262if recursion == 0
263        if signal_stack not empty
267
268        if entry queue not empty
270\end{pseudo}
271\end{multicols}
272\caption{Entry and exit routine for monitors with internal scheduling and external scheduling}
273\label{lst:entry3}
274\end{figure}
275
276\begin{figure}
277\begin{multicols}{2}
278Destructor Entry
279\begin{pseudo}
280if monitor is free
281        enter
283        increment recursion
284        return
285create wait context
288        push self to AS-stack
289        baton pass
290else
291        wait
292increment recursion
293\end{pseudo}
294\columnbreak
295Waitfor
296\begin{pseudo}
297lock all monitors
299        if found destructor
300                push destructor to AS-stack
301                unlock all monitors
302        else
303                push self to AS-stack
304                baton pass
305        return
306
307if non-blocking
308        Unlock all monitors
309        Return
310
311push self to AS-stack