1 | % ====================================================================== |
---|
2 | % ====================================================================== |
---|
3 | \chapter{Channels}\label{s:channels} |
---|
4 | % ====================================================================== |
---|
5 | % ====================================================================== |
---|
6 | |
---|
7 | Most modern concurrent programming languages do not subscribe to just one style of communication among threads and provide features that support multiple approaches. |
---|
8 | Channels are a concurrent-language feature used to perform \Newterm{message-passing concurrency}: a model of concurrency where threads communicate by sending data as messages (mostly non\-blocking) and synchronizing by receiving sent messages (blocking). |
---|
9 | This model is an alternative to shared-memory concurrency, where threads communicate directly by changing shared state. |
---|
10 | |
---|
11 | Channels were first introduced by Kahn~\cite{Kahn74} and extended by Hoare~\cite{Hoare78} (CSP). |
---|
12 | Both papers present a pseudo (unimplemented) concurrent language where processes communicate using input/output channels to send data. |
---|
13 | Both languages are highly restrictive. |
---|
14 | Kahn's language restricts a reading process to only wait for data on a single channel at a time and different writing processes cannot send data on the same channel. |
---|
15 | Hoare's language restricts both the sender and receiver to explicitly name the process that is the destination of a channel send or the source of a channel receive. |
---|
16 | These channel semantics remove the ability to have an anonymous sender or receiver. |
---|
17 | Additionally all channel operations in CSP are synchronous (no buffering). |
---|
18 | Advanced channels as a programming language feature has been popularized in recent years by the language Go~\cite{Go}, which encourages the use of channels as its fundamental concurrent feature. |
---|
19 | It was the popularity of Go channels that lead me to implement them in \CFA. |
---|
20 | Neither Go nor \CFA channels have the restrictions in early channel-based concurrent systems. |
---|
21 | |
---|
22 | \section{Producer-Consumer Problem} |
---|
23 | A channel is an abstraction for a shared-memory buffer, which turns the implementation of a channel into the producer-consumer problem. |
---|
24 | The producer-consumer problem, also known as the bounded-buffer problem, was introduced by Dijkstra~\cite[\S~4.1]{Dijkstra65}. |
---|
25 | In the problem, threads interact with a buffer in two ways: producing threads insert values into the buffer and consuming threads remove values from the buffer. |
---|
26 | In general, a buffer needs protection to ensure a producer only inserts into a non-full buffer and a consumer only removes from a non-empty buffer (synchronization). |
---|
27 | As well, a buffer needs protection from concurrent access by multiple producers or consumers attempting to insert or remove simultaneously (MX). |
---|
28 | |
---|
29 | \section{Channel Size}\label{s:ChannelSize} |
---|
30 | Channels come in three flavours of buffers: |
---|
31 | \begin{enumerate} |
---|
32 | \item |
---|
33 | Zero sized implies the communication is synchronous, \ie the producer must wait for the consumer to arrive or vice versa for a value to be communicated. |
---|
34 | \item |
---|
35 | Fixed sized (bounded) implies the communication is asynchronous, \ie the producer can proceed up to the buffer size and vice versa for the consumer with respect to removal. |
---|
36 | \item |
---|
37 | Infinite sized (unbounded) implies the communication is asynchronous, \ie the producer never waits but the consumer waits when the buffer is empty. |
---|
38 | Since memory is finite, all unbounded buffers are ultimately bounded; |
---|
39 | this restriction must be part of its implementation. |
---|
40 | \end{enumerate} |
---|
41 | |
---|
42 | In general, the order values are processed by the consumer does not affect the correctness of the producer-consumer problem. |
---|
43 | For example, the buffer can be LIFO, FIFO, or prioritized with respect to insertion and removal. |
---|
44 | However, like MX, a buffer should ensure every value is eventually removed after some reasonable bounded time (no long-term starvation). |
---|
45 | The simplest way to prevent starvation is to implement the buffer as a queue, either with a cyclic array or linked nodes. |
---|
46 | |
---|
47 | \section{First-Come First-Served} |
---|
48 | As pointed out, a bounded buffer requires MX among multiple producers or consumers. |
---|
49 | This MX should be fair among threads, independent of the FIFO buffer being fair among values. |
---|
50 | Fairness among threads is called \gls{fcfs} and was defined by Lamport~\cite[p.~454]{Lamport74}. |
---|
51 | \gls{fcfs} is defined in relation to a doorway~\cite[p.~330]{Lamport86II}, which is the point at which an ordering among threads can be established. |
---|
52 | Given this doorway, a CS is said to be \gls{fcfs}, if threads access the shared resource in the order they proceed through the doorway. |
---|
53 | A consequence of \gls{fcfs} execution is the elimination of \Newterm{barging}, where barging means a thread arrives at a CS with waiting threads, and the MX protecting the CS allows the arriving thread to enter the CS ahead of one or more of the waiting threads. |
---|
54 | |
---|
55 | \gls{fcfs} is a fairness property that prevents unequal access to the shared resource and prevents starvation, however it comes at a cost. |
---|
56 | Implementing an algorithm with \gls{fcfs} can lead to \Newterm{double blocking}, where arriving threads block outside the doorway waiting for a thread in the lock entry-protocol and inside the doorway waiting for a thread in the CS. |
---|
57 | An analogue is boarding an airplane: first you wait to get through security to the departure gates (short term), and then wait again at the departure gate for the airplane (long term). |
---|
58 | As such, algorithms that are not \gls{fcfs} (barging) can be more performant by skipping the wait for the CS and entering directly; |
---|
59 | however, this performance gain comes by introducing unfairness with possible starvation for waiting threads. |
---|
60 | |
---|
61 | \section{Channel Implementation} |
---|
62 | Currently, only the Go programming language provides user-level threading where the primary communication mechanism is channels. |
---|
63 | Experiments were conducted that varied the producer-consumer problem algorithm and lock type used inside the channel. |
---|
64 | With the exception of non-\gls{fcfs} algorithms, no algorithm or lock usage in the channel implementation was found to be consistently more performant that Go's choice of algorithm and lock implementation. |
---|
65 | Therefore, the low-level channel implementation in \CFA is largely copied from the Go implementation, but adapted to the \CFA type and runtime systems. |
---|
66 | As such the research contributions added by \CFA's channel implementation lie in the realm of safety and productivity features. |
---|
67 | |
---|
68 | \PAB{Discuss the Go channel implementation. Need to tie in FIFO buffer and FCFS locking.} |
---|
69 | |
---|
70 | In this work, all channel sizes \see{Sections~\ref{s:ChannelSize}} are implemented with bounded buffers. |
---|
71 | However, only non-zero-sized buffers are analysed because of their complexity and higher usage. |
---|
72 | |
---|
73 | \section{Safety and Productivity} |
---|
74 | Channels in \CFA come with safety and productivity features to aid users. |
---|
75 | The features include the following. |
---|
76 | |
---|
77 | \begin{itemize} |
---|
78 | \item Toggle-able statistic collection on channel behaviour that count channel and blocking operations. |
---|
79 | Tracking blocking operations helps illustrate usage for tuning the channel size, where the aim is to reduce blocking. |
---|
80 | |
---|
81 | \item Deadlock detection on channel deallocation. |
---|
82 | If threads are blocked inside a channel when it terminates, this case is detected and the user is informed, as this can cause a deadlock. |
---|
83 | |
---|
84 | \item A @flush@ routine that delivers copies of an element to all waiting consumers, flushing the buffer. |
---|
85 | Programmers use this mechanism to broadcast a sentinel value to multiple consumers. |
---|
86 | Additionally, the @flush@ routine is more performant then looping around the @insert@ operation since it can deliver the elements without having to reacquire mutual exclusion for each element sent. |
---|
87 | \end{itemize} |
---|
88 | |
---|
89 | \subsection{Toggle-able Statistics} |
---|
90 | \PAB{Discuss toggle-able statistics.} |
---|
91 | |
---|
92 | |
---|
93 | \subsection{Deadlock Detection} |
---|
94 | \PAB{Discuss deadlock detection.} |
---|
95 | |
---|
96 | \subsection{Program Shutdown} |
---|
97 | % The other safety and productivity feature of \CFA channels deals with concurrent termination. |
---|
98 | Terminating concurrent programs is often one of the most difficult parts of writing concurrent code, particularly if graceful termination is needed. |
---|
99 | The difficulty of graceful termination often arises from the usage of synchronization primitives that need to be handled carefully during shutdown. |
---|
100 | It is easy to deadlock during termination if threads are left behind on synchronization primitives. |
---|
101 | Additionally, most synchronization primitives are prone to \gls{toctou} issues where there is race between one thread checking the state of a concurrent object and another thread changing the state. |
---|
102 | \gls{toctou} issues with synchronization primitives often involve a race between one thread checking the primitive for blocked threads and another thread blocking on it. |
---|
103 | Channels are a particularly hard synchronization primitive to terminate since both sending and receiving to/from a channel can block. |
---|
104 | Thus, improperly handled \gls{toctou} issues with channels often result in deadlocks as threads trying to perform the termination may end up unexpectedly blocking in their attempt to help other threads exit the system. |
---|
105 | |
---|
106 | % C_TODO: add reference to select chapter, add citation to go channels info |
---|
107 | \paragraph{Go channels} provide a set of tools to help with concurrent shutdown. |
---|
108 | Channels in Go have a @close@ operation and a \Go{select} statement that both can be used to help threads terminate. |
---|
109 | The \Go{select} statement is discussed in \ref{waituntil}, where \CFA's @waituntil@ statement is compared with the Go \Go{select} statement. |
---|
110 | |
---|
111 | The @close@ operation on a channel in Go changes the state of the channel. |
---|
112 | When a channel is closed, sends to the channel panic along with additional calls to @close@. |
---|
113 | Receives are handled differently where receivers never block on a closed channel and continue to remove elements from the channel. |
---|
114 | Once a channel is empty, receivers can continue to remove elements, but receive the zero-value version of the element type. |
---|
115 | To avoid unwanted zero-value elements, Go provides the ability to iterate over a closed channel to remove the remaining elements. |
---|
116 | These Go design choices enforce a specific interaction style with channels during termination: careful thought is needed to ensure additional @close@ calls do not occur and no sends occur after a channel is closed. |
---|
117 | These design choices fit Go's paradigm of error management, where users are expected to explicitly check for errors, rather than letting errors occur and catching them. |
---|
118 | If errors need to occur in Go, return codes are used to pass error information up call levels. |
---|
119 | Note, panics in Go can be caught, but it is not the idiomatic way to write Go programs. |
---|
120 | |
---|
121 | While Go's channel closing semantics are powerful enough to perform any concurrent termination needed by a program, their lack of ease of use leaves much to be desired. |
---|
122 | Since both closing and sending panic once a channel is closed, a user often has to synchronize the senders to a channel before the channel can be closed to avoid panics. |
---|
123 | However, in doing so it renders the @close@ operation nearly useless, as the only utilities it provides are the ability to ensure receivers no longer block on the channel and receive zero-valued elements. |
---|
124 | This functionality is only useful if the zero-typed element is recognized as a sentinel value, but if another sentinel value is necessary, then @close@ only provides the non-blocking feature. |
---|
125 | To avoid \gls{toctou} issues during shutdown, a busy wait with a \Go{select} statement is often used to add or remove elements from a channel. |
---|
126 | Due to Go's asymmetric approach to channel shutdown, separate synchronization between producers and consumers of a channel has to occur during shutdown. |
---|
127 | |
---|
128 | \paragraph{\CFA channels} have access to an extensive exception handling mechanism~\cite{Beach21}. |
---|
129 | As such \CFA uses an exception-based approach to channel shutdown that is symmetric for both producers and consumers, and supports graceful shutdown. |
---|
130 | |
---|
131 | Exceptions in \CFA support both termination and resumption. |
---|
132 | \Newterm{Termination exception}s perform a dynamic call that unwinds the stack preventing the exception handler from returning to the raise point, such as in \CC, Python and Java. |
---|
133 | \Newterm{Resumption exception}s perform a dynamic call that does not unwind the stack allowing the exception handler to return to the raise point. |
---|
134 | In \CFA, if a resumption exception is not handled, it is reraised as a termination exception. |
---|
135 | This mechanism is used to create a flexible and robust termination system for channels. |
---|
136 | |
---|
137 | When a channel in \CFA is closed, all subsequent calls to the channel raise a resumption exception at the caller. |
---|
138 | If the resumption is handled, the caller attempts to complete the channel operation. |
---|
139 | However, if channel operation would block, a termination exception is thrown. |
---|
140 | If the resumption is not handled, the exception is rethrown as a termination. |
---|
141 | These termination exceptions allow for non-local transfer that is used to great effect to eagerly and gracefully shut down a thread. |
---|
142 | When a channel is closed, if there are any blocked producers or consumers inside the channel, they are woken up and also have a resumption thrown at them. |
---|
143 | The resumption exception, @channel_closed@, has a couple fields to aid in handling the exception. |
---|
144 | The exception contains a pointer to the channel it was thrown from, and a pointer to an element. |
---|
145 | In exceptions thrown from remove the element pointer will be null. |
---|
146 | In the case of insert the element pointer points to the element that the thread attempted to insert. |
---|
147 | This element pointer allows the handler to know which operation failed and also allows the element to not be lost on a failed insert since it can be moved elsewhere in the handler. |
---|
148 | Furthermore, due to \CFA's powerful exception system, this data can be used to choose handlers based which channel and operation failed. |
---|
149 | Exception handlers in \CFA have an optional predicate after the exception type which can be used to optionally trigger or skip handlers based on the content of an exception. |
---|
150 | It is worth mentioning that the approach of exceptions for termination may incur a larger performance cost during termination that the approach used in Go. |
---|
151 | This should not be an issue, since termination is rarely an fast-path of an application and ensuring that termination can be implemented correctly with ease is the aim of the exception approach. |
---|
152 | |
---|
153 | \section{\CFA / Go channel Examples} |
---|
154 | To highlight the differences between \CFA's and Go's close semantics, an example program is presented. |
---|
155 | The program is a barrier implemented using two channels shown in Figure~\ref{f:ChannelBarrierTermination}. |
---|
156 | Both of these examples are implemented using \CFA syntax so that they can be easily compared. |
---|
157 | Figure~\ref{l:cfa_chan_bar} uses \CFA-style channel close semantics and Figure~\ref{l:go_chan_bar} uses Go-style close semantics. |
---|
158 | In this problem it is infeasible to use the Go @close@ call since all threads are both potentially producers and consumers, causing panics on close to be unavoidable. |
---|
159 | As such in Figure~\ref{l:go_chan_bar} to implement a flush routine for the buffer, a sentinel value of @-1@ has to be used to indicate to threads that they need to leave the barrier. |
---|
160 | This sentinel value has to be checked at two points. |
---|
161 | Furthermore, an additional flag @done@ is needed to communicate to threads once they have left the barrier that they are done. |
---|
162 | This use of an additional flag or communication method is common in Go channel shutdown code, since to avoid panics on a channel, the shutdown of a channel often has to be communicated with threads before it occurs. |
---|
163 | In the \CFA version~\ref{l:cfa_chan_bar}, the barrier shutdown results in an exception being thrown at threads operating on it, which informs the threads that they must terminate. |
---|
164 | This avoids the need to use a separate communication method other than the barrier, and avoids extra conditional checks on the fast path of the barrier implementation. |
---|
165 | Also note that in the Go version~\ref{l:go_chan_bar}, the size of the barrier channels has to be larger than in the \CFA version to ensure that the main thread does not block when attempting to clear the barrier. |
---|
166 | |
---|
167 | \begin{figure} |
---|
168 | \centering |
---|
169 | |
---|
170 | \begin{lrbox}{\myboxA} |
---|
171 | \begin{cfa}[aboveskip=0pt,belowskip=0pt] |
---|
172 | struct barrier { |
---|
173 | channel( int ) barWait, entryWait; |
---|
174 | int size; |
---|
175 | }; |
---|
176 | void ?{}( barrier & this, int size ) with(this) { |
---|
177 | barWait{size}; entryWait{size}; |
---|
178 | this.size = size; |
---|
179 | for ( i; size ) |
---|
180 | insert( entryWait, i ); |
---|
181 | } |
---|
182 | void wait( barrier & this ) with(this) { |
---|
183 | int ticket = remove( entryWait ); |
---|
184 | |
---|
185 | if ( ticket == size - 1 ) { |
---|
186 | for ( i; size - 1 ) |
---|
187 | insert( barWait, i ); |
---|
188 | return; |
---|
189 | } |
---|
190 | ticket = remove( barWait ); |
---|
191 | |
---|
192 | if ( size == 1 || ticket == size - 2 ) { // last ? |
---|
193 | for ( i; size ) |
---|
194 | insert( entryWait, i ); |
---|
195 | } |
---|
196 | } |
---|
197 | void flush(barrier & this) with(this) { |
---|
198 | @close( barWait ); close( entryWait );@ |
---|
199 | } |
---|
200 | enum { Threads = 4 }; |
---|
201 | barrier b{Threads}; |
---|
202 | |
---|
203 | thread Thread {}; |
---|
204 | void main( Thread & this ) { |
---|
205 | @try {@ |
---|
206 | for () |
---|
207 | wait( b ); |
---|
208 | @} catch ( channel_closed * ) {}@ |
---|
209 | } |
---|
210 | int main() { |
---|
211 | Thread t[Threads]; |
---|
212 | sleep(10`s); |
---|
213 | |
---|
214 | flush( b ); |
---|
215 | } // wait for threads to terminate |
---|
216 | \end{cfa} |
---|
217 | \end{lrbox} |
---|
218 | |
---|
219 | \begin{lrbox}{\myboxB} |
---|
220 | \begin{cfa}[aboveskip=0pt,belowskip=0pt] |
---|
221 | struct barrier { |
---|
222 | channel( int ) barWait, entryWait; |
---|
223 | int size; |
---|
224 | }; |
---|
225 | void ?{}( barrier & this, int size ) with(this) { |
---|
226 | barWait{size + 1}; entryWait{size + 1}; |
---|
227 | this.size = size; |
---|
228 | for ( i; size ) |
---|
229 | insert( entryWait, i ); |
---|
230 | } |
---|
231 | void wait( barrier & this ) with(this) { |
---|
232 | int ticket = remove( entryWait ); |
---|
233 | @if ( ticket == -1 ) { insert( entryWait, -1 ); return; }@ |
---|
234 | if ( ticket == size - 1 ) { |
---|
235 | for ( i; size - 1 ) |
---|
236 | insert( barWait, i ); |
---|
237 | return; |
---|
238 | } |
---|
239 | ticket = remove( barWait ); |
---|
240 | @if ( ticket == -1 ) { insert( barWait, -1 ); return; }@ |
---|
241 | if ( size == 1 || ticket == size - 2 ) { // last ? |
---|
242 | for ( i; size ) |
---|
243 | insert( entryWait, i ); |
---|
244 | } |
---|
245 | } |
---|
246 | void flush(barrier & this) with(this) { |
---|
247 | @insert( entryWait, -1 ); insert( barWait, -1 );@ |
---|
248 | } |
---|
249 | enum { Threads = 4 }; |
---|
250 | barrier b{Threads}; |
---|
251 | @bool done = false;@ |
---|
252 | thread Thread {}; |
---|
253 | void main( Thread & this ) { |
---|
254 | for () { |
---|
255 | @if ( done ) break;@ |
---|
256 | wait( b ); |
---|
257 | } |
---|
258 | } |
---|
259 | int main() { |
---|
260 | Thread t[Threads]; |
---|
261 | sleep(10`s); |
---|
262 | done = true; |
---|
263 | flush( b ); |
---|
264 | } // wait for threads to terminate |
---|
265 | \end{cfa} |
---|
266 | \end{lrbox} |
---|
267 | |
---|
268 | \subfloat[\CFA style]{\label{l:cfa_chan_bar}\usebox\myboxA} |
---|
269 | \hspace*{3pt} |
---|
270 | \vrule |
---|
271 | \hspace*{3pt} |
---|
272 | \subfloat[Go style]{\label{l:go_chan_bar}\usebox\myboxB} |
---|
273 | \caption{Channel Barrier Termination} |
---|
274 | \label{f:ChannelBarrierTermination} |
---|
275 | \end{figure} |
---|
276 | |
---|
277 | Listing~\ref{l:cfa_resume} is an example of a channel closing with resumption. |
---|
278 | The @Producer@ thread-main knows to stop producing when the @insert@ call on a closed channel raises exception @channel_closed@. |
---|
279 | The @Consumer@ thread-main knows to stop consuming after all elements of a closed channel are removed and the call to @remove@ would block. |
---|
280 | Hence, the consumer knows the moment the channel closes because a resumption exception is raised, caught, and ignored, and then control returns to @remove@ to return another item from the buffer. |
---|
281 | Only when the buffer is drained and the call to @removed@ would block is a termination exception raised to stop consuming. |
---|
282 | The same program in Go would require explicit synchronization among producers and consumers by a mechanism outside the channel to ensure all elements are removed before threads terminate. |
---|
283 | |
---|
284 | \begin{cfa}[caption={\CFA channel resumption usage},label={l:cfa_resume}] |
---|
285 | channel( int ) chan{ 128 }; |
---|
286 | thread Producer {}; |
---|
287 | void main( Producer & this ) { |
---|
288 | @try {@ |
---|
289 | for ( i; 0~$@$ ) |
---|
290 | insert( chan, i ); |
---|
291 | @} catch( channel_closed * ) {}@ $\C[3in]{// channel closed}$ |
---|
292 | } |
---|
293 | thread Consumer {}; |
---|
294 | void main( Consumer & this ) { |
---|
295 | size_t runs = 0; |
---|
296 | @try {@ |
---|
297 | for () { |
---|
298 | int i = remove( chan ); |
---|
299 | } |
---|
300 | @} catchResume( channel_closed * ) {}@ $\C{// remaining item in buffer \(\Rightarrow\) remove it}$ |
---|
301 | @catch( channel_closed * ) {}@ $\C{// blocking call to remove \(\Rightarrow\) buffer empty}$ |
---|
302 | } |
---|
303 | int main() { |
---|
304 | enum { Processors = 8 }; |
---|
305 | processor p[Processors - 1]; $\C{// one processor per thread, have one processor}$ |
---|
306 | Consumer c[Processors / 2]; $\C{// share processors}$ |
---|
307 | Producer p[Processors / 2]; |
---|
308 | sleep( 10`s ); |
---|
309 | @close( chan );@ $\C{// stop producer and consumer}\CRT$ |
---|
310 | } |
---|
311 | \end{cfa} |
---|
312 | |
---|
313 | \section{Performance} |
---|
314 | |
---|
315 | Given that the base implementation of the \CFA channels is very similar to the Go implementation, this section aims to show the performance of the two implementations are comparable. |
---|
316 | The microbenchmark for the channel comparison is similar to listing~\ref{l:cfa_resume}, where the number of threads and processors is set from the command line. |
---|
317 | The processors are divided equally between producers and consumers, with one producer or consumer owning each core. |
---|
318 | The number of cores is varied to measure how throughput scales. |
---|
319 | |
---|
320 | The results of the benchmark are shown in Figure~\ref{f:chanPerf}. |
---|
321 | The performance of Go and \CFA channels on this microbenchmark is comparable. |
---|
322 | Note, the performance should decline as the number of cores increases as the channel operations occur in a critical section, so increasing cores results in higher contention with no increase in parallelism. |
---|
323 | |
---|
324 | \begin{figure} |
---|
325 | \centering |
---|
326 | \subfloat[AMD \CFA Channel Benchmark]{ |
---|
327 | \resizebox{0.5\textwidth}{!}{\input{figures/nasus_Channel_Contention.pgf}} |
---|
328 | \label{f:chanAMD} |
---|
329 | } |
---|
330 | \subfloat[Intel \CFA Channel Benchmark]{ |
---|
331 | \resizebox{0.5\textwidth}{!}{\input{figures/pyke_Channel_Contention.pgf}} |
---|
332 | \label{f:chanIntel} |
---|
333 | } |
---|
334 | \caption{The channel contention benchmark comparing \CFA and Go channel throughput (higher is better).} |
---|
335 | \label{f:chanPerf} |
---|
336 | \end{figure} |
---|
337 | |
---|
338 | % Local Variables: % |
---|
339 | % tab-width: 4 % |
---|
340 | % End: % |
---|