source: doc/theses/colby_parsons_MMAth/text/channels.tex @ 9586d22

% ======================================================================
% ======================================================================
\chapter{Channels}\label{s:channels}
% ======================================================================
% ======================================================================

Channels were first introduced by Hoare in his paper Communicating Sequentual Processes~\cite{Hoare78}, where he proposes a concurrent language that communicates across processes using input/output channels to send data. 
Channels are a concurrent language feature used to perform message passing concurrency, a model of concurrency where threads communicate by sending data as messages, and synchronizing via the message passing mechanism. 
This is an alternative to shared memory concurrency, where threads can communicate directly by changing shared memory state. 
Most modern concurrent programming languages do not subscribe to just one style of communication between threads, and provide features that support both. 
Channels as a programming language feature has been popularized in recent years due to the language Go, which encourages the use of channels as its fundamental concurrent feature.

\section{Producer-Consumer Problem}
Most channels in modern programming languages are built on top of a shared memory buffer. 
While it is possible to create a channel that contains an unbounded buffer, most implementations opt to only support a fixed size channel, where the size is given at the time of channel creation. 
This turns the implementation of a channel into the producer-consumer problem. 
The producer-consumer problem, also known as the bounded buffer problem, was introduced by Dijkstra in his book Cooperating Sequential Processes\cite{Dijkstra65}. 
In the problem threads interact with the buffer in two ways, either consuming values by removing them from the buffer, or producing values and inserting them in the buffer. 
The buffer needs to be protected from concurrent access since each item in the buffer should only be produced and consumed once. 
Additionally, a consumer can only remove from a non-empty buffer and a producer can only insert into a non-full buffer.

\section{First-Come First-Served}
The channel implementations that will be discussed are \gls{fcfs}. 
This term was defined by Lamport~\cite{Lamport74}. 
\gls{fcfs} is defined in relation to a doorway~\cite[p.~330]{Lamport86II}, which is the point at which an ordering among threads can be established. 
Given this doorway, a critical section is said to be \gls{fcfs}, if threads access the shared resource in the order they proceed through the doorway. 
\gls{fcfs} is a fairness property which prevents unequal access to the shared resource and prevents starvation, however it can come at a cost. 
Implementing an algorithm with \gls{fcfs} can lead to double blocking, where entering threads may need to block to allow other threads to proceed first, resulting in blocking both inside and outside the doorway. 
As such algorithms that are not \gls{fcfs} may be more performant but that performance comes with the downside of likely introducing starvation and unfairness.

\section{Channel Implementation}
The channel implementation in \CFA is a near carbon copy of the Go implementation. 
Experimentation was conducted that varied the producer-consumer problem algorithm and lock type used inside the channel. 
With the exception of non-\gls{fcfs} algorithms, no algorithm or lock usage in the channel implementation was found to be consistently more performant that Go's choice of algorithm and lock implementation. 
As such the research contributions added by \CFA's channel implementation lie in the realm of safety and productivity features.

\section{Safety and Productivity}
Channels in \CFA come with safety and productivity features to aid users. 
The features include the following.

\begin{itemize}
\item Toggle-able statistic collection on channel behvaiour that counts channel operations, and the number of the operations that block. 
Tracking blocking operations helps users tune their channel size or channel usage when the channel is used for buffering, where the aim is to have as few blocking operations as possible.
\item Deadlock detection on deallocation of the channel. 
If any threads are blocked inside the channel when it terminates it is detected and informs the user, as this would cause a deadlock.
\item A \code{flush} routine that delivers copies of an element to all waiting consumers, flushing the buffer. 
Programmers can use this to easily to broadcast data to multiple consumers. 
Additionally, the \code{flush} routine is more performant then looping around the \code{insert} operation since it can deliver the elements without having to reaquire mutual exclusion for each element sent.
\end{itemize}

The other safety and productivity feature of \CFA channels deals with concurrent termination. 
Terminating concurrent programs is often one of the most difficult parts of writing concurrent code, particularly if graceful termination is needed. 
The difficulty of graceful termination often arises from the usage of synchronization primitives which need to be handled carefully during shutdown. 
It is easy to deadlock during termination if threads are left behind on synchronization primitives. 
Additionally, most synchronization primitives are prone to \gls{toctou} issues where there is race between one thread checking the state of a concurrent object and another thread changing the state. 
\gls{toctou} issues with synchronization primitives often involve a race between one thread checking the primitive for blocked threads and another thread blocking on it. 
Channels are a particularly hard synchronization primitive to terminate since both sending and receiving off a channel can block. 
Thus, improperly handled \gls{toctou} issues with channels often result in deadlocks as threads trying to perform the termination may end up unexpectedly blocking in their attempt to help other threads exit the system.

% C_TODO: add reference to select chapter, add citation to go channels info
Go channels provide a set of tools to help with concurrent shutdown. 
Channels in Go have a \code{close} operation and a \code{select} statement that both can be used to help threads terminate. 
The \code{select} statement will be discussed in \ref{}, where \CFA's \code{waituntil} statement will be compared with the Go \code{select} statement. 
The \code{close} operation on a channel in Go changes the state of the channel. 
When a channel is closed, sends to the channel will panic and additional calls to \code{close} will panic. 
Receives are handled differently where receivers will never block on a closed channel and will continue to remove elements from the channel. 
Once a channel is empty, receivers can continue to remove elements, but will receive the zero-value version of the element type. 
To aid in avoiding unwanted zero-value elements, Go provides the ability to iterate over a closed channel to remove the remaining elements. 
These design choices for Go channels enforce a specific interaction style with channels during termination, where careful thought is needed to ensure that additional \code{close} calls don't occur and that no sends occur after channels are closed. 
These design choices fit Go's paradigm of error management, where users are expected to explicitly check for errors, rather than letting errors occur and catching them. 
If errors need to occur in Go, return codes are used to pass error information where they are needed. 
Note that panics in Go can be caught, but it is not considered an idiomatic way to write Go programs.

While Go's channel closing semantics are powerful enough to perform any concurrent termination needed by a program, their lack of ease of use leaves much to be desired. 
Since both closing and sending panic, once a channel is closed, a user often has to synchronize the senders to a channel before the channel can be closed to avoid panics. 
However, in doing so it renders the \code{close} operation nearly useless, as the only utilities it provides are the ability to ensure that receivers no longer block on the channel, and will receive zero-valued elements. 
This can be useful if the zero-typed element is recognized as a sentinel value, but if another sentinel value is preferred, then \code{close} only provides its non-blocking feature. 
To avoid \gls{toctou} issues during shutdown, a busy wait with a \code{select} statement is often used to add or remove elements from a channel. 
Due to Go's asymmetric approach to channel shutdown, separate synchronization between producers and consumers of a channel has to occur during shutdown.

In \CFA, exception handling is an encouraged paradigm and has full language support \cite{Beach21}.
As such \CFA uses an exception based approach to channel shutdown that is symmetric for both producers and consumers, and supports graceful shutdown.Exceptions in \CFA support both termination and resumption.Termination exceptions operate in the same way as exceptions seen in many popular programming languages such as \CC, Python and Java.
Resumption exceptions are a style of exception that when caught run the corresponding catch block in the same way that termination exceptions do.
The difference between the exception handling mechanisms arises after the exception is handled. 
In termination handling, the control flow continues into the code following the catch after the exception is handled. 
In resumption handling, the control flow returns to the site of the \code{throw}, allowing the control to continue where it left off. 
Note that in resumption, since control can return to the point of error propagation, the stack is not unwound during resumption propagation. 
In \CFA if a resumption is not handled, it is reraised as a termination. 
This mechanism can be used to create a flexible and robust termination system for channels.

When a channel in \CFA is closed, all subsequent calls to the channel will throw a resumption exception at the caller. 
If the resumption is handled, then the caller will proceed to attempt to complete their operation. 
If the resumption is not handled it is then rethrown as a termination exception. 
Or, if the resumption is handled, but the subsequent attempt at an operation would block, a termination exception is thrown. 
These termination exceptions allow for non-local transfer that can be used to great effect to eagerly and gracefully shut down a thread. 
When a channel is closed, if there are any blocked producers or consumers inside the channel, they are woken up and also have a resumption thrown at them. 
The resumption exception, \code{channel_closed}, has a couple fields to aid in handling the exception. 
The exception contains a pointer to the channel it was thrown from, and a pointer to an element. 
In exceptions thrown from remove the element pointer will be null. 
In the case of insert the element pointer points to the element that the thread attempted to insert. 
This element pointer allows the handler to know which operation failed and also allows the element to not be lost on a failed insert since it can be moved elsewhere in the handler. 
Furthermore, due to \CFA's powerful exception system, this data can be used to choose handlers based which channel and operation failed. 
Exception handlers in \CFA have an optional predicate after the exception type which can be used to optionally trigger or skip handlers based on the content of an exception. 
It is worth mentioning that the approach of exceptions for termination may incur a larger performance cost during termination that the approach used in Go. 
This should not be an issue, since termination is rarely an fast-path of an application and ensuring that termination can be implemented correctly with ease is the aim of the exception approach.

To highlight the differences between \CFA's and Go's close semantics, an example program is presented. 
The program is a barrier implemented using two channels shown in Listings~\ref{l:cfa_chan_bar} and \ref{l:go_chan_bar}. 
Both of these exaples are implmented using \CFA syntax so that they can be easily compared. 
Listing~\ref{l:go_chan_bar} uses go-style channel close semantics and Listing~\ref{l:cfa_chan_bar} uses \CFA close semantics. 
In this problem it is infeasible to use the Go \code{close} call since all tasks are both potentially producers and consumers, causing panics on close to be unavoidable. 
As such in Listing~\ref{l:go_chan_bar} to implement a flush routine for the buffer, a sentinel value of $-1$ has to be used to indicate to threads that they need to leave the barrier. 
This sentinel value has to be checked at two points. 
Furthermore, an additional flag \code{done} is needed to communicate to threads once they have left the barrier that they are done. 
This use of an additional flag or communication method is common in Go channel shutdown code, since to avoid panics on a channel, the shutdown of a channel often has to be communicated with threads before it occurs. 
In the \CFA version~\ref{l:cfa_chan_bar}, the barrier shutdown results in an exception being thrown at threads operating on it, which informs the threads that they must terminate. 
This avoids the need to use a separate communication method other than the barrier, and avoids extra conditional checks on the fast path of the barrier implementation. 
Also note that in the Go version~\ref{l:go_chan_bar}, the size of the barrier channels has to be larger than in the \CFA version to ensure that the main thread does not block when attempting to clear the barrier.

\begin{cfa}[tabsize=3,caption={\CFA channel barrier termination},label={l:cfa_chan_bar}]
struct barrier {
    channel( int ) barWait;
    channel( int ) entryWait;
    int size;
}
void ?{}(barrier & this, int size) with(this) {
    barWait{size};
    entryWait{size};
    this.size = size;
    for ( j; size )
        insert( *entryWait, j );
}

void flush(barrier & this) with(this) {
    close(barWait);
    close(entryWait);
}
void wait(barrier & this) with(this) {
    int ticket = remove( *entryWait );
    if ( ticket == size - 1 ) {
        for ( j; size - 1 )
            insert( *barWait, j );
        return;
    }
    ticket = remove( *barWait );

    // last one out
    if ( size == 1 || ticket == size - 2 ) {
        for ( j; size )
            insert( *entryWait, j );
    }
}
barrier b{Tasks};

// thread main
void main(Task & this) {
    try {
        for ( ;; ) {
            wait( b );
        }
    } catch ( channel_closed * e ) {}
}

int main() {
    {
        Task t[Tasks];

        sleep(10`s);
        flush( b );
    } // wait for tasks to terminate
    return 0;
}
\end{cfa}

\begin{cfa}[tabsize=3,caption={Go channel barrier termination},label={l:go_chan_bar}]

struct barrier {
    channel( int ) barWait;
    channel( int ) entryWait;
    int size;
}
void ?{}(barrier & this, int size) with(this) {
    barWait{size + 1};
    entryWait{size + 1};
    this.size = size;
    for ( j; size )
        insert( *entryWait, j );
}

void flush(barrier & this) with(this) {
    insert( *entryWait, -1 );
    insert( *barWait, -1 );
}
void wait(barrier & this) with(this) {
    int ticket = remove( *entryWait );
    if ( ticket == -1 ) {
        insert( *entryWait, -1 );
        return;
    }
    if ( ticket == size - 1 ) {
        for ( j; size - 1 )
            insert( *barWait, j );
        return;
    }
    ticket = remove( *barWait );
    if ( ticket == -1 ) {
        insert( *barWait, -1 );
        return;
    }

    // last one out
    if ( size == 1 || ticket == size - 2 ) {
        for ( j; size )
            insert( *entryWait, j );
    }
}
barrier b;

bool done = false;
// thread main
void main(Task & this) {
    for ( ;; ) {
        if ( done ) break;
        wait( b );
    }
}

int main() {
    {
        Task t[Tasks];

        sleep(10`s);
        done = true;

        flush( b );
    } // wait for tasks to terminate
    return 0;
}
\end{cfa}

In Listing~\ref{l:cfa_resume} an example of channel closing with resumption is used. 
This program uses resumption in the \code{Consumer} thread main to ensure that all elements in the channel are removed before the consumer thread terminates. 
The producer only has a \code{catch} so the moment it receives an exception it terminates, whereas the consumer will continue to remove from the closed channel via handling resumptions until the buffer is empty, which then throws a termination exception. 
If the same program was implemented in Go it would require explicit synchronization with both producers and consumers by some mechanism outside the channel to ensure that all elements were removed before task termination.

\begin{cfa}[tabsize=3,caption={\CFA channel resumption usage},label={l:cfa_resume}]
channel( int ) chan{ 128 };

// Consumer thread main
void main(Consumer & this) {
    size_t runs = 0;
    try {
        for ( ;; ) {
            remove( chan );
        }
    } catchResume ( channel_closed * e ) {}
    catch ( channel_closed * e ) {} 
}

// Producer thread main
void main(Producer & this) {
    int j = 0;
    try {
        for ( ;;j++ ) {
            insert( chan, j );
        }
    } catch ( channel_closed * e ) {} 
}

int main( int argc, char * argv[] ) {
    {
        Consumers c[4];
        Producer p[4];

        sleep(10`s);

        for ( i; Channels )
            close( channels[i] );
    }
    return 0;
}
\end{cfa}

\section{Performance}

Given that the base implementation of the \CFA channels is very similar to the Go implementation, this section aims to show that the performance of the two implementations are comparable. 
One microbenchmark is conducted to compare Go and \CFA. 
The benchmark is a ten second experiment where producers and consumers operate on a channel in parallel and throughput is measured. 
The number of cores is varied to measure how throughtput scales. 
The cores are divided equally between producers and consumers, with one producer or consumer owning each core. 
The results of the benchmark are shown in Figure~\ref{f:chanPerf}. 
The performance of Go and \CFA channels on this microbenchmark is comparable. 
Note, it is expected for the performance to decline as the number of cores increases as the channel operations all occur in a critical section so an increase in cores results in higher contention with no increase in parallelism.


\begin{figure}
    \centering
    \begin{subfigure}{0.5\textwidth}
        \centering
        \scalebox{0.5}{\input{figures/nasus_Channel_Contention.pgf}}
        \subcaption{AMD \CFA Channel Benchmark}\label{f:chanAMD}
    \end{subfigure}\hfill
    \begin{subfigure}{0.5\textwidth}
        \centering
        \scalebox{0.5}{\input{figures/pyke_Channel_Contention.pgf}}
        \subcaption{Intel \CFA Channel Benchmark}\label{f:chanIntel}
    \end{subfigure}
    \caption{The channel contention benchmark comparing \CFA and Go channel throughput (higher is better).}
    \label{f:chanPerf}
\end{figure}