% ======================================================================
% ======================================================================
\chapter{Conclusion}\label{s:conclusion}
% ======================================================================
% ======================================================================

This thesis presented a suite of safe and efficient concurrency tools that provide users with the means to write scalable programs in \CFA through multiple avenues.
If users prefer the message passing paradigm of concurrency, \CFA now provides message passing tools in the form of a performant actor system and channels.
For shared memory concurrency, the mutex statement provides a safe and easy-to-use interface for mutual exclusion.
The @waituntil@ statement aids in writing concurrent programs in both the message passing and shared memory paradigms of concurrency.
Furthermore, no other language provides a synchronous multiplexing tool polymorphic over resources like \CFA's @waituntil@.

On overview of the contributions in this thesis include the following:
\begin{enumerate}
\item The mutex statement, which provides performant and deadlock-free multiple lock acquisition.
\item Channels with comparable performance to Go, that have safety and productivity features including deadlock detection, and an easy-to-use exception-based channel @close@ routine.
\item An in-memory actor system that achieved the lowest latency message send of systems tested due to the novel copy-queue data structure. The actor system presented has built-in detection of six common actor errors, and it has good performance compared to other systems on all benchmarks.
\item A @waituntil@ statement which tackles the hard problem of allowing a thread to safely synchronously wait for some set of concurrent resources.
\end{enumerate}

From the novel copy-queue data structure in the actor system and the plethora of user-supporting safety features, all these utilities build upon existing concurrent tooling with value added.
Performance results verify that each new feature is comparable or better than similar features in other programming languages.
All in all, this suite of concurrent tools expands users' ability to easily write safe and performant multi-threaded programs in \CFA.

\section{Future Work}

\subsection{Further Implicit Concurrency}

This thesis only scratches the surface of implicit concurrency by providing an actor system.
There is room for more implicit concurrency tools in \CFA.
User-defined implicit concurrency in the form of annotated loops or recursive concurrent functions exists in many other languages and libraries~\cite{uC++,OpenMP}.
Similar implicit concurrency mechanisms could be implemented and expanded on in \CFA.
Additionally, the problem of automatic parallelism of sequential programs via the compiler is an interesting research space that other languages have approached~\cite{wilson94,haskell:parallel} and could be explored in \CFA.

\subsection{Advanced Actor Stealing Heuristics}

In this thesis, two basic victim-selection heuristics are chosen when implementing the work stealing actor system.
Good victim selection is an active area of work stealing research, especially when taking into account NUMA effects and cache locality~\cite{barghi18,wolke17}.
The actor system in \CFA is modular and exploration of other victim-selection heuristics for queue stealing in \CFA could be provided by pluggable modules.
Another question in work stealing is: when should a worker thread steal?
Work stealing systems can often be too aggressive when stealing, causing the cost of the steal to be have a negative rather positive effect on performance.
Given that thief threads often have cycles to spare, there is room for a more nuanced approaches when stealing.
Finally, there is the very difficult problem of blocking and unblocking idle threads for workloads with extreme oscillations in CPU needs.

\subsection{Synchronously Multiplexing System Calls}

There are many tools that try to synchronously wait for or asynchronously check I/O, since improvements in this area pay dividends in many areas of computer science~\cite{linux:select,linux:poll,linux:epoll,linux:iouring}.
Research on improving user-space tools to synchronize over I/O and other system calls is an interesting area to explore in the world of concurrent tooling.
Specifically, incorporating I/O into the @waituntil@ to allow a network server to work with multiple kinds of asynchronous I/O interconnects without using tradition event loops.

\subsection{Better Atomic Operations}

When writing low-level concurrent programs, especially lock/wait-free programs, low-level atomic instructions need to be used.
In C, the gcc-builtin atomics~\cite{gcc:atomics} are commonly used, but leave much to be desired. 
Some of the problems include the following.
Archaic and opaque macros often have to be used to ensure that atomic assembly is generated instead of locks.
The builtins are polymorphic, but not type safe since they use void pointers.
The semantics and safety of these builtins require careful navigation since they require the user to have a deep understanding of concurrent memory-ordering models.
Furthermore, these atomics also often require a user to understand how to fence appropriately to ensure correctness.
All these problems and more could benefit from language support in \CFA.
Adding good language support for atomics is a difficult problem, which if solved well, would allow for easier and safer writing of low-level concurrent code.