Index: doc/theses/andrew_beach_MMath/features.tex
===================================================================
--- doc/theses/andrew_beach_MMath/features.tex	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ doc/theses/andrew_beach_MMath/features.tex	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -2,29 +2,71 @@
 
 This chapter covers the design and user interface of the \CFA
-exception-handling mechanism.
-
-\section{Virtuals}
-Virtual types and casts are not part of the exception system nor are they
-required for an exception system. But an object-oriented style hierarchy is a
-great way of organizing exceptions so a minimal virtual system has been added
-to \CFA.
-
-The pattern of a simple hierarchy was borrowed from object-oriented
-programming was chosen for several reasons.
-The first is that it allows new exceptions to be added in user code
-and in libraries independently of each other. Another is it allows for
-different levels of exception grouping (all exceptions, all IO exceptions or
-a particular IO exception). Also it also provides a simple way of passing
-data back and forth across the throw.
-
-Virtual types and casts are not required for a basic exception-system but are
-useful for advanced exception features. However, \CFA is not object-oriented so
-there is no obvious concept of virtuals. Hence, to create advanced exception
-features for this work, I needed to design and implement a virtual-like
-system for \CFA.
-
-% NOTE: Maybe we should but less of the rational here.
-Object-oriented languages often organized exceptions into a simple hierarchy,
-\eg Java.
+exception-handling mechanism (EHM). % or exception system.
+
+% We should cover what is an exception handling mechanism and what is an
+% exception before this. Probably in the introduction. Some of this could
+% move there.
+\paragraph{Raise / Handle}
+An exception operation has two main parts: raise and handle.
+These are the two parts that the user will write themselves and so
+might be the only two pieces of the EHM that have any syntax.
+These terms are sometimes also known as throw and catch but this work uses
+throw/catch as a particular kind of raise/handle.
+
+\subparagraph{Raise}
+The raise is the starting point for exception handling and usually how
+Some well known examples include the throw statements of \Cpp and Java and
+the raise statement from Python.
+
+For this overview a raise does nothing more kick off the handling of an
+exception, which is called raising the exception. This is inexact but close
+enough for the broad strokes of the overview.
+
+\subparagraph{Handle}
+The purpose of most exception operations is to run some sort of handler that
+contains user code.
+The try statement of \Cpp illistrates the common features
+Handlers have three common features: a region of code they apply to, an
+exception label that describes what exceptions they handle and code to run
+when they handle an exception.
+Each handler can handle exceptions raised in that region that match their
+exception label. Different EHMs will have different rules to pick a handler
+if multipe handlers could be used such as ``best match" or ``first found".
+
+\paragraph{Propagation}
+After an exception is raised comes what is usually the biggest step for the
+EHM, finding and setting up the handler. This can be broken up into three
+different tasks: searching for a handler, matching against the handler and
+installing the handler.
+
+First the EHM must search for possible handlers that could be used to handle
+the exception. Searching is usually independent of the exception that was
+thrown and instead depends on the call stack, the current function, its caller
+and repeating down the stack.
+
+Second it much match the exception with each handler to see which one is the
+best match and hence which one should be used to handle the exception.
+In languages where the best match is the first match these two are often
+intertwined, a match check is preformed immediately after the search finds
+a possible handler.
+
+Third, after a handler is chosen it must be made ready to run.
+What this actually involves can vary widely to fit with the rest of the
+design of the EHM. The installation step might be trivial or it could be
+the most expensive step in handling an exception. The latter tends to be the
+case when stack unwinding is involved.
+
+As an alternate third step if no appropriate handler is found then some sort
+of recovery has to be preformed. This is only required with unchecked
+exceptions as checked exceptions can promise that a handler is found. It also
+is also installing a handler but it is a special default that may be
+installed differently.
+
+\subparagraph{Hierarchy}
+In \CFA the EHM uses a hierarchial system to organise its exceptions.
+This stratagy is borrowed from object-orientated languages where the
+exception hierarchy is a natural extension of the object hierarchy.
+
+Consider the following hierarchy of exceptions:
 \begin{center}
 \setlength{\unitlength}{4000sp}%
@@ -43,33 +85,86 @@
 \end{picture}%
 \end{center}
-The hierarchy provides the ability to handle an exception at different degrees
-of specificity (left to right). Hence, it is possible to catch a more general
-exception-type in higher-level code where the implementation details are
-unknown, which reduces tight coupling to the lower-level implementation.
-Otherwise, low-level code changes require higher-level code changes, \eg,
-changing from raising @underflow@ to @overflow@ at the low level means changing
-the matching catch at the high level versus catching the general @arithmetic@
-exception. In detail, each virtual type may have a parent and can have any
-number of children. A type's descendants are its children and its children's
-descendants. A type may not be its own descendant.
-
-The exception hierarchy allows a handler (@catch@ clause) to match multiple
-exceptions, \eg a base-type handler catches both base and derived
-exception-types.
-\begin{cfa}
-try {
-	...
-} catch(arithmetic &) {
-	... // handle arithmetic, underflow, overflow, zerodivide
-}
-\end{cfa}
-Most exception mechanisms perform a linear search of the handlers and select
-the first matching handler, so the order of handers is now important because
-matching is many to one.
-
-Each virtual type needs an associated virtual table. A virtual table is a
-structure with fields for all the virtual members of a type. A virtual type has
-all the virtual members of its parent and can add more. It may also update the
-values of the virtual members and often does.
+
+A handler labelled with any given exception can handle exceptions of that
+type or any child type of that exception. The root of the exception hierarchy
+(here \texttt{exception}) acts as a catch-all, leaf types catch single types
+and the exceptions in the middle can be used to catch different groups of
+related exceptions.
+
+This system has some notable advantages, such as multiple levels of grouping,
+the ability for libraries to add new exception types and the isolation
+between different sub-hierarchies. So the design was adapted for a
+non-object-orientated language.
+
+% Could I cite the rational for the Python IO exception rework?
+
+\paragraph{Completion}
+After the handler has finished the entire exception operation has to complete
+and continue executing somewhere else. This step is usually very simple
+both logically and in its implementation as the installation of the handler
+usually does the heavy lifting.
+
+The EHM can return control to many different places.
+However, the most common is after the handler definition and the next most
+common is after the raise.
+
+\paragraph{Communication}
+For effective exception handling, additional information is usually required
+as this base model only communicates the exception's identity. Common
+additional methods of communication are putting fields on an exception and
+allowing a handler to access the lexical scope it is defined in (usually
+a function's local variables).
+
+\paragraph{Other Features}
+Any given exception handling mechanism is free at add other features on top
+of this. This is an overview of the base that all EHMs use but it is not an
+exaustive list of everything an EHM can do.
+
+\section{Virtuals}
+Virtual types and casts are not part of the exception system nor are they
+required for an exception system. But an object-oriented style hierarchy is a
+great way of organizing exceptions so a minimal virtual system has been added
+to \CFA.
+
+The virtual system supports multiple ``trees" of types. Each tree is
+a simple hierarchy with a single root type. Each type in a tree has exactly
+one parent - except for the root type which has zero parents - and any
+number of children.
+Any type that belongs to any of these trees is called a virtual type.
+
+% A type's ancestors are its parent and its parent's ancestors.
+% The root type has no ancestors.
+% A type's decendents are its children and its children's decendents.
+
+Every virtual type also has a list of virtual members. Children inherit
+their parent's list of virtual members but may add new members to it.
+It is important to note that these are virtual members, not virtual methods.
+However as function pointers are allowed they can be used to mimic virtual
+methods as well.
+
+The unique id for the virtual type and all the virtual members are combined
+into a virtual table type. Each virtual type has a pointer to a virtual table
+as a hidden field.
+
+Up until this point the virtual system is a lot like ones found in object-
+orientated languages but this where they diverge. Objects encapsulate a
+single set of behaviours in each type, universally across the entire program,
+and indeed all programs that use that type definition. In this sense the
+types are ``closed" and cannot be altered.
+
+However in \CFA types do not encapsulate any behaviour. Traits are local and
+types can begin to statify a trait, stop satifying a trait or satify the same
+trait in a different way with each new definition. In this sense they are
+``open" as they can change at any time. This means it is implossible to pick
+a single set of functions that repersent the type.
+
+So we don't try to have a single value. The user can define virtual tables
+which are filled in at their declaration and given a name. Anywhere you can
+see that name you can use that virtual table; even if it is defined locally
+inside a function, although in that case you must respect its lifetime.
+
+An object of a virtual type is ``bound" to a virtual table instance which
+sets the virtual members for that object. The virtual members can be accessed
+through the object.
 
 While much of the virtual infrastructure is created, it is currently only used
@@ -83,5 +178,5 @@
 \Cpp syntax for special casts. Both the type of @EXPRESSION@ and @TYPE@ must be
 a pointer to a virtual type.
-The cast dynamically checks if the @EXPRESSION@ type is the same or a subtype
+The cast dynamically checks if the @EXPRESSION@ type is the same or a sub-type
 of @TYPE@, and if true, returns a pointer to the
 @EXPRESSION@ object, otherwise it returns @0p@ (null pointer).
@@ -107,5 +202,5 @@
 
 The function @get_exception_vtable@ is actually a constant function.
-Recardless of the value passed in (including the null pointer) it should
+Regardless of the value passed in (including the null pointer) it should
 return a reference to the virtual table instance for that type.
 The reason it is a function instead of a constant is that it make type
@@ -139,5 +234,5 @@
 and their use will be detailed there.
 
-However all three of these traits can be trickly to use directly.
+However all three of these traits can be tricky to use directly.
 There is a bit of repetition required but
 the largest issue is that the virtual table type is mangled and not in a user
@@ -152,5 +247,5 @@
 list will be passed to both types.
 In the current set-up the base name and the polymorphic arguments have to
-match so these macros can be used without losing flexability.
+match so these macros can be used without losing flexibility.
 
 For example consider a function that is polymorphic over types that have a
@@ -185,5 +280,5 @@
 It is dynamic, non-local goto. If a throw is successful then the stack will
 be unwound and control will (usually) continue in a different function on
-the call stack. They are commonly used when an error has occured and recovery
+the call stack. They are commonly used when an error has occurred and recovery
 is impossible in the current function.
 
@@ -196,5 +291,5 @@
 \end{cfa}
 The expression must return a reference to a termination exception, where the
-termination exception is any type that satifies @is_termination_exception@
+termination exception is any type that satisfies @is_termination_exception@
 at the call site.
 Through \CFA's trait system the functions in the traits are passed into the
@@ -203,5 +298,5 @@
 
 The throw will copy the provided exception into managed memory. It is the
-user's responcibility to ensure the original exception is cleaned up if the
+user's responsibility to ensure the original exception is cleaned up if the
 stack is unwound (allocating it on the stack should be sufficient).
 
@@ -220,5 +315,5 @@
 }
 \end{cfa}
-When viewed on its own a try statement will simply exceute the statements in
+When viewed on its own a try statement will simply execute the statements in
 @GUARDED_BLOCK@ and when those are finished the try statement finishes.
 
@@ -232,5 +327,5 @@
 Exception matching checks the representation of the thrown exception-type is
 the same or a descendant type of the exception types in the handler clauses. If
-it is the same of a descendent of @EXCEPTION_TYPE@$_i$ then @NAME@$_i$ is
+it is the same of a descendant of @EXCEPTION_TYPE@$_i$ then @NAME@$_i$ is
 bound to a pointer to the exception and the statements in @HANDLER_BLOCK@$_i$
 are executed. If control reaches the end of the handler, the exception is
@@ -255,5 +350,5 @@
 closure will be taken from up the stack and executed, after which the throwing
 function will continue executing.
-These are most often used when an error occured and if the error is repaired
+These are most often used when an error occurred and if the error is repaired
 then the function can continue.
 
@@ -263,9 +358,9 @@
 \end{cfa}
 The semantics of the @throwResume@ statement are like the @throw@, but the
-expression has return a reference a type that satifies the trait
+expression has return a reference a type that satisfies the trait
 @is_resumption_exception@. The assertions from this trait are available to
 the exception system while handling the exception.
 
-At runtime, no copies are made. As the stack is not unwound the exception and
+At run-time, no copies are made. As the stack is not unwound the exception and
 any values on the stack will remain in scope while the resumption is handled.
 
@@ -331,5 +426,5 @@
 search and match the handler in the @catchResume@ clause. This will be
 call and placed on the stack on top of the try-block. The second throw then
-throws and will seach the same try block and put call another instance of the
+throws and will search the same try block and put call another instance of the
 same handler leading to an infinite loop.
 
@@ -337,6 +432,6 @@
 can form with multiple handlers and different exception types.
 
-To prevent all of these cases we mask sections of the stack, or equvilantly
-the try statements on the stack, so that the resumption seach skips over
+To prevent all of these cases we mask sections of the stack, or equivalently
+the try statements on the stack, so that the resumption search skips over
 them and continues with the next unmasked section of the stack.
 
@@ -373,5 +468,5 @@
 The symmetry with termination is why this pattern was picked. Other patterns,
 such as marking just the handlers that caught, also work but lack the
-symmetry whih means there is more to remember.
+symmetry which means there is more to remember.
 
 \section{Conditional Catch}
@@ -396,6 +491,6 @@
 }
 \end{cfa}
-Note, catching @IOFailure@, checking for @f1@ in the handler, and reraising the
-exception if not @f1@ is different because the reraise does not examine any of
+Note, catching @IOFailure@, checking for @f1@ in the handler, and re-raising the
+exception if not @f1@ is different because the re-raise does not examine any of
 remaining handlers in the current try statement.
 
@@ -448,5 +543,5 @@
 @return@ that causes control to leave the finally block. Other ways to leave
 the finally block, such as a long jump or termination are much harder to check,
-and at best requiring additional run-time overhead, and so are mearly
+and at best requiring additional run-time overhead, and so are mealy
 discouraged.
 
@@ -506,5 +601,5 @@
 this point.}
 
-The recommended way to avoid the abort is to handle the intial resumption
+The recommended way to avoid the abort is to handle the initial resumption
 from the implicate join. If required you may put an explicate join inside a
 finally clause to disable the check and use the local
Index: libcfa/src/concurrency/clib/cfathread.cfa
===================================================================
--- libcfa/src/concurrency/clib/cfathread.cfa	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ libcfa/src/concurrency/clib/cfathread.cfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -58,5 +58,5 @@
 	this.themain = themain;
 	this.arg = arg;
-	((thread&)this){"C-thread", cl};
+	(this.self){"C-thread", cl};
 	__thrd_start(this, main);
 }
@@ -102,5 +102,5 @@
 	this.init = init;
 	this.arg = arg;
-	((thread&)this){"Processir Init"};
+	(this.self){"Processir Init"};
 
 	// Don't use __thrd_start! just prep the context manually
@@ -312,5 +312,6 @@
 
 	ssize_t cfathread_write(int fildes, const void *buf, size_t nbyte) {
-		return cfa_write(fildes, buf, nbyte, CFA_IO_LAZY);
+		// Use send rather then write for socket since it's faster
+		return cfa_send(fildes, buf, nbyte, 0, CFA_IO_LAZY);
 	}
 
@@ -335,6 +336,7 @@
 
 	ssize_t cfathread_read(int fildes, void *buf, size_t nbyte) {
-		return cfa_read(fildes, buf, nbyte, CFA_IO_LAZY);
-	}
-
-}
+		// Use recv rather then read for socket since it's faster
+		return cfa_recv(fildes, buf, nbyte, 0, CFA_IO_LAZY);
+	}
+
+}
Index: libcfa/src/concurrency/invoke.h
===================================================================
--- libcfa/src/concurrency/invoke.h	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ libcfa/src/concurrency/invoke.h	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -225,5 +225,5 @@
 
 		static inline $thread * volatile & ?`next ( $thread * this )  __attribute__((const)) {
-			return this->seqable.back;
+			return this->seqable.next;
 		}
 
Index: libcfa/src/concurrency/kernel.hfa
===================================================================
--- libcfa/src/concurrency/kernel.hfa	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ libcfa/src/concurrency/kernel.hfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -69,4 +69,7 @@
 	// Cluster from which to get threads
 	struct cluster * cltr;
+
+	// Id within the cluster
+	unsigned cltr_id;
 
 	// Set to true to notify the processor should terminate
Index: libcfa/src/concurrency/kernel/startup.cfa
===================================================================
--- libcfa/src/concurrency/kernel/startup.cfa	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ libcfa/src/concurrency/kernel/startup.cfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -486,5 +486,5 @@
 
 		// Adjust the ready queue size
-		ready_queue_grow( cltr, target );
+		this.cltr_id = ready_queue_grow( cltr, target );
 
 	// Unlock the RWlock
Index: libcfa/src/concurrency/kernel_private.hfa
===================================================================
--- libcfa/src/concurrency/kernel_private.hfa	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ libcfa/src/concurrency/kernel_private.hfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -278,5 +278,5 @@
 //-----------------------------------------------------------------------
 // Increase the width of the ready queue (number of lanes) by 4
-void ready_queue_grow  (struct cluster * cltr, int target);
+unsigned ready_queue_grow  (struct cluster * cltr, int target);
 
 //-----------------------------------------------------------------------
Index: libcfa/src/concurrency/locks.hfa
===================================================================
--- libcfa/src/concurrency/locks.hfa	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ libcfa/src/concurrency/locks.hfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -20,7 +20,122 @@
 
 #include "bits/weakso_locks.hfa"
+#include "containers/queueLockFree.hfa"
+
+#include "thread.hfa"
 
 #include "time_t.hfa"
 #include "time.hfa"
+
+//-----------------------------------------------------------------------------
+// Semaphores
+
+// '0-nary' semaphore
+// Similar to a counting semaphore except the value of one is never reached
+// as a consequence, a V() that would bring the value to 1 *spins* until
+// a P consumes it
+struct Semaphore0nary {
+	__spinlock_t lock; // needed to protect
+	mpsc_queue($thread) queue;
+};
+
+static inline bool P(Semaphore0nary & this, $thread * thrd) __attribute__((artificial));
+static inline bool P(Semaphore0nary & this, $thread * thrd) {
+	/* paranoid */ verify(!(thrd->seqable.next));
+	/* paranoid */ verify(!(thrd`next));
+
+	push(this.queue, thrd);
+	return true;
+}
+
+static inline bool P(Semaphore0nary & this) __attribute__((artificial));
+static inline bool P(Semaphore0nary & this) {
+    $thread * thrd = active_thread();
+    P(this, thrd);
+    park();
+    return true;
+}
+
+static inline $thread * V(Semaphore0nary & this, const bool doUnpark = true) __attribute__((artificial));
+static inline $thread * V(Semaphore0nary & this, const bool doUnpark = true) {
+	$thread * next;
+	lock(this.lock __cfaabi_dbg_ctx2);
+		for (;;) {
+			next = pop(this.queue);
+			if (next) break;
+			Pause();
+		}
+	unlock(this.lock);
+
+	if (doUnpark) unpark(next);
+	return next;
+}
+
+// Wrapper used on top of any sempahore to avoid potential locking
+struct BinaryBenaphore {
+	volatile ssize_t counter;
+};
+
+static inline {
+	void ?{}(BinaryBenaphore & this) { this.counter = 0; }
+	void ?{}(BinaryBenaphore & this, zero_t) { this.counter = 0; }
+	void ?{}(BinaryBenaphore & this, one_t ) { this.counter = 1; }
+
+	// returns true if no blocking needed
+	bool P(BinaryBenaphore & this) { return __atomic_fetch_sub(&this.counter, 1, __ATOMIC_SEQ_CST) > 0; }
+	bool tryP(BinaryBenaphore & this) {
+		ssize_t c = this.counter;
+		return (c >= 1) && __atomic_compare_exchange_n(&this.counter, &c, c-1, false, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED);
+	}
+
+	// returns true if notify needed
+	bool V(BinaryBenaphore & this) {
+		ssize_t c = 0;
+		for () {
+			if (__atomic_compare_exchange_n(&this.counter, &c, c+1, false, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST)) {
+				if (c == 0) return true;
+				/* paranoid */ verify(c < 0);
+				return false;
+			} else {
+				if (c == 1) return true;
+				/* paranoid */ verify(c < 1);
+				Pause();
+			}
+		}
+	}
+}
+
+// Binary Semaphore based on the BinaryBenaphore on top of the 0-nary Semaphore
+struct ThreadBenaphore {
+	BinaryBenaphore ben;
+	Semaphore0nary  sem;
+};
+
+static inline void ?{}(ThreadBenaphore & this) {}
+static inline void ?{}(ThreadBenaphore & this, zero_t) { (this.ben){ 0 }; }
+static inline void ?{}(ThreadBenaphore & this, one_t ) { (this.ben){ 1 }; }
+
+static inline bool P(ThreadBenaphore & this)              { return /* P(this.ben) ? false : */ P(this.sem); }
+static inline bool P(ThreadBenaphore & this, $thread * t) { return /* P(this.ben) ? false : */ P(this.sem, t ); }
+static inline bool tryP(ThreadBenaphore & this)           { return tryP(this.ben); }
+static inline bool P(ThreadBenaphore & this, bool wait)   { return wait ? P(this) : tryP(this); }
+
+static inline $thread * V(ThreadBenaphore & this, const bool doUnpark = true) {
+	// if (V(this.ben)) return 0p;
+	return V(this.sem, doUnpark);
+}
+
+//-----------------------------------------------------------------------------
+// Semaphore
+struct semaphore {
+	__spinlock_t lock;
+	int count;
+	__queue_t($thread) waiting;
+};
+
+void  ?{}(semaphore & this, int count = 1);
+void ^?{}(semaphore & this);
+bool   P (semaphore & this);
+bool   V (semaphore & this);
+bool   V (semaphore & this, unsigned count);
 
 //----------
@@ -54,4 +169,88 @@
 static inline size_t get_recursion_count( owner_lock & this ) { return get_recursion_count( (blocking_lock &)this ); }
 
+struct fast_lock {
+	$thread * volatile owner;
+	ThreadBenaphore sem;
+};
+
+static inline bool $try_lock(fast_lock & this, $thread * thrd) {
+    $thread * exp = 0p;
+    return __atomic_compare_exchange_n(&this.owner, &exp, thrd, false, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED);
+}
+
+static inline void $lock(fast_lock & this, $thread * thrd) {
+	/* paranoid */verify(thrd != this.owner);
+
+	for (;;) {
+		if ($try_lock(this, thrd)) return;
+		P(this.sem, thrd);
+	}
+}
+
+static inline void lock( fast_lock & this ) {
+	$thread * thrd = active_thread();
+	/* paranoid */verify(thrd != this.owner);
+
+	for (;;) {
+		if ($try_lock(this, thrd)) return;
+		P(this.sem);
+	}
+}
+
+static inline void try_lock ( fast_lock & this ) {
+	$thread * thrd = active_thread();
+	/* paranoid */ verify(thrd != this.owner);
+	return $try_lock(this, thrd);
+}
+
+static inline void unlock( fast_lock & this ) {
+	$thread * thrd = active_thread();
+	/* paranoid */ verify(thrd == this.owner);
+	$thread * next = V(this.sem, false); // implicit fence
+	// open 'owner' only after fence
+	this.owner = 0p;
+
+	// Unpark the next person (can be 0p, unpark handles it)
+	unpark(next);
+}
+
+static inline void on_wait( fast_lock & this ) {
+	unlock(this);
+	#warning this is broken
+}
+
+static inline void on_notify( fast_lock & this, struct $thread * t ) {
+	$lock(this, t);
+	#warning this is broken
+}
+
+static inline void   set_recursion_count( fast_lock & this, size_t recursion ) {}
+static inline size_t get_recursion_count( fast_lock & this ) { return 0; }
+
+struct mcs_node {
+	mcs_node * volatile next;
+	single_sem sem;
+};
+
+static inline void ?{}(mcs_node & this) { this.next = 0p; }
+
+static inline mcs_node * volatile & ?`next ( mcs_node * node ) {
+	return node->next;
+}
+
+struct mcs_lock {
+	mcs_queue(mcs_node) queue;
+};
+
+static inline void lock(mcs_lock & l, mcs_node & n) {
+	if(push(l.queue, &n))
+		wait(n.sem);
+}
+
+static inline void unlock(mcs_lock & l, mcs_node & n) {
+	mcs_node * next = advance(l.queue, &n);
+	if(next) post(next->sem);
+}
+
 //-----------------------------------------------------------------------------
 // is_blocking_lock
@@ -121,16 +320,2 @@
 	bool wait( condition_variable(L) & this, L & l, uintptr_t info, Time time );
 }
-
-//-----------------------------------------------------------------------------
-// Semaphore
-struct semaphore {
-	__spinlock_t lock;
-	int count;
-	__queue_t($thread) waiting;
-};
-
-void  ?{}(semaphore & this, int count = 1);
-void ^?{}(semaphore & this);
-bool   P (semaphore & this);
-bool   V (semaphore & this);
-bool   V (semaphore & this, unsigned count);
Index: libcfa/src/concurrency/ready_queue.cfa
===================================================================
--- libcfa/src/concurrency/ready_queue.cfa	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ libcfa/src/concurrency/ready_queue.cfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -39,5 +39,5 @@
 #endif
 
-#define BIAS 16
+#define BIAS 4
 
 // returns the maximum number of processors the RWLock support
@@ -252,5 +252,5 @@
 		preferred =
 			//*
-			kernelTLS().this_processor ? kernelTLS().this_processor->id * 4 : -1;
+			kernelTLS().this_processor ? kernelTLS().this_processor->cltr_id : -1;
 			/*/
 			thrd->link.preferred * 4;
@@ -330,5 +330,5 @@
 	#if defined(BIAS)
 		// Don't bother trying locally too much
-		preferred = kernelTLS().this_processor->id * 4;
+		preferred = kernelTLS().this_processor->cltr_id;
 	#endif
 
@@ -352,8 +352,5 @@
 
 		#if !defined(__CFA_NO_STATISTICS__)
-			if(locali) {
-				__tls_stats()->ready.pick.pop.local++;
-			}
-			if(localj) {
+			if(locali && localj) {
 				__tls_stats()->ready.pick.pop.local++;
 			}
@@ -528,5 +525,8 @@
 
 // Grow the ready queue
-void ready_queue_grow  (struct cluster * cltr, int target) {
+unsigned ready_queue_grow(struct cluster * cltr, int target) {
+	unsigned preferred;
+	size_t ncount;
+
 	/* paranoid */ verify( ready_mutate_islocked() );
 	__cfadbg_print_safe(ready_queue, "Kernel : Growing ready queue\n");
@@ -543,5 +543,11 @@
 		// Find new count
 		// Make sure we always have atleast 1 list
-		size_t ncount = target >= 2 ? target * 4: 1;
+		if(target >= 2) {
+			ncount = target * 4;
+			preferred = ncount - 4;
+		} else {
+			ncount = 1;
+			preferred = 0;
+		}
 
 		// Allocate new array (uses realloc and memcpies the data)
@@ -578,4 +584,5 @@
 
 	/* paranoid */ verify( ready_mutate_islocked() );
+	return preferred;
 }
 
Index: libcfa/src/containers/queueLockFree.hfa
===================================================================
--- libcfa/src/containers/queueLockFree.hfa	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ libcfa/src/containers/queueLockFree.hfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -20,11 +20,12 @@
 		// Adds an element to the list
 		// Multi-Thread Safe, Lock-Free
-		T * push(mcs_queue(T) & this, T & elem) {
-			/* paranoid */ verify(!(&elem)`next);
+		T * push(mcs_queue(T) & this, T * elem) __attribute__((artificial));
+		T * push(mcs_queue(T) & this, T * elem) {
+			/* paranoid */ verify(!(elem`next));
 			// Race to add to the tail
-			T * prev = __atomic_exchange_n(&this.tail, &elem, __ATOMIC_SEQ_CST);
+			T * prev = __atomic_exchange_n(&this.tail, elem, __ATOMIC_SEQ_CST);
 			// If we aren't the first, we need to tell the person before us
 			// No need to
-			if (prev) prev`next = &elem;
+			if (prev) prev`next = elem;
 			return prev;
 		}
@@ -33,13 +34,19 @@
 		// Passing an element that is not the head is undefined behavior
 		// NOT Multi-Thread Safe, concurrent pushes are safe
-		T * advance(mcs_queue(T) & this, T & elem) {
-			T * expected = &elem;
+		T * advance(mcs_queue(T) & this, T * elem) __attribute__((artificial));
+		T * advance(mcs_queue(T) & this, T * elem) {
+			T * expected = elem;
 			// Check if this is already the last item
 			if (__atomic_compare_exchange_n(&this.tail, &expected, 0p, false, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST)) return 0p;
 
-			// If not wait for next item to show-up
-			// added by push
-			while (!(&elem)`next) Pause();
-			return (&elem)`next;
+			// If not wait for next item to show-up, filled by push
+			while (!(elem`next)) Pause();
+
+			// we need to return if the next link was empty
+			T * ret = elem`next;
+
+			// invalidate link to reset to initial state
+			elem`next = 0p;
+			return ret;
 		}
 	}
@@ -64,7 +71,8 @@
 		// Added a new element to the queue
 		// Multi-Thread Safe, Lock-Free
-		T * push(mpsc_queue(T) & this, T & elem) {
+		T * push(mpsc_queue(T) & this, T * elem) __attribute__((artificial));
+		T * push(mpsc_queue(T) & this, T * elem) {
 			T * prev = push((mcs_queue(T)&)this, elem);
-			if (!prev) this.head = &elem;
+			if (!prev) this.head = elem;
 			return prev;
 		}
@@ -74,4 +82,5 @@
 		// next is set to the new head of the queue
 		// NOT Multi-Thread Safe
+		T * pop(mpsc_queue(T) & this, T *& next) __attribute__((artificial));
 		T * pop(mpsc_queue(T) & this, T *& next) {
 			T * elem = this.head;
@@ -84,4 +93,7 @@
 				// force memory sync
 				__atomic_thread_fence(__ATOMIC_SEQ_CST);
+
+				// invalidate link to reset to initial state
+				elem`next = 0p;
 			}
 			// Otherwise, there might be a race where it only looks but someone is enqueuing
@@ -91,5 +103,5 @@
 				// after that point, it could overwrite the write in push
 				this.head = 0p;
-				next = advance((mcs_queue(T)&)this, (*elem));
+				next = advance((mcs_queue(T)&)this, elem);
 
 				// Only write to the head if there is a next element
@@ -98,7 +110,4 @@
 				if (next) this.head = next;
 			}
-
-			// invalidate link
-			elem`next = 0p;
 
 			// return removed element
Index: tests/Makefile.am
===================================================================
--- tests/Makefile.am	(revision 2644610515a94ce16f004c9137eada1d5d1badeb)
+++ tests/Makefile.am	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -75,9 +75,10 @@
 	pybin/tools.py \
 	long_tests.hfa \
-	.in/io.data \
 	io/.in/io.data \
+	io/.in/many_read.data \
 	avltree/avl.h \
 	avltree/avl-private.h \
 	concurrent/clib.c \
+	concurrent/clib_tls.c \
 	exceptions/with-threads.hfa \
 	exceptions/except-io.hfa
Index: tests/collections/.expect/atomic_mpsc.txt
===================================================================
--- tests/collections/.expect/atomic_mpsc.txt	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
+++ tests/collections/.expect/atomic_mpsc.txt	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -0,0 +1,3 @@
+Starting
+Done!
+Match!
Index: tests/collections/atomic_mpsc.cfa
===================================================================
--- tests/collections/atomic_mpsc.cfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
+++ tests/collections/atomic_mpsc.cfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -0,0 +1,88 @@
+#include <fstream.hfa>
+#include <queueLockFree.hfa>
+#include <thread.hfa>
+
+const unsigned int num_times = 50000;
+
+struct some_node {
+	some_node * volatile next;
+	single_sem sem;
+	unsigned value;
+};
+
+void ?{}(some_node & this) { this.next = 0p; }
+
+static inline some_node * volatile & ?`next ( some_node * node ) {
+	return node->next;
+}
+
+mpsc_queue(some_node) global_queue;
+
+void thrash() {
+	unsigned t[100];
+	for(i; 100) {
+		t[i] = 0xDEADBEEF;
+	}
+}
+
+thread Producer {
+	size_t sum;
+};
+void ?{}( Producer & this ) { this.sum = 0; }
+
+void wait(Producer & this) {
+	some_node node;
+	node.value = (unsigned)&this;
+	push(global_queue, &node);
+	wait(node.sem);
+	this.sum += node.value;
+}
+
+void main(Producer & this) {
+	for(num_times) {
+		thrash();
+		wait(this);
+		thrash();
+		yield(random(10));
+	}
+}
+
+thread Consumer {
+	size_t sum;
+};
+void ?{}(Consumer & this) { this.sum = 0; }
+void main(Consumer & this) {
+	LOOP: for() {
+		waitfor( ^?{} : this) {
+			break LOOP;
+		}
+		or else {}
+
+		some_node * node = pop(global_queue);
+		if(node) {
+			this.sum += node->value;
+			post(node->sem);
+		}
+		else yield();
+	}
+}
+
+int main() {
+	unsigned psum = 0;
+	unsigned csum = -32;
+	processor p[2];
+	sout | "Starting";
+	{
+		Consumer cons;
+		{
+			Producer prods[13];
+			for(i; 13) {
+				psum += join(prods[i]).sum;
+			}
+		}
+		csum = join(cons).sum;
+	}
+	sout | "Done!";
+	if(psum == csum) sout | "Match!";
+	else sout | "No Match!";
+}
Index: tests/concurrent/.expect/spinaphore.txt
===================================================================
--- tests/concurrent/.expect/spinaphore.txt	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
+++ tests/concurrent/.expect/spinaphore.txt	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -0,0 +1,3 @@
+Starting
+Done!
+Match!
Index: tests/concurrent/spinaphore.cfa
===================================================================
--- tests/concurrent/spinaphore.cfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
+++ tests/concurrent/spinaphore.cfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -0,0 +1,97 @@
+#include <fstream.hfa>
+#include <locks.hfa>
+#include <thread.hfa>
+
+enum { num_blockers = 17, num_unblockers = 13 };
+
+void thrash() {
+	unsigned t[100];
+	for(i; 100) {
+		t[i] = 0xDEADBEEF;
+	}
+}
+
+Semaphore0nary sem;
+
+const unsigned int num_blocks = 1625;
+
+thread Blocker {
+	size_t sum;
+};
+void main(Blocker & this);
+
+Blocker * from_thread($thread * t) {
+	Blocker & nullb = *(Blocker*)0p;
+	$thread & nullt = (thread&)nullb;
+	uintptr_t offset  = (uintptr_t)&nullt;
+	uintptr_t address = ((uintptr_t)t) - offset;
+	return (Blocker*)address;
+}
+
+void main(Blocker & this) {
+	$thread * me = active_thread();
+	Blocker * me1 = &this;
+	Blocker * me2 = from_thread(me);
+	if( me1 != me2 ) sout | "Bad casting!" | me1 | "vs" | me2;
+	this.sum = 0;
+	for(num_blocks) {
+		// sout | "b P" | me;
+		P(sem);
+		if(((thread&)this).seqable.next != 0p) sout | "Link not invalidated";
+		// sout | "a P" | me;
+	}
+}
+
+const unsigned int num_unblocks = 2125;
+
+thread Unblocker {
+	size_t sum;
+};
+
+void main(Unblocker & this) {
+	this.sum = 0;
+	unsigned me = (unsigned)&this;
+	for(num_unblocks) {
+		$thread * t = V(sem, false);
+		Blocker * b = from_thread(t);
+		b->sum += me;
+		this.sum += (unsigned)b;
+		unpark(t);
+		// sout | "a V" | t;
+		yield(random(10));
+	}
+}
+
+int main() {
+	size_t usum = 0;
+	size_t bsum = 0;
+
+	if((num_unblocks * num_unblockers) != (num_blocks * num_blockers)) sout | "Mismatched Operations: " | (num_unblocks * num_unblockers) | "vs" | (num_blocks * num_blockers);
+
+	sout | "Starting";
+	{
+		Blocker   blockers  [num_blockers  ];
+		Unblocker unblockers[num_unblockers];
+
+		for(i;num_blockers) {
+			for(num_blocks)
+				usum += (unsigned)&blockers[i];
+		}
+
+		for(i;num_unblockers) {
+			for(num_unblocks)
+				bsum += (unsigned)&unblockers[i];
+		}
+
+		for(i;num_blockers) {
+			bsum -= join(blockers[i]).sum;
+		}
+
+		for(i;num_unblockers) {
+			usum -= join(unblockers[i]).sum;
+		}
+	}
+	sout | "Done!";
+	if(bsum == 0 && usum == 0) sout | "Match!";
+	else sout | "No Match!" | usum | "/" | bsum;
+}
Index: tests/unified_locking/.expect/mcs.txt
===================================================================
--- tests/unified_locking/.expect/mcs.txt	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
+++ tests/unified_locking/.expect/mcs.txt	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -0,0 +1,3 @@
+Starting
+Done!
+Match!
Index: tests/unified_locking/mcs.cfa
===================================================================
--- tests/unified_locking/mcs.cfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
+++ tests/unified_locking/mcs.cfa	(revision 1da73971dbde341615a5a04d2b19b654d5757091)
@@ -0,0 +1,66 @@
+#include <fstream.hfa>
+#include <locks.hfa>
+#include <thread.hfa>
+
+const unsigned int num_times = 50000;
+
+struct MutexObj {
+	mcs_lock l;
+	$thread * id;
+	size_t sum;
+};
+
+MutexObj mo;
+
+void trash() {
+	unsigned t[100];
+	for(i; 100) {
+		t[i] = 0xDEADBEEF;
+	}
+}
+
+unsigned cs() {
+	$thread * me = active_thread();
+	unsigned value = (unsigned)me;
+	mcs_node n;
+	lock(mo.l, n);
+	{
+		size_t tsum = mo.sum;
+		mo.id = me;
+		yield(random(5));
+		if(mo.id != me) sout | "Intruder!";
+		mo.sum = tsum + value;
+	}
+	unlock(mo.l, n);
+	return value;
+}
+
+thread LockCheck {
+	size_t sum;
+};
+
+void main(LockCheck & this) {
+	this.sum = 0;
+	for(num_times) {
+		trash();
+		this.sum += cs();
+		trash();
+		yield(random(10));
+	}
+}
+
+int main() {
+	size_t sum = -32;
+	mo.sum = -32;
+	processor p[2];
+	sout | "Starting";
+	{
+		LockCheck checkers[13];
+		for(i;13) {
+			sum += join(checkers[i]).sum;
+		}
+	}
+	sout | "Done!";
+	if(sum == mo.sum) sout | "Match!";
+	else sout | "No Match!" | sum | "vs" | mo.sum;
+}
