Index: doc/proposals/exceptions-assert.md
===================================================================
--- doc/proposals/exceptions-assert.md	(revision 2ec81ca5d342fd71e6a860a7a29ad59792257970)
+++ doc/proposals/exceptions-assert.md	(revision 63917abc8d84f43ab52f2b99b5f7c3774273a32b)
@@ -61,58 +61,327 @@
 throws no exceptions, that means all throws will be matched to a try.
 
-Syntax and Usage
-----------------
-The throws annotations is new a syntax. The design is not final, but for
-examples we will use: `throws (EXCEPTIONS)`, where EXCEPTIONS is a comma
-separated list of exception signatures. This can happen within the scope of a
-forall clause so it can be polymorphic with the rest of the function.
-
-There is no syntax to declare exceptions, although one could be added, they
-are effectively declared in the throws annotations.
-
-Throw statements remain, but now they are joined by throw expressions. The
-throw expressions return their response value. Throw statements can
-(Throw expressions with a response of `void` can be a void expression, in the
-few places where a void expression is allowed.)
-```
-throw Name(Exprs,...)
-```
-Along with the addition of the expression form, the main change is that the
-exception is not an expression, but a literal name and a series of
-expressions.
-
-Try statements are the same as the current design except for the catch clauses.
-```
-catch Name(Arguments...) { ... }
-```
-Exceptions thrown in the try block are checked for matching handlers. The
-match is simply preformed by label name. (Although with type annotations on
-the arguments or for the return type, could be extended to involve full
-resolution on the signature of the exception.) When an exception is thrown
-the matching catch is put onto the stack and run. Running off the end, or
-otherwise exiting the catch block, causes a termination and unwinds the
-stack.
-
-In addition there is a new statement that can only be used inside a catch
-statement (and only in one that can return to the throw).
-```
-resume [EXPR];
-```
-This is like a return statement, the expression is required if the exception
-expects a value in response. If the expression is present, it is evaluated
-before the resume occurs. When the resume occurs, we exit the handler and the
-throw finishes execution, resulting in the evaluated value as appropriate.
-
-### Exception Signature Polymorphism
-We have everything to make a functional exception handling mechanism.
-However, to get a flexible (especially in regards to higher order functions)
-system we need a basic level of polymorphism.
-
+Features
+--------
+The detailed design of the proposal organized by feature.
+There are core features (exception signatures, the throw expression and
+try statement) followed by extra features. The core features form the minimal
+working version of this system and should be understood together.
+
+All syntax given is a mock-up and can be changed.
+
+### Exception Signatures
+Exceptions exist as assertion-like addition to a function signature.
+
+An exception signature describes an exception, and consists of a response
+type, a identifier as the name and the parenthesized list of comma separated
+parameter types.
+```
+RESPONSE_TYPE NAME(PARAMETER_TYPES)
+```
+Each exception signature notes an exception that can be thrown.
+The parameter types are the types of the values passed from the throw to
+the handler and the response type is the type of the value that might be
+passed from the handler back to the throw.
+
+These are put into the throws clause of a function,
+an optional part of the function signature.
+The function signature is just the keyword throws followed by a parenthesized
+and comma separated list of exception signatures.
+```
+throws(EXCEPTION_SIGNATURES)
+```
+By convention, this goes after the parameter list of the function,
+unless the function has a forall list in which case it goes after that.
+
+This doesn't actual do anything, it just annotes what exceptions can come
+from this function. Or rather propagate from call expressions that use it.
+Expressions and statements also - implicitly - have their own throws clause
+of the exceptions that can come from them.
+For most expressions and statements it is simply the combined set of
+exceptions of their components.
+
+The function's throws clause should match that of its body statement.
+Any missing exceptions is a compile time error.
+Any extra exceptions should be a warning,
+some notation to silence the warning would be provided.
+
+The throws clause does not participate in overload resolution. Once the
+overload is choosen, the throws clauses are checked and errors raised
+when there is a mismatch.
+
+Two exceptions are the same exception if their name, parameter types and
+response type are the same. It should follow the same pattern as function
+resolution.
+
+No Return Signature (iterator):
+Consider an iterator interface, that returns a value and updates the iterator
+in place to produce a value, and throws an exception to indicate the end of
+iteration.
+```
+forall(T, I)
+trait Iterator {
+    I next(T &) throws(noreturn end_of_iteration());
+}
+```
+
+Function Exception Separation (too-much-overloading):
+Exceptions don't exist in the same space as functions and variables.
+The places they can be used only accept exception names so there is no
+confusion.
+```
+int f(char) throws(int f(char));
+
+void g() throws(int f(char)) {
+    ...
+    int i = f('a');
+    ...
+}
+```
+This will always call the function as exceptions are in their own namespace
+and do not interact with function names, variable names or trait names.
+
+### Throw Expression
+Exceptions come from throw expressions.
+
+A throw expression is made of the keyword `throw`,
+the exception name and the parenthesized list of argument expressions.
+```
+throw NAME(ARGUMENTS)
+```
+When the throw expression is evaluated all the arguments are evaluated
+and then the exception is thrown.
+If the handler responds to it, that is the value the expression evaluates to.
+If an unwinding is triggered during the handling,
+the unwinding starts at this expression.
+
+The parameter types and response type are not explicitly given.
+Since the exception must be given in context, it can resolve the expression
+against the list of possible exceptions. Annotation casts can be used for
+disambiguation when needed, but it shouldn't be a common occurrence.
+
+Although throws are expressions they can, and in some cases should, be used
+at the top level as a statement.
++   Value Returning: These should only be used as expressions, it is why they
+    are expressions, so the response can be captured and used.
+    There should be a warning if the value is not captured.
++   Void Returning: This can be use either as an expression (in a place where
+    a void expression is allowed) or as a statement.
++   Non-Returning: As control will never return, they could be put anywhere
+    but should only be used at the statement level for clarity of control
+    flow.
+
+Value Throw Example (lookup):
+Here is a simple example of a throw that both passes a value and uses the
+response. If the lookup finds a value, it is returns it, otherwise it uses
+a throw to try and find a default value.
+```
+forall(K, V | relational(K)) throws(V NotFound(K &));
+V lookup(map(K, V) const & mapping, K & key) {
+    // Internal code equivilent too:
+    node(K, V) * node = lookup_internal(mapping, key);
+
+    return ((node) ? node->value : throw NotFound(key));
+}
+```
+
+No Return Throw Example (iterator):
+An iterator that uses an exception to note that the iterator is empty.
+```
+int next(Iter & iter) throws (noreturn end_of_iteration()) {
+    if (iter.size <= iter.position) {
+        throw end_of_iteration();
+    }
+    ++iter.position;
+    return iter.data[iter.position - 1];
+}
+```
+
+### Try Statement
+The exceptions are handled by try statements.
+
+The try statement is made up of the try keyword, the try block
+and then one or more catch clauses.
+```
+try {
+    BODY
+} catch NAME(ARGS) {
+    HANDLER
+}
+```
+When the try statement is executed, the BODY is evaluated and then control
+continues after the try statement. The try statement is the same as wrapping
+the BODY in a compound statement unless an exception is thrown while BODY
+is being executed.
+
+When an exception is thrown in the BODY (either directly or indirectly)
+that matches one of the catch clauses then that exception is handled by this
+try statement. This means the exception does not continue outwards from here.
+That binds to the values passed to the throw to the names in ARGS and then
+runs the HANDLER code.
+
+Within the HANDLER, you can also respond to the exception with a resume
+statement, which is passing to it the response value.
+```
+resume EXPR;
+```
+(We also allow just `resume;` for void responses.)
+You cannot respond to a noreturn statement, so you cannot use a resume
+statement inside a handler for a noreturn expression.
+
+Value Try Example (lookup):
+Although you would probably want to implement a separate lookup with default
+function, you could implement one as a wrapper:
+```
+forall(K, V | relational(K))
+V lookup(map(K, V) const & mapping, K & key, V & defaultValue) {
+  try {
+    return lookup(mapping, key);
+  } catch NotFound(_key) {
+    resume defaultValue;
+  }
+}
+```
+
+Void Resume/Terminate Example (Out of Memory):
+Here is an example of void resume (not passing back a value) and also a
+termination, where the handler can trigger unwinding.
+```
+try {
+  ...
+} catch NoMem(size_t target_size, Buffer * buff) {
+  if (void * new_buff = realloc(buff->data, target_size)) {
+    buff->data = new_buff;
+    resume;
+  }
+  // If the realloc fails, we go to a recovery path.
+}
+```
+Here, the error cannot be fixed with information, but rather a correction to
+the container state. This also shows how resumption is optional (for both
+void and value throws) and the handler can instead decide to terminate and
+unwind the stack.
+
+No Return Try Example (succ):
+There are few functions that are very hard to make complete, any could
+require an awkward check before calling it, or the function can report that
+with an exception.
+
+For example, a checked successor function for enumerations:
+`SomeEnum succ(SomeEnum) throws(noreturn out_of_range())`
+
+This can be used as a check on a loop:
+```
+int sum = 0;
+try {
+    SomeEnum e = minimum;
+    for () {
+        e = succ(e);
+        sum += transform(e);
+    }
+} catch OutOfRange() {
+    return sum;
+}
+```
+
+This pattern could also be used with iterators and be wrapped up in a
+control-flow structure for ease of use, like with Python's StopIteration.
+
+### Rethrow (Conditional Catch)
+Sometimes the decision if an exception can be handled does not come down
+to the location in code and the exception in question; which is how they
+base try statements decide it.
+
+This could be implemented one of two ways: a rethrow or conditional catch.
+The conditional catch is a bit more straightforward so we will start there.
+
+A conditional catch is part of a conditional clause, and adds an if keyword
+followed by a parenthesized boolean expression, the condition.
+```
+catch NAME(ARGS) if(COND) {
+    HANDLER
+}
+```
+When the exception matches a catch clause, after the arguments are bound
+to their names but before it commits to handling the exception,
+the condition is evaluated.
+If the condition evaluates to true the handler runs,
+otherwise the exception continues to propagate out and up.
+
+This is a change from the existing syntax `catch NAME(...; ...)` because
+the declaration section of the clause (before the `;`) now is a comma
+separated list, which could make it harder to see the `;` separator.
+That might be fine but it was enough to change my mock-up syntax.
+
+Conditional Catch Example (special key):
+Although the predicate can be as complex as it needs to be, using catch
+arguments, local varaibles and function calls, it is ultimately just a
+conditional expression:
+```
+catch NotFound(key) if (is_special_key(key)) {
+    resume make_special_value(key);
+}
+```
+
+The rethrow is an equivalent feature, but is organized differently, in a way
+that is more flexible but has the same structure. When the rethrow statement
+(a `throw` with no exception) is executed the handler stops executing and
+the exception propagation continues.
+```
+catch NAME(...) {
+    ...
+    throw;
+    ...
+}
+```
+Unlike in some other languages, a rethrow just continues the original throw
+and doesn't create a new throw with the same or a new exception.
+
+Rethrow Example (special key):
+```
+catch NotFound(key) {
+    if (is_special_key(key)) {
+        resume make_special_value(key);
+    }
+    throw;
+}
+```
+
+Rethrow Variable Example (metadata):
+The rethrow can weave in other code into the check more easily.
+```
+catch NotFound(key) {
+    KeyData meta = get_metadata(key);
+    if (is_set("ignore-default", meta)) {
+        throw;
+    }
+    resume get_string("default", meta);
+}
+```
+
+### Polymorphism
+Functions polymorphic over exceptions in throws signature.
+
+We extend forall clauses to include a new type of polymorphic parameters,
+that declare exception sets: `exception VAR`. The declared variable may be
+used in any throws clause, as you would include an exception in the list.
+
+For example, a higher order function that passes all of its exceptions
+through to its caller works like this:
 ```
 forall(T, [N], exception E) throws(E)
 void foreach(void op(T &) throws(E), array(T, N) & data) {
-  for (i; N) op(data[i]);
-}
-```
+    for (i; N) op(data[i]);
+}
+```
+
+It is important to note it is an exception set, not a single exception. If
+the inner function throws one or more exceptions, then foreach throws the
+same exceptions. If the inner function doesn't throw anything, then foreach
+doesn't throw anything and its caller doesn't have to handle anything.
+
+It should be noted that these exception sets to not interact with concrete
+exceptions within the polymorphic functions themselves. They are logically
+"packed" when they move from monomorphic to polymorphic code and "unpacked"
+when it moves back from polymorphic to monomorphic code.
 
 A new type of polymorphic parameter is added `exception` which is polymorphic
@@ -125,120 +394,8 @@
 throw and catch the exception, passing through the polymorphic section.
 
-Examples
---------
-
-### Noreturn Example (Enumeration succ)
-A simple example from the enumeration traits, there is a `succ_unsafe`
-function in the Serial trait that is wrapped by a `succ` function that adds
-some checks and aborts if they fail.
-
-Instead of aborting, it could have an assertion-like exception:
-```
-int succ(int value) throws(noreturn out_of_range()) {
-    if (value == MAX) {
-        throw out_of_range();
-    }
-    return value + 1;
-}
-```
-This serves the same role as the existing function, but combines the succ
-with a range check the caller can respond to.
-
-For an example of catching this, consider creating a wrapper to get the
-current succeed or abort implementation:
-```
-forall(T | { T succ(T) throws(noreturn out_of_range(); })
-T succ_abort(T val) {
-  try {
-    return succ(val);
-  } catch out_of_range() {
-    abort("Attempt to succ to non-existant successor.");
-  }
-}
-```
-
-### Value Resume Example (lookup)
-For an example of catching and resuming with a value, consider a map lookup
-that throws when you attempt to lookup a key not in the map and a wrapper
-that provides a default value.
-```
-forall(K, V | relational(K)) throws(V NotFound(K &));
-V lookup(map(K, V) const & mapping, K & key) {
-  // Internal code equivilent too:
-  node(K, V) * node = lookup_internal(mapping, key);
-
-  return ((node) ? node->value : throw NotFound(key));
-}
-
-forall(K, V | relational(K))
-V lookup(map(K, V) const & mapping, K & key, V & defaultValue) {
-  try {
-    return lookup(mapping, key);
-  } catch NoFound(_key) {
-    resume defaultValue;
-  }
-}
-```
-
-This may not be the most efficient way to implement a lookup-with-default
-function, but it would work. Practically, one would handle the NoFound
-directly, even running more complex code or using information not passed
-to the lookup, as showed here:
-
-```
-// local_var is declared up here.
-try {
-  function_that_calls_lookup(args);
-} catch NoFound(key) {
-  if (is_special_key(key)) {
-    resume build_default_value(key, local_var);
-  } else {
-    resume simple_default_value;
-  }
-}
-```
-
-### Void Resume/Terminate Example (Out of Memory)
-The third type of resumption is a void return, which you may resume but do
-not pass any data back to the throw.
-
-```
-try {
-  ...
-} catch NoMem(size_t target_size, Buffer * buff) {
-  if (void * new_buff = realloc(buff->data, target_size)) {
-    buff->data = new_buff;
-    resume;
-  }
-}
-```
-
-Here, the error cannot be fixed with information, but rather a correction to
-the container state. This also shows how resumption is optional (for both
-void and value throws) and the handler can instead decide to terminate and
-unwind the stack.
-
-### Polymorphism Example (Save/Restore)
-Higher order functions can handle types simply need to polymorphic on the
-same exception type as the functions they take as parameters.
-```
-forall(T, exception E)
-T isolate_state(T (*func)() throws (E), State state) {
-  // SavedState wraps a State and restores it on deconstruction.
-  SavedState _saved = save_state();
-  set_state(state);
-  return func();
-}
-```
-This isolates a function by saving and restoring state around the function,
-where that state is some generic global state. The wrapped function can raise
-any of those exceptions and they will pass through the isolateState. When
-one of those exceptions terminate or func returns, _saved's destructor runs.
-
-### Polymorphic Error Example (foreach)
+Polymorphic Error Example (foreach):
 Here is a simple example of polymorphism over a higher order function.
-
-```
-void print_widget(Widget &) throws (noreturn FormatError());
+```
+void print_widget(Widget &) throws (void FormatError(Widget &));
 
 forall(T, [N], exception E) throws(E)
@@ -247,38 +404,152 @@
 }
 
+// Incorrect wrapping:
+forall([N])
+void print_widgets_ERROR(array(Widget, N) & widgets) {
+    // FormatError not handled.
+    foreach(print_widget, widgets);
+}
+
 forall([N])
 void print_widgets(array(Widget, N) & widgets) {
-  // FormatError not handled.
-  foreach(print_widget, widgets);
-}
-```
-It is infact too simple as foreach will have the same exception signature
-as print_widget and that means print_widgets is not handling the function
-nor passing it to its caller.
-
-However, this example (or a version that catches FormatError) will work
-because the exception signatures all match.
-```
+    try {
+        foreach(print_widget, widgets);
+    } catch FormatError(Widget &) {
+        sout | "Opaque Widget";
+        resume;
+    }
+}
+```
+
+### Exception Tunnelling
+Exceptions can pass through functions they should not interact with.
+
+(Credit to Mike for the original idea.)
+
+Higher order functions that take functions as parameters can "erase"
+exceptions from the throws list (including all of them).
+If it does so those exceptions can no longer be handled by that function or
+any of its callees. If the passed function raises any exceptions, they
+skip forward and appear from the call expression where the "erasing" happens.
+
+Here is an overview of some interactions:
+```
+int high(int op(int, int) throws(A, C)) throws(D);
+int passed(int, int) throws(A, B);
+
+int res = high(passed);
+```
++   The exception(s) A are not thrown from this expression.
+    Thrown by passed and handled by high. Either by being caught by high (or
+    some other function between it and the calling of passed) or by also
+    being part of D.
++   The exception(s) B are thrown from this expression.
+    These are the tunnelled exceptions that pass from the call site of passed
+    to this call site of high.
++   The exception(s) C are not thrown from this expression.
+    This is just to show that throwing extra exceptions is fine.
++   The exception(s) D are thrown from this expression.
+    No interaction with exception tunnelling, they come from high.
+
+Tunnelling Error Example (foreach):
+Here is an example of an exception tunnelling though a higher order function.
+```
+void print_widget(Widget &) throws (void FormatError(Widget &));
+
+forall(T, [N])
+void foreach(void op(T &), array(T, N) & data) {
+  for (i; N) op(data[i]);
+}
+
+// Incorrect wrapping:
 forall([N])
-void print_widgets(array(Wiget, N) & widgets) throws (noreturn FormatError()) {
-  foreach(print_widget, widgets);
-}
-```
+void print_widgets_ERROR(array(Widget, N) & widgets) {
+    // FormatError not handled.
+    foreach(print_widget, widgets);
+}
+
+forall([N])
+void print_widgets(array(Widget, N) & widgets) {
+    try {
+        foreach(print_widget, widgets);
+    } catch FormatError(Widget &) {
+        sout | "Opaque Widget";
+        resume;
+    }
+}
+```
+
+Tunnelling Thunk Expansion (sort):
+Although the implementation is not set,
+the behaviour of tunnelling can also be described by creating a thunk.
+
+Consider the following example were a sort is called with a less than
+operation that throws in some way the sort did not expect.
+```
+forall(T, [N] | { bool ?<?(T &, T &); })
+void sort(array(T, N) & data);
+
+struct SomeType { ... };
+
+bool ?<?(SomeType & lhs, SomeType & rhs) throw(RType except(AType)) {
+    ...
+}
+
+void example_function() {
+    ...
+    sort(data);
+    ...
+}
+```
+
+Internally this creates an thunk that contains some special code, probably
+in the form of some special annotations, but we can represent most of it
+with a try statement.
+```
+void example_function() {
+    ...
+    RaisePoint: bool ?<?'thunk(SomeType & lhs, SomeType & rhs) {
+        try {
+            return ?<?(lhs, rhs);
+        } catch except(AType data) {
+            // Captures the frame and code location.
+            RType value = magic_resume_from RaisePoint except(data);
+            resume value;
+        }
+    }
+    sort(data);
+    ...
+}
+```
+
+Thunks do lead to issues in life-time management and contribute to the general
+thunk problem. (Thunks may also be created when we pass a function to a
+function pointer that throws more exceptions, even though there should be
+no change in behaviour.)
+
+### Finally Clauses
+You could add `finally` clauses to try statements to run when they unwind.
+
+This is a classic tool used to ensure proper scoping of resources, a dual to
+destructors which are a OO-ish (not true OO) way of handling the problem.
+We can bring the syntax over to the new system:
+```
+try
+    TRY_BLOCK
+finally
+    FINALLY_BLOCK
+```
+
+However, although this does fire with the same timing, when the try statement
+is removed from the stack, this does interact with the change in unwind
+timing in a perhaps unexpected way. It will run after the exception passes
+it and the handler decides to unwind.
+
+A separate construct would be needed if you wanted to run code when the
+search leaves that section of the stack, because then you might need a paired
+construct to run code when you re-enter it.
 
 Details And Additional Notes
 ----------------------------
-
-### Rethrows and Conditional Catch
-As extension to the core system, we could add rethrowing or conditional
-catching to try blocks. There purpose is the same, to allow a handler to
-use non-type information to decide if it can handle an exception or not.
-
-If an exception is rethrown or the condition evaluates to false, exception
-propogation continues up the stack. As far as control flow goes this should
-be the same as not catching it at all.
-
-A rethrowing or conditional catch does not remove the exception it could
-catch and handle from the unhandled exception set. Because it may not, so
-the type system has to account for that.
 
 ### Performance (Where to Pay Costs)
