Elaborating Statements

Next in our elaboration story is the elaboration of block statements into checked trees. We will do so by elaborating a small imperative language consisting of: Boolean values, a local stack, ‘global’ heap, and conditionals

Concrete & Abstract Syntax Trees

We begin by establishing the concrete and abstract syntax trees we will be using.

Here are the types our language will support.

data Ty = BOOL | INT | UNIT | REF Ty

Here is the concrete term syntax we will be using.

namespace Concrete
  public export
  data Expr : (state : Type) -> Type where
    B : (state : a) -> (b : Bool) -> Expr a

    Var : (state : a)
       -> (name  : String)
                -> Expr a

    Fetch : (state : a)
         -> (expr  : Expr a)
                  -> Expr a

  public export
  data KIND = LEAF | BLOCK

  public export
  data Stmt : (kind : KIND) -> (state : Type) -> Type where
    Return : (state : a)
          -> (expr  : Expr a)
                   -> Stmt LEAF a

    Let : (state : a)
       -> (name  : String)
       -> (type  : (a, Ty))
       -> (expr  : Expr a)
                -> Stmt LEAF a

    LetVar : (state : a)
          -> (name  : String)
          -> (type  : (a, Ty))
          -> (expr  : Expr a)
                   -> Stmt LEAF a

    Mutate : (state : a)
          -> (name  : String)
          -> (val   : Expr a)
                   -> Stmt LEAF a

    Cond : (state : a)
        -> (expr  : Expr a)
        -> (whenT : Stmt BLOCK a)
                 -> Stmt LEAF  a

    Block : (state : a)
         -> (block : List (Stmt LEAF a))
                  -> Stmt BLOCK a

Here is the intrinsically-typed syntax we will be elaborating to.

data Expr : (ctxt : SnocList Ty)
         -> (type : Ty)
                 -> Type
  where
    B : (b : Bool) -> Expr ctxt BOOL

    Var : (idx : Var type ctxt)
              -> Expr ctxt type

    Fetch : (expr : Expr ctxt (REF type))
                 -> Expr ctxt      type

    Alloc : (expr : Expr ctxt      type)
                 -> Expr ctxt (REF type)

    Mutate : (ref : Expr ctxt (REF type))
          -> (val : Expr ctxt      type)
                 -> Expr ctxt UNIT

data HowEnd = STOP | RETURN

data Stmt : (he   : HowEnd)
         -> (ctxt : SnocList Ty)
         -> (type : Ty)
                 -> Type
  where
    Return : (expr : Expr        ctxt type)
                  -> Stmt RETURN ctxt type

    Stop : Stmt STOP ctxt type

    Seq : (this : Expr   ctxt UNIT)
       -> (next : Stmt a ctxt type)
               -> Stmt a ctxt type

    Cond : (this : Expr   ctxt BOOL)
        -> (wheT : Stmt a ctxt type)
        -> (kont : Stmt b ctxt type)
                -> Stmt b ctxt type

    Let : (this : Expr    ctxt    typeA)
       -> (body : Stmt a (ctxt :< typeA) typeB)
               -> Stmt a  ctxt           typeB

We have purposefully designed the concrete and abstract syntaxes to have different shape.

Within our abstract syntax for expressions, we have variables, constants, and fetching. Our statements detail our two binding forms, mutation, conditionals, and returning.

Our intrinsically-typed expressions collate all the operations for memory, and our statements have a single binding form, sequencing and conditionals. When extracting intrinsically-typed terms, we transform from one syntax to the other.

Importantly, as we are dealing with a language that does not support type inference we require mutation to have a name on the left hand side. With inference we can infer the type for the LHS and check that it is a reference type. With checking we do not know the inner type of the reference. Thus, we must cheat ever so slightly.

A Type Checking Algorithm and Proof for Expressions

Our algorithm for checking expressions in the concrete syntax follows the same ideas we saw in the previous section. We will skip the details and leave them as an exercise, noting that the type constructor will be:

data Expr : (ctxt : SnocList (String, Ty))
         -> (type : Ty)
         -> (expr : Expr a)
                 -> Type
  where

That we will also need unique typing of expressions:

unique : Expr ctxt typeA expr
      -> Expr ctxt typeB expr
      -> Equal typeA typeB

and the proof is:

namespace Expr
  export
  check : (ctxt : SnocList (String, Ty))
       -> (type : Ty)
       -> (expr : Expr a)
               -> Dec (Error a)
                      (Expr ctxt type expr)

Term extraction will also be the same:

namespace Expr
   export
   toTerm : Expr                  ctxt  type expr
         -> Expr (map Builtin.snd ctxt) type

Complete the definition of Expr, the implementation of unique, and the implementations for check and toTerm.

Intermezzo: Ensuring a Type is a Reference.

Later, when checking the type of a variable from the context, we will see that we do not know a priori what the returned type will be. Although we could use decidable equality, we can only do so for atomic types. For compound types such as REF, we do not know what the inner type will be.

We can construct predicates, datatypes to assert some thing about a value, to not only assert if the type is a reference type, but to also extract the type of the value itself.

We do so by pattern matching at the type-level on instances of Ty but also ensuring that we only care about references.

public export
data IsRef : Ty -> Ty -> Type where
  YesIsRef : IsRef (REF type) type

We know that, given two instances of the reference type, the inner type will be the same.

unique : IsRef x a -> IsRef x b -> a === b
unique YesIsRef YesIsRef = Refl

We also know, by construction, that both UNIT and BOOL are atomic, and thus have no inner type.

isRefTyUnit : IsRef UNIT x -> Void
isRefTyUnit (YesIsRef) impossible

isRefTyB : IsRef BOOL a -> Void
isRefTyB (YesIsRef) impossible

Such that, if given a type it is decidable if that type is a reference type and what the inner type will be.

isRef : (type : Ty)
             -> Dec (DPair Ty (IsRef type))

We know that references are references.

isRef (REF x)
  = Yes (_ ** YesIsRef)

Whilst, the other types are not:

isRef UNIT
  = No (\case (ty ** YesIsRef) => isRefTyUnit YesIsRef)

isRef BOOL
  = No (\case (ty ** YesIsRef) => isRefTyB YesIsRef)

A Type Checking Algorithm and Proof for Statements

We will now look at describing how to type of statements for our concrete syntax. Importantly, we will also look at transforming our block structure to a tree one.

As with expressions, we will describe a type constructor where the type comes first and the list of statements second. We will also record how the statement ends, using HowEnd.

data Stmt : (he   : HowEnd)
         -> (ctxt : SnocList (String, Ty))
         -> (type : Ty)
         -> (stmt : List (Stmt LEAF a))
                 -> Type
  where

We can now state describing how we can type statement blocks. Key to our algorithm is that we will be left-folding over the blocks and transforming them into our tree.

Leaf Statements for Stopping and Returning

Recall, also, that in our statement based world, a block may not return a result. When parsing we can encode empty blocks with the empty list []. When we reach the empty list we can Stop. As when defining our intrinsically-typed statements, Stop ends and accepts the type it is being checked against.

    Stop : Stmt STOP ctxt type []

Our next statement is Return, which is the other leaf statement. We need to ensure that there are other statements after return, which we do by treating our block as a singleton list. Finally, we have to check that the expression specified satisfies the input type.

    Return : (expr : Expr        ctxt type            ast)
                  -> Stmt RETURN ctxt type [Return st ast]

Let-and-Variable Binding

We now move onto our two binding forms: Let and Var bindings. Both forms have the same shape. We first check the expression being bound (expr) against its type annotation (type). Next we can then check the scope with an extended context, extended with a name type pair. We then, and this is a neat trick, check the rest of the list and use that as the inductively defined scope and returning the result of how the continuation ended (he).

For variables, we must extend the scope with the type being a reference type.


    Let : (expr  : Expr     ctxt          ty        ast)
       -> (scope : Stmt he (ctxt :< (naam,ty)) type rest)
                -> Stmt he  ctxt               type (Let st naam (a,ty) ast :: rest)

    LetVar : (expr  : Expr     ctxt          ty        ast)
          -> (scope : Stmt he (ctxt :< (naam,REF ty)) type rest)
                   -> Stmt he  ctxt                   type (LetVar st naam (a,ty) ast :: rest)

Conditional

Conditionals are representative of statements with inner blocks that are not a continuation. We first ensure that the condition is an expression and has type BOOL. When then type both the inner branch (whenT) and the continuation (kont) under the same type and context, but noting that they may end differently.

    Cond : {a,b : _}
        -> (expr  : Expr    ctxt BOOL cond)
        -> (whenT : Stmt a ctxt type astT)
        -> (kont  : Stmt b ctxt type astK)
                 -> Stmt b ctxt type (Cond st cond (Block st' astT) :: astK)

That the we can safely pattern match on the statement for the true branch as dependent types ensures that it can only be a block.

We have also implicitly bound how the statements return as we will need them at the value level for checking.

Mutations

Finally mutations. As we are statically typing, we lookup the name of the reference within the typing context and ensure that the type itself is a reference (using IsRef) and extracting the type of the value. We can then use that value to type the expression on the right-hand side, and check the rest of the list for the continuation.

    Mutate : {ty'  : _}
          -> (idx   : Exists name type ctxt)
          -> (prf   : IsRef type ty')
          -> (val   : Expr    ctxt      ty'  astV)
          -> (scope : Stmt he ctxt      ty    rest)
                   -> Stmt he ctxt      ty    (Mutate state name astV :: rest)

Extracing Intrinsically-Typed Terms from Propositions

Before we look at proving that elaboration is complete, we will look at constructing terms from propositions.

Our function type signature has the same construction as with expressions, ensuring that we are moving from a named context to a nameless one.

toTerm : Stmt he                  ctxt  type expr
      -> Stmt he (map Builtin.snd ctxt) type

Leaf Terms

Both Stop and Return map to their intrinsically-typed terms, ensuring that expressions are also turned into typed-terms.

toTerm Stop
  = Stop

toTerm (Return x)
  = Return (toTerm x)

Let-and-Variable Bindings

Let-bindings are a direct translation of the bound expression and scope into intrinsically-typed terms.

toTerm (Let x scope)
  = Let (toTerm x) (toTerm scope)

Variable bindings are, however, different. When translating the bound expression, we must wrap it in our allocation expression to ensure that the value is placed on the heap.

toTerm (LetVar x scope)
  = Let (Alloc $ toTerm x)
        (toTerm scope)

Variable bindings are a good example of showing how we can do type-safe transformations between different term representations.

However, these transformations are not well-described and we are specifying these transformations by hand. It would be better to have a more type-level type-driven approach to these transformations.

Conditionals

Conditionals, like let-bindings and our leaf terms, do not require any rewrites.

toTerm (Cond x whenT kont)
  = Cond (toTerm x)
         (toTerm whenT)
         (toTerm kont)

Mutations

Finally, we consider mutations. We construct a Var expression from the Exists instance, wrap the mutation in a mutation expression and then sequence the rest.

toTerm (Mutate idx YesIsRef val rest)
  = Seq (Mutate (Var $ existsToVar idx)
                (toTerm val))
        (toTerm rest)

Like variable bindings, mutation are a good example of rewriting our concrete syntax into a smaller more expression core language.

Proving Elaboration is complete.

We can now move on to proving that elaboration of statements is sound and complete. Much like checking expressions, the type signature will remain the same except that we are checking a list of statements and determining how the statement will finally end.

check : (ctxt : SnocList (String, Ty))
     -> (type : Ty)
     -> (stmt : List (Stmt LEAF FileContext))
             -> Dec Error (he ** Stmt he ctxt type stmt)

We will now examine the list of statements, examining the head to form our branches and using the tail to construct the child nodes.

Leaf Terms

An empty list signals the end of computation.

check ctxt type []
  = Yes (STOP ** Stop)

With return, we must examine two cases for the tail. The first is when the tail is empty, signifying that the statement has finishes and there are no more statements to process. The valid case. The second is an error case when the programmer added code after a return statement.

The valid case requires that we check the type of the expression. If the type is incorrect, we return an error message.

check ctxt type ((Return state expr) :: []) with (check ctxt type expr)
  check ctxt type ((Return state expr) :: []) | (Yes x)
    = Yes (RETURN ** Return x)

  check ctxt type ((Return state expr) :: []) | (No msg no)
    = No (Stack state msg)
         (\case ((RETURN ** (Return expr))) => f expr)

For the error case, we can return a generic error message. To construct the proof of false, we need a proof that our proposition has been designed to reject statements where there are statements after a return statement has been specified. We will do this in a special way such that we can say that the idea is absurd.

check ctxt type ((Return state expr) :: (x :: xs))
  = No (Stack state $ E "Cannot do stuff after return")
       (\case (he ** stmt) => absurd stmt)

Intermezzo: Uninhabited

Within Idris we can use the Uninhabited interface to declare that certain propositions cannot be constructed from their type-level values. For Stmt we need to complete the following code:

Uninhabited (Stmt he ctxt type (Return st expr :: this :: that)) where
  uninhabited pat => ?rhs

If we pattern match on pat, easiest using Idris’ completion tooling, we will see that all cases are impossible for a return statement followed by other statements.

Let-and-Variable Bindings

We now move onto bindings. We know that both binding forms share a common computational structure but differ on how to extend the environment.

first, check the bound expression;
then check the scope with an extended context.

Let-bindings

check ctxt type ((Let state name (state',ty) expr) :: xs) with (check ctxt ty expr)
  check ctxt type ((Let state name (state', ty) expr) :: xs) | (Yes expr) with (check (ctxt :< (name,ty)) type xs)
    check ctxt type ((Let state name (state', ty) expr) :: xs) | (Yes expr) | (Yes (he ** scope))
      = Yes (he ** Let expr scope)
    check ctxt type ((Let state name (state', ty) expr) :: xs) | (Yes expr) | (No msg no)
      = No (Stack state $ msg)
           (\case (he ** (Let expr scope)) => no (he ** scope))

  check ctxt type ((Let state name (state', ty) expr) :: xs) | (No msg no)
    = No (Stack state $ Stack state' msg)
         (\case (he ** (Let expr scope)) => no expr)

Variable bindings

check ctxt type ((LetVar state name (state',ty) expr) :: xs) with (check ctxt ty expr)
  check ctxt type ((LetVar state name (state', ty) expr) :: xs) | (Yes expr) with (check (ctxt :< (name, REF ty)) type xs)
    check ctxt type ((LetVar state name (state', ty) expr) :: xs) | (Yes expr) | (Yes (he ** scope))
      = Yes (he ** LetVar expr scope)
    check ctxt type ((LetVar state name (state', ty) expr) :: xs) | (Yes expr) | (No msg no)
      = No (Stack state $ msg)
           (\case (he ** (LetVar expr scope)) => no (he ** scope))

  check ctxt type ((LetVar state name (state', ty) expr) :: xs) | (No msg no)
    = No (Stack state $ Stack state' msg)
         (\case (he ** (LetVar expr scope)) => no expr)

Mutations

Mutating variables is special as we have access to the name of the variable being mutated. The process requires that we:

Check that the name does refer to a bound variable.
Check that the type of the variable is a reference type.
Check that the new value shares the same type as the reference type.
Check that the continuation has the correct type.

check ctxt type ((Mutate st naam val) :: xs) with (isBound naam ctxt)
  check ctxt type ((Mutate st naam val) :: xs) | (Yes (ty ** idx)) with (isRef ty)
    check ctxt type ((Mutate st naam val) :: xs) | (Yes (ty ** idx)) | (Yes (innerTy ** prf)) with (check ctxt innerTy val)
      check ctxt type ((Mutate st naam val) :: xs) | (Yes (ty ** idx)) | (Yes (innerTy ** prf)) | (Yes val) with (check ctxt type xs)
        check ctxt type ((Mutate st naam val) :: xs) | (Yes (ty ** idx)) | (Yes (innerTy ** prf)) | (Yes val) | (Yes ((he ** cont)))
          = Yes (he ** Mutate idx prf val cont)
        check ctxt type ((Mutate st naam val) :: xs) | (Yes (ty ** idx)) | (Yes (innerTy ** prf)) | (Yes val) | (No msg no)
          = No (Stack st msg)
               (\case (he ** (Mutate idx prf val scope)) => f (no ** scope))
      check ctxt type ((Mutate st naam val) :: xs) | (Yes (ty ** idx)) | (Yes (innerTy ** prf)) | (No msg no)

A few of the no cases are interesting to note.

First, if the value being bound has the wrong type then we must use the knowledge that indexing contexts and ensuring that reference inner types are unique. We need to help idris see this when proving the no case.

        = No (Stack st msg)
             (\case (he ** (Mutate idx' prf' val scope)) =>
                           case unique idx idx' of
                             Refl => case unique prf prf' of
                                          Refl => no val)

Similarly, if a variable does not exist then we need to use the knowledge that indexing contexts is unique.

    check ctxt type ((Mutate st naam val) :: xs) | (Yes (ty ** idx)) | (No contra)
      = No (Stack st $ E "Expected a reference")
           (\case (fst ** (Mutate idx' YesIsRef val scope)) =>
                          case unique idx idx' of
                            Refl => contra (_ ** YesIsRef))

  check ctxt type ((Mutate st naam val) :: xs) | (No contra)
    = No (Stack st (E "Not Bound"))
         (\case (fst ** (Mutate idx YesIsRef val scope)) => contra (_ ** idx))

Conditionals

We end with conditionals. The algorithm for checking conditionals requires that we:

check that the condition has type BOOL
check the true branch has the required type;
check that the rest of the list, the continuation, has the required type,

check ctxt type ((Cond state expr whenT) :: xs) with (check ctxt BOOL expr)
  check ctxt type ((Cond state expr (Block state' block)) :: xs) | (Yes cond) with (check ctxt type block)
    check ctxt type ((Cond state expr (Block state' block)) :: xs) | (Yes cond) | (Yes (heb ** whenT)) with (check ctxt type xs)
      check ctxt type ((Cond state expr (Block state' block)) :: xs) | (Yes cond) | (Yes (heb ** whenT)) | (Yes (hek ** kont))
        = Yes $ (y ** Cond cond whenT kont)
      check ctxt type ((Cond state expr (Block state' block)) :: xs) | (Yes cond) | (Yes (heb ** whenT)) | (No msg no)
        = No (Stack state $ Stack state' msg)
             (\case (he ** (Cond cond whenT kont)) => no (he ** kont))

    check ctxt type ((Cond state expr (Block state' block)) :: xs) | (Yes cond) | (No msg no)
      = No (Stack state msg)
           (\case (fst ** (Cond cond whenT kont)) => f (_ ** whenT))

  check ctxt type ((Cond state expr whenT) :: xs) | (No msg no)
    = No (Stack state msg)
         (\case ((he ** (Cond cond whenT kont))) => f cond)

Exercises

The exercises from intrinsic statements asked you to extend Stmt with while-loops and single branch conditionals.

Use that version of Stmt to extend elaboration of statements but with two-branch conditionals instance of single branch.

Integrate your definition of Expr from the previous section