Principles of Programming Languages www cs bgu ac

Principles of Programming Languages www. cs. bgu. ac. il/~ppl 192 Lesson 13 –Substitution Interpreter

L 2 Semantics The syntax of L 2 extends that of L 1 with two new expression types if-exp and proc-exp: <program> : : = (L 2 <exp>+) // program(exps: List(exp)) <exp> : : = <define-exp> | <cexp> <define-exp> : : = (define <var-decl> <cexp>) // def-exp(var: var-decl, val: cexp) <cexp> : : = <num-exp> // num-exp(val: Number) | <bool-exp> // bool-exp(val: Boolean) | <prim-op> // prim-op(op: string) | <var-ref> // var-ref(var: string) | (if <exp>) // if-exp(test, then, else) | (lambda (<var-decl>*) <cexp>+) // procexp(params: List(var-decl), body: List(cexp)) ##### L 2 | (<cexp>*) // app-exp(rator: cexp, rands: List(cexp)) <prim-op> : : = + | - | * | / | < | > | = | not <num-exp> : : = a number token <bool-exp> : : = #t | #f <var-ref> : : = an identifier token <var-decl> : : = an identifier token

L 2 Value Type (L 2 (define square (lambda (x) (* x x))) (+ (square 2) (square 3))) we examine the new expression types. If. Exp expressions return the consequent or the alternative, which are both cexp expressions. Thus it never returns new types of values. Proc. Exp expressions return a new type named closure. We extend the definition of the Value type to include closure values: L 2 Value = Number | Boolean | Prim-op | Void | Closure We define the closure data type as a record with two fields: Params: a list of var-decl values Body: a list of cexp values

Closures We define the closure data type as a record with two fields: Params: a list of var-decl values Body: a list of cexp values Closure : : = (Closure (<var-decl>*) <cexp>+) // closure(params: List(vardecl), bo dy: List(cexp)) Note that a Proc. Exp (lambda (x) (* x x)) is an expression while a closure is a value. Closures are the result of a computation!

Evaluation of Conditional Expressions We use the following rule for if expressions. eval(If. Exp(test, then, else), env) => ; ; test, then, else are of type cexp let c: Value = eval(test, env) If c is considered a true value: return eval(then, env) else return eval(else, env) Note that we must define what counts as a true value. This is a semantic decision, and the answer varies across languages. In Scheme, a true value is anything that is not #f. We implement this in this procedure in our interpreter code: // Purpose: Define what is considered a true value in an if-exp export const is. True. Value = (x: Value | Error): boolean | Error => is. Error(x) ? x : ! (x === false);

Evaluation of Conditional Expressions In Javascript, truth value is any value that is: not false, not undefined, not +0, not -0, not Na. N, not an empty string. Evaluation of Procedure Expressions 1. eval(proc-exp(params, body), env) => ; ; Construct a closure value return make. Closure(params, body) Observe that when we compute the value of a procedure expression, we do not calculate the function. The body is not computed as it should only be computed once the procedure is invoked. Therefore, we can delay the computation of an expression by wrapping it inside a procedure.

Evaluation of Procedure Application Due to the introduction of closures, we also need to change procedure application. In the L 1 case we only had primitive procedures. eval(App. Exp(rator, rands) => ; ; rator is of type Cexp ; ; rands is of type List(Cexp) let proc = eval(rator, env) args = [eval(r, env) for r in rands] return apply. Proc(proc, args) Apply. Proc defines how a procedure value is applied to values. We must now define how a closure value is applied to argument values. Observe that we perform an applicative order evaluation. That is we first evaluate the elements of the combination and then apply the closure (which is the value of the operator) to the arguments (which are the values of the operands). . This strategy is called the standard evaluation strategy but we will discuss an alternative strategy in the next section (normal evaluation).

Evaluation of Procedure Application To apply a closure to arguments, we define the substitution model: evaluate the body of the closure with each formal parameter replaced by the corresponding argument. Example: (define square (lambda (x) (* x x)) (square 5) The evaluation process is the following: 1. Evaluate define. Exp: 1. Evaluate (lambda (x) (* x x)) => (closure (x) (* x x)) 2. Bind square to the value (closure (x) (* x x)) in the global environment 2. Evaluate (square 5) (an App. Exp) in the global environment: 1. Evaluate square (a Var. Ref expression) => (closure (x) (* x x)) 2. Evaluate 5 (a Num. Exp expression) => 5 3. Apply. Proc[ (closure (x) (* x x)) (5) ] ; ; To clarify the process let us use full AST for the closure elements

Substitute Expressions instead of Values Consider the types of values in the substations: ; ; (closure [(Var. Decl x)] [(App. Exp (Prim. Op *) [(Var. Ref x), (Var. Ref x)])]) 1. Substitute the Var. Ref free occurrences of the Var. Decl in body with the corresponding value Substituted-body = [(App. Exp (Prim. Op *) [5, 5])] The apply. Proc procedure receives arguments which are all of type Value (proc is a Value which can be either a Prim. Op or a Closure value, rands is a list of Values). The body of the closure is a list of CExp expressions. Our objective is to replace all Var. Ref occurrences in the body with the corresponding values of the arguments (in our example, we want to replace (Var. Ref x) with 5). There is a typing problem with this operation: 5 is a Value, while (Var. Ref x) is an expression. If we replace (Var. Ref x) with the value 5 (a number), the resulting body is not a valid AST.

Substitute Expressions instead of Values To address this discrepancy, we must map the values of the arguments to corresponding expressions. This mapping is performed in our interpreter with the following function: In [1]: // Purpose: Transform a value into a literal expression denoting this value // Pre-conditions: val is not void const value. To. Lit. Exp = (v: Value): Num. Exp | Bool. Exp | Str. Exp | Lit. Exp | Prim. Op | Proc. Exp => is. Number(v) ? make. Num. Exp(v) : is. Boolean(v) ? make. Bool. Exp(v) : is. String(v) ? make. Str. Exp(v) : is. Prim. Op(v) ? v : is. Closure(v) ? make. Proc. Exp(v. params, v. body) : make. Lit. Exp(v); Out[1]: undefined

Substitute Expressions instead of Values As a result, the closure application above is processed as follows: ; ; (closure [(Var. Decl x)] [(App. Exp (Prim. Op *) [(Var. Ref x), (Var. Ref x)])]) 1. Substitute the Var. Ref free occurrences of the Var. Decl in body with the corresponding value Substituted-body = [(App. Exp (Prim. Op *) [Num. Exp(5), Num. Exp(5)])] In summary, the substitution procedure has type: // // @Pre: vars and exps have the same length export const substitute = (body: CExp[], vars: string[], exps: CExp[]): CExp[]

Free Variable Occurrences in the Body When we apply a closure to arguments, we consider the body and the params of the closure separately. E. g. , in (closure (x) (* x y)) x occurs and y occurs free. If we now look at the body separately - (* x y) - then the values which were bound to the params now appear free in the body. We must replace only free variables with values. Example 2: (closure (x) ; 1 ((lambda (x) (* x x)) ; 2 (+ x x))) ; 3 In this case, the var-ref occurrences in line 2 are bound in the body to the var -decl in line 2, while the occurrences in line 3 are free. When we apply this closure to the value 2, we must replace the free occurrences in line 3 but leave those in line 2 unchanged.

Free Variable Occurrences in the Body The substitution algorithm is implemented below, Var. Ref is the only expression type which is changed. Observe how the code of the transformation is similar to the code of apply. Env we discussed in the previous lecture. // @Pre: vars and exps have the same length export const substitute = (body: CExp[], vars: string[], exps: CExp[]): CExp[] => { const sub. Var. Ref = (e: Var. Ref): CExp => { const pos = vars. index. Of(e. var); return ((pos > -1) ? exps[pos] : e); }; const sub. Proc. Exp = (e: Proc. Exp): Proc. Exp => { const arg. Names = map((x) => x. var, e. args); const subst = zip(vars, exps); const free. Subst = filter((ve) => arg. Names. index. Of(first(ve)) === -1, subst); return make. Proc. Exp(e. args, substitute(e. body, map(first, free. Subst), map(second, free. Su bst))); }; const sub = (e: CExp): CExp => is. Num. Exp(e) ? e : is. Bool. Exp(e) ? e : is. Prim. Op(e) ? e : is. Lit. Exp(e) ? e : is. Str. Exp(e) ? e : is. Var. Ref(e) ? sub. Var. Ref(e) : is. If. Exp(e) ? make. If. Exp(sub(e. test), sub(e. then), sub(e. alt)) : is. Proc. Exp(e) ? sub. Proc. Exp(e) : is. App. Exp(e) ? make. App. Exp(sub(e. rator), map(sub, e. rands)) : e; return map(sub, body); };

Free Variables and substitution Consider the following program: (define z (lambda (x) (* x x))) (((lambda (x) (lambda (z) (x z))) ; 1 (lambda (w) (z w))) ; 2 2) If we apply the substitution model when computing the 2 nd expression, we get the following: (lambda (z) ((lambda (w) (z w)) z)) The problem in this substitution is that the inner var-ref z coming from the function (lambda (w) (z w)) is now captured by the (lambda (z). . . ) context in which we operated the substitution. As a result, this z var-ref now refers to the (lambda (z). . . ) var-decl instead of referring to the global(define z. . . ) var-decl as it should. This effect is called free variable capture and we must avoid it.

Free Variables and substitution The simplest solution to address this problem is to ensure that before we perform substitution, we rename consistently all the bound variables that occur in the body with fresh names. Why does it works?

Renaming Algorithm /* Purpose: create a generator of new strings of the form v n with n is incremented at each call. Note the typical usage of a closure with side effect of the closed variable. */ export const make. Var. Gen = (): (v: string) => string => { let count: number = 0; return (v: string) => { count++; return `${v} ${count}`; } } /* Purpose: Consistently rename bound variables in 'exps' to fresh names. Start numbering at 1 for all new var names. */ export const rename. Exps = (exps: CExp[]): CExp[] => { const var. Gen = make. Var. Gen(); const replace = (e: CExp): CExp => is. If. Exp(e) ? make. If. Exp(replace(e. test), replace(e. then), replace(e. alt)) : is. App. Exp(e) ? make. App. Exp(replace(e. rator), map(replace, e. rands)) : is. Proc. Exp(e) ? replace. Proc(e) : e; // Rename the params and substitute old params with renamed ones. // First recursively rename all Proc. Exps inside the body. const replace. Proc = (e: Proc. Exp): Proc. Exp => { const old. Args = map((arg: Var. Decl): string => arg. var, e. args); const new. Args = map(var. Gen, old. Args); const new. Body = map(replace, e. body); return make. Proc. Exp(make. Var. Decl, new. Args), substitute(new. Body, old. Args, map(make. Var. Ref, new. Args))); } return map(replace, exps); };

The Apply Procedure Putting all elements discussed above together, the apply-procedure implements the following algorithm: const L 3 apply. Procedure = (proc: Value | Error, args: Array<Value | Error>, env: Env): V Error => is. Error(proc) ? proc : !has. No. Error(args) ? Error(`Bad argument: ${get. Error. Messages(args)}`) : is. Prim. Op(proc) ? apply. Primitive(proc, args) : is. Closure(proc) ? apply. Closure(proc, args, env) : Error("Bad procedure " + JSON. stringify(proc)) // @Pre: none of the args is an Error (checked in apply. Procedure) const apply. Closure = (proc: Closure, args: Value[], env: Env): Value | Error => { let vars = map((v: Var. Decl) => v. var, proc. params); let body = rename. Exps(proc. body); let lit. Args = map(value. To. Lit. Exp, args); return eval. Exps(substitute(body, vars, lit. Args), env); }; 1. we make sure the body is renamed. 2. we map arguments to lit-exps 3. then we perform the substitution We use the following terminology: Substitute formal parameters, Reduce (evaluate the substituted body) alue |

Renaming Correct Renaming: (lambda (x) x) ==> (lambda (x 1)

Renaming Correct Renaming: (lambda (x) x) ==> (lambda (x 1) (+ x ( (lambda (x) (+ x y)) 4)) ==> (+ x ( (lambda (x 1) (+ x 1 y)) 4))

Renaming Correct Renaming: (lambda (x) x) ==> (lambda (x 1) (+ x ( (lambda (x) (+ x y)) 4)) ==> (+ x ( (lambda (x 1) (+ x 1 y)) 4)) incorrect Renaming: (+ x ( (lambda (y) (+ y y)) 4)) ==> (+ x 1 ( (lambda (x 1) (+ x 1 y)) 4)))

Substitution Substitute is an operation which replaces free occurrences of variable references in an expression by other expressions. Definition: A substitution s is a mapping from a finite set of variables to a finite set of expressions. Substitutions are denoted using set notation. For example: {x = 3, y = a, z = #t, w = (lambda (x) (+ x 1))} is a substitution {x = 3, x = a, z = #t, w = (lambda (x) (+ x 1))} is not a substitution because the variable x is mapped to 2 distinct values. NOTE: We abbreviate substitutions e. g. , {x = 3, y = a, z = #t} Is: {x = (num-exp 3), y = (lit-exp a), z = (bool-exp #t)}

Substitution Composition The composition of substitutions s and s ′ (denoted s ∘ s ′ ) is a substitution that extends s with a binding <x; s'(x)> for every variable x for which s(x) is not defined. For example: {x = 3, y = a} o {z = #t, w = (lambda (x) (+ x 1))} = {x = 3, y = a, z = #t, w = (lambda (x) (+ x 1))} The empty substitution {} is the neutral element of the substitution-composition operation: For every substitution s, { } ∘ s = s ∘ { } = s. How we use composition: The substitute operation operates an expressions E, and s. 1. Consistently rename E and s. 2. Replaces all the variables of s in the renamed E, by their value in s.

Substitution Examples 10 o {x = 5} = 10: No renaming; no replacement. (+ x y) o {x = 5} = (+ 5 y): No renaming; just replacement. (+ x y) o {x = 5, y = 'x} = (+ 5 'x): No renaming; just replacement. ((+ x ((lambda (x) (+ x 3)) 4))) o {x = 5} = 1. Renaming: ((+ x ((lambda (x 1) (+ x 1 3)) 4))) 2. Substitute: ((+ 5 ((lambda (x 1) (+ x 1 3)) 4))) (lambda (y) (((lambda (x) x) y) x)) o {x = (lambda (x) (y x))} =: y in the substitution is free. 1. Renaming: the substitution turns into: {x = (lambda (x 1) (y x 1))}; E turns into(lambda (y 2) (((lambda (x 3) y 2) x)) 2. Substitute: E turns into (lambda (y 2) ( ((lambda (x 3) y 2) (lambda (x 1) (y x 1))) What would be the result without renaming?

Notes NOTE: In manual derivation exercies, we may skip the renaming step if it is not needed. For example: ((+ x ((lambda (x) (+ x 3)) 4))) o {x = 5} = ((+ 5 ((lambda (x) (+ x 3)) 4))) NOTE 2: The substitution model – applicative order uses the call-by-value method for parameter passing. This is the standard evaluation model in Scheme, and the most frequent method in other languages as well (Java. Script, C++, Java).

Eval Example (define y (define f (g y))))) (define h (f h) ==> 4) (lambda (g) (lambda (y) (+ y (lambda (x) (+ x y))) <Closure (y 1) (+ y 1 ((lambda (x) (+ x y)) y 1))> Trace of the algorithm: applicative-eval[ (f h) ] ==> applicative-eval[ f ] ==> <Closure (g) (lambda (y) (+ y (g y)))> applicative-eval[ h ] ==> <Closure (x) (+ x y)> ==> Substitute – rename both expressions and replace: Map the closure value of f to the corresponding lambda expression (lambda (y 2) (+ y 2 (g y 2))) o {g = (lambda (x 1) (+ x 1 y))} ==> Reduce - applicative-eval[ (lambda (y 2) (+ y 2 ((lambda (x 1) (+ x 1 y)) y 2 ) )) ] ==> <Closure (y 2) (+ y 2 (<Closure (x 1) (+ x 1 y)> y 2) ) >

L 3: Compound Values and Lists Recall that we introduced value constructors and accessors as primitives: (cons, car, cdr), Type predicates: (list? , symbol? ) and equality predicate (eq? ) We also introduced side effect primitives: (display and newline). Note that the empty list special value (which is a value which is not a number, not a boolean and not a symbol) must be supported in the syntax. The last modification that is required is to support literal expressions for the new compound values. We use the Scheme quote special operator to support these. An expression:

L 3: Compound Values and Lists We use the Scheme quote special operator to support these. An expression: (quote <sexp>) is a special expression which is computed according to the following computation rule: eval((quote <sexp>)) => return <sexp> For example: (quote a) => 'a (quote (a b)) => '(a b) The special form (quote <sexp>) is abbriviated to'<sexp> - for example, 'a for a symbol or '(a b) for a list.

L 3: Compound Values and Lists To support the apply-procedure and substitution of values back as expressions, we must update the procedure that turns values into expressions with a special case for Literal Expressions that wrap SExp values: const value. To. Lit. Exp = (v: Value): Num. Exp | Bool. Exp | Str. Exp | Lit. Exp | Prim. Op | Proc. Exp => is. Number(v) ? make. Num. Exp(v) : is. Boolean(v) ? make. Bool. Exp(v) : is. String(v) ? make. Str. Exp(v) : is. Prim. Op(v) ? v : is. Closure(v) ? make. Proc. Exp(v. params, v. body) : make. Lit. Exp(v); Since SExps are not a type that exists in Type. Script, we must implement this type as part of the possible values computed by L 3. This is implemented in the definition of the L 3 -Value module:

Implementation // ============================ // SExp export interface Compound. SExp { tag: "Compound. Sexp"; val: SExp[]; }; export interface Empty. SExp { tag: "Empty. SExp"; }; export interface Symbol. SExp { tag: "Symbol. SExp"; val: string; }; export type SExp = number | boolean | string | Prim. Op | Closure | Symbol. SExp | Empty. SEx Compound. SExp; export const is. SExp = (x: any): x is SExp => typeof(x) === 'string' || typeof(x) === 'boolean' || typeof(x) === 'number' || is. Symbol. SExp(x) || is. Compound. SExp(x) || is. Empty. SExp(x) || is. Prim. Op(x) || is. Closure( x); export const make. Compound. SExp = (val: SExp[]): Compound. SExp => ({tag: "Compound. Sexp", val: val}); export const is. Compound. SExp = (x: any): x is Compound. SExp => x. tag === "Compound. Sexp"; export const make. Empty. SExp = (): Empty. SExp => ({tag: "Empty. SExp"}); export const is. Empty. SExp = (x: any): x is Empty. SExp => x. tag === "Empty. SExp"; export const make. Symbol. SExp = (val: string): Symbol. SExp => ({tag: "Symbol. SExp", val: val}); export const is. Symbol. SExp = (x: any): x is Symbol. SExp => x. tag === "Symbol. SExp"; p

Implementation The following function extends the s. Exp for lists. // In L 3 -AST export const parse. Lit. Exp = (sexps: any[]): Lit. Exp | Error => safe. F(make. Lit. Exp)(parse. SExp(second(sexps ))); // x is the output of p (sexp parser) export const parse. SExp = (x: any): SExp | Error => x === "#t" ? true : x === "#f" ? false : is. Numeric. String(x) ? +x : is. Sexp. String(x) ? x. to. String() : is. String(x) ? make. Symbol. SExp(x) : x. length === 0 ? make. Empty. SExp() : is. Array(x) ? make. Compound. SExp(map(parse. SExp, x)) : Error(`Bad literal expression: ${x}`); Example L 3 Program: (L 3 (define empty? (lambda (x) (eq? x '()))) (define filter (lambda (pred l) (if (empty? l) l (if (pred (car l)) (cons (car l) (filter pred (cdr l)))))) (filter (lambda (x) (not (= x 2))) '(1 2 3 2)))

Error Handling We decide to implement the parser with the following signature: export const parse. L 3 = (x: string): Parsed | Error That is, the parser either returns a valid Parsed result (a Program or an Expression) or an Error. Similarly, we define the interpreter with the following signature: const L 3 applicative. Eval = (exp: CExp | Error, env: Env): Value | Error That is, the interpreter can obtain as parameter a valid CExp or an Error, and it returns either a valid Value or an Error. Note that we do not change the definition of the AST data type or the Value data type to include Error as a possible value. There are two reasons for this decision: 1. Errors are not real ASTs or Values. 2. Errors are absorbing elements of the interpreter. Absorbing – means that if we compose an error with any value then the result is an error.

Error Handling Example: evaluate if with errors: const eval. If = (exp: If. Exp, env: Env): Value | Error => { const test = L 3 applicative. Eval(exp. test, env); return is. Error(test) ? test : is. True. Value(test) ? L 3 applicative. Eval(exp. then, env) : L 3 applicative. Eval(exp. alt, env); }; The two procedures is. Error and has. No. Error are used to handle the case where we want to guard against further processing after an error has been returned at runtime - either for a single value or for a list of values. Note the typing of the procedure - it receives parameters which are unchecked Value | Error. After the guards is. Error or has. No. Error have been checked, the parameters are validated, and are known by the type system of Type. Script as Value only - so that, for example, apply. Closure is defined with a type where args has type Value[] (without Error).

Error Handling In the code of the parser we use the following pattern for error handling: export const parse. Lit. Exp = (sexps: any[]): Lit. Exp | Error => safe. F(make. Lit. Exp)(parse. SExp(second(sexps))); The general structure of the parser is: 1. Recognize the type of compound expression we are traversing (in the case of Scheme this is easy - by looking up the first keyword in a compound S-expression) 2. Parse the sub-expressions recursively 3. Build a compound AST value. Error handling in this context means that the recursively parsed sub-expressions can end up being of type Error instead of a valid AST type. In this case, we want to guard the 3 rd step from invoking the constructor of the compound AST type with an Error argument. We use the safe. F (and safe. FL) functional abstraction to do so in a type-safe manner:

Error Handling // Make a safe version of f: apply f to x but check if x is an error before applying i t. export const safe. F: <T 1, T 2>(f: (x: T 1) => T 2) => (x: T 1 | Error) => T 2 | Error = (f) = > (x) => { if (is. Error(x)) return x; else return f(x); } safe. F abstracts away the guard of checking with is. Error to avoid invoking the strict function when the parameter is an error. In our example - make. Lit. Exp is a strict function - a constructor which only accepts valid SExp values as argument. safe. F(make. Lit. Exp) is a function which accepts an unchecked argument SExp | Error and returns the union Lit. Exp | Error. That is we handle errors without using exception in the meta-language. In a sense, we have explained what throwing an exception would mean in the semantic domain.

Normal Order Evaluation Algorithm applicative-eval implements an eager approach to evaluation: arguments are evaluated immediately before the closure is reduced. Alternatively, a lazy approach avoids evaluating arguments until the value is necessary in the evaluation process. That is it evaluate arguments when: • • 1. We need to decide a computation branch. We need to apply a primitive procedure. The normal evaluation algorithm is similar to the applicative-eval but moves the step of argument evaluation to just before a primitive procedure is applied. The algorithm is unchanged otherwise. This is a small change to the algorithm and a deep change to the language behavior.

Normal Order Evaluation Algorithm We change a single evaluation rule for application expressions: normal-eval(app-exp(rator, rands)) => ; ; rator is of type cexp ; ; rands is of type List(cexp) let proc = normal-eval(rator, env) ; ; invoke the procedure on the arguments WITHOUT evaluating them return normal-apply-proc(proc, rands) We also adapt the apply procedure to this slight change: apply-proc(proc: Closure, rands: CExp): Value if proc is a primitive: ; ; We must evaluate all the args to apply a primitive let args = [normal-eval(r) for r in rands] return apply-primitive(proc, args) else ; ; proc is a closure ; ; Substitute the rands and reduce let subst-body = substitute params in body of proc with rands return normal-eval(subst-body)

Normal Order Evaluation Algorithm Observe how the same computations are repeated in the normal evaluation algorithm, while they were processed only once in applicative order: for example (* 5 2) when it is passed to the function square is not computed before the substitution into the body of square - which leads to the computation of (* (* 5 2)) in normal order instead of (* 10 10) in applicative order. Normal Order Parameter Passing Mode: Call by Name The normal order strategy of passing arguments to procedures without pre-computing them is called call by name - as opposed to the call by value defined by applicative-eval. Normal-order evaluation is also called lazy evaluation because it delays the evaluation of arguments to the last moment when it is needed.