Winter 2007 2008 Compiler Construction T 5 AST

Winter 2007 -2008 Compiler Construction T 5 – AST Mooly Sagiv and Roman Manevich School of Computer Science Tel-Aviv University

Today ic Lexical Analysis Syntax Analysis Parsing IC AST Symbol Table etc. Inter. Rep. (IR) Code Generation exe Executable Language n Today: n n n EBNF AST construction AST traversal n code n Next week: n n n Annotating ASTs Symbol tables Type-checking Visitor pattern 2

EBNF n Extended Backus–Naur form n n n Extends BNF with regular expressions R* R+ (ABC) R? [R] stands for optional part (same as R? ) Not supported by CUP Translate EBNF rules to BNF n list : : = x* list : : = | x list n x : : = y [z] x : : = y | y z 3

Abstract Syntax Trees n n Intermediate program representation Defines a tree n n n What is the root of the AST? Preserves program hierarchy Node types defined by class hierarchy Generated by parser Keywords and punctuation symbols not stored (not relevant once parse tree exists) Provides clear interface to other compiler phases 4

Partial AST hierarchy for IC ASTNode Program ICClass Statement Virtual. Method Field Static. Method . . . Library. Method Expression Literal New Call . . . Statement. Block Local. Variable declaration If While . . . 5

Eclipse type hierarchy view 6

AST node contents abstract class ASTNode { int get. Line(); . . . } class Program extends ASTNode { List<ICClass> classes; . . . } used for error reporting Java 1. 5 Generics class ICClass extends ASTNode { String name; List<Field> fields; List<Method> methods; // ICClass superclass; . . . } 7

Actions part of IC. cup non terminal Program program; non terminal ICClass class; non terminal List<ICClass> class_list; program : : = class_list: cl {: RESULT = new Program(get. Line(), cl); : } ; class_list : : = class: c {: RESULT = new Linked. List<ICClass>(); RESULT. add(c); : } | class_list: cl class: c {: cl. add(c); RESULT = cl; : } ; Symbol stack object stack class_list c cl Reduce class_list : : = class_list: cl class: c {: RESULT = cl. add(c); : } In reality, use value field of Symbol object stack Symbol stack RESULT pop class_list push class_list 8

Actions part of IC. cup non terminal Program program; non terminal ICClass class; non terminal List<ICClass> class_list; program : : = class_list: cl {: RESULT = new Program(get. Line(), cl); : } ; class_list : : = class: c {: RESULT = new Linked. List<ICClass>(); RESULT. add(c); : } | class_list: cl class: c {: cl. add(c); RESULT = cl; : } ; Reduce class_list : : = class_list: cl class: c {: RESULT = cl. add(c); : } object stack Symbol stack RESULT class_list 9

AST traversal n Once AST stable want to operate on tree n n AST traversal for type-checking AST traversal for transformation (IR) AST traversal for pretty-printing (-dump-ast) Each operation in separate pass 10

Non-Object Oriented approach pretty. Print(ASTNode node) { if (node instanceof Program) { Program prog = (Program) node; for (ICClass icc : prog. classes) { pretty. Print(icc); } } else if (node instanceof ICClass) { ICClass icc = (ICClass) node; print. Class(icc); } else if (node instanceof Binary. Expression) { Binary. Expression be = (Binary. Expression) node; pretty. Print(be. lhs); System. out. println(be. operator); pretty. Print(be. rhs); }. . . } n Messy code n instanceof + down-casting error-prone n Not extensible 11

Visitor Pattern n n Separate operations on objects of a data structure from object representation Each operation (pass) may be implemented as separate visitor Use double-dispatch to find right method for object Instance of a design pattern n Recommended book Design Patterns / Gang of Four 12

Single dispatch - polymorphism conceptually, one-dimensional table class A { void op() {…} } Java 1. 5 annotation class B extends A { @Override void op() {…} } class C extends A { @Override void op() {…} } op index A B C 13

What if we need more operations? class A { void op 1() {…} void op 2() {…} void op 3() {…} } class B extends A { @Override void op 1() {…} @Override void op 2() {…} @Override void op 3() {…} } Want to separate complicated operations from data structures class C extends A { @Override void op 1() {…} @Override void op 2() {…} @Override void op 3() {…} } 14

What if we need more operations? class A { } class op 1 { … // lots of code } class B extends A { } class op 2 { … // lots of code } class C extends A { } × class op 3 { … // lots of code } Problem: OO languages support only single-polymorphism. We seem to need double-polymorphism 15

Visitor pattern in Java interface visit(A visit(B visit(C } class A { A x; accept(Visitor v) { v. visit(this); } } class B extends A { accept(Visitor v) { v. visit(this); } } class C extends A { accept(Visitor v) { v. visit(this); } } × Visitor { a); c); class op 1 visit(A visit(B visit(C } implements Visitor { a) {…} c) {…} class op 2 visit(A visit(B visit(C } implements Visitor { a) {…} c) {…} class op 3 visit(A visit(B visit(C } implements Visitor { a) {…} c) {…} 16

Double dispatch example 1 st dispatch class B { accept(Visitor v) { // always calls visit(B b) v. visit(this); } } Visitor v = new op 1(); // op 1/2/3 A x = new B(); // x can be A/B/C x. accept(v); class op 1 implements Visitor { visit(A a) { } visit(B b) { … } } 2 nd dispatch 17

Double dispatch example 1 st Visitor v = new op 1(); // op 1/2/3 A x = new B(); // x can be A/B/C x. accept(v); dispatch class B { accept(Visitor v) { // always calls visit(B b) v. visit(this); } } class op 1 implements Visitor { visit(A a) { } visit(B b) { … } } 2 nd dispatch x. accept(v) op 1 1 st dispatch v. visit(this) 2 nd dispatch A B C op 1. visit(B b) op 2 op 3 Visitor pattern conceptually implements two-dimensional table 18

Straight Line Program example prog stmt_list ASTNode stmt_list stmt var = expr; stmt print(expr); expr + expr - expr * expr / expr - expr ( expr ) expr number expr readi() expr var Stmt Print. Stmt Assign. Stmt. List Expr Binary. Op. Expr Unary. Op. Expr Number. Expr Read. IExpr Var. Expr (Code available on web site. Demonstrates scanning, parsing, AST + visitors) 19

Printing visitor example Java 1. 5 enhanced for loop public class Pretty. Printer implements Visitor { public void print(ASTNode root) { root. accept(this); } public void visit(Stmt. List stmts) { for (Stmt s : stmts. statements) { s. accept(this); System. out. println(); } } // x = 2*7 public void visit(Assign. Stmt stmt) { stmt. var. Expr. accept(this ); System. out. print("="); stmt. rhs. accept(this); System. out. print("; "); } // x public void visit(Var. Expr expr) { System. out. print(expr. name); } // 2*7 public void visit(Binary. Op. Expr expr) { expr. lhs. accept(this); System. out. print(expr. op); expr. rhs. accept(this); }. . . interface Visitor { void visit(Stmt. List stmts); void visit(Stmt stmt); void visit(Print. Stmt stmt); void visit(Assign. Stmt stmt); void visit(Expr expr); void visit(Read. IExpr expr); void visit(Var. Expr expr); void visit(Number. Expr expr); void visit(Unary. Op. Expr expr); void visit(Binary. Op. Expr expr); } } 20

Visitor variations interface Propagating. Visitor { /** Visits a statement node with a given * context object (book-keeping) * and returns the result * of the computation on this node. */ Object visit(Stmt st, Object context); Object visit(Expr e, Object context); Object visit(Binary. Op. Expr e, Object context); . . . } n Propagate values down the AST (and back) 21

Evaluating visitor example public class SLPEvaluator implements Propagating. Visitor { public void evaluate(ASTNode root) { class Environment { root. accept(this); Integer get(Var. Expr ve) {…} } void update(Var. Expr ve, int value) {…} } /** x = 2*7 */ public Object visit(Assign. Stmt stmt, Object env) { Expr rhs = stmt. rhs; Integer expression. Value = (Integer) rhs. accept(this, env); Var. Expr var = stmt. var. Expr; ((Environment)env). update(var , expression. Value); return null; } /** expressions like 2*7 and 2*y */ public Object visit(Binary. Op. Expr expr, Object env) { Integer lhs. Value = (Integer) expr. lhs. accept(this, env); Integer rhs. Value = (Integer) expr. rhs. accept(this, env); int result; switch (expr. op) { case PLUS: result = lhs. Value. int. Value() + rhs. Value. int. Value() ; . . . } return new Integer(result); }. . . } 22

AST traversal 1+2+x root 6 Binary. Op. Expr + left right 3 visit (lhs) class Binary. Op. Expr extends Expression { Object accept(Visitor v) { return v. visit(this); } Expression lhs, rhs; } class Number. Expr extends Expression { Object accept(Visitor v) { return v. visit(this); } int val; } 3 visit (rhs) public class SLPEvaluator … { public Object visit(Binary. Op. Expr e, Object env) { Integer lhs. Value=(Integer)e. lhs. accept(this, env ); left right 2 Integer rhs. Value=(Integer)e. rhs. accept(this, env ); 1 visit int result; visit (rhs) switch (expr. op) { (lhs) case PLUS: result=lhs. Value. int. Value()+rhs. Value. int. Value (); . . . Var. Expr Number. Expr } return new Integer(result); name = x value = 1 value= 2 } public Object visit(Number. Expr e, Object env) { return e. value; } SLPEvaluator ev = new SLPEvaluator(); Integer result = (Integer)root. accept(ev); public Object visit(Var. Expr e, Object env) { return ((Environment)env). get(e); } (alternative: let accept do tree walking) 23 Binary. Op. Expr +

Visitor + Generics interface Up. Type. . . } Propagating. Visitor<Down. Type, Up. Type> { visit(Stmt st, Down. Type d); visit(Expr e, Down. Type d); visit(Var. Expr ve, Down. Type d); public class SLPEvaluator implements Propagating. Visitor<Environment, Integer> { public Integer visit(Var. Expr expr, Environment env) { return env. get(expr); }. . . } 24

See you next week 25