Syntax Directed Translation Chapter 5 1 ContextFree Languages

  • Slides: 93
Download presentation
Syntax Directed Translation Chapter 5 1

Syntax Directed Translation Chapter 5 1

Context-Free Languages Context-free languages LR(k) ≡ LR(1) Deterministic languages (LR(k)) LL(k) languages Simple precedence

Context-Free Languages Context-free languages LR(k) ≡ LR(1) Deterministic languages (LR(k)) LL(k) languages Simple precedence languages LL(1) languages Operator precedence languages The inclusion hierarchy for context-free languages 2

Context-Free Grammars Context-free grammars Floyd-Evans Parsable Unambiguous CFGs Operator Precedence LR(k) LL(k) LR(1) LALR(1)

Context-Free Grammars Context-free grammars Floyd-Evans Parsable Unambiguous CFGs Operator Precedence LR(k) LL(k) LR(1) LALR(1) • Operator precedence includes some ambiguous grammars • LL(1) is a subset of SLR(1) LL(1) LR(0) The inclusion hierarchy for context-free grammars 3

Beyond Syntax • There is a level of correctness that is deeper than grammar

Beyond Syntax • There is a level of correctness that is deeper than grammar fie(a, b, c, d) int a, b, c, d; {…} fee() { int f[3], g[0], h, i, j, k; char *p; fie(h, i, “ab”, j, k); k = f * i + j; h = g[17]; printf(“<%s, %s>. n”, p, q); p = 10; } What is wrong with this program? (let me count the ways …) 4

Semantics • There is a level of correctness that is deeper than grammar fie(a,

Semantics • There is a level of correctness that is deeper than grammar fie(a, b, c, d) int a, b, c, d; {…} fee() { int f[3], g[0], h, i, j, k; char *p; fie(h, i, “ab”, j, k); k = f * i + j; h = g[17]; printf(“<%s, %s>. n”, p, q); p = 10; } What is wrong with this program? (let me count the ways …) • declared g[0], used g[17] • wrong number of args to fie() • “ab” is not an int • wrong dimension on use of f • undeclared variable q • 10 is not a character string All of these are “deeper than syntax” 5

Syntax-directed translation • In syntax-directed translation, we attach ATTRIBUTES to grammar symbols. • The

Syntax-directed translation • In syntax-directed translation, we attach ATTRIBUTES to grammar symbols. • The values of the attributes are computed by SEMANTIC RULES associated with grammar productions. • Conceptually, we have the following flow: • In practice, however, we do everything in a single pass. 6

Syntax-directed translation • There are two ways to represent the semantic rules we associate

Syntax-directed translation • There are two ways to represent the semantic rules we associate with grammar symbols. • SYNTAX-DIRECTED DEFINITIONS (SDDs) do not specify the order in which semantic actions should be executed • TRANSLATION SCHEMES explicitly specify the ordering of the semantic actions. • SDDs are higher level; translation schemes are closer to • an implementation 7

Syntax-Directed Definitions 8

Syntax-Directed Definitions 8

Syntax-directed definitions • The SYNTAX-DIRECTED DEFINITION (SDD) is a generalization of the context-free grammar.

Syntax-directed definitions • The SYNTAX-DIRECTED DEFINITION (SDD) is a generalization of the context-free grammar. • Each grammar symbol in the CFG is given a set of ATTRIBUTES. • Attributes can be any type (string, number, memory loc, etc) • SYNTHESIZED attributes are computed from the values of the CHILDREN of a node in the parse tree. • INHERITED attributes are computed from the attributes of the parents and/or siblings of a node in the parse tree. 9

Attribute dependencies • Given a SDD, we can describe the dependencies between the attributes

Attribute dependencies • Given a SDD, we can describe the dependencies between the attributes with a DEPENDENCY GRAPH. • A parse tree annotated with attributes is called an ANNOTATED parse tree. • Computing the attributes of the nodes in a parse tree is called ANNOTATING or DECORATING the tree. 10

Form of a syntax-directed definition • • • In a SDD, each grammar production

Form of a syntax-directed definition • • • In a SDD, each grammar production A -> α has associated with it semantic rules b : = f( c 1, c 2, …, ck ) where f() is a function, and either 1. b is a synthesized attribute of A, and c 1, c 2, …, are attributes of the grammar symbols of α, or 2. b is an inherited attribute of one of the symbols on the RHS, and c 1, c 2, … are attributes of the grammar symbols of α • in either case, we say b DEPENDS on c 1, c 2, …, ck. 11

Semantic rules • Usually we actually write the semantic rules with expressions instead of

Semantic rules • Usually we actually write the semantic rules with expressions instead of functions. • If the rule has a SIDE EFFECT, e. g. updating the symbol table, we write the rule as a procedure call. • When a SDD has no side effects, we call it an ATTRIBUTE GRAMMAR. 12

Synthesized attributes • Synthesized attributes depend only on the attributes of • children. They

Synthesized attributes • Synthesized attributes depend only on the attributes of • children. They are the most common attribute type. • If a SDD has synthesized attributes ONLY, it is called a S-ATTRIBUTED DEFINITION. • S-attributed definitions are convenient since the attributes can be calculated in a bottom-up traversal of the parse tree. 13

Example SDD: desk calculator • Production • L -> E newline • E ->

Example SDD: desk calculator • Production • L -> E newline • E -> E 1 + T • E -> T • T -> T 1 * F • T -> F • F -> ( E ) • F -> number Semantic Rule print( E. val ) E. val : = E 1. val + T. val E. val : = T 1. val x F. val T. val : = F. val : = E. val F. val : = number. lexval • Notice the similarity to the yacc spec from last lecture. 14

Calculating synthesized attributes Input string: 3*5+4 newline Annotated tree: 15

Calculating synthesized attributes Input string: 3*5+4 newline Annotated tree: 15

Inherited attributes • An inherited attribute is defined in terms of the attributes of

Inherited attributes • An inherited attribute is defined in terms of the attributes of the node’s parents and/or siblings. • Inherited attributes are often used in compilers for passing contextual information forward, for example, the type keyword in a variable declaration statement. 16

Example SDD with an inherited attribute • Suppose we want to describe decls like

Example SDD with an inherited attribute • Suppose we want to describe decls like “real x, y, z” • Production • D -> T L • T -> int • T -> real • L -> L 1, id • • L -> id Semantic Rules L. in : = T. type : = integer T. type : = real L 1. in : = L. in addtype( id. entry, L. in ) addtype() is just a procedure that sets the type field in the symbol table. • L. in is inherited since it depends on a sibling or parent. 17

Annotated parse tree for real id 1, id 2, id 3 What are the

Annotated parse tree for real id 1, id 2, id 3 What are the dependencies? 18

Dependency Graphs 19

Dependency Graphs 19

Dependency graphs • If an attribute b depends on attribute c, then attribute b

Dependency graphs • If an attribute b depends on attribute c, then attribute b has to be evaluated AFTER c. • DEPENDENCY GRAPHS visualize these requirements. • Each attribute is a node • We add edges from the node for attribute c to the node for attribute b, if b depends on c. • For procedure calls, we introduce a dummy synthesized attribute that depends on the parameters of the procedure calls. 20

Dependency graph example • Production • E -> E 1 + E 2 Semantic

Dependency graph example • Production • E -> E 1 + E 2 Semantic Rule E. val : = E 1. val + E 2. val • Wherever this rule appears in the parse, tree we draw: 21

Example dependency graph 22

Example dependency graph 22

Example dependency graph 23

Example dependency graph 23

Finding a valid evaluation order • A TOPOLOGICAL SORT of a directed acyclic graph

Finding a valid evaluation order • A TOPOLOGICAL SORT of a directed acyclic graph orders the nodes so that for any nodes a and b such that a -> b, a appears BEFORE b in the ordering. • There are many possible topological orderings for a DAG. • Each of the possible orderings gives a valid order for evaluation of the semantic rules. 24

Example dependency graph 25

Example dependency graph 25

S-Attributed Definitions • Given an SDD and a parse tree, it is easy to

S-Attributed Definitions • Given an SDD and a parse tree, it is easy to tell (by doing a topological sort) whether a suitable evaluation exists (and to find one). • However, a very difficult problem is, given an SDD, are there any parse trees with cycles in their dependency graphs, i. e. , are there suitable evaluation orders for all parse trees. • Fortunately, there are classes of SDDs for which a suitable evaluation order is guaranteed. • SDD is S-attributed if every attribute is synthesized. • For these SDDs all attributes are calculated from attribute values at the children since the other possibility, the tail attribute is at the same node, is impossible since the tail attribute must be inherited for such arrows. • Thus no cycles are possible and the attributes can be evaluated by a postorder traversal of the parse tree. 26

S-Attributed Definitions postorder (N) { for ( each child C of N, from the

S-Attributed Definitions postorder (N) { for ( each child C of N, from the left ) postorder(C); evaluate the attributes associated with node N; } • Since postorder corresponds to the actions of an LR parser when reducing the body of a production to its head, it is often convenient to evaluate synthesized attributes during an LR parse. 27

L-Attributed Definitions • Unfortunately, it is hard to live without inherited attributes. So we

L-Attributed Definitions • Unfortunately, it is hard to live without inherited attributes. So we define a class that permits certain kinds of inherited attributes. 28

L-Attributed Definitions Definition: An SDD is L-Attributed if each attribute is either 1. Synthesized.

L-Attributed Definitions Definition: An SDD is L-Attributed if each attribute is either 1. Synthesized. 2. Inherited from the left, and hence the name L-attributed. If the production is A → X 1 X 2. . . Xn, then the inherited attributes for Xj can depend only on a. Inherited attributes of A, the LHS. b. Any attribute of X 1, . . . , Xj-1, i. e. only on symbols to the left of Xj. 3. Attributes of Xj, *BUT* you must guarantee (separately) that these attributes do not by themselves cause a cycle. • Case three must be handled specially whenever it occurs. • The tree on the previous slide illustrates what the first two cases look like and suggest why there cannot be any cycles. 29

Evaluating L-Attributed Definitions • do a depth first traversal of the tree. • The

Evaluating L-Attributed Definitions • do a depth first traversal of the tree. • The first time you visit a node, evaluate its inherited attributes (since you will know the value of everything it depends on), • and the last time you visit it, evaluate the synthesized attributes. • Suppose we have a production A → B C D. Each of the four nonterminals has two attributes s, which is synthesized, and i, which is inherited. For each set of rules below, tell whether the rules are consistent with (i) an S-attributed definition, (ii) an L-attributed definition, (iii) any evaluation order at all. • A. s = B. i + C. i • A. s = B. i + C. s and D. i = A. i + B. s • A. s = B. s + D. s 30

Semantic Rules with Controlled Side Effects • When we have side effects such as

Semantic Rules with Controlled Side Effects • When we have side effects such as printing or adding an entry to a table we must ensure that we have not added a constraint to the evaluation order that causes a cycle. • For example, the left-recursive SDD shown in the table on the right propagates type information from a declaration to entries in an identifier table. • Production Semantic Rule Type D→TL L. type = T. type inherited T → INT T. type = integer synthesized L 1. type = L. type inherited add. Type(ID. entry, L. type) synthesized, side effect L → L 1 , ID L → ID 31

Semantic Rules with Controlled Side Effects • The function add. Type adds the type

Semantic Rules with Controlled Side Effects • The function add. Type adds the type information in the second argument to the identifier table entry specified in the first argument. Note that the side effect, adding the type info to the table, does not affect the evaluation order. • Draw the dependency graph 32

Syntax Trees 33

Syntax Trees 33

Application: syntax tree construction • One thing SDDs are useful for is construction of

Application: syntax tree construction • One thing SDDs are useful for is construction of SYNTAX TREES. • Recall from Lecture 1 that a syntax tree is a condensed form of parse tree. • Syntax trees are useful for representing programming language constructs like expressions and statements. • They help compiler design by decoupling parsing from translation. 34

Syntax trees • Leaf nodes for operators and keywords are removed. • Internal nodes

Syntax trees • Leaf nodes for operators and keywords are removed. • Internal nodes corresponding to uninformative non-terminals are replaced by the more meaningful operators. 35

SDD for syntax tree construction • We need some functions to help us build

SDD for syntax tree construction • We need some functions to help us build the syntax tree: • mknode(op, left, right) constructs an operator node with label op, and two children, left and right • mkleaf(id, entry) constructs a leaf node with label id and a pointer to a symbol table entry • mkleaf(num, val) constructs a leaf node with label num and the token’s numeric value val • Use these functions to build a syntax tree for a-4+c: • P 1 : = mkleaf( id, st_entry_for_a ) • P 2 : = … 36

SDD for syntax tree construction • Production • E -> E 1 + T

SDD for syntax tree construction • Production • E -> E 1 + T • E -> E 1 - T • E -> T • T -> ( E ) • T -> id • T -> num Semantic Rules E. nptr : = mknode( ‘+’, E 1. nptr, T. nptr) E. nptr : = mknode( ‘-’, E 1. nptr, T. nptr) E. nptr : = T. nptr : = E. nptr T. nptr : = mkleaf( id, id. entry ) T. nptr : = mkleaf( num, num. val ) • Note that this is a S-attributed definition. • Try to derive the annotated parse tree for a-4+c. 37

Evaluating SDDs Bottom-up 38

Evaluating SDDs Bottom-up 38

Bottom-up evaluation of S-attributed defn • How can we build a translator for a

Bottom-up evaluation of S-attributed defn • How can we build a translator for a given SDD? • For S-attributed definitions, it’s pretty easy! • A bottom-up shift-reduce parser can evaluate the (synthesized) attributes as the input is parsed. • We store the computed attributes with the grammar symbols and states on the stack. • When a reduction is made, we calculate the values of any synthesized attributes using the already-computed attributes from the stack. 39

Bottom-up evaluation of S-attributed defns • In the scheme, our parser’s stack now stores

Bottom-up evaluation of S-attributed defns • In the scheme, our parser’s stack now stores grammar symbols AND attribute values. • For every production A -> XYZ with semantic rule A. a : = f( X. x, Y. y, Z. z ), before XYZ is reduced to A, we should already have X. x Y. y and Z. z on the stack. 40

Desk calculator example • If attribute values are placed on the stack as described,

Desk calculator example • If attribute values are placed on the stack as described, it is now easy to implement the semantic rules for the desk calculator. • Production Semantic Rule • L -> E newline print( E. val ) Code print val[top-1] • E -> E 1 + T E. val : = E 1. val + T. val[newtop] = val[top-2]+val[top] • E -> T E. val : = T. val /*newtop==top, so nothing to do*/ • T -> T 1 * F T. val : = T 1. val x F. val[newtop] = val[top-2]+val[top] • T -> F T. val : = F. val /*newtop==top, so nothing to do*/ • F -> ( E ) F. val : = E. val[newtop] = val[top-1] • F -> number F. val : = number. lexval /*newtop==top, so nothing to do*/ 41

Desk calculator example • For input 3 * 5 + 4 newline, what happens?

Desk calculator example • For input 3 * 5 + 4 newline, what happens? • Assume when a terminal’s attribute is shifted when it is. • Input • 3*5+4 n • • • • States number Values 3 Action shift reduce F->number reduce T->F shift reduce F->number reduce T->T*F reduce E->T shift reduce F->number reduce T->F reduce E->E+T shift reduce E->En 42

L-attributed definitions • S-attributed definitions only allow synthesized attributes. • We saw earlier that

L-attributed definitions • S-attributed definitions only allow synthesized attributes. • We saw earlier that inherited attributes are useful. • But we prefer definitions that can be evaluated in one pass. • L-ATTRIBUTED definitions are the set of SDDs whose attributes can be evaluated in a DEPTH-FIRST traversal of the parse tree. 43

Depth-first traversal • algorithm dfvisit( node n ) { • for each child m

Depth-first traversal • algorithm dfvisit( node n ) { • for each child m of n, in left-to-right order, do { • evaluate the inherited attributes of m • dfvisit( m ) • } • evaluate the synthesized attributes of n • } 44

L-attributed definitions • If a definition can be evaluated by dfvisit() we say it

L-attributed definitions • If a definition can be evaluated by dfvisit() we say it is L-attributed. • Another way of putting it: a SDD is L-attributed if each INHERITED attribute of Xi on the RHS of a production A -> X 1 X 2 … Xn depends ONLY on: • The attributes of X 1, …, Xi-1 (to the LEFT of Xi in the production) • The INHERITED attributes of A. • Since S-attributed definitions have no inherited attributes, they are necessarily L-attributed. 45

L-attributed definitions • Is the following SDD L-attributed? • Production • A -> L

L-attributed definitions • Is the following SDD L-attributed? • Production • A -> L M • • • A -> Q R • • Semantic Rules L. i : = l(A. i) M. i : = m(L. s) A. s : = f(M. s) R. i : = r(A. i) Q. i : = q(R. s) A. s : = f(Q. s) 46

Translation Schemes 47

Translation Schemes 47

Translation schemes • Translation schemes are another way to describe syntax-directed translation. • Translation

Translation schemes • Translation schemes are another way to describe syntax-directed translation. • Translation schemes are closer to a real implementation because the specify when, during the parse, attributes should be computed. • Example, for conversion of INFIX expressions to POSTFIX: • E -> T R • R -> addop T { print ( addop. lexeme ) } | ε • T -> num { print( num. val ) } • This translation scheme will turn 9 -5+2 into 95 -2+ 48

Turning a SDD into a translation scheme • For a translation scheme to work,

Turning a SDD into a translation scheme • For a translation scheme to work, it must be the case that an attribute is computed BEFORE it is used. • If the SDD is S-attributed, it is easy to create the translation scheme implementing it: • • Production T -> T 1 * F Semantic Rule T. val : = T 1. val x F. val • Translation scheme: • T -> T 1 * F { T. val = T 1. val * F. val } • That is, we just turn the semantic rule into an action and add at the far right hand side. This DOES NOT WORK for inherited attribs! 49

Turning a SDD into a translation scheme With inherited attributes, the translation scheme designer

Turning a SDD into a translation scheme With inherited attributes, the translation scheme designer needs to follow three rules: 1. An inherited attribute for a symbol on the RHS MUST be computed in an action BEFORE the occurrence of the symbol. 2. An action MUST NOT refer to the synthesized attribute of a symbol to the right of the action. 3. A synthesized attribute for the LHS nonterminal can ONLY be computed in an action FOLLOWING the symbols for all the attributes it references. • 50

Example • This translation scheme does NOT follow the rules: • • S ->

Example • This translation scheme does NOT follow the rules: • • S -> A 1 A 2 A -> a { A 1. in = 1; A 2. in = 2 } { print( A. in ) } • If we traverse the parse tree depth first, A 1. in has not been set when referred to in the action print( A. in ) • • S -> { A 1. in = 1 } A 1 { A 2. in = 2 } A 2 A -> a { print( A. in ) } 51

Bottom-up evaluation of inherited attributes • The first step is to convert the SDD

Bottom-up evaluation of inherited attributes • The first step is to convert the SDD to a valid translation scheme. • Then a few “tricks” have to be applied to the translation scheme. • It is possible, with the right tricks, to do one-pass bottom-up attribute evaluation for ALL LL(1) grammars and MOST LR(1) grammars, if the SDD is L-attributed. • This means when adding semantic actions to your yacc specifications, you might run into trouble. See section 5. 6 of the text! 52

Beyond Syntax • These questions are part of context-sensitive analysis • Answers depend on

Beyond Syntax • These questions are part of context-sensitive analysis • Answers depend on values, not parts of speech • Questions & answers involve non-local information • Answers may involve computation • How can we answer these questions? • Use formal methods • Context-sensitive grammars? • Attribute grammars? (attributed grammars? ) • Use ad-hoc techniques • Symbol tables • Ad-hoc code (action routines) • In scanning & parsing, formalism won; different story here. 53

Beyond Syntax • Telling the story • The attribute grammar formalism is important •

Beyond Syntax • Telling the story • The attribute grammar formalism is important • Succinctly makes many points clear • Sets the stage for actual, ad-hoc practice • The problems with attribute grammars motivate practice • Non-local computation • Need for centralized information • Some folks in the community still argue for attribute grammars • We will cover attribute grammars, then move on to ad-hoc ideas 54

Attribute Grammars • What is an attribute grammar? • A context-free grammar augmented with

Attribute Grammars • What is an attribute grammar? • A context-free grammar augmented with a set of rules • Each symbol in the derivation has a set of values, or attributes • The rules specify how to compute a value for each attribute Example grammar Number → Sign List Sign →+ | – List → List Bit | Bit → 0 |1 This grammar describes signed binary numbers We would like to augment it with rules that compute the decimal value of each valid input string 55

Examples For “-1” Number For “-101” → → Sign List – Bit – 1

Examples For “-1” Number For “-101” → → Sign List – Bit – 1 Number Sign – Number → Sign List Bit → Sign List 1 → Sign List Bit 1 → Sign List 1 1 → Sign Bit 0 1 → Sign 1 0 1 → – 101 Number Sign List – Bit List Bit 0 1 Bit 1 1 56

Attribute Grammars • Add rules to compute the decimal value of a signed binary

Attribute Grammars • Add rules to compute the decimal value of a signed binary number Productions Attribution Rules Symbol Attributes Number → Sign List. pos ← 0 If Sign. neg then Number. val ← – List. val else Number. val ← List. val Number val Sign neg List pos, val →+ Sign. neg ← false Bit pos, val | – Sign. neg ← true Sign List 0 → List 1 Bit | Bit → 0 | 1 List 1. pos ← List 0. pos + 1 Bit. pos ← List 0. pos List 0. val ← List 1. val + Bit. val Bit. pos ← List. pos List. val ← Bit. val ← 0 Bit. val ← 2 Bit. pos 57

Back to The Examples Rules + parse tree imply an attribute dependence graph For

Back to The Examples Rules + parse tree imply an attribute dependence graph For “-1” Number. val ← –List. val≡ – 1 Number neg ← true Sign List. pos ← 0 List. val ← Bit. val≡ 1 – Bit. pos ← 0 Bit. val ← 2 Bit. pos ≡ 1 One possible evaluation order: 1. 2. 3. 4. 5. 6. List. pos Sign. neg Bit. pos Bit. val List. val Number. val Other orders are possible 1 • Knuth suggested a data-flow model for evaluation • Independent attributes first Evaluation order must be consistent with the attribute dependence graph • Others in order as input values become available 58

Back to the Examples This is the complete attribute dependence graph for “– 101”.

Back to the Examples This is the complete attribute dependence graph for “– 101”. It shows the flow of all attribute values in the example. Some flow downward → inherited attributes Some flow upward → synthesized attributes A rule may use attributes in the parent, children, or siblings of a node 59

The Rules of The Game • • • Attributes associated with nodes in parse

The Rules of The Game • • • Attributes associated with nodes in parse tree Rules are value assignments associated with productions Attribute is defined once, using local information Label identical terms in production for uniqueness Rules & parse tree define an attribute dependence graph • Graph must be non-circular • This produces a high-level, functional specification • Synthesized attribute • Depends on values from children • Inherited attribute • Depends on values from siblings & parent 60

Using Attribute Grammars • • Attribute grammars can specify context-sensitive actions Take values from

Using Attribute Grammars • • Attribute grammars can specify context-sensitive actions Take values from syntax Perform computations with values Insert tests, logic, … Synthesized Attributes • Use values from children & from constants • S-attributed grammars • Evaluate in a single bottom-up pass Good match to LR parsing Inherited Attributes • Use values from parent, constants, & siblings • directly express context • can rewrite to avoid them • Thought to be more natural Not easily done at parse time We want to use both kinds of attribute 61

Evaluation Methods • • • Dynamic, dependence-based methods Build the parse tree Build the

Evaluation Methods • • • Dynamic, dependence-based methods Build the parse tree Build the dependence graph Topological sort the dependence graph Define attributes in topological order • • Rule-based methods Analyze rules at compiler-generation time Determine a fixed (static) ordering Evaluate nodes in that order • • • Oblivious methods (passes, dataflow) Ignore rules & parse tree Pick a convenient order (at design time) & use it (treewalk) 62

Back to the Example Number Sign – List Bit 1 0 Bit 1 For

Back to the Example Number Sign – List Bit 1 0 Bit 1 For “-101” 63

Back to the Example Number Sign – List Bit 1 val: neg: List pos:

Back to the Example Number Sign – List Bit 1 val: neg: List pos: val: pos: 0 val: List pos: val: Bit pos: val: 1 0 For “-101” 64

Back to the Example Number Sign – List Bit 1 val: -5 Inherited Attributes

Back to the Example Number Sign – List Bit 1 val: -5 Inherited Attributes neg: true List pos: 2 val: 4 List pos: 0 val: 5 pos: 1 val: 4 Bit pos: 1 val: 0 pos: 0 val: 1 1 0 For “-101” 65

Back to the Example Number Sign – List Bit 1 val: -5 Synthesized Attributes

Back to the Example Number Sign – List Bit 1 val: -5 Synthesized Attributes neg: true List pos: 2 val: 4 List pos: 0 val: 5 pos: 1 val: 4 Bit pos: 1 val: 0 pos: 0 val: 1 1 0 For “-101” 66

Back to the Example Number Sign – List Bit 1 val: -5 Synthesized Attributes

Back to the Example Number Sign – List Bit 1 val: -5 Synthesized Attributes neg: true List pos: 2 val: 4 List pos: 0 val: 5 pos: 1 val: 4 Bit pos: 1 val: 0 pos: 0 val: 1 1 0 For “-101” 67

Back to the Example Number Sign – List Bit 1 val: -5 If we

Back to the Example Number Sign – List Bit 1 val: -5 If we show the computation … neg: true List pos: 2 val: 4 List pos: 0 val: 5 pos: 1 val: 4 Bit pos: 1 val: 0 pos: 0 val: 1 & then peel away the parse tree … 1 0 For “-101” 68

Back to the Example All that is left is the attribute dependence graph. val:

Back to the Example All that is left is the attribute dependence graph. val: -5 neg: true pos: 2 val: 4 1 pos: 0 val: 1 pos: 1 val: 4 – This succinctly represents the flow of values in the problem instance. pos: 0 val: 5 pos: 1 val: 0 The dynamic methods sort this graph to find independent values, then work along graph edges. 1 The rule-based methods try to discover “good” orders by analyzing the rules. 0 For “-101” The oblivious methods ignore the structure of this graph. The dependence graph must be acyclic 69

Circularity • We can only evaluate acyclic instances • We can prove that some

Circularity • We can only evaluate acyclic instances • We can prove that some grammars can only generate instances with acyclic dependence graphs • Largest such class is “strongly non-circular” grammars (SNC ) • SNC grammars can be tested in polynomial time • Failing the SNC test is not conclusive • Many evaluation methods discover circularity dynamically • ⇒ Bad property for a compiler to have • SNC grammars were first defined by Kennedy & Warren 70

A Circular Attribute Grammar Productions Attribution Rules Number → List. a ← 0 List

A Circular Attribute Grammar Productions Attribution Rules Number → List. a ← 0 List 1. a ← List 0. a + 1 List 0. b ← List 1. b List 1. c ← List 1. b + Bit. val → List 1 Bit | Bit → 0 | 1 List 0. b ← List 0. a + List 0. c + Bit. val ← 0 Bit. val ← 2 Bit. pos 71

An Extended Example (§ 4. 3. 3) Grammar for a basic block Block 0

An Extended Example (§ 4. 3. 3) Grammar for a basic block Block 0 → | Assign → Expr 0 → | | Term 0 → | | Factor → | | Block 1 Assign Ident = Expr ; Expr 1 + Term Expr 1 – Term 1 * Factor Term 1 / Factor ( Expr ) Number Identifier Let’s estimate cycle counts • Each operation has a COST • Add them, bottom up • Assume a load per value • Assume no reuse Simple problem for an AG Hey, this looks useful ! 72

An Extended Example Adding attribution rules Block 0 → Block 1 Assign | Assign

An Extended Example Adding attribution rules Block 0 → Block 1 Assign | Assign → Ident = Expr 0 ; → Expr 1 + Term | Expr 1 – Term 0 | Term → Term 1 * Factor | Term 1 / Factor | → | | Factor ( Expr ) Number Identifier Block 0. cost ← Block 1. cost + Assign. cost Block 0. cost ← Assign. cost ← COST(st ore) + Expr. cost Expr 0. cost ← Expr 1. cost + COST(add) + Term. cost Expr 0. cost ← Term. cost Term 0. cost ← Term 1. cost + COST(mult ) + Factor. cost Term 0. cost ← Term 1. cost + COST(div) + Factor. cost Term 0. cost ← Factor. cost ← Expr. cost Factor. cost ← COST(load. I) Factor. cost ← COST(load) All these attributes are synthesized! 73

An Extended Example • Properties of the example grammar • All attributes are synthesized

An Extended Example • Properties of the example grammar • All attributes are synthesized ⇒ S-attributed grammar • Rules can be evaluated bottom-up in a single pass • Good fit to bottom-up, shift/reduce parser • Easily understood solution • Seems to fit the problem well • What about an improvement? • Values are loaded only once per block (not at each use) • Need to track which values have been already loaded 74

A Better Execution Model • Adding load tracking • Need sets Before and After

A Better Execution Model • Adding load tracking • Need sets Before and After for each production • Must be initialized, updated, and passed around the tree Factor → ( Expr ) | Number | Identifier Factor. cost ← Expr. cost ; Expr. Before ← Factor. Before ; Factor. After ← Expr. After Factor. cost ← COST(loadi) ; Factor. After ← Factor. Before if(Identifier. name ∉ Factor. Before) then Factor. cost ← COST(load); Factor. After ← Factor. Before ∪ Identifier. name else Factor. cost ← 0 Factor. After ← Factor. Before This looks more complex! 75

A Better Execution Model • Load tracking adds complexity • But, most of it

A Better Execution Model • Load tracking adds complexity • But, most of it is in the “copy rules” • Every production needs rules to copy Before & After • A Sample Production Expr 0 → Expr 1 + Term Expr 0. cost ← Expr 1. cost + COST(add) + Term. cost ; Expr 1. Before ← Expr 0. Before ; Term. Before ← Expr 1. After; Expr 0. After ← Term. After These copy rules multiply rapidly Each creates an instance of the set Lots of work, lots of space, lots of rules to write 76

An Even Better Model • What about accounting for finite register sets? • Before

An Even Better Model • What about accounting for finite register sets? • Before & After must be of limited size • Adds complexity to Factor→Identifier • Requires more complex initialization • Jump from tracking loads to tracking registers is small • Copy rules are already in place • Some local code to perform the allocation 77

The Extended Example • Tracking loads • Introduced Before and After sets to record

The Extended Example • Tracking loads • Introduced Before and After sets to record loads • Added ≥ 2 copy rules per production • Serialized evaluation into execution order • Made the whole attribute grammar large & cumbersome • • Finite register set Complicated one production (Factor → Identifier) Needed a little fancier initialization Changes were quite limited • Why is one change hard and the other easy? 78

The Moral of the Story • Non-local computation needed lots of supporting rules •

The Moral of the Story • Non-local computation needed lots of supporting rules • Complex local computation was relatively easy • The Problems • Copy rules increase cognitive overhead • Copy rules increase space requirements • Need copies of attributes • Can use pointers, for even more cognitive overhead • Result is an attributed tree • Must build the parse tree • Either search tree for answers or copy them to the root 79

Addressing the Problem • Ad-hoc techniques • Introduce a central repository for facts •

Addressing the Problem • Ad-hoc techniques • Introduce a central repository for facts • Table of names • Field in table for loaded/not loaded state • Avoids all the copy rules, allocation & storage headaches • All inter-assignment attribute flow is through table • • Clean, efficient implementation Good techniques for implementing the table When its done, information is in the table ! Cures most of the problems (hashing, § B. 4) • Unfortunately, this design violates the functional paradigm • Do we care? 80

The Realist’s Alternative • Ad-hoc syntax-directed translation • Associate a snippet of code with

The Realist’s Alternative • Ad-hoc syntax-directed translation • Associate a snippet of code with each production • At each reduction, the corresponding snippet runs • Allowing arbitrary code provides complete flexibility • Includes ability to do tasteless & bad things • To make this work • Need names for attributes of each symbol on lhs & rhs • Typically, one attribute passed through parser + arbitrary code (structures, globals, statics, …) • Yacc introduced $$, $1, $2, … $n, left to right ANTLR allows defining variables • Need an evaluation scheme • Should fit into the parsing algorithm 81

Reworking the Example Block 0 Assign Expr 0 Term 0 Factor → Block 1

Reworking the Example Block 0 Assign Expr 0 Term 0 Factor → Block 1 Assign | Assign → Ident = Expr ; → Expr 1 + Term | Expr 1 – Term | Term → Term 1 * Factor | Term 1 / Factor | Factor → ( Expr ) | Number | Identifier cost← cost + COST(store); cost← cost + COST(add); cost← cost + COST(sub); cost← cost + COST(mult); cost← cost + COST(div); cost← cost + COST(loadi); { i← hash(Identifier); if(Table[i]. loaded = false) then { cost ← cost + COST(load); Table[i]. loaded ← true; } } This looks cleaner & simpler than the AG sol’n ! One missing detail: initializing cost 82

Reworking the Example Start Init Block 0 → Init Block →ε → Block 1

Reworking the Example Start Init Block 0 → Init Block →ε → Block 1 Assign | Assign → Ident = Expr ; cost ← 0; cost← cost + COST(store); … and so on as in the previous version of the example … • Before parser can reach Block, it must reduce Init • Reduction by Init sets cost to zero • This is an example of splitting a production to create a reduction • in the middle — for the sole purpose of hanging an action routine • there! 83

Reworking the Example Block 0 Assign Expr 0 Term 0 Factor → Block 1

Reworking the Example Block 0 Assign Expr 0 Term 0 Factor → Block 1 Assign | Assign → Ident = Expr ; → Expr 1 + Term | Expr 1 – Term | Term → Term 1 * Factor | Term 1 / Factor | Factor → ( Expr ) | Number | Identifier $$ ← $1 + $2 ; $$ ← $1 ; $$ ← COST(store) + $3; $$ ← $1 + COST(add) + $3; $$ ← $1 + COST(sub) + $3; $$ ← $1 + COST(mult) + $3; $$ ← $1 + COST(div) + $3; $$ ← $1; $$ ← $2; $$ ← COST(loadi); { i← hash(Identifier); if (Table[i]. loaded = false) then { $$ ← COST(load); Table[i]. loaded ← true; } else $$ ← 0 } This version passes the values through attributes. It avoids the need for initializing “cost” 84

Reworking the Example • Assume constructors for each node • Assume stack holds pointers

Reworking the Example • Assume constructors for each node • Assume stack holds pointers to nodes • Assume yacc syntax Goal Expr → → | | Term → | | Factor → | | Expr + Term Expr – Term * Factor Term / Factor ( Expr ) number id $$ $$ $$ = = = = = $1; Make. Add. Node($1, $3); Make. Sub. Node($1, $3); $1; Make. Mul. Node($1, $3); Make. Div. Node($1, $3); $1; $2; Make. Num. Node(token); Make. Id. Node(token); 85

Reality • Most parsers are based on this ad-hoc style of context-sensitive analysis •

Reality • Most parsers are based on this ad-hoc style of context-sensitive analysis • Advantages • Addresses the shortcomings of the AG paradigm • Efficient, flexible • • Disadvantages Must write the code with little assistance Programmer deals directly with the details Annotate action code with grammar rules 86

Typical Uses • Building a symbol table • Enter declaration information as processed •

Typical Uses • Building a symbol table • Enter declaration information as processed • At end of declaration syntax, do some post processing • Use table to check errors as parsing progresses • Simple error checking/type checking • • Define before use → lookup on reference Dimension, type, . . . → check as encountered Type conformability of expression → bottom-up walk Procedure interfaces are harder • Build a representation for parameter list & types • Create list of sites to check • Check offline, or handle the cases for arbitrary orderings 87

Is This Really “Ad-hoc” ? • Relationship between practice and attribute grammars • •

Is This Really “Ad-hoc” ? • Relationship between practice and attribute grammars • • Similarities Both rules & actions associated with productions Application order determined by tools, not author (Somewhat) abstract names for symbols • • Differences Actions applied as a unit; not true for AG rules Anything goes in ad-hoc actions; AG rules are functional AG rules are higher level than ad-hoc actions 88

Making Ad-hoc SDT Work • What about a rule that must work in mid-production?

Making Ad-hoc SDT Work • What about a rule that must work in mid-production? • Can transform the grammar • Split it into two parts at the point where rule must go • Apply the rule on reduction to the appropriate part • Can also handle reductions on shift actions • Add a production to create a reduction • Was: fee → fum • Make it: fee → fie → fum and tie action to this reduction • ANTLR supports the above automatically • Together, these let us apply rule at any point in the parse 89

Limitations of Ad-hoc SDT • Forced to evaluate in a given order: postorder •

Limitations of Ad-hoc SDT • Forced to evaluate in a given order: postorder • Left to right only • Bottom up only • Implications • Declarations before uses • Context information cannot be passed down • How do you know what rule you are called from within? • Example: cannot pass bit position from right down • Could you use globals? • Requires initialization & some re-thinking of the solution • Can we rewrite it in a form that is better for the ad-hoc sol’n 90

Alternative Strategy • Build an abstract syntax tree • Use tree walk routines •

Alternative Strategy • Build an abstract syntax tree • Use tree walk routines • Use “visitor” design pattern to add functionality Tree. Node. Visitor Visit. Assignment(Assignment. Node) Visit. Variable. Ref(Variable. Ref. Node) Type. Check. Visitor Analysis. Visitor Visit. Assignment(Assignment. Node) Visit. Variable. Ref(Variable. Ref. Node) 91

Visitor Treewalk • Parallel structure of tree: • Separates treewalk code from node handling

Visitor Treewalk • Parallel structure of tree: • Separates treewalk code from node handling code • Facilitates change in processing without change to tree structure Tree. Node Accept(Node. Visitor) Assignment. Node Variable. Ref. Node Accept(Node. Visitor v) v. Visit. Assignment(this) v. Visit. Variable. Ref(this) 92

Summary • Wrap-up of parsing • More example to build LR(1) table • Attribute

Summary • Wrap-up of parsing • More example to build LR(1) table • Attribute Grammars • Pros: Formal, powerful, can deal with propagation strategies • Cons: Too many copy rules, no global tables, works on parse tree • Ad-hoc SDT • Annotate production with ad-hoc action code • Postorder Code Execution • Pros: Simple and functional, can be specified in grammar, does not require parse tree • Cons: Rigid evaluation order, no context inheritance • Generalized Tree Walk • Pros: Full power and generality, operates on abstract syntax tree (using Visitor pattern) • Cons: Requires specific code for each tree node type, more complicated • Powerful tools like ANTLR can help with this 93