Syntax 2 Clite Grammar lexical level 1 Identifier

Syntax (2)

Clite Grammar lexical level (1) Identifier Letter { Letter | Digit } Letter a | b |. . . | z | A | B |. . . | Z Digit 0 | 1 |. . . | 9 Literal Integer | Boolean | Float | Char Integer Digit { Digit } Boolean true | False Float Integer Char ‘ ASCII Char ‘ 2

Issues Not Addressed by this Grammar o o Comments Whitespace Distinguishing one token <= from two tokens < = Distinguishing identifiers from keywords like if 3

Lexical Syntax (Lexer ) o o Input: a stream of characters from the ASCII set, keyed by a programmer. Output: a stream of tokens 4

Classes of Tokens o o o Identifiers: Stack, x, i, push Literals : 123, 'x', 3. 25, true Keywords : bool false true char int float if else while main Operators : = || && == != < <= > >= + - * / ! Punctuation: ; , { } ( ) 5

Whitespace o Whitespace is any: n n o o space Tab end-of-line character (or characters) character sequence inside a comment No token may contain embedded whitespace (unless it is a character or string literal) Example: >= one token > = two tokens 6

Whitespace Examples in Pascal o while a < b do n o legal - spacing between tokens whilea < b do n n Whilea whilea < – valid identifier token -invalid statement prefix 7

Keywords and Identifiers o o Both an identifier and a keyword are lexically the same. if is a keyword and it is also an identifier In most languages keywords are reserved and cannot be used as identifiers main in C and C++ is not a reserved word but it is special identifier 8

Integer and float values Range and storage o o o Not limited in Clite grammar No idea about storage both limits and storage space are semantic issues 9

Concrete Syntax (Parser) o o o Based on BNF/EBNF grammar Input: tokens Output: Concrete Syntax (parse) tree or Abstract Syntax tree 10

Concrete Syntax of Clite o o Metabraces {} imply left associativity Metabrackets [] makes Equ. Op and Rel. Op non-associative n In C++, the expression: o o o if (a < x < b) is not equivalent to if (a < x && x < b) But it is error-free! Clite Differs from C/C++ n n Fewer operators Equality and relational are non-associative 11

Clite Grammar Expressions Expression Conjunction { || Conjunction } Operator Conjunction Equality { && Equality } Unary - ! Equality Relation [ Equ. Op Relation ] */ Equ. Op == | != Relation Addition [ Rel. Op Addition ] +Rel. Op < | <= | >= < <= > >= Addition Term { Add. Op Term } == != Add. Op + | && Term Factor { Mul. Op Factor } Mul. Op * | / | % || Factor [ Unary. Op ] Primary Unary. Op - | ! Primary Identifier | Literal | ( Expression ) | Type ( Expression ) (2) Associativity none left 12

Clite Grammar Statements (3) Program int main ( ) { Declarations Statements } Declarations { Declaration } Declaration Type Identifier { , Identifier } Type int | bool | float | char Statements { Statement } Statement ; | Block | Assignment | If. Statement | While. Statement Block { Statements } Assignment Identifier = Expression ; If. Statement if ( Expression ) Statement [ else Statement ] While. Statement while ( Expression ) Statement 13

Abstract Syntax o Removes “syntactic redundancies ” and keeps essential elements of a language. Pascal while i < n do begin i : = i + 1; end; o C/C++ while (i < n) { i = i + 1; } The only essential information n It is a loop A terminating condition i < n A body increments the current value of i. 14

Parse and Abstract Syntax trees o o o Parse tree is inefficient The shape of the parse tree reveals the meaning of the program. So we want a tree that removes its inefficiency and keeps its shape. n n n Remove separator/punctuation terminal symbols Remove all trivial root nonterminals Replace remaining nonterminals with leaf terminals 15

Example: z = x + 2*y; Parse Tree Abstract Syntax Tree 16

Partial Abstract Syntax of Clite o o o o Assignment = Variable target; Expression source Expression = Variable | Value | Binary | Unary Variable = String id Value = Integer Value Binary = Operator op; Expression term 1, term 2 Unary = Operator op; Expression term Operator = + | - | * | / | ! 17

Example Abstract Syntax Tree for z = x+2*y Assignment = Variable target; Expression source Expression = Variable | Value | Binary | Unary Variable = String id Value = Integer Value Binary = Operator op; Expression term 1, term 2 Unary = Operator op; Expression term Operator = + | - | * | / | ! Assignment Variable z Binary Operator Variable x + Binary Operator * Value 2 Variable y 18