Lex Yacc COP 3402 Spr 2010 Lex Symbol

Lex & Yacc COP 3402, Spr 2010

Lex – Symbol Table • When lexer has to recognize many tokens and their number is changing use a symbol table – stores lexemes and info about them, such as type of token – use a single pattern to match any word and then consult symbol table which token is found • Declaration start with type of token followed by the lexeme(s) – e. g:

Lex – Symbol Table (Cont’d) • Symbol table (ST) maintenance routines and variables: – func. add_word(): adds a new lexeme into ST – func. lookup_word(): looks up a word already in ST – var. state: keeps track of whether we are looking up words (state =LOOKUP) or declaring them (state = token type)

Lex Example -Definition Section

Lex Example -Rules Section

Lex Example –User Subroutines Section

Lex Example –User Subroutines Section

Lex Example - Test

Yacc • For simple token recognition a lexer might be sufficient. • When specific sequences of tokens have to be recognized, a parser is necessary. • Grammar - a description of sequences of tokens and actions associated with them – e. g:

Yacc • Provides a general tool for imposing structure on the input to a computer program • Takes a grammar specified by the user and writes a parser that recognizes valid “sentences” in that grammar • When lex scanner and yacc parser are used together, the parser is the higher level routine – parser calls lexer yylex() whenever it needs a token – lexer scans through input recognizing tokens

Yacc • Lexer & parser have to decide on tokens codes

Example – Lexer Part

Example – Lexer Part Cont’d

Example – Parser Part