Lexical analyzer generator Last Lecture Review State Regular

Lexical analyzer generator Last Lecture Review ？ State Regular expression Computer realization convention graph Regular expression grammar Equivalence Subset construction Nondeterministic finite automaton Language Deterministic finite automaton Sate enumeration method Combining the undistinguishable states Minimalist deterministic finite automaton

2. 5 Lexical Analyzer Generator Creating a lexical analyzer with Lex source code lex. l lex. yy. c Input Stream Lex compiler lex. yy. c C compiler a. out Sequence of tokens

2. 5 Lexical Analyzer Generator Lex program has three parts Declarations ％％ Translation rules ％％ auxiliary functions Lex translation rules p 1 p 2 … pn {action 1} {action 2} … {action n}

2. 5 Lexical Analyzer Generator ex---Declarations %{ /* defination of manifest constants LT, LE, EQ, NE, GT, GE, WHILE, DO, ID, NUMBER, RELOP*/ %} /* regular definations */ delim [ t n ] ws {delim}+ letter [A Za z] digit [0 9] id {letter}({letter}|{digit})* number {digit}+(. {digit}+)? (E[+ ]? {digit}+)?

2. 5 Lexical Analyzer Generator ex---translation rule {ws} {/* no actions，no return */} while {return (WHILE); } do {return (DO); } {id} {yylval = install_id ( ); return (ID); } {number} {yylval = install_num( ); return (NUMBER); } “<” {yylval = LT; return (RELOP); } “ <= ” {yylval = LE; return (RELOP); } “=” {yylval = EQ; return (RELOP); } “ <> ” {yylval = NE; return (RELOP); } “>” {yylval = GT; return (RELOP); } “ >= ” {yylval = GE; return (RELOP); }

2. 5 Lexical Analyzer Generator ex---auxiliary fuction install_ id ( ) { /* function to install the lexeme。 yytext points to the first character of lexeme， yyleng is the length of lexeme */ } install_num ( ) { /* similar to install_id, but puts numerical constrants into a separate table */ }

2. 5 Lexical Analyzer Generator Example for experiment : example. l int num_lines = 0, num_chars = 0; %% n ++num_lines; ++num_chars; %% main() { yylex(); printf("# of lines = %d, # of chars = %dn ", num_lines, num_chars); }

2. 5 Lexical Analyzer Generator Example for experiment : example. l hello world wo ai tian an men hello world i love lex. yy. exe # of lines = 3, # of chars = 49

Computer Realization Chapter Checklist Source combination pattern code tokens lexemes characters Name Lex Non-Formalization Description set Alphabet Table Formalization Description Regular Expression Syntaxdirected Nondeterministic Finite Automata Manual Transition Diagrams equal Minimization of Finite Automata Merge Subset undistinguished construction combination set state letters string Language Deterministic State Enumeration Finite Union LUM Automata conjunction exponent concatenation LM closure L* Positive closure L+