Lex Fall 2006 Costas Busch RPI 1 Lex

Lex Fall 2006 Costas Busch - RPI 1

Lex: a lexical analyzer • A Lex program recognizes strings • For each kind of string found the lex program takes an action Fall 2006 Costas Busch - RPI 2

Output Input Var = 12 + 9; if (test > 20) temp = 0; else while (a < 20) temp++; Fall 2006 Lex program Costas Busch - RPI Identifier: Var Operand: = Integer: 12 Operand: + Integer: 9 Semicolumn: ; Keyword: if Parenthesis: ( Identifier: test. . 3

In Lex strings are described with regular expressions Lex program Regular expressions Fall 2006 “+” “-” “=“ /* operators */ “if” “then” /* keywords */ Costas Busch - RPI 4

Lex program Regular expressions (0|1|2|3|4|5|6|7|8|9)+ (a|b|. . |z|A|B|. . . |Z)+ Fall 2006 /* integers */ /* identifiers */ Costas Busch - RPI 5

integers (0|1|2|3|4|5|6|7|8|9)+ Fall 2006 Costas Busch - RPI [0 -9]+ 6

identifiers (a|b|. . |z|A|B|. . . |Z)+ Fall 2006 Costas Busch - RPI [a-z. A-Z]+ 7

Each regular expression has an associated action (in C code) Examples: Regular expression Action n linenum++; [0 -9]+ prinf(“integer”); [a-z. A-Z]+ printf(“identifier”); Fall 2006 Costas Busch - RPI 8

Default action: ECHO; Prints the string identified to the output Fall 2006 Costas Busch - RPI 9

A small lex program %% [ tn] ; [0 -9]+ printf(“Integern”); [a-z. A-Z]+ printf(“Identifiern”); Fall 2006 /*skip spaces*/ Costas Busch - RPI 10

Output Input 1234 Integer Identifier Integer test var 566 78 9800 Fall 2006 Costas Busch - RPI 11

%{ int linenum = 1; %} Another program %% [ t] n ; /*skip spaces*/ linenum++; [0 -9]+ prinf(“Integern”); [a-z. A-Z]+ printf(“Identifiern”); . Fall 2006 printf(“Error in line: %dn”, Costas Buschlinenum); - RPI 12

Output Input 1234 Integer Identifier Integer Error in line: 3 Identifier test var 566 78 9800 + temp Fall 2006 Costas Busch - RPI 13

Lex matches the longest input string Example: Regular Expressions Input: Matches: Fall 2006 ifend if “ifend” “if” Costas Busch - RPI “if” “ifend” 14

Internal Structure of Lex Regular expressions NFA DFA Minimal DFA The final states of the DFA are associated with actions Fall 2006 Costas Busch - RPI 15