Factor Oracle Suffix Oracle Factor Oracle Suffix Oracle

  • Slides: 22
Download presentation
Factor Oracle Suffix Oracle Factor Oracle, Suffix Oracle 1

Factor Oracle Suffix Oracle Factor Oracle, Suffix Oracle 1

Outline Factor oracle definition n Construction methods n Suffix oracle n Factor oracle for

Outline Factor oracle definition n Construction methods n Suffix oracle n Factor oracle for a set of words n Applications in string matching n Factor Oracle, Suffix Oracle 2

Data Structures that Represent the Factors of a String n n Suffix trie –

Data Structures that Represent the Factors of a String n n Suffix trie – tree representing all the suffixes of the string. Suffix automaton (DAWG) – the minimal automaton recognizing all the suffixes of the string. Figure 1. Suffix Trie of the string abbc Figure 2. Suffix Automaton of the string abbc n Both the suffix automaton and the factor oracle can be obtained from the suffix trie. Figure 3. Factor Oracle for the string abbc Factor Oracle, Suffix Oracle 3

Factor Oracle – Basic Ideas The factor oracle is a data structure used for

Factor Oracle – Basic Ideas The factor oracle is a data structure used for indexing all the factors of a given word. n An automaton built on a string p that acts like an oracle on the factors of the string. n If a string is accepted by the automaton it may be a factor of p – weak factor recognition. n All the correct factors are accepted. n Factor Oracle, Suffix Oracle 4

Factor Oracle – Example Figure 4. Factor oracle for abbbaab ba is a factor

Factor Oracle – Example Figure 4. Factor oracle for abbbaab ba is a factor of baab so a transition from 2 to 5 by a is added n n n Factor oracle for the string abbbaab. All states are considered final. The word abba is accepted although it is not a factor of abbbaab. Factor Oracle, Suffix Oracle 5

Factor Oracle – Formal Definition Figure 5. High level construction algorithm of Oracle(p). The

Factor Oracle – Formal Definition Figure 5. High level construction algorithm of Oracle(p). The algorithm has a quadratic time complexity. Definition 1. The factor oracle of a string is the automaton built by the algorithm Build_Oracle, where all the states are terminal. Factor Oracle, Suffix Oracle 6

Factor Oracle – Properties Acyclic homogenous deterministic automaton. 2. Recognizes at least the factors

Factor Oracle – Properties Acyclic homogenous deterministic automaton. 2. Recognizes at least the factors of p, the string that it was built for. 3. Has the fewest states possible (for a string p of length m there are precisely m+1 states). 4. Has a linear number of transitions (the total number ranges between m and 2 m-1). 1. Factor Oracle, Suffix Oracle 7

Factor Oracle – Construction n In the sequential construction the letters of the word

Factor Oracle – Construction n In the sequential construction the letters of the word are read from left to right and the automaton is upgraded at each step. n We denote n We define a function on the states of the automaton called supply function that maps each state i of Oracle(p) to the state j where the reading of ends. the longest suffix of that appears at least twice in it. Factor Oracle, Suffix Oracle 8

Factor Oracle – Construction Algorithm Buid_Oracle_Sequential( 1. 2. 3. 4. 5. 6. 7. 8.

Factor Oracle – Construction Algorithm Buid_Oracle_Sequential( 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. ) create initial state 0, set for i=1 to m do create new state i add a new transition from i-1 to i by set while and there is no transition from k by add new transition from k to i by set endwhile if k = -1 then set else set endfor Factor Oracle, Suffix Oracle do 9

Construction of the Factor Oracle for the string abbbaab Add a new transition from

Construction of the Factor Oracle for the string abbbaab Add a new transition from 0 to 2 by b No new transition is needed Factor Oracle, Suffix Oracle 10

Construction of the Factor Oracle for the string abbbaab Add new transitions from 3

Construction of the Factor Oracle for the string abbbaab Add new transitions from 3 and 2 to 5 by a Add new transition from 1 to 6 by a No new transition is needed Factor Oracle, Suffix Oracle 11

Suffix Oracle - Definition n We mark some states in the factor oracle for

Suffix Oracle - Definition n We mark some states in the factor oracle for the string p as final in order to recognize suffixes of p. The new structure is called suffix oracle. n A state q of the suffix oracle is terminal if and only if there is a path labeled by a suffix of p from the initial state leading to q. n Terminal states are determined by following the supply function from state m of Oracle( ). Factor Oracle, Suffix Oracle 12

Suffix Oracle – Example Figure 6. Suffix oracle for the string abbbaab. Double circled

Suffix Oracle – Example Figure 6. Suffix oracle for the string abbbaab. Double circled states are terminal. n The suffix oracle is a little more complicated to implement than the factor oracle. Also, it requires more memory space. Factor Oracle, Suffix Oracle 13

Factor Oracle for a Set of Words n n The factor oracle can be

Factor Oracle for a Set of Words n n The factor oracle can be extended for a set of words so that it contains at least all the factors of the words from the set. We set an order on the words from the set, in order to avoid the uniqueness problem. The oracle is built on a trie of all the words which is updated similarly to the factor oracle for one word. The supply function maps each state i of the oracle to the state j where the reading of the longest repeated suffix that appears in one of the words ends. Factor Oracle, Suffix Oracle 14

Factor Oracle for a Set of Words Example Figure 7. Trie for the set

Factor Oracle for a Set of Words Example Figure 7. Trie for the set {abbba, baaa} Figure 8. Intermediate phase in the construction of the factor oracle for the set {abbba, baaa} Figure 9. Factor oracle for the set {abbba, baaa} Factor Oracle, Suffix Oracle 15

Backward Oracle Matching Algorithm n Version of the BDM algorithm using the factor oracle

Backward Oracle Matching Algorithm n Version of the BDM algorithm using the factor oracle instead of the suffix automaton. n Fast in practice for very long patterns and small alphabets. n Preprocessing phase linear in time and space complexity. n Optimal on average (conjecture. ) Factor Oracle, Suffix Oracle 16

BOM – Main Idea The search uses the oracle of the reversed pattern. The

BOM – Main Idea The search uses the oracle of the reversed pattern. The search stops when the word is no longer recognized by the oracle (which shows it is certainly not a factor of the reversed pattern). The search window is shifted beyond the point the search failed (safe shift). Factor Oracle, Suffix Oracle 17

BOM – Facts n n The suffix oracle of the reversed pattern can be

BOM – Facts n n The suffix oracle of the reversed pattern can be used instead of the factor oracle. The shifts are longer but there are more operations needed. Worst case complexity of BOM is O(mn), where m is the length of the pattern, and n the total length of the text. Because the factor oracle accepts some words that are not really factors of the pattern in some cases the total number of inspections is greater than in BDM. Turbo. BOM combines BOM with KMP to obtain an algorithm linear in the worst case. Factor Oracle, Suffix Oracle 18

Factor Oracle – Applications n Finding the repeats in a string ¨ Data compression

Factor Oracle – Applications n Finding the repeats in a string ¨ Data compression ¨ Bioinformatics ¨ Machine improvisation Factor Oracle, Suffix Oracle 19

Factor Oracle – Open Problems Figure 10. The factor oracle for the string abbb

Factor Oracle – Open Problems Figure 10. The factor oracle for the string abbb accepts exactly all the factors of the string. n What is the automaton-independent characterization of the language recognized by the oracle. Factor Oracle, Suffix Oracle 20

Factor Oracle – Open Problems Figure 11. The factor oracle for the string abcacdace

Factor Oracle – Open Problems Figure 11. The factor oracle for the string abcacdace has 8 extra transitions Figure 12. A similar automaton with 7 extra transitions n The factor oracle is not the minimal homogenous automaton which recognizes at least the factors of the string. Factor Oracle, Suffix Oracle 21

References 1. Cyril Allauzen, Maxime Crochemore, Mathieu Raffinot Efficient Experimental String Matching by Weak

References 1. Cyril Allauzen, Maxime Crochemore, Mathieu Raffinot Efficient Experimental String Matching by Weak Factor Recognition in Proceedings of 12 th conference on Combinatorial Pattern Matching, 2001 2. Cyril Allauzen, Mathieu Raffinot Oracle des facteurs d’un ensemble de mots Technical report 99 -11, Institut Gaspard Monge Universite Marne la Valee, 1999 3. Loek Cleophas, Gerard Zwaan, Bruce Watson Constructing Factor Oracles in Proceedings of the Prague Stringology Conference 2003, 2003 4. Arnaud Levebvre, Thierry Lecroq Computing repeated factors with a factor oracle in Proceedings of 11 th Australian Workshop on Combinatorial Algorithms, 2000 5. G. Assayag, S. Dubnov Using Factor Oracles for Machine Improvisation Soft Computing, 2004 Factor Oracle, Suffix Oracle 22