9162020 LCS 1 v Given two strings S

9/16/2020 LCS 1

v. Given two strings S 1 of length m and S 2 of length n over the same alphabeth. The Longest Common Substring problem is to find the longest substring of S 1 that is also a substring of S 2. v. A generalization is the k-common substring problem. Given the set of strings S={S 1, S 2, ……………, Sk}. where |Si|=ni. Σ ni=N. Find for each 2 ≤ k ≤ K , the longest string which occur as substring of all strings. 9/16/2020 LCS 2

1. Brute Force Technique 2. Dynamic Programming 3. Suffix Tree 9/16/2020 LCS 3

9/16/2020 LCS 4

LCS(S[1…m-1], T[1…n-1])+1 If S[m]=T[n] LCS(S[1. . . m], T[1…n]) = 0 9/16/2020 Otherwise LCS 5

j i B A A B A B 0 0 0 0 1 0 1 0 2 0 0 0 2 0 3 0 0 0 3 0 A B Longest Common Substring 9/16/2020 A B Longest Common Substring LCS 6

$ a c # b b a b b b c c c # $ $ 9/16/2020 # $ LCS # 7

Common Sub-Strings ‘a’ ‘b’ ‘c’ ‘ab’ ‘bc’ Longest Common Sub-String ‘ab’ ‘bc’ 9/16/2020 LCS 8

LCS compares two strings and finds the longest run of characters that occurs in both of them. We can then declare the two documents as near duplicates if the ratio of the common substring length to the length of the documents exceeds some threshold. Consider the Example Below Selling a beautiful house in California. Buying a beautiful chip in California. The longest common substring is " in California. " (it is 15 characters long, whereas " a beautiful " comes in second at 13 characters long). The first string is 40 characters long. So, you could assess how similar the strings are by taking the ratio: 15/40 = 0. 375. Best part about this application is that user can decide threshold level interactively 9/16/2020 Target Audience of this Application *Ideal for Universities which do not have access to turn it in. *Students who do not have access to turn it. LCS 9

Medical record linkage is becoming increasingly important as clinical data is distributed across independent sources. Two corresponding fields within a record are said to agree only if all characters match; otherwise the fields are considered as mismatches. LCS score for the names ‘TAMMY SHACKELFORD’ ‘TAMMIE SHACKLEFORD’ The total length of the common substrings is [5 (SHACK) + 4 (TAMM) + 4 (FORD)] = 13. The length of the shorter name string (ignoring white space) is 16, therefore the LCS score is (13÷ 16) = 0. 8125 9/16/2020 LCS 10

9/16/2020 Approach Worst Case Time Complexity Brute Force O(n^3) Dynamic Programming O(m n) Suffix Array O(n log n) Suffix Tree O(n) LCS 11

Results were Measured on Intel Core i-7 2. 00 GHZ processor 4 GB Ram System Approach Time Complexity Time(ms) Basic Operations Execution Time (mille seconds) Brute Force n^3 129 Dynamic Programming m*n 68 Suffix Tree n 29 Approach Time Complexity Time(ms) Basic Operations Execution Time (mille seconds) Brute Force n^3 273 Dynamic Programming m*n 120 Suffix Tree n 60 9/16/2020 LCS 12

q In Dynamic programming following changes can be done to exiting algorithm to reduce the memory usage of an implementation : ØKeep only the last and current row of the Dynamic Programming table to save memory O(min(m, n)) instead of O(n m)). ØStore only non-zero values in the rows. This can be done using hash tables instead of arrays. This is useful for large alphabets. q. Exiting Ukkonen’s suffix-tree implementation of longest common substring problem can be modified using Mc. Creight and Weiner to see marginal improvement in time and space complexities. q. Hybrid algorithm's performance can be compared with exiting performance results and see if there is any significant change in time and space complexity using rolling hash and suffix arrays 9/16/2020 LCS 13

Longest common substring problem http: //en. wikipedia. org/wiki/Longest_common_substring_problem#See_also On–line construction of Su�x trees http: //www. cs. helsinki. fi/u/ukkonen/Suffix. T 1 with. Figs. pdf Generalized suffix tree http: //en. wikipedia. org/wiki/Generalized_suffix_tree Real World Performance of Approximate String Comparators for use in Patient Matching http: //www. cs. mun. ca/~harold/Courses/Old/CS 6772. F 04/Diary/5604 Grannis. pdf 9/16/2020 LCS 14

9/16/2020 LCS 15