Length Reduction in Binary Transforms Amihood Amir Bar

Length Reduction in Binary Transforms Amihood Amir Bar Ilan University and Johns Hopkins University Oren Kapah Ely Porat Amir Rothschild

Motivation

Error in Content: U. of Minnesota Bar Ilan University Error in Address: U. of Minnesota Bar Ilan University

Motivation: Architecture. Assume distributed memory. Our processor has text and requests pattern of length m. Pattern arrives in m asynchronous packets, of the form: <symbol, addr> Example: <A, 3>, <B, 0>, <A, 4>, <C, 1>, <B, 2> Pattern: BCBAA

Our Model… Text: T[0], T[1], …, T[n] Pattern: P[0]=<C[0], A[0]>, P[1]=< C[1], A[1]>, …, P[m]=<C[m], A[m]>; P[i] є ∑, I[i] є {1, …, m}. Standard pattern Matching: no error in A. Asynchronous Pattern Matching: no error in C. Eventually: error in both.

Register log m bits What does “bad” mean? 1. bit “flips” its value. 2. bit sometimes flips its value. 3. Transient error. “bad” bits

We will now concentrate on consistent bit flips Example: Let ∑={a, b} T[0] a T[1] a T[2] b T[3] b P[0] b P[1] b P[2] a P[3] a

Example: BAD P[0] P[1] P[2] P[3] b P[00] b P[01] a P[10] a P[11] P[00] b P[01] b P[10] a P[11] a

Example: GOOD P[0] P[1] P[2] P[3] b P[00] b P[01] a P[10] a P[11] P[00] a P[01] a P[10] b P[11] b

Example: BEST P[0] P[1] P[2] P[3] b P[00] b P[01] a P[10] a P[11] P[00] a P[01] a P[10] b P[11] b

Naïve Algorithm log m For each of the 2 = m different bit combinations try matching. Choose match with minimum bits. Time: O(m 2 ).

Approximate Pattern Matching Hamming distance: For every location, write number of mismatches Text: ABBABCBABBC Pattern: ABCBA

Approximate Pattern Matching Hamming distance: For every location, write number of mismatches Text: ABBABCBABBC Pattern: ABCBA 3

Approximate Pattern Matching Hamming distance: For every location, write number of mismatches Text: Pattern: ABBABCBABBC ABCBA 3

Approximate Pattern Matching Hamming distance: For every location, write number of mismatches Text: Pattern: ABBABCBABBC ABCBA 5

Approximate Pattern Matching Hamming distance: For every location, write number of mismatches Text: Pattern: ABBABCBABBC ABCBA 0

Approximate Pattern Matching Hamming distance: For every location, write number of mismatches Text: Pattern: ABBABCBABBC ABCBA 4 Naïve Algorithm Time: O(nm)

In Pattern Matching Polynomial Multiplication: b 0 Naïve Time: O(nm) bb 01 bb 102 b 21 b 2

What do the Two Examples have in Common? What Really Happened? P[0] P[1] P[2] P[3] 0 0 0 T[0] T[1] T[2] T[3] C[-2] C[-1] C[0] C[1] C[2] C[3] Dot products array: 0 0 0

What Really Happened? P[0] P[1] P[2] P[3] 0 0 0 T[0] T[1] T[2] T[3] C[-2] C[-1] C[0] C[1] C[2] C[3] 0 0 0

Another way of defining the transform: Where we define: P[x]=0 for x<0 and x>m.

FFT solution to the “shift” convolution: 1. Compute in time O(m log m) (values of X at roots of unity). 2. For polynomial multiplication compute values of product polynomial at roots of unity in time O(m log m). 3. Compute the coefficient of the product polynomial, again in time O(m log m).

A General Convolution Cf Bijections ; j=1, …. , O(m)

Consistent bit flip as a Convolution Construct a mask of length log m that has 0 in every bit except for the bad bits where it has a 1. Example: Assume the bad bits are in indices i, j, k є{0, …, log m}. Then the mask is i j k 00000100001000 An exclusive OR between the mask and a pattern index Gives the target index.

Example: Mask: 0010 Index: 1010 1000 Index: 1000 1010

Our Case: Denote our convolution by: Our convolution: For each of the 2 let jє{0, 1}log m =m masks,

To compute min bit flip: Let T, P be over alphabet {0, 1}: For each j, is a permutation of P. Thus, only the j ’s for which are valid flips. = number of 1 ‘s in T Since for them all 1’s match 1’s and all 0’s match 0’s. Choose valid j with minimum number of 1’s.

Time All convolutions can be computed in time O(m 2) After preprocessing the permutation functions as tables. Can we do better? (As in the FFT, for example)

Idea – Divide and Conquer. Walsh Transform 1. Split T and P to the length m/2 arrays: 2. Compute 3. Use their values to compute 4. in time O(m). 5. Time: Recurrence: t(m)=2 t(m/2)+m 6. 7. Closed Form: t(m)=O(m log m)

Sparse Transform Applications where most of the input is 0. The locations where there are “ 1”s are given as inputs We are only interested in the transform results for the locations where all pattern “ 1”s match text “ 1”s.

Motivation – Point Set Matching 1 -D Point Set Matching: T: (t 1, t 2, …, tn) P: (p 1, p 2, …, pm) 2 -D Point Set Matching – Searching in Music:

Notations: Length of text: Length of Pattern: N M Number of “ 1”s in text: n Number of “ 1”s in pattern: m.

Idea: Map text and pattern to small text and pattern: Hash function h

Idea: Do fast transform on the small text and pattern

Idea: Map results onto transform result of original text and pattern. h-1

Length Reduction in DFT Goal: Given two vectors V 1&V 2, obtain two vectors V’ 1&V’ 2 of size O(n’) such that all non-zero in V 1 and in V 2 will appear as singletons respectively while maintaining the distance property. The Distance Property: If V’ 2[h(0)] is aligned with V’ 1[h(i )], then is aligned with V’ 1[h(fi (j))] = V’ 1[f(i +j)]. V’ 2[h(j)] Using the reduced size vectors, matching can be done in time O(n’ log n’) using the FFT algorithm.

Example: Length Reduction The vectors are given as sets of pairs: (index, value) V 1: (0, 5), (6, 2), (13, 3), (19, 1) V 2: (0, 2), (7, 3) Length Reduction Hash Function: mod(5) V’ 1: 5 2 0 3 1 V’ 2: 2 0 3 0 0

The Randomized Algorithm of Cole & Hariharan [STOC 02] Idea: Find a set of log(n) short vectors, in which with high probability, each non-zero in V, appears as a singleton in at least one of the vectors. Hash functions: (ax mod(q))mod(s). Where q is a large prime number, and s is O(n). If s is c·n, then the probability of a non-zero appearing as a multiple is constant. Using log(n) different hash functions will reduce the failure probability exponentially.

Problem For the Walsh Transform, the mod function is useless. The distance property has to do with exclusive or, not addition!

IDEA Instead of the modulo function Do an exclusive or( ) of the index bits with a random bit string.

Example address Text Pattern 000 001 010 0 1 0 0 011 0 0 100 0 0 101 0 0 110 1 0 Let’s do the Walsh Transform 111 0 1

Location 000 address Text address Pattern Dot Product 000 001 010 0 1 0 001 010 1 0 011 0 100 0 101 0 110 1 110 0 111 1

Location 001 -XOR address Text address Pattern Dot Product 000 001 010 011 100 101 0 0 0 0 001 000 011 010 101 100 1 0 0 0 110 1 111 0 110 1

Location 001 -dot product address Text address Pattern Dot Product 000 001 010 011 100 101 0 0 0 0 001 000 011 010 101 100 1 0 0 0 2 110 1 111 0 110 1

Location 010 -XOR address Text address Pattern Dot Product 000 001 010 011 100 0 1 0 010 011 000 001 110 1 0 0 0 2 101 0 110 1 100 0 111 0 101 1

Location 010 -dot product address Text address Pattern Dot Product 000 001 010 011 100 0 1 0 010 011 000 001 110 1 0 0 0 2 0 101 0 110 1 100 0 111 0 101 1

Location 011 -XOR address Text address Pattern Dot Product 000 001 010 011 100 0 1 0 011 010 001 000 111 1 0 0 0 2 0 101 0 110 1 101 0 111 0 100 1

Location 011 -dot product address Text address Pattern Dot Product 000 001 010 011 100 0 1 0 011 010 001 000 111 1 0 0 0 2 0 0 101 0 110 1 101 0 111 0 100 1

Location 100 -XOR address Text address Pattern Dot Product 000 001 010 0 100 101 110 1 0 0 0 2 0 011 100 101 110 0 1 111 000 001 010 0 0 111 0 011 1

Location 100 -dot product address Text address Pattern Dot Product 000 001 010 0 100 101 110 1 0 0 0 2 0 011 100 101 110 0 1 111 000 001 010 0 0 0 111 0 011 1

Location 101 -XOR address Text address Pattern Dot Product 000 001 010 0 101 100 111 1 0 0 0 2 0 011 0 110 0 0 101 110 0 0 1 000 011 0 0 111 0 010 1

Location 101 -dot product address Text address Pattern Dot Product 000 001 010 0 101 100 111 1 0 0 0 2 0 011 0 110 0 0 101 110 0 0 1 000 011 0 0 0 111 0 010 1

Location 110 -XOR address Text address Pattern Dot Product 000 001 010 0 110 111 100 1 0 0 0 2 0 011 0 101 0 0 100 0 010 0 0 101 110 111 0 011 000 001 0 0 1 0

Location 110 -dot product address Text address Pattern Dot Product 000 001 010 0 110 111 100 1 0 0 0 2 0 011 0 101 0 0 100 0 010 0 0 101 110 111 0 011 000 001 0 0 1 0 2

Location 111 -XOR address Text address Pattern Dot Product 000 001 010 011 100 101 110 111 0 0 0 0 111 110 101 100 011 010 001 000 1 0 0 0 1 0 2 0 0 2

Location 111 -dot product address Text address Pattern Dot Product 000 001 010 011 100 101 110 111 0 0 0 0 111 110 101 100 011 010 001 000 1 0 0 0 1 0 2 0

The Length Reduction Reduce the length by half. Choose a mask of log n - 1 bits at random, add to it a MSB 1, and XOR it with each index in the second half. This will randomly hash all 1’s in the second half to the first half.

Length Reduction - Example address Text Pattern 000 001 010 0 1 0 0 011 0 0 100 0 0 101 0 0 110 1 0 111 0 1 011 000 011 010 Let mask be 101 address 000 001 010 Text 0 1 0 Pattern 1 0 0 0 1

Reduced Strings - Example address Text Pattern 000 001 010 0 1 0 0 011 0 0 mask is 101 address 000 001 010 011 Text 0 1 Pattern 1 0 100 0 0 101 0 0 110 1 0 111 0 1

Reduced Strings - Example Walsh Transform of reduced string: address 000 001 010 011 Text 0 1 Pattern 1 0 0 1 0 Walsh transform

Reduced Strings - Example Walsh Transform of reduced string: address 000 001 010 011 Text 0 1 Pattern 1 0 0 2 1 0 Walsh transform

Reduced Strings - Example Walsh Transform of reduced string: address 000 001 010 011 Text 0 1 Pattern 1 0 0 2 1 0 0 Walsh transform

Reduced Strings - Example Walsh Transform of reduced string: address Text Pattern Walsh transform 000 001 010 Questions: 011 1. Does the 0 1 distance property 1 0 hold? 2. 0 2 Which of these results is “legal”? 3. Where should it be mapped?

Answers: Distance property The Distance Property: If T[h(0)] is aligned with P[h(i )], then T[h(j)] is aligned with P[h(fi (j))] = P[f(i j)]. Holds because both h and f are XOR functions and because of the commutativity and associativity of XOR.

Answers: which are “legal”? address Text Pattern 000 001 010 0 1 0 0 011 0 0 100 0 0 101 0 0 110 1 0 111 0 1 011 000 011 010 Let mask be 101 address 000 001 010 Text 0 1 0 Pattern 1 0 0 0 1

Answers: which are “legal”? address Text Pattern 000 001 010 0 1 0 0 011 0 0 mask is 101 address Text Pattern 000 001 010 011 0 100 0 0 101 0 0 110 1 0 111 0 1 Original tenants Johnny-come-latelies

Answers: which are “legal”? address Text Pattern 000 001 010 0 1 0 0 011 0 0 mask is 101 address Text Pattern 000 001 010 011 0 s 0 m 0 100 0 0 101 0 0 110 1 0 111 0 1 Original tenants Johnny-come-latelies

Answers: which are “legal”? address Text Pattern 000 001 010 0 1 0 0 011 0 0 mask is 101 address 000 001 010 011 Text 0 s 0 m Pattern s 0 m 0 100 0 0 101 0 0 110 1 0 111 0 1 Observation: Legal multiplications are: all s in text by s in pattern and m in text by m in pattern or all s in text by m in pattern and m in text by s in pattern.

Answers: which are “legal”? Observation: Legal multiplications are: 1. all s in text by s in pattern and m in text by m in pattern or 2. all s in text by m in pattern and m in text by s in pattern. This can be checked by a constant number of binary DWT’s with an added benefit: 1. means result stays. 2. means result is moved to its address XOR with the mask.

Reduced Strings - Example Walsh Transform of reduced string: address 000 001 010 011 Text 0 1 Pattern 1 0 0 2 1 0 Walsh transform Result in correct address

Reduced Strings - Example Walsh Transform of reduced string: address 000 001 010 011 Text 0 1 Pattern 1 0 0 2 Walsh transform Result belongs in address 011 101 = 110

Reminder – the dot product address Text address Pattern Dot Product 000 001 010 011 100 101 110 111 0 0 0 0 111 110 101 100 011 010 001 000 1 0 0 0 1 0 2 0

Analysis: We could continue this process recursively and analyze probability of clash of masked element with an element that is already there but… More elegant solution:

Polynomials over a finite field Consider indices as elements in F 2 L In F 2 L: x+y = x y. . Length Reduction: Every index in F 2 L is written as a polynomial in F 2ℓ[X] of degree d = L/ℓ - 1.

Length reduction example: Index = 17 In binary = 10001 Take ℓ=2 10001 The polynomial: 1·X 2 + 00·X + 01· 1= X 2+1 Choose a value for X from F 2ℓ at random and evaluate the polynomial.

Length reduction example: Recall that in F 2ℓ: addition is exclusive or multiplication is polynomial multiplication modulo some irreducible polynomial. So, evaluating a polynomial at X gives a number with ℓ bits.

Probability of Collision: Probability of collision of index i and j = Probability that the chosen value of X is the root of the difference polynomial Pi(x)-Pj(x) Where Pi, Pj are the polynomials of index i and j, resp. Degree of difference polynomial = d So probability= d/2ℓ.

Moral of the Story: Polynomials are a good candidate for locality preserving length reductions for discrete transforms.

The End