Algorithms Lecture 9 Divideandconquer Computational arithmetic Fast exponentiation

Algorithms Lecture 9

Divide-and-conquer: Computational arithmetic

Fast exponentiation • Cryptography deals with huge numbers • Frequent operation: compute an mod b • Naïve algorithm: O(n) – n ≈ 22000 is typical (!) – Linear-in-n time is horrible here… – We want an algorithm that is polynomial in the length of n, i. e. , an O(polylog n)-time algorithm

Divide-and-conquer algorithm • Exp(a, n, b) – If n=1 return a mod b – Else • Tmp 1 = Exp(a, n/2 , b) • Tmp 2 = Exp(a, n/2 , b) • Return Tmp 1 * Tmp 2 mod b • T(n) ≤ 2 T(n/2) + c • T(n) = O(n) – No better than the naïve algorithm…

Divide-and-conquer algorithm • Exp(a, n, b) – If n=1 return a mod b – Else if n even • Tmp = Exp(a, n/2 , b) • Return Tmp * Tmp mod b – Else if n odd • Tmp = Exp(a, (n-1)/2, b) • Return a * Tmp mod b • T(n) ≤ T(n/2) + c • T(n) = O(log n) – Huge improvement!

Big-integer arithmetic • What is the complexity of basic arithmetic? • Motivations – Numbers in crypto can be huge, e. g. , 1000+ bits – Fundamental problem… • How long does it take to add two k-bit (positive) integers? • Is this optimal?

Multiplication • How long does it take to multiply two k-bit integers? • “Grade-school algorithm” takes time O(k 2) • Is this optimal?

Karatsuba’s algorithm • Idea: note that x y = (x 1 2 k/2 + x 0)(y 1 2 k/2 + y 0) = x 1 y 1 2 k + (x 1 y 0 + x 0 y 1) 2 k/2 + x 0 y 0 – Note that bit-shifts are easy… • If we use 4 multiplications of k/2 -bit integers we haven’t gained anything… • Instead, compute x 1 y 1, x 0 y 0, and (x 0+x 1)(y 0+y 1) – Then x 1 y 0 + x 0 y 1 = (x 0+x 1)(y 0+y 1) – x 1 y 1 – x 0 y 0 (!) – Only 3 multiplications needed! • T(k) = 3 T(k/2) + ck T(k) = O(klog 2 3) = O(k 1. 59)

Integer multiplication • Is this optimal? • Breakthrough [2019]: O(k log k)-time algorithm!

Matrix multiplication • Let A, B be n n matrices • What is the complexity of computing AB? – Measure in terms of number of multiplications • Straightforward algorithm uses O(n 3) time

Matrix multiplication •

Strassen’s algorithm • Compute – M 1 = (A 1, 1 + A 2, 2)(B 1, 1 + B 2, 2) – M 2 = (A 2, 1 + A 2, 2)B 1, 1 – M 3 = A 1, 1(B 1, 2 – B 2, 2) – M 4 = A 2, 2(B 2, 1 – B 1, 1) – M 5 = (A 1, 1 + A 1, 2)B 2, 2 – M 6 = (A 2, 1 – A 1, 1)(B 1, 1 + B 1, 2) – M 7 = (A 1, 2 – A 2, 2)(B 2, 1 + B 2, 2) • (7 recursive calls…)

Strassen’s algorithm • Then compute – C 1, 1 = M 1 + M 4 – M 5 + M 7 – C 1, 2 = M 3 + M 5 – C 2, 1 = M 2 + M 4 – C 2, 2 = M 1 – M 2 + M 3 + M 6 • T(n) = 7 T(n/2) + c n 2 T(n) = O(nlog 2 7) = O(n 2. 808)

Matrix multiplication • Is this optimal? • Current best: O(n 2. 3729) • Lower bound: (n 2)

Fast Fourier Transform (FFT)

Convolutions • Problem: given (a 0, …, an-1) and (b 0, …, bn-1), compute (c 0, …, c 2 n-2), where ck = i+j=k aibj • What is the complexity for computing this? – Measure in terms of number of multiplications • Naïve algorithm runs in O(n 2) time – Each ck can be computed in O(n) time – Output vector c has O(n) coordinates

Convolutions • Why would we want to do this? – It is exactly polynomial multiplication – Also used in signal processing, cryptography, statistics, … • From now on, we treat a, b, c as polynomials – E. g. , a(X) = a 0 + a 1 X + a 2 X 2 + … + an-1 Xn-1 – a, b are degree-(n-1) polynomials, and c is a degree-(2 n-2) polynomial

FFT idea 0 • Recall that a degree-(n-1) polynomial p(X) is defined by its value at n points – I. e. , for any distinct w 0, w 1, …, wn-1, the values p(w 0), p(w 1), …, p(wn-1) uniquely define a degree-(n-1) polynomial p(X) • If we fix some distinct {wi}, we have two ways to represent a degree-(n-1) polynomial: (p 0, p 1, …, pn-1) or (p(w 0), p(w 1), …, p(wn-1)) – “Coefficient” and “evaluation” representations

FFT idea 1 • Given the eval. representations of a, b (as degree-(2 n-2) polynomials), we can compute the evaluation representation of c in O(n) time • Simply compute c(w 0) = a(w 0) b(w 0), …, c(w 2 n-2) = a(w 2 n-2) b(w 2 n-2)

FFT idea 1 • The overall structure of the algorithm will be: 1. Convert a, b to their eval representations 2. Compute eval representation of c 3. Convert c to its coefficient representation • We have seen how to do step #2 in O(n) time

FFT idea 2 • Converting a(X) to its eval representation requires evaluating a(X) at O(n) points – Evaluating a(X) at a point takes O(n) time • Observation: we can simultaneously evaluate a(X) at O(n) points in O(n log n) time if we pick the points cleverly – In particular, use complex numbers – (Nothing so far assumed real numbers)

FFT idea 2 • Consider the equation X 2 n – 1 = 0 – This has 2 n distinct complex roots – The roots are 1, e 2 i/2 n, (e 2 i/2 n)2, …, (e 2 i/2 n)2 n-1 (i. e. , the (2 n)th roots of unity), which are equally spaced on the unit circle in the complex plane – Let wk, j = (e 2 i/k)j // this is different from the book(!) • Square of a (2 n)th root of unity is an nth root of unity – We will evaluate on w 2 n, 0, w 2 n, 1, …, w 2 n, 2 n-1 using divide-and-conquer

FFT idea 2 • Eval(a) // evaluate a degree-(2 n-1) polynomial on the (2 n)th roots of unity; assume n is a power of 2 – Let aeven(X) = a 0 + a 2 X + a 4 X 2 + … + a 2 n-2 Xn-1 aodd(X) = a 1 + a 3 X + a 5 X 2 + … + a 2 n-1 Xn-1 // Note that a(X) = aeven(X 2) + X aodd(X 2) – Call Eval(aeven) and Eval(aodd) • We get aeven and aodd evaluated on the nth roots of unity – a(w 2 n, i) = aeven(w 22 n, i) + w 2 n, i aodd(w 22 n, i) = aeven(wn, i) + w 2 n, i aodd(wn, i) • T(n) = 2 T(n/2) + c n T(n) = O(n log n)

FFT idea 3 • We now have the eval. representation of c – c(w 2 n, 0), …, c(w 2 n, 2 n-1) – Define di = c(w 2 n, i) • Want to convert to coefficient representation c(X) = c 0 + c 1 + … + c 2 n-1 X 2 n-1 – This is just polynomial interpolation – But we want to do it in O(n log n) time • Idea: view the {di} as coefficients of a polynomial d(X) and evaluate it at the (2 n)th roots of unity(!!)

Lemma •

FFT idea 3 • Let d(X) = d 0 + d 1 X + … + d 2 n-1 X 2 n-1, where di = c(w 2 n, i) for some c(X) = c 0 + c 1 X + … + c 2 n-1 X 2 n-1 • Claim: for j=1, …, 2 n, d(w 2 n, j) = 2 n c 2 n-j • Proof: d(w 2 n, j) = s ds ws 2 n, j = s t ct wt 2 n, s ws 2 n, j = t ct ( s ws 2 n, t+j) // term in parentheses is 0 unless w 2 n, t+j = 1 (i. e. , unless t=2 n-j), in which case it equals 2 n = 2 n c 2 n-j

(Small) example • a(X)=3+2 X, b(X) = -1 + 3 X • 4 th roots of unity are {1, i, -1, -i} (in that order) • a(1) = 5; a(i) = 3+2 i; a(-1) = 1; a(-i) = 3 -2 i b(1) = 2; b(i) = -1+3 i; b(-1) = -4; b(-i)=-1 -3 i • So c(1) = 10; c(i) = -9+7 i; c(-1) = -4; c(-i) = -9 -7 i • Set d(X) = 10 + (-9+7 i)X – 4 X 2 – (9+7 i)X 3 • d(i)/4 = 0 = c 3 ; d(-1)/4 = 6 = c 2; d(-i)/4 = 7 = c 1; d(1)/4 = -3 = c 0 • Check: -3 + 7 X + 6 X 2 = a(X) b(X)