Algorithms in the Real World Error Correcting Codes

Algorithms in the Real World Error Correcting Codes II – Cyclic Codes – Reed-Solomon Codes Page 1

Viewing Messages as Polynomials A (n, k, n-k+1) code: Consider the polynomial of degree k-1 p(x) = ak-1 xk-1 + L + a 1 x + a 0 Message: (ak-1, …, a 1, a 0) Codeword: (p(y 0), p(y 1), …, p(yn-1)) for distinct y 0, …, yn-1 To keep the p(yi) fixed size, we use yi, ai GF(pr) To make the yi distinct, n < pr Unisolvence Theorem: Any subset of size k of (p(y 0), p(y 2), …, p(yn-1)) is enough to (uniquely) reconstruct p(x) using polynomial interpolation, e. g. , La. Grange’s Formula. Page 2

Polynomial-Based Code A (n, k, 2 s +1) code: k 2 s n Can detect 2 s errors Can correct 2 s erasures Generally can correct erasures and b errors if + 2 b 2 s 2 s = n-k, so this is an (n, k, n-k+1) code Page 3

Correcting Errors Correcting s errors: 1. Find k + s symbols that agree on a polynomial p(x). These must exist since originally k + 2 s symbols agreed and only s are in error 2. There are no k + s symbols that agree on the wrong polynomial p’(x) - Any subset of k symbols will define p’(x) - Since at most s out of the k+s symbols are in error, p’(x) = p(x) Page 4

A Systematic Code Message: (m 0, m 1, …, mk-1) Find polynomial p(x) = ak-1 xk-1 + L + a 1 x + a 0 such that p(y 0)= m 0, p(y 2)= m 2, …, p(yk-1) = m 0 Codeword: (m 0, m 1, …, mk-1, p(yk), p(yk+1), …, p(yn-1)) This has the advantage that if we know there are no errors (e. g. , all points lie on the same degree-(k-1) polynomial), decoding is trivial. The version of RS used in practice uses something slightly different. This will allow us to use the “Parity Check” ideas from linear codes (i. e. , Hc. T = 0? ) to quickly test for errors. Page 5

Reed-Solomon Codes in the Real World (204, 188, 17)256 : ITU J. 83(A)2 (128, 122, 7)256 : ITU J. 83(B) (255, 223, 33)256 : Common in Practice – Note that they are all byte based (i. e. , symbols are from GF(28)). Decoding rate on 1. 8 GHz Pentium 4: – (255, 251, 5) = 89 Mbps – (255, 223, 33) = 18 Mbps Dozens of companies sell hardware cores that operate 10 x faster (or more) – (204, 188, 17) = 320 Mbps (Altera decoder) 296. 3 Page 6

Applications of Reed-Solomon Codes • • • Storage: CDs, DVDs, “hard drives”, Wireless: Cell phones, wireless links Sateline and Space: TV, Mars rover, … Digital Television: DVD, MPEG 2 layover High Speed Modems: ADSL, . . Good at handling burst errors. Other codes are better for random errors. – e. g. , Gallager codes, Turbo codes Page 7

RS and “burst” errors Let’s compare to Hamming Codes (which are “optimal”). code bits check bits RS (255, 253, 3)256 2040 16 Hamming (211 -1, 211 -11 -1, 3)2 2047 11 They can both correct 1 error, but not 2 random errors. – The Hamming code does this with fewer check bits However, RS can fix 8 contiguous bit errors in one byte – Much better than lower bound for 8 arbitrary errors Page 8

Discrete Fourier Transform (DFT) Evaluating polynomial at n points via matrix multiply: is a primitive nth root of unity ( n = 1) – a generator Evaluate polynomial mk-1 xk-1 + + m 1 x + m 0 at n distinct roots of unity, 1, , 2, 3, , n-1 DFT: Inverse DFT: FFT is fast way to compute DFT Page 9

Galois Field GF(23) with irreducible polynomial: x 3 + x + 1 = x is a generator 0 0 000 y 0 x 010 y 1 2 x 2 100 y 2 3 x+1 011 y 3 4 x 2 + x 110 y 4 5 x 2 + x +1 111 y 5 6 x 2 + 1 101 y 6 7 1 001 y 7 Will use this as an example. Page 10

DFT Example = x is 7 th root of unity in GF(23)/x 3 + x + 1 (i. e. , multiplicative group, which excludes additive inverse) Recall = y 1, 2 = y 2, … Should be clear that c = T (m 0, m 1, …, mk-1, 0, …)T is the same as evaluating p(x) = m 0 + m 1 x + … + mk-1 xk-1 at n points y 0, y 1, …, yn-1. Page 11

Decoding Why is it hard? Brute Force: try each. possibilities and solve for Page 12

Cyclic Codes A linear code is cyclic if: (c 0, c 1, …, cn-1) C (cn-1, c 0, …, cn-2) C Both Hamming and Reed-Solomon codes are cyclic. Note: we might have to reorder the columns to make the code “cyclic”. Motivation: Cyclic codes are easier to decode than general codes. Page 13

Linear Code Generator and Parity Check Matrices View message, codeword as vectors (m 0, m 1, …, mk-1) and (c 0, c 1, …, cn-1) Generator Matrix: A k x n matrix G such that: C = {m G | m åk} Made from stacking the basis vectors Parity Check Matrix: A (n – k) x n matrix H such that: C = {v ån | H v. T = 0} Codewords are the nullspace of H These always exist for linear codes H GT = 0 Page 14

RS Generator and Parity Check Polynomials View message (m 0, m 1, …, mk-1) as polynomial m 0 + m 1 x + … + mk-1 xk-1, codeword (c 0, c 1, …, cn-1) as polynomial c 0 + c 1 x + … + cn-1 xn-1 Generator Polynomial: A degree (n-k) polynomial g(x) = g 0 + g 1 x + … + gn-kxn-k such that g | xn – 1 C = {m g | m m 0 + m 1 x + … + mk-1 xk-1} Parity Check Polynomial: A degree k polynomial h(x) = h 0 + h 1 x + … + hkxk such that h | xn - 1 C = {v ån [x] | h v = 0 (mod xn – 1)} These always exist for linear cyclic codes h g = xn - 1 Page 15

Poly multiplication via matrix multiplication If g(x) = g 0 + g 1 x + … + gn-kxn-k We can put this generator in matrix form (k x n): k-1 n-k+1 k-1 Write m = m 0 + m 1 x +…+ mk-1 xk-1 as (m 0, m 1, …, mk-1) Then c = m. G Page 16

g generates cyclic codes Codes consist of all linear combinations of the rows: c = (c 0, c 1, …, cn-1) = m 0 g + m 1 xg + m 2 x 2 g+ … + mk-1 xk-1 g Claim: c’ = (cn-1, c 0, c 1, …, cn-2) is also a codeword. Right shift of every row but last is next row. Right shift of last row is xkg mod (xn – 1) = gn-k, 0, …, g 0, g 1, …, gn-k-1 Will show xkg mod (xn – 1) is a linear combination of other rows. Consider h = h 0 + h 1 x + … + hkxk (gh = xn – 1) h 0 g + (h 1 x)g + … + (hk-1 xk-1)g + (hkxk)g = xn – 1 (hkxk)g = (xn – 1) – (h 0 g + (h 1 x)g + … + (hk-1 xk-1)g ) (hkxk)g mod (xn – 1) = -(h 0 g + h 1(xg) + … + hk-1(xk-1 g)) xkg mod (xn – 1) = -hk-1(h 0 g + h 1(xg) + … + hk-1(xk-1 g)) Therefore right cyclic shift of every row is a linear Page 17 combination of other rows.

Viewing h as a matrix If h = h 0 + h 1 x + … + hkxk we can put this parity check poly. in matrix form ((n-k) x n) : n-k-1 k+1 n-k-1 Hc. T = 0 (syndrome gives coefficients of xn-1 through xk in h c, which are the same as in h c mod xn-1) Page 18

Hamming Codes Revisited The Hamming (7, 4, 3)2 code. g = 1 + x 3 h = x 4 + x 2 + x + 1 gh = x 7 – 1, GHT = 0 The columns are not identical to the previous example Hamming code. (And note that coefficients are from GF(2), but n > 2. ) Page 19

Factors of xn -1 Intentionally left blank Page 20

Another way to write g Let a be a generator of GF(pr). Let n = pr - 1 (the size of the multiplicative group) Then we can write a generator polynomial as g(x) = (x- )(x- 2) … (x - n-k), h = (x- n-k+1)…(x- n) Lemma: g | xn – 1, h | xn – 1, gh = xn – 1 (a | b means a divides b) Proof: – n = 1 (because of the size of the group) n – 1 = 0 root of xn – 1 (x - ) | xn -1 – similarly for 2, 3, …, n – therefore xn - 1 is divisible by (x - )(x - 2) … Page 21

Back to Reed-Solomon Consider a generator polynomial g GF(pr)[x], s. t. g | (xn – 1) Recall that n – k = 2 s (the degree of g is n-k, n-k+1 coefficients) Encode (trick to make code systematic): – m’ = m x 2 s (basically shift by 2 s) – b = m’ (mod g), m’ = qg + b for some q – c = m’ – b = (mk-1, …, m 0, -b 2 s-1, …, -b 0) – Note that c is a cyclic code based on g - c = m’ – b = qg (i. e. , given m we found another message q such that qg is “systematic” for m) Parity check: - hc=0? Page 22

Example Lets consider the (7, 3, 5)8 Reed-Solomon code. We use GF(23)/x 3 + x + 1 x 010 2 x 2 100 3 x+1 011 4 x 2 + x 110 5 x 2 + x + 1 111 6 x 2 + 1 101 7 1 001 Page 23

Example RS (7, 3, 5)8 n = 7, k = 3, n-k = 2 s = 4, d = 2 s+1 = 5 g = (x - )(x - 2)(x - 3)(x - 4) 010 4 3 3 2 3 = x + x + 2 100 h = (x - 5)(x - 6)(x - 7) 3 011 = x 3 + 3 x 2 + 2 x + 4 4 110 gh = x 7 - 1 5 111 Consider the message: 110 000 110 6 101 m = ( 4, 0, 4) = 4 x 2 + 4 7 001 m’ = x 4 m = 4 x 6 + 4 x 4 = ( 4 x 2 + x + 3)g + ( 3 x 3 + 6 x + 6) c = ( 4, 0, 4, 3, 0, 6) ch = 0 (mod x 7 – 1) = 110 000 110 011 000 101 Page 24

A useful theorem Theorem: For any b, if g(b) = 0 then b 2 sm(b) = b(b) Proof: x 2 sm(x) = m’(x) = g(x)q(x) + b(x) b 2 sm(b) = g(b)q(b) + b(b) = b(b) Corollary: b 2 sm(b) = b(b) for b { , 2, 3, …, 2 s=n-k} Proof: { , 2, …, 2 s} are the roots of g by definition. Page 25

Fixing erasures Theorem: Any k symbols from c can reconstruct c and hence m Proof: We can write 2 s equations involving m (cn-1, …, c 2 s) and b (c 2 s-1, …, c 0). These are 2 s m( ) = b( ) 4 s m( 2) = b( 2) … 2 s(2 s) m( 2 s) = b( 2 s) We have at most 2 s unknowns (erasures), so we can solve for them. (I’m skipping showing that the equations are linearly independent). Page 26

Efficient Decoding I don’t plan to go into the Reed-Solomon decoding algorithm, other than to mention the steps. c Syndrome Calculator Error Polynomial Error Locations Error Magnitudes Berlekamp Massy Chien Search Forney Algorithm Error Corrector This is the hard part. CD players use this algorithm. (Can also use Euclid’s algorithm. ) Page 27 m