MITSUBISHI ELECTRIC RESEARCH LABORATORIES Cambridge Massachusetts ML decoding

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Cambridge, Massachusetts ML decoding via mixed-integer adaptive linear programming Stark Draper (MERL & UW Madison) Jonathan Yedidia (MERL) Yige Wang (U Hawaii Manoa)

MITSUBISHI ELECTRIC RESEARCH LABORATORIES BSC results for (155, 64) LDPC: BP, LP, ML low medium high noise 10 103 105 MITSUBISHI ELECTRIC 2 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Agenda Linear programming (LP) decoding – cf. Feldman Ph. D thesis ’ 03; Feldman, Wainwright, Karger IT ‘ 05 Adaptive LP (ALP) decoding – cf. Taghavi and Siegel ISIT ‘ 06 Mixed-integer ALP Experimental setup Results for (155, 64) LDPC – cf. Tanner, Sridhara, Fuja ICSTA ‘ 01 MITSUBISHI ELECTRIC 3 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES ML decoding is a LP with integer constraints ML decoding: Linear cost w/ integer (codeword) constraints MITSUBISHI ELECTRIC 4 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Relax integer constraints to make tractable ML cost function: Relax to non-integer values: Need even parity: • For every check j • For every odd-sized subset a of neighbors of check j 1. Decoding performance similar to BP 2. There are 5 odd-sized subset (a lot) MITSUBISHI ELECTRIC Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES In adaptive LP add only violated constraints Don’t start with all constraints per check Start with “hard-decoding” set: Identify violated constraints, add to set, re-solve Taghavi & Siegel show (i) at most one violated constraint per check node per iteration, and (ii) it’s easy to find Initialize Perform LP decoding Find all violated constraints of current solution True 6 If one or more violated constraints are found, add to constraints Codeword False MITSUBISHI ELECTRIC Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES All-integer LP optimum is ML & LP tightenings If LP optimum all-integer then is equal to ML solution If LP optimum non-integer then “pseudo-codeword” found In latter case can tighten relaxation & resolve, e. g. , – add redundant parity-checks, RPCs, Feldman et al. ; Taghavi et al. – add integer constraints to certain bits, Yang, Feldman, Wang JSAC ’ 06 We tighten by adding to the symbol s. t. We then re-run ALP with the updated constraints MITSUBISHI ELECTRIC 7 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES ML decoding via mixed-integer ALP Codeword ALP decoder (possibly w/ integer constraints) Yes Initialize Constraint set Update constraint set Perform LP decoding Find all violated constraints of current solution Additional constraints Any violated constraints found? Yes No Is ALP solution integral? No Identify I *, and add to the constraint set MITSUBISHI ELECTRIC 8 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Experimental Setup Tested algorithm on (N=155, k=64, d=20) LDPC code • cf. Tanner, Sridhara and Fuja, ICSTA ’ 01 • excellent minimum distance (d=20) for block-length • pseudo-codewords impair performance of LP or BP Results for a BSC • ML decoding succeeds if bit flips • simulated until got 200 ML errors at fixed number bit flips from 23 bit flips (~capacity) down to 11 • estimated ML error rate at 10 bit flips • calculated WER as function of Pr [bit flip] MITSUBISHI ELECTRIC 9 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Raw results for fixed # bit flips Estimate ML performance for BSC by weighting by probability MITSUBISHI of seeing each # of flips – get down to lower WERs ELECTRIC 10 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES BSC results for BP, LP, ML low medium high noise 10 capacity 103 105 MITSUBISHI ELECTRIC 11 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Distribution of # of integer constraints required Integer constraints slow LP solver considerably Most applicable when few integer constraints 12 MITSUBISHI ELECTRIC Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES At high SNR negligible increase in decoding time Sample average decoding times Number bit flips 12 14 16 LP decoding (sec) 0. 12 0. 15 0. 23 ML decoding (sec) 0. 14 0. 22 0. 87 Performed simulations on 3 GHz Intel processors Used GNU Linear Programming Kit MITSUBISHI ELECTRIC 13 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES In paper we provide more statistics Average, median, and maximum ML decoding times – also classified by whether ML decoding is correct MITSUBISHI ELECTRIC 14 Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Conclusions & future directions Combine adaptive LP decoding with suitably selected integer constraints to get ML decoder for moderate block lengths Even if resulting ML decoding too slow for application, provides benchmark of optimal decoder performance Future directions: • more intelligent choice of integer constraints • hybrid BP/LP approach • compare to other tightenings Caveat: • While ALP decoders work on high density parity-check codes, our mixed-integer ALP decoder becomes intolerably MITSUBISHI slow. E. g. , it appears useless for BCH codes. ELECTRIC 15 Changes for the better