DSP Chapter7 Recursive Least Squares Algorithms Marc Moonen

DSP Chapter-7 : Recursive Least Squares Algorithms Marc Moonen Dept. E. E. /ESAT-STADIUS, KU Leuven marc. moonen@esat. kuleuven. be www. esat. kuleuven. be/stadius/

Part-III : Optimal & Adaptive Filters Chapter-6 Wieners Filters & the LMS Algorithm • • Introduction / General Set-Up Applications Optimal Filtering: Wiener Filters Adaptive Filtering: LMS Algorithm Chapter-7 – Recursive Least Squares Algorithms • • Least Squares Estimation Recursive Least Squares (RLS) Square Root Algorithms Fast RLS Algorithms DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 2 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 3 / 32

u[k] e[k] DSP 2016 / Chapter-7: Recursive Least Squares Algorithms y[k] d[k] PS: Shorthand notation uk=u[k], yk=y[k], dk=d[k], ek=e[k], Filter coefficients (‘weights’) are wl (replacing bl of previous chapters) For adaptive filters wl also have a time index wl[k] Least Squares & RLS Estimation 4 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 5 / 32

Least Squares & RLS Estimation uk dk ek dk u k. T DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 6 / 32

Least Squares & RLS Estimation ‘Normal equations’ (L+1 equations in L+1 unknowns) DSP 2016 / Chapter-7: Recursive Least Squares Algorithms This is the ‘Least Squares Solution’ 7 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 8 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 9 / 32

Least Squares & RLS Estimation Matrices and vectors now with time index added k Can LS solution @ time k be computed from solution @ time k-1 ? DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 10 / 32

Least Squares & RLS Estimation-1 nk s a R ate d up (check ‘matrix inversion lemma’ in Wikipedia) With this it is proved that: DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 11 / 32

Least Squares & RLS Estimation : Goal is to give a smaller weight to ‘older’ data, i. e. Which leads to… DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 12 / 32

Least Squares & RLS Estimation i. e. exponential weighting hardly changes RLS formulas. . (easy!) DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 13 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 14 / 32

Least Squares & RLS Estimation square * rectangular * square DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 15 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 16 / 32

(**) orthogonal transformation preserves norm Least Squares & RLS Estimation triangular backsubstitution This is a numerically better way of computing the LS solution, better than DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 17 / 32

Least Squares & RLS Estimation k-1 DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 18 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 19 / 32

Least Squares & RLS Estimation =‘triangular backsubstitution’ DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 20 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 21 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 22 / 32

Least Squares & RLS Estimation 3 -by-3 example [k] DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 23 / 32

Least Squares & RLS Estimation u[k-1] u[k-2] u[k-3] d[k] 4 -by-4 example u[k] DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 24 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 25 / 32

Least Squares & RLS Estimation u[k] u[k-1] u[k-2] u[k-3] d[k] DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 26 / 32

Least Squares & RLS Estimation DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 27 / 32

Fast Recursive Least Squares Algorithms DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 28 / 32

Fast Recursive Least Squares Algorithms 31) DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 29 / 32

See p. 31 for a signal flow graph of this Fast Recursive Least Squares Algorithms DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 30 / 32

Least Squares & RLS Estimation See p. 31 for a signal flow graph of this Derivation omitted… DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 31 / 32

Fast Recursive Least Squares Algorithms DSP 2016 / Chapter-7: Recursive Least Squares Algorithms 32 / 32