DSPCIS PartIV Filter Banks TimeFrequency Transforms Chapter13 Frequency
DSP-CIS Part-IV : Filter Banks & Time-Frequency Transforms Chapter-13 : Frequency Domain Filtering Marc Moonen Dept. E. E. /ESAT-STADIUS, KU Leuven marc. moonen@kuleuven. be www. esat. kuleuven. be/stadius/
Part-IV : Filter Banks & Time-Frequency Transforms Chapter-11 Filter Bank Preliminaries Chapter-12 Filter Bank Design Chapter-13 Frequency Domain Filtering • Frequency Domain FIR Filter Realization • Overlap-Add & Overlap-Save Revisited • Frequency Domain Adaptive FIR Filters • Background: Block-LMS • FDAF • PB-FDAF Chapter-14 Time-Frequency Analysis & Scaling DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 2 / 40
5 r e t p a h C Frequency Domain FIR Filter Realization =Construct (realize) LTI system (with delay elements, adders and multipliers), such that I/O behavior is given by. . o t n r u t e R er t l i 1 f + f L r o of e mb tead u , n ins s) e nc w L ula e i o m n n e or f s v i on nts sier c r a e Fo ffici (…e e co Several possibilities exist… 1. Direct form 2. Transposed direct form 3. Lattice realization (LPC lattice) 4. Lossless lattice realization 5. Frequency domain realization: see Part IV DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 3 / 40
y l l Frequency Domain FIR Filter Realization a i c e p L s e e g n FIR Filter Realization r o a i l t ) a =Construct (realize) LTI system (with idelay y z elements, r l e a v adders and multipliers), such that I/O behavior is given by. . e ( r. r e. e i t fil ers, IR filt F n i R Several possibilities exist… I a F m g o 1. Direct form n d o l y c fordirect form 2. Transposed n e t u 3. Lattice realization (LPC lattice) n q a e ev r F 4. Lossless lattice realization l e r 5. Frequency domain realization: see Part IV DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 4 / 40
Overlap-Add & Overlap-Save Have to know a theorem from linear algebra here: • A `circulant’ matrix is a matrix where each row is obtained from the previous row using a right-shift (by 1 position), the rightmost element which spills over is circulated back to become the leftmost element • The eigenvalue decomposition of a circulant matrix is always given as. . . (4 x 4 example) with F the DFT-matrix. This means that the eigenvectors are equal to the column-vectors of the IDFT-matrix, and that then eigenvalues are obtained as the DFT of the first column of the circulant matrix (proof by Matlab) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 5 / 40
Overlap-Add & Overlap-Save FIR Filter Realization (example L=4, similar for other L) Consider a 'block processing’ where a block of LB output samples are computed at once, with ‘block length’ LB=L: DSP-CIS 2020 -2021 / Chapter-13: Domain Filtering PS: formulas given for L=LFrequency B=4, for conciseness (but without loss of generality) 6 / 40
Overlap-Add & Overlap-Save Now some matrix manipulation… DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 7 / 40
Overlap-Add & Overlap-Save • This means that a block of LB=L output samples can be computed as follows (read previous formula from right to left) : – Compute DFT of 2 L-vector with input samples, i. e. last L samples combined (‘overlapped’) with previous L samples – Perform component-wise multiplication with… (=freq. domain representation of the FIR filter) ✪ – Compute IDFT – Throw away 1 st half of result, select (‘save’) 2 nd half • This is referred to as an ‘overlap-save’ procedure (and ‘frequency domain filter realization’ because of the DFT/IDFT) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 8 / 40
Overlap-Add & Overlap-Save • This corresponds to a filter bank-type realization as follows. . . u[k] 4 4 4 4 + y[k] Analysis bank: Subband processing: Synthesis bank: This is a 2 L-channel filter bank, with L-fold downsampling The analysis FB is a 2 L-channel uniform DFT filter bank (see Chapter-12, p. 29 -32) The synthesis FB is matched to the analysis bank, for PR: DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 9 / 40
Overlap-Add & Overlap-Save • PS: Note that in analysis filter bank prototype filter is . . . whereas synthesis filter bank prototype filter is Both are lowpass filters, but bandwidth of F 0(z) is twice bandwidth of H 0(z) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 10 / 40
Overlap-Add & Overlap-Save • PS: Note that in p. 9 if. . . B(z)=1 (b 0=1, b 1=b 2=b 3=0) then. . . Y(z)=z-3. U(z) (as in Chapter-11, p. 24) with (check!) (see ✪ p. 8) • PS: Note that in p. 9 if. . . then. . . Y(z)=z-7. U(z)=z-3. B(z). U(z) with B(z)=z-4 (b 0=b 1=b 2=b 3=0, b 4=1) (b 4 to be added in derivation p. 5 -7) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering (check!) 11 / 40
Overlap-Add & Overlap-Save Derivation on p. 7 can also be modified as follows…… with ’relaxed’ notation z-1. u[k]=u[k-1] DSP-CIS 2020 -2021 / Chapter-13: Domain Filtering PS: formulas given for L=LFrequency B=4, for conciseness (but without loss of generality) 12 / 40
Overlap-Add & Overlap-Save • This means that a block of LB=L output samples can be computed as follows (read previous formula from right to left) : – Compute DFT of 2 L-vector with L input samples & L zero samples – Perform component-wise multiplication with… (=freq. domain representation of the FIR filter) ✪ – Compute IDFT – Use 2 nd half of result, add 1 st half of result from previous IDFT • This is referred to as an ‘overlap-add’ procedure (and ‘frequency domain filter realization’ because of the DFT/IDFT) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 13 / 40
Overlap-Add & Overlap-Save • This corresponds to a filter bank-type realization as follows. . . u[k] 4 4 4 4 + y[k] Analysis bank: Subband processing: Synthesis bank: This is a 2 L-channel filter bank, with L-fold downsampling The analysis FB is a 2 L-channel uniform DFT filter bank (see Chapter-12, p. 29 -32) The synthesis FB is matched to the analysis bank, for PR: DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 14 / 40
Overlap-Add & Overlap-Save • Overlap-save/-add procedure is very efficient for large L=LB – Computational complexity (with FFT/IFFT i. o. DFT/IDFT) is 2. [α. 2 L. log(2 L)] + 2 L multiplications for L output samples, i. e. O(log(L)) per sample for large L – Compare to computational complexity for direct form realization: L multiplications per output sample, i. e. O(L) per sample • Overlap-save/-add procedure introduces O(L)=O(LB) processing delay/latency (e. g. y[k-L+1] only available sometime after time k) . . . which may not be tolerable DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 15 / 40
Overlap-Add & Overlap-Save • Conclusion: For large L, complexity reduction is large, but latency is also large • Therefore, in the next slides, will derive ‘intermediate’ realizations, with a smaller latency at the expense of a smaller complexity reduction. This will be based on a polyphase decomposition of B(z) … DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 16 / 40
Part-IV : Filter Banks & Time-Frequency Transforms Chapter-11 Filter Bank Preliminaries Chapter-12 Filter Bank Design Chapter-13 Frequency Domain Filtering • Frequency Domain FIR Filter Realization • Overlap-Add & Overlap-Save Revisited • Frequency Domain Adaptive FIR Filters • Background: Block-LMS • FDAF • PB-FDAF Chapter-14 Time-Frequency Analysis & Scaling DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 17 / 40
Overlap-Add & Overlap-Save Revisited A compact derivation will rely on a result from filter bank theory ( return to Chapter-11…) u[k] 4 4 4 4 + T(z)*u[k-3] ( …and now let B(z) take the place of ‘distortion function’ T(z)) This means that a filter (specified with D-fold polyphase decomposition) can be realized in a multirate structure, based on a pseudo-circulant matrix DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering (but without loss of generality) PS: formulas given for D=4, for conciseness 18 / 40
Overlap-Add & Overlap-Save Revisited Now some matrix manipulation… (compare to p. 6) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 19 / 40
p. to Overlap-Add & Overlap-Save Revisited e r a p m o C • A filter bank representation of this is. . . u[k] 4 4 4 4 + y[k] Analysis bank: Subband processing: Synthesis bank: This is a 2 D-channel filter bank, with D-fold downsampling The analysis FB is a 2 D-channel DFT filter bank (see Chapter-11) The synthesis FB is matched to the analysis bank, for PR: DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 20 / 40
Overlap-Add & Overlap-Save Revisited • This is again known as an `overlap-save’ realization : – Analysis bank: performs 2 D-point DFT (FFT) of a block of D samples, together with the previous block of D samples (hence `overlap’) `block’ `previous block’ – Synthesis bank: performs 2 D-point IDFT (IFFT), throws away the 1 st half of the result, saves the 2 nd half (hence `save’) `throw away’ `save’ – Subband processing corresponds to `frequency domain’ operation • Complexity/latency? See p. 25 DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 21 / 40
Overlap-Add & Overlap-Save Revisited Derivation on p. 19 can also be modified as follows… DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 22 / 40
1 p. to Overlap-Add & Overlap-Save Revisited re pa m o • C A filter bank representation of this is. . . u[k] 4 4 4 4 + y[k] Analysis bank: Subband processing: Synthesis bank: This is a 2 D-channel filter bank, with D-fold downsampling The analysis FB is a 2 D-channel DFT filter bank (see Chapter-11) The synthesis FB is matched to the analysis bank, for PR: DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 23 / 40
Overlap-Add & Overlap-Save Revisited • This is again known as an `overlap-add’ realization : – Analysis bank: performs 2 D-point DFT (FFT) of a block of D samples, padded with D zero samples `block’ `zero padding’ – Synthesis bank: performs 2 D-point IDFT (IFFT), adds 2 nd half of the result to 1 st half of previous IDFT (hence `add’) `overlap’ `add’ – Subband processing corresponds to `frequency domain’ operation DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 24 / 40
Overlap-Add & Overlap-Save Revisited • Computational complexity is (with FFT/IFFT i. o. DFT/IDFT, plus subband processing) 2. [α. 2 D. log(2 D)] + 2 L multiplications for D output samples, i. e. O(log(D))+O(L/D) per sample For large D≈L this is O(log(D)) i. e. dominated by FFT/IFFT (cheap!) For D<<L this is O(L/D), i. e. dominated by subband processing • Processing delay/latency is O(D) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 25 / 40
Overlap-Add & Overlap-Save Revisited • Standard overlap-add/-save(= p. 9 & p. 14) realizations are derived when 0 th order polyphase components are used in the above derivation (D=L, i. e. each polyphase component has only L/D=1 filter coefficient). For large L, this leads to a large complexity reduction, but also a large latency (=O(L)) • In the more general case, with higher-order polyphase components (D<<L, i. e. each polyphase component has L/D>>1 filter coefficients) a smaller complexity reduction is achieved, but the latency is also smaller (=O(D)). DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 26 / 40
Part-IV : Filter Banks & Time-Frequency Transforms Chapter-11 Filter Bank Preliminaries Chapter-12 Filter Bank Design Chapter-13 Frequency Domain Filtering • Frequency Domain FIR Filter Realization • Overlap-Add & Overlap-Save Revisited • Frequency Domain Adaptive FIR Filters • Background: Block-LMS • FDAF • PB-FDAF Chapter-14 Time-Frequency Analysis & Scaling DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 27 / 40
Background: Block-LMS • A similar derivation can be made for LMS-based adaptive filtering with block processing ('Block-LMS'). The adaptive filter then consist in a filtering operation plus an adaptation operation, which corresponds to a correlation operation. Both operations can be performed cheaply in the frequency domain. . • Starting point is the LMS update equation DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 28 / 40
Background: Block-LMS Consider block processing with so-called 'Block-LMS' • Remember that LMS is a ‘stochastic gradient’ algorithm, where instantaneous estimates of the autocorrelation matrix and cross-correlation vector are used to compute a gradient vector (=steepest descent vector) • Block-LMS uses averaged estimates, with averaging over a block of LB (=‘block length’) samples, and hence an averaged gradient vector. The update formula is then (n is the block index) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 29 / 40
Background: Block-LMS • Complexity of Block-LMS ≈ complexity of LMS • Compared to LMS, Block-LMS does fewer updates (one per LB samples), but with (presumably) better gradient estimates. Overall, convergence could be faster or slower (=unpredictable). • The important thing is that Block-LMS can be realized cheaply… DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 30 / 40
Part-IV : Filter Banks & Time-Frequency Transforms Chapter-11 Filter Bank Preliminaries Chapter-12 Filter Bank Design Chapter-13 Frequency Domain Filtering • Frequency Domain FIR Filter Realization • Overlap-Add & Overlap-Save Revisited • Frequency Domain Adaptive FIR Filters • Background: Block-LMS • FDAF • PB-FDAF Chapter-14 Time-Frequency Analysis & Scaling DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 31 / 40
FDAF =frequency domain filtering Will consider case where block length LB = filter length L The update formulas are then given as follows 1) Compute a priori residuals (example LB=L=4 , similar for other L) with Wi=… DSP-CIS 2020 -2021 / Chapter-13: Domain Filtering PS: formulas given for L=LFrequency B=4, for conciseness (but without loss of generality) 32 / 40
FDAF with Ei=… DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering =frequency domain correlation Will consider case where block length LB = filter length L The update formulas are then given as follows 2) Filter update (example LB=L=4 , similar for other L) 33 / 40
FDAF This is referred to as FDAF (’Frequency Domain Adaptive Filtering’) • FDAF is functionally equivalent to Block-LMS cheaper, see below) (but • Convergence: Instead of using one and the same stepsize for all ‘frequency bins’ (=subbands), frequency dependent stepsizes can be applied. . – In the update formula, is removed and Ei is replaced by i. Ei – Stepsize i dependent on the energy in the ith frequency bin – Leads to increased convergence speed at only a small extra cost DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 34 / 40
FDAF • Complexity ≈ 5 (I)FFT’s (size 2 L) per block of L output samples (check!) Hence for large L, FDAF is very efficient/cheap, only O(log(L)) multiplications per output sample (compared to O(L) for (Block-)LMS) Example: LB=L=1024, then ! • Processing delay/latency is again O(L). Example: LB=L=1024 and fs=8000 Hz, then delay is 256 ms ! In cases where this is objectionable (e. g. acoustic echo cancellation), need ‘intermediate’ algorithms with smaller latency and smaller complexity reduction, based on a polyphase decomposition…(read on) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 35 / 40
Part-IV : Filter Banks & Time-Frequency Transforms Chapter-11 Filter Bank Preliminaries Chapter-12 Filter Bank Design Chapter-13 Frequency Domain Filtering • Frequency Domain FIR Filter Realization • Overlap-Add & Overlap-Save Revisited • Frequency Domain Adaptive FIR Filters • Background: Block-LMS • FDAF • PB-FDAF Chapter-14 Time-Frequency Analysis & Scaling DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 36 / 40
PB-FDAF For large L, a block length of LB=L may lead to a too large latency. . . If a D-fold polyphase decomposition of the adaptive filter is considered (hence with LP=L/D coefficients per polyphase component), then a frequency domain adaptive filtering algorithm with block length LB=D can derived as follows. . . (where “D takes the place of L”) Example LB=D=4, i. e. (as on p. 9) with DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering (but without loss of generality) PS: formulas given for D=4, for conciseness 37 / 40
PB-FDAF (compare to p. 29 -30) The update formulas are given as follows 1) Compute a priori residuals (example LB=D=4 , similar for other D) with Pij=… DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 38 / 40
PB-FDAF (compare to p. 29 -30) The update formulas are given as follows 2) Filter update (example LB=D=4 , similar for other D) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 39 / 40
PB-FDAF This is referred to as PB-FDAF (’Partitioned Block Frequency Domain Adaptive Filtering’) • PB-FDAF is functionally equivalent to Block-LMS • Complexity ≈ 3+2. LP (I)FFT’s (size 2 D)) per block of D output samples Example: L=1024, D=128, then (check!) ! • Processing delay/latency is O(D). Example: L=1024, D=128 and fs=8000 Hz, then delay is 32 ms (used in commercial acoustic echo cancellers) DSP-CIS 2020 -2021 / Chapter-13: Frequency Domain Filtering 40 / 40
- Slides: 40