Digital Audio Signal Processing DASP Chapter2 SingleChannel Noise

Digital Audio Signal Processing DASP Chapter-2 Single-Channel Noise Reduction Marc Moonen Dept. E. E. /ESAT-STADIUS, KU Leuven marc. moonen@kuleuven. be homes. esat. kuleuven. be/~moonen/

Single-Channel Noise Reduction • Microphone signal is desired signal s[k] y[k] desired signal noise contribution ? desired signal estimate noise signal(s) • Goal: Estimate s[k] based on y[k] • Applications: Speech enhancement in conferencing, handsfree telephony, hearing aids, … Digital audio restoration Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 2/40

Single-Channel Noise Reduction desired signal s[k] y[k] desired signal noise contribution ? desired signal estimate noise signal(s) • Will consider methods that do not rely on a priori information (e. g. expected noise spectrum) • Will consider speech applications: s[k] = speech signal Speech is on/off signal, where speech pauzes can be used to estimate noise spectrum Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 3/40

Overview • Spectral Subtraction Methods – Spectral subtraction basics (=spectral filtering) – Features: gain functions, realization, musical noise, … • Iterative Wiener Filter Based Noise Reduction • Kalman Filter Based Noise Reduction • Signal Subspace Methods Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 4/40

Spectral Subtraction Methods: Basics • Signal segmented into `frames’ (e. g. 10. . 20 msec), for each frame a frequency domain representation (=spectrum) is (i-th frame) • However, as speech signal is an on/off signal, some frames have speech+noise, i. e. some frames have noise only, i. e. • A speech detection algorithm (a. k. a. ‘voice activity detection’, VAD) is needed to distinguish between these 2 types of frames (based on energy/dynamic range/statistical properties, …) Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 5/40

Spectral Subtraction Methods: Basics • Definition: average magnitude of noise spectrum • Assumption: i. e. noise is (short-term) stationary • Hence (also assuming ergodicity) ( ) can be estimated by averaging over noise-only frames (*) • For each frame estimate clean speech spectrum Si( ) by using corrupted speech spectrum Yi( ) together with estimated ( ): based on `gain function’ Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 6/40

Spectral Subtraction Methods: Basics • PS: Applying a gain function like • can improve signal-to-noise ratio (SNR) of the signal as a whole (i. e. all radial frequencies). . but does not improve the SNR for a particular radial frequency (i. e. speech and noise are equally scaled) • hence impact on speech intelligibility is found to be minimal (or not-existing). . but ‘listening comfort’ is said to be improved For true SNR & speech intelligibility improvement, see multi-channel noise reduction (Chapter 3 -4) Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 7/40

Spectral Subtraction: Gain Functions Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 8/40

s Spectral Subtraction: Gain Functions la u m r o 1: Ephraim-Malah Suppression Rule (EMSR) • Example f p i sk with: • • modified Bessel functions This corresponds to a MMSE (*) estimation of the ‘speech spectral amplitude’ |Si( )| based on observation Yi( ) ( estimate equal to E{ |Si( )| | Yi( ) } ) assuming Gaussian a priori distributions for Si( ) and Ni( ) [Ephraim & Malah 1984] Similar formula for MMSE ‘log-spectral amplitude’ estimation [Ephraim & Malah 1985] (*) minimum mean squared error Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 9/40

Spectral Subtraction: Gain Functions • Example 2: Magnitude Subtraction – Signal model: – Estimation of clean speech spectrum: – PS: half-wave rectification Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 10/40

Spectral Subtraction: Gain Functions • Example 3: Wiener Estimation – Linear MMSE estimation: find linear filter Gi( ) to minimize MSE – Solution: cross-correlation auto-correlation Assume speech s[k] and noise n[k] are uncorrelated, then. . . – PS: half-wave rectification Digital Audio Signal Processing Version 2019 -2020 =‘instantaneous estimate’ (see also p. 14) Chapter-2: Single-Channel Noise Reduction 11/40

Spectral Subtraction: Realization requires time-frequency analysis (see DSP-CIS Chapter 14) Example: ‘Weigthed Overlap-Add’ (WOLA) based on Short-Time Fourier Transform with analysis/synthesis DFT-modulated filter bank based on analysis and synthesis window Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 12/40

Spectral Subtraction: Realization w[k] and v[k] = length-N analysis and synthesis window = prototype defining DFT modulated analysis and synthesis filter bank N = number of filter bank channels = frequency resolution D = downsampling factor = ‘window shift’ (e. g. 2 -fold oversampling: D=N/2, 3 -fold: D=N/3) yn[i] = n-th subband signal = estimate for Y( n ) at time i (i-th frame) G[n, i] = subband processing in n-th subband = gain for n at time i (i-th frame) y[k] Example (N=4, D=2) 2 2 yn[i] x x 2 2 x x Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction + 13/40

Spectral Subtraction: Musical Noise • Audio demo: magnitude subtraction • Artifact: musical noise What? Short-time estimates of |Yi( )| fluctuate randomly in noise-only frames, resulting in random gains Gi( ) statistical analysis shows that broadband noise is transformed into signal composed of short-lived tones with randomly distributed frequencies (=musical noise) Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 14/40

Spectral Subtraction: Musical Noise Solutions? – Magnitude averaging: replace Yi( ) in calculation of Gi( ) by a local average over frames average instantaneous – EMSR (see p. 9) – Augment Gi( ) with soft-decision VAD: Gi( ) P(H 1 | Yi( )). Gi( ) –… probability that speech is present, given observation Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 15/40

Overview • Spectral Subtraction Methods – Spectral subtraction basics (=spectral filtering) – Features: gain functions, realization, musical noise, … • Iterative Wiener Filter Based Noise Reduction • Kalman Filter Based Noise Reduction • Signal Subspace Methods Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 16/40

Iterative Wiener Filter Based Noise Reduction Example of signal model-based spectral subtraction… • Basic: Wiener filtering based spectral subtraction (p. 11), with (improved) spectra estimation based on parametric model • Procedure: 1. Estimate parameters of a speech model from noisy signal y[k] 2. Using estimated speech parameters, perform noise reduction (e. g. Wiener estimation, p. 11) 3. Re-estimate parameters of speech model from the speech signal estimate 4. Iterate 2 & 3 Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 17/40

Iterative Wiener Filter Based Noise Reduction pulse train voiced … … pitch period unvoiced all-pole filter u[k] speech signal x white noise generator frequency domain: time domain: = linear prediction parameters Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 18/40

Iterative Wiener Filter Based Noise Reduction For each frame (vector) y[m] 1. Estimate (i=iteration nr. ) and 2. Construct Wiener Filter (p. 11) Repeat until some error criterion is satisfied with: • • estimated during noise-only periods 3. Filter speech frame y[m] Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 19/40

Overview • Spectral Subtraction Methods – Spectral subtraction basics (=spectral filtering) – Features: gain functions, realization, musical noise, … • Iterative Wiener Filter Based Noise Reduction • Kalman Filter Based Noise Reduction • Signal Subspace Methods Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 20/40

Kalman Filter Recap 1/4 State space model of a time-varying discrete-time system process noise measurement noise with v[k] and w[k]: mutually uncorrelated, zero mean, white noises Then: given A[k], B[k], C[k], D[k], V[k], W[k] and input/output-observations u[k], y[k], k=0, 1, 2, . . . then Kalman filter produces MMSE estimates of internal states x[k], k=0, 1, . . . Digital Audio Processing Version 2019 -2020 i. e. x Chapter-2: PS: will. Signal often use shorthand notation, k, yk , . . Single-Channel instead of Noise x[k], Reduction y[k], . . 21/40

Kalman Filter Recap 2/4 Definition: • `FILTERING’ = MMSE-estimate of xk using all available data up until time l = estimate • `PREDICTION’ = estimate • `SMOOTHING’ = estimate Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 22/40

Kalman Filter Recap 3/4 Initalization: =error covariance matrix ‘Conventional’ Kalman Filter: For k=0, 1, 2, . . Given and corresponding error covariance matrix Step 1: Measurement Update : (produces ‘filtered’ estimate) (compare to standard RLS!) Step 2: Time Update (produces ‘ 1 -step prediction’) Better: ‘Square Root’ Algorithm (see DSP-CIS Chapter 14) Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 23/40

Kalman Filter Recap 4/4 Kalman Smoother Estimate states x[1], x[2], …, x[T] based on data u[k], y[k], k = 1, …T, T+1, …, N With ‘Conventional’ Kalman Filter: 1. forward run: apply previous equations for k = 1, 2, … N 2. backward run: apply following equations for k = N, N -1, … 1 Result: (better) estimates With ‘Square Root’ Kalman Filter: Full backsubstitution (details omitted) Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 24/40

Kalman Filter Based Noise Reduction • Assume AR model of speech and noise u[k], w[k] = zero mean, unit variance, white noise • Equivalent state-space model is… y[k] = microphone signal Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 25/40

Kalman Filter Based Noise Reduction Digital Audio Signal Processing s[k] and n[k] are included in state vector, hence can be estimated by Kalman Filter with: Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 26/40

Kalman Filter Based Noise Reduction Iterative algorithm (compare to iterative Wiener Filter) (details omitted) iterations y[k] split signal in frames estimate parameters reconstruct signal Kalman Filter/Smoother i-th frame Disadvantages: • complexity • delay Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 27/40

Kalman Filter Based Noise Reduction Sequential algorithm (details omitted) iteration index time index (i. e. no iterations) D State Estimator (Kalman Filter) Parameters Estimator (Kalman Filter) D Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 28/40

Overview • Spectral Subtraction Methods – Spectral subtraction basics (=spectral filtering) – Features: gain functions, realization, musical noise, … • Iterative Wiener Filter Based Noise Reduction • Kalman Filter Based Noise Reduction • Signal Subspace Methods Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 29/40

Signal Subspace Methods • Signal model: • Construct Toeplitz matrix (Lx. M, N=L+M-1) : Hence: Digital Audio Signal Processing (with S and N similarly constructed) Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 30/40

Signal Subspace Methods Assumptions: Consider L>>M (‘tall thin’ matrices) – Clean signal matrix S is `rank deficient‘: rank(S) = K < M – Clean signal matrix S is orthogonal (≈uncorrelated) to noise: ST N = 0 – White noise (p. 38 for coloured noise): NT N = 2 noise. IM Assumptions better satisfied as L ∞ Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 31/40

Signal Subspace Methods Tool: Singular value decomposition (SVD) of Y U is Lx. M orthogonal matrix UT. U=IMx. M Columns of U are ‘left singular vectors’ V is Mx. M orthogonal matrix VT. V=IMx. M Columns of V are ‘right singular vectors’ å is Mx. M diagonal matrix Diagonal elements are ‘singular values’ i 1 has K largest i corresponding to ‘signal+noise subspace’ 2 has M-K smallest I ≈ noise corresponding to ‘noise subspace’ Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 32/40

Signal Subspace Methods Procedure • Step-1 : From SVD of Y, find estimate of S – Least squares estimate – Minimum variance estimate • Step-2 : Construct Digital Audio Signal Processing Version 2019 -2020 from Chapter-2: Single-Channel Noise Reduction 33/40

Signal Subspace Methods • Step-1 (v 1. 0) : Least squares estimator YLS – Goal: approximate Y by a matrix YLS of rank K – Solution is obtained by setting M-K smallest singular values to zero (`truncation’) – Problem: proper determination of order K Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 34/40

Signal Subspace Methods • Step-1 (v 2. 0): Minimum variance estimator YMV – Find a matrix T that minimizes (assume S is given, then i-th column of T is optimal linear filter for i-th column of S) – Then: (*) PS : rank-deficiency condition not needed here PS : noise has to be known (e. g. estimated during noise-only) Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 35/40

Signal Subspace Methods • Step-2 : Signal Reconstruction Goal: Given Problem: , construct does not have Toeplitz structure Solution: Restore Toeplitz structure by arithmetically averaging every diagonal of the matrix Hence: (summation starts in column α) (summation ends in column β) Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 36/40

Signal Subspace Methods • Audio Demo Subspace Algorithm Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 37/40

Signal Subspace Methods • PS: What if coloured noise NT N 2 noise. IM ? – Estimate NTN during noise-only periods – Similar procedure, now including ‘pre-whitening’ of Hankel/Toeplitz matrix prior to SVD and ‘de-whitening’ step after estimation of – Computational scheme can use ‘generalized SVD’ instead of SVD (with ‘built-in’ pre-/de-whitening) – Details omitted… Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 38/40

Signal Subspace Methods • PS: Link with spectral subtraction? – Compare (*) p. 35 with gain function on p. 11 (Wiener) – Interpretation: ‘ Signal subspace method provides a ‘spectral subtraction’ in signal dependent filter bank (analysis filter bank VT, synthesis filter bank V) instead of signal independent WOLA filter bank – Details omitted… Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 39/40

Conclusion • Many single-channel noise reduction procedures available • Performance never spectacular • Seek performance improvement with multi-channel processing (=Chapter 3 -4) Digital Audio Signal Processing Version 2019 -2020 Chapter-2: Single-Channel Noise Reduction 40/40