Digital Audio Signal Processing Lecture4 Noise Reduction Marc

Digital Audio Signal Processing Lecture-4: Noise Reduction Marc Moonen/Alexander Bertrand Dept. E. E. /ESAT-STADIUS, KU Leuven marc. moonen@esat. kuleuven. be homes. esat. kuleuven. be/~moonen/

Overview • Spectral subtraction for single-micr. noise reduction – – Single-microphone noise reduction problem Spectral subtraction basics (=spectral filtering) Features: gain functions, implementation, musical noise, … Iterative Wiener filter based on speech signal model • Multi-channel Wiener filter for multi-micr. noise red. – Multi-microphone noise reduction problem – Multi-channel Wiener filter (=spectral+spatial filtering) • Kalman filter based noise reduction – Kalman filters for noise reduction Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 2

Single-Microphone Noise Reduction Problem • Microphone signal is desired signal s[k] y[k] desired signal noise contribution ? desired signal estimate noise signal(s) • Goal: Estimate s[k] based on y[k] • Applications: Speech enhancement in conferencing, handsfree telephony, hearing aids, … Digital audio restoration • Will consider speech applications: s[k] = speech signal Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 3

Spectral Subtraction Methods: Basics • Signal chopped into `frames’ (e. g. 10. . 20 msec), for each frame a frequency domain representation is (i-th frame) • However, speech signal is an on/off signal, hence some frames have speech +noise, i. e. some frames have noise only, i. e. • A speech detection algorithm is needed to distinguish between these 2 types of frames (based on energy/dynamic range/statistical properties, …) Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 4

Spectral Subtraction Methods: Basics • Definition: ( ) = average amplitude of noise spectrum • Assumption: noise characteristics change slowly, hence estimate ( ) by (long-time) averaging over (M) noise-only frames • Estimate clean speech spectrum Si( ) (for each frame), using corrupted speech spectrum Yi( ) (for each frame, i. e. short-time estimate) + estimated ( ): based on `gain function’ Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 5

Spectral Subtraction: Gain Functions Magnitude Subtraction Spectral Subtraction Wiener Estimation Maximum Likelihood Non-linear Estimation Ephraim-Malah = most frequently used in practice Digital Audio Signal Processing Version 2014 -2015 see next slide Lecture-4: Noise Reduction p. 6

s Spectral Subtraction: Gain Functions la r fo u m • Example 1: Ephraim-Malah Suppression Rule (EMSR) p i sk with: • • modified Bessel functions This corresponds to a MMSE (*) estimation of the speech spectral amplitude |Si( )| based on observation Yi( ) ( estimate equal to E{ |Si( )| | Yi( ) } ) assuming Gaussian a priori distributions for Si( ) and Ni( ) [Ephraim & Malah 1984]. Similar formula for MMSE log-spectral amplitude estimation [Ephraim & Malah 1985]. (*) minimum mean squared error Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 7

Spectral Subtraction: Gain Functions • Example 2: Magnitude Subtraction – Signal model: – Estimation of clean speech spectrum: – PS: half-wave rectification Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 8

Spectral Subtraction: Gain Functions • Example 3: Wiener Estimation – Linear MMSE estimation: find linear filter Gi( ) to minimize MSE – Solution: <- cross-correlation in i-th frame <- auto-correlation in i-th frame Assume speech s[k] and noise n[k] are uncorrelated, then. . . – PS: half-wave rectification Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 9

Spectral Subtraction: Implementation y[k] Y[n, i] Short-time analysis Gain functions Short-time synthesis Short-time Fourier Transform (=uniform DFT-modulated analysis filter bank) = estimate for Y( n ) at time i (i-th frame) N=number of frequency bins (channels) n=0. . N-1 M=downsampling factor K=frame length h[k] = length-K analysis window (=prototype filter) frames with 50%. . . 66% overlap (i. e. 2 -, 3 -fold oversampling, N=2 M. . 3 M) subband processing: synthesis bank: matched to analysis bank (see DSP-CIS) Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 10

Spectral Subtraction: Musical Noise • Audio demo: car noise magnitude subtraction • Artifact: musical noise What? Short-time estimates of |Yi( )| fluctuate randomly in noise-only frames, resulting in random gains Gi( ) statistical analysis shows that broadband noise is transformed into signal composed of short-lived tones with randomly distributed frequencies (=musical noise) Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 11

Spectral Subtraction: Musical Noise • Solutions? - Magnitude averaging: replace Yi( ) in calculation of Gi( ) by a local average over frames average instantaneous - EMSR (p 7) - augment Gi( ) with soft-decision VAD: Gi( ) P(H 1 | Yi( )). Gi( ) … probability that speech is present, given observation Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 12

Spectral Subtraction: Iterative Wiener Filter Example of signal model-based spectral subtraction… • Basic: Wiener filtering based spectral subtraction (p. 9), with (improved) spectra estimation based on parametric models • Procedure: 1. Estimate parameters of a speech model from noisy signal y[k] 2. Using estimated speech parameters, perform noise reduction (e. g. Wiener estimation, p. 9) 3. Re-estimate parameters of speech model from the speech signal estimate 4. Iterate 2 & 3 Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 13

Spectral Subtraction: Iterative Wiener Filter pulse train voiced … … pitch period unvoiced all-pole filter u[k] speech signal x white noise generator frequency domain: time domain: = linear prediction parameters Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 14

Spectral Subtraction: Iterative Wiener Filter For each frame (vector) y[m] 1. Estimate (i=iteration nr. ) and 2. Construct Wiener Filter (p. 9) Repeat until some error criterion is satisfied with: • • estimated during noise-only periods 3. Filter speech frame y[m] Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 15

Overview • Spectral subtraction for single-micr. noise reduction – – Single-microphone noise reduction problem Spectral subtraction basics (=spectral filtering) Features: gain functions, implementation, musical noise, … Iterative Wiener filter based on speech signal model • Multi-channel Wiener filter for multi-micr. noise red. – Multi-microphone noise reduction problem – Multi-channel Wiener filter (=spectral+spatial filtering) • Kalman filter based noise reduction – Kalman filters for noise reduction Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 16

Multi-Microphone Noise Reduction Problem speech source ? (some) speech estimate microphone signals noise source(s) speech part Digital Audio Signal Processing Version 2014 -2015 noise part Lecture-4: Noise Reduction p. 17

Multi-Microphone Noise Reduction Problem Will estimate speech part in microphone 1 (*) (**) ? (*) Estimating s[k] is more difficult, would include dereverberation. . . (**) This is similar to single-microphone model (p. 3), where additional microphones (m=2. . M) help to get a better estimate Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 18

Multi-Microphone Noise Reduction Problem • Data model: See Lecture-2 on multi-path propagation, with q left out for conciseness. Hm(ω) is complete transfer function from speech source position to m -the microphone Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 19

Multi-Channel Wiener Filter (MWF) • Data model: • Will use linear filters to obtain speech estimate (as in Lecture-2) • Wiener filter (=linear MMSE approach) Note that (unlike in DSP-CIS) `desired response’ signal S 1(w) is unknown here (!), hence solution will be `unusual’… Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 20

Multi-Channel Wiener Filter (MWF) • Wiener filter solution is (see DSP-CIS) compute during speech+noise periods compute during noise-only periods – All quantities can be computed ! – Special case of this is single-channel Wiener filter formula on p. 9 Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 21

Multi-Channel Wiener Filter (MWF) • MWF combines spatial filtering (as in Lecture-2) with single-channel spectral filtering (as in single-channel noise reduction) if then… Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 22

Multi-Channel Wiener Filter (MWF) • …then it can be shown that • represents a spatial filtering (*) Compare to superdirective & delay-and-sum beamforming (Lecture-2) – Delay-and-sum beamf. maximizes array gain in white noise field – Superdirective beamf. maximizes array gain in diffuse noise field – MWF maximizes array gain in unknown (!) noise field. MWF is operated without invoking any prior knowledge (steering vector/noise field) ! (the secret is in the voice activity detection… (explain)) (*) Note that spatial filtering can improve SNR, spectral filtering never improves SNR (at one frequency) Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 23

Multi-Channel Wiener Filter (MWF) • • …then it can be shown that (continued) represents an additional `spectral post-filter’ i. e. single-channel Wiener estimate (p. 9), applied to output signal of spatial filter (prove it!) Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 24

Multi-Channel Wiener Filter: Implementation • Implementation with short-time Fourier transform: see p. 10 • Implementation with time-domain linear filtering: filter coefficients Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 25

Multi-Channel Wiener Filter: Implementation • Implementation with time-domain linear filtering: Solution is… compute during speech+noise periods compute during noise-only periods Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 26

Overview • Spectral subtraction for single-micr. noise reduction – – Single-microphone noise reduction problem Spectral subtraction basics (=spectral filtering) Features: gain functions, implementation, musical noise, … Iterative Wiener filter based on speech signal model • Multi-channel Wiener filter for multi-micr. noise red. – Multi-microphone noise reduction problem – Multi-channel Wiener filter (=spectral+spatial filtering) • Kalman filter based noise reduction – Kalman filters for noise reduction Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 27

Kalman Filter 1/12 State space model of a time-varying discrete-time system process noise measurement noise with v[k] and w[k]: mutually uncorrelated, zero mean, white noises Then: given A[k], B[k], C[k], D[k], V[k], W[k] and input/output-observations u[k], y[k], k=0, 1, 2, . . . then Kalman filter produces MMSE estimates of internal states x[k], k=0, 1, . . . Digital Audio Signal Processing PS: will use shorthand notation here, i. e. Version 2014 -2015 Lecture-4: Noise Reduction xk, yk , . . instead of x[k], y[k], . . p. 28

Kalman Filter 2/12 Definition: • `FILTERING’ = MMSE-estimate of xk using all available data up until time l = estimate • `PREDICTION’ = estimate • `SMOOTHING’ = estimate Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 29

Kalman Filter 3/12 Initalization: =error covariance matrix ‘Conventional Kalman Filter’ operation @ time k (k=0, 1, 2, . . ) Given and corresponding error covariance matrix , Compute and using uk, yk : Step 1: Measurement Update (produces ‘filtered’ estimate) (compare to standard RLS!) Step 2: Time Update (produces ‘ 1 -step prediction’) Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 30

Kalman Filter 4/12 PS: ‘Standard RLS’ is a special case of ‘Conventional KF’ Digital Audio Signal Processing Version 2014 -2015 Internal state vector is FIR filter coefficients vector, which is assumed to be time-invariant Lecture-4: Noise Reduction p. 31

Kalman Filter 5/12 State estimation @ time k corresponds to overdetermined set of linear equations, where vector of unknowns contains all previous state vectors : Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 32

Kalman Filter 6/12 Technical detail… State estimation @ time k corresponds to overdetermined set of linear equations, where vector of unknowns contains all previous state vectors : Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 33

Kalman Filter 7/12 State estimation @ time k corresponds to overdetermined set of linear equations, where vector of unknowns contains all previous state vectors : Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 34

Kalman Filter 8/12 Propagated from time k-1 to time k ‘Square-Root’ Kalman Filter . . hence requires only lower-right/lower part Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 35

Kalman Filter 9/12 Digital Audio Signal Processing Version 2014 -2015 Propagated from time k-1 to time k ‘Square-Root’ Kalman Filter Lecture-4: Noise Reduction p. 36

Kalman Filter 10/12 Digital Audio Signal Processing Propagated from time k to time k+1 Propagated from time k-1 to time k ‘Square-Root’ Kalman Filter Version 2014 -2015 Lecture-4: Noise Reduction p. 37

Kalman Filter 11/12 QRD- =QRDDigital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 38

Kalman Filter 12/12 can be derived from square-root KF equations is . : can be worked into measurement update eq. : can be worked into state update eq. [details omitted] Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 39

Kalman filter for Speech Enhancement • Assume AR model of speech and noise u[k], w[k] = zero mean, unit variance, white noise • Equivalent state-space model is… y=microphone signal Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 40

Kalman filter for Speech Enhancement with: Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 41

Kalman filter for Speech Enhancement Iterative algorithm y[k] split signal in frames iterations estimate parameters Kalman Filter reconstruct signal Disadvantages iterative approach: • complexity • delay Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 42

Kalman filter for Speech Enhancement Sequential algorithm iteration index time index (no iterations) D State Estimator: Kalman Filter Parameters Estimator (Kalman Filter) D Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 43

CONCLUSIONS • Single-channel noise reduction – Basic system is spectral subtraction – Only spectral filtering, hence can only exploit differences in spectra between noise and speech signal: • noise reduction at expense of speech distortion • achievable noise reduction may be limited • Multi-channel noise reduction – Basic system is MWF, – Provides spectral + spatial filtering (links with beamforming!) • Iterative Wiener filter & Kalman filtering – Signal model based approach Digital Audio Signal Processing Version 2014 -2015 Lecture-4: Noise Reduction p. 44