Speech and Audio Signal Processing Lab Multiplicative Update

语音与音频信号处理研究室 Speech and Audio Signal Processing Lab Multiplicative Update of AR gains in Codebookdriven Speech Enhancement Qi He 1, Changchun Bao 1, and Feng Bao 2 1 Beijing University of Technology, China 2 The University of Auckland, New Zealand 2016 -3 -25 http: //www. bjut. edu. cn/sci/voice/index. htm

Outline l Speech Enhancement Review Ø Background Ø Traditional Methods l Multiplicative Update of AR Gains in Codebook-driven Speech Enhancement Ø Estimation of spectral shape of noise Ø Estimation of AR gains Ø Bayesian MMSE estimation Ø Codebook-driven Wiener filter Ø Experimental Results 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 2 http: //www. bjut. edu. cn/sci/voice/index. htm

Background Noises exist everywhere St r no eet ise y or t c e Fa ois n Ba b no ble ise e fic e f O is no 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 3 http: //www. bjut. edu. cn/sci/voice/index. htm

Background l Speech enhancement applications Mobile phone/ Communication Hearing aids Robust speech / speaker/ language recognition, etc. 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 4 http: //www. bjut. edu. cn/sci/voice/index. htm

Background l Speech enhancement aims at Ø suppressing the noise in noisy speech Ø improving the quality and intelligibility of enhanced speech Speech Noisy Speech enhancement Enhanced speech Noise (1) where n is the frame index. 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 5 http: //www. bjut. edu. cn/sci/voice/index. htm

Traditional Methods l Traditional speech enhancement methods Ø Spectral subtraction Ø Wiener filtering No a Priori Information Ø Subspace method Ø …… Performance of these methods: l For Stationary Noises: Good l For Non-Stationary Noises: Bad 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 6 http: //www. bjut. edu. cn/sci/voice/index. htm

Traditional Methods l Codebook-based methods 1) Codebook-based method using ML estimator[1]. 2) Codebook-based method using Bayesian MMSE estimator[2]. Speech corpus AR: auto-regressive Noise corpus Speech codebook Noise codebook Noisy speech FFT AR gains estimate Noisy spectrum ML or Bayesian MMSE estimation Wiener filter 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 7 IFFT Enhanced speech http: //www. bjut. edu. cn/sci/voice/index. htm

Traditional Methods l Traditional method for AR gains estimation For each pair of code-words from speech and noise codebooks, the corresponding AR gains should be obtained by (2) with observed noisy spectrum modeled noisy spectrum 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 8 http: //www. bjut. edu. cn/sci/voice/index. htm

Traditional Methods l Traditional method for AR gains estimation Since there is no closed-form solution for optimal speech and noise AR gains estimation, the conventional codebook-driven methods indirectly obtain the AR gain estimation based on the log-spectral (LS) distortion, which has a closed-form solution by applying the series expansion. That is (3) with 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 9 http: //www. bjut. edu. cn/sci/voice/index. htm

Traditional Methods l Traditional method for AR gains estimation By differentiating Eq. 3 with respect to the AR gains and setting the results to zero, the AR gains can be calculated by (4) After getting the AR gains corresponding to each code-word combination, we can use the following ML estimator or Bayesian MMSE estimator to obtain the AR parameters of speech and noise (5) 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 10 http: //www. bjut. edu. cn/sci/voice/index. htm

Traditional Methods l Traditional method for de-noising A Wiener filter constructed by the estimated AR parameters of speech and noise is used to enhance noisy speech. (6) Although the codebook-driven speech enhancement methods are more suitable for eliminating non-stationary noise, there are still some problems to be addressed. 1) Noise classification; 2) The accuracy of gain estimation can be further improved; 3) The residual noise between the harmonics of noisy speech should be further suppressed ; 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 11 http: //www. bjut. edu. cn/sci/voice/index. htm

Proposed Method l The estimation of spectral shape of noise To solve the problem of noise classification, the spectral shape of noise is estimated online by the Minima Controlled Recursive Averaging(MCRA) algorithm in the proposed method. (7) (8) 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 12 http: //www. bjut. edu. cn/sci/voice/index. htm

Proposed Method l The estimation of AR gains In this paper, we use a multiplicative update rule[3 -4] to obtain approximately closed-form solution of IS distortion. Since we only train the shape codebook of speech spectrum offline and the spectral shape of noise is estimated online, for each speech code-word, we can rewrite the modeled noisy spectrum as follows (9) By expressing the Eq. 9 in matrix form, we can get: with 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 13 http: //www. bjut. edu. cn/sci/voice/index. htm

Proposed Method l The estimation of AR gains The IS distortion is rewritten as (10) By differentiating Eq. 10 with respect to gain matrices, we have [3 -4]： (11) . The symbol ‘ ’ indicates the point-wise multiplication. By simplifying the above formula, we can get: (12) 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 14 http: //www. bjut. edu. cn/sci/voice/index. htm

Proposed Method l The estimation of AR gains The and are obtained by iterating the following multiplicative rules to minimize the IS distortion: (13) Then we have (14) 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 15 http: //www. bjut. edu. cn/sci/voice/index. htm

Proposed Method l The estimation of AR gains An example of average IS distortion is illustrated in Fig. 1. The average IS distortion is defined as follows The Nx is the size of speech codebook. The AR gains are estimated by the conventional and proposed methods, respectively. The speech material is corrupted by white noise with the SNR of 5 d. B. Fig. 1 the average IS distortion comparison 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 16 http: //www. bjut. edu. cn/sci/voice/index. htm

Proposed Method l Bayesian MMSE estimation Let θx denote the random variable corresponding to the speech AR coefficients. And let gx and gw denote the random variables corresponding to the speech and noise AR gains, respectively. Let θ=[θx, gw] denote the set of random variables. After getting each , the desired Bayesian MMSE estimate can be written as follows (15) with and 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 17 http: //www. bjut. edu. cn/sci/voice/index. htm

Proposed Method Fig. 2 AR gain estimation of clean speech 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab

Proposed Method l Modified codebook-driven Wiener filter Conventional codebook-driven Wiener filter is constructed by the estimated spectral envelopes of speech and noise, which usually causes an inaccurate fitting for the spectra between the harmonics of speech. Consequently, the residual noise still remains between the harmonics of the enhanced speech. In this section, we introduce the SPP to modify the traditional codebook-driven Wiener filter for suppressing the residual noise. (16) with and where 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 19 http: //www. bjut. edu. cn/sci/voice/index. htm

Performance Evaluation l Experiments Four types of noise: white, babble, office, and street The test materials : 9 utterances from 4 female speakers and 5 male speakers. The sampling rate: 8 KHz The size of speech codebook: 6 bit TABLE. 1. TEST RESULTS OF PESQ Enhancement Methods Noisy ML-CB [1] MMSE-CB [2] Proposed Average PESQ 0 d. B 5 d. B 10 d. B 1. 87 2. 20 2. 55 2. 05 2. 45 2. 71 2. 33 2. 64 2. 90 2. 40 2. 76 3. 06 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 20 http: //www. bjut. edu. cn/sci/voice/index. htm

Performance Evaluation TABLE. 2. TEST RESULTS OF SSNR IMPROVEMENT Enhancement Methods Noisy ML-CB [1] MMSE-CB [2] Proposed Average SSNR Improvement 0 d. B 5 d. B 10 d. B -- -- -- 9. 74 8. 71 7. 59 13. 23 11. 97 10. 69 16. 42 15. 46 14. 16 TABLE. 3. TEST RESULTS OF LSD Enhancement Methods Noisy ML-CB [1] MMSE-CB [2] Proposed Average LSD 0 d. B 5 d. B 10 d. B 14. 57 12. 66 10. 88 10. 68 8. 99 7. 92 9. 09 7. 73 6. 46 7. 41 6. 11 5. 15 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 21 http: //www. bjut. edu. cn/sci/voice/index. htm

Demos (a) clean speech (b) noisy speech (white noise, SNR=10 d. B), (c) enhanced speech using ML-CB, (d) enhanced speech using MMSE-CB, (e) enhanced speech using our method without SPP, (f) enhanced speech using our method with SPP. 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 22 http: //www. bjut. edu. cn/sci/voice/index. htm

Demos (a) clean speech (b) noisy speech (babble noise, SNR=10 d. B), (c) enhanced speech using ML-CB, (d) enhanced speech using MMSE-CB, (e) enhanced speech using our method without SPP, (f) enhanced speech using our method with SPP. 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 23 http: //www. bjut. edu. cn/sci/voice/index. htm

References [1] S. Srinivasan, J. Samuelsson, and W. B. Kleijn, “Codebook driven short-term predictor parameter estimation for speech enhancement”, IEEE Trans. Audio, Speech, Lang. Process. , vol. 14, no. 1, pp. 163– 176, Jan. 2006. [2] S. Srinivasan, J. Samuelsson, and W. B. Kleijn, “Codebook-based Bayesian speech enhancement for nonstationary environments” , IEEE Trans. Audio, Speech, Lang. Process. , vol. 15, no. 2, pp. 441– 452, Feb. 2007 [3] Daniel D. Lee and H. Sebastian Seung, “Algorithms for non-negative matrix factorization, ” in NIPS, 2000, pp. 556– 562. [4] C. Févotte, N. Bertin, and J. L. Durrieu, “Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis, ” Neural Comput. , vol. 21, pp. 793– 830, 2009. 语音与音频信号处理研究室 Speech and Audio Signal Processing Lab 24 http: //www. bjut. edu. cn/sci/voice/index. htm

语音与音频信号处理研究室 Speech and Audio Signal Processing Lab Q&A http: //www. bjut. edu. cn/sci/voice/index. htm