Speech Audio Processing PartII Digital Audio Signal Processing

Speech & Audio Processing / Part-II Digital Audio Signal Processing DASP Marc Moonen Dept. E. E. /ESAT-STADIUS, KU Leuven marc. moonen@esat. kuleuven. be homes. esat. kuleuven. be/~moonen/

Outline • Aims/scope • Contents • Case study: Hearing instruments • Lectures/Exercise Sessions/Exam Website/Questions Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 2 / 32

Aims/Scope Aim is 2 -fold. . • Speech & audio per se S & A industry in Belgium/Europe Topics: Noise Reduction / Acoustic Echo & Feedback Cancellation / Active Noise Control / Sound Reproduction /. . etc • Develop basic signal processing tools/principles which are also used in many other fields Spatial filter design (for wireless comms) Adaptive filter algorithms, filtered-X LMS (for active vibration control) Kalman filters (for automatic control, navigation, . . ) Time-frequency analysis/processing . . etc Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 3 / 32

Outline • Aims/scope • Contents • Case study: Hearing instruments • Lectures/Exercise Sessions/Exam Website/Questions Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 4 / 32

Contents: Chapter-2 Noise Reduction-I Single-Channel Noise Reduction desired signal s[k] y[k] desired signal noise contribution ? desired signal estimate noise signal(s) • Spectral subtraction methods (spectral filtering) • Iterative methods based on speech modeling (Wiener & Kalman Filters) Applications: smartphones, conferencing, hearing aids, … Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 5 / 32

Contents: Chapter-3 Noise Reduction-II Microphone Array Processing - Fixed Beamforming Referred to as ‘spatial filtering’ (similar to ‘spectral filtering’) Filter-and-sum beamformer Applications: smartphones, conferencing, hearing aids, … Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 6 / 32

Contents: Chapter-4 Noise Reduction-III Adaptive Beamforming & Multi-Channel Noise Reduction • Adaptive Beamforming – Known (fixed) speaker position – Unknown (time-varying) noise field • Multi-channel noise reduction – Wiener filtering approach = spectral+spatial filtering Applications: smartphones, conferencing, hearing aids, … Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 7 / 32

Contents: Chapter-5 Acoustic Echo & Feedback Cancellation Adaptive filtering problem: • Non-stationary/wideband/… speech signals • Non-stationary/long/… acoustic channels Adaptive filtering algorithms AEC Control AEC Post-processing Stereo AEC Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 8 / 32

Contents: Chapter-5 Acoustic Echo & Feedback Cancellation • Hearing aids, public address (PA) systems • Correlation between filter input (`x ’) and near-end signal (‘ n ’) • Fixes : noise injection, pitch shifting, notch filtering, … Model F Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 9 / 32

Contents: Chapter-6 Active Noise Control • Solution based on `filtered-X LMS’ • Application : active headsets/ear defenders Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 10 / 32

Contents: Chapter-7 Distributed Signal Processing in Wireless Acoustic Sensor Networks Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 11 / 32

Contents: Chapter-8 Guest Lecture: Dr. Enzo De Sena, University of Surrey, UK ‘Sound Field Recording and Reproduction’ Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 12 / 32

Outline • Aims/scope • Contents • Case study: Hearing instruments • Lectures/Exercise Sessions/Exam Website/Questions Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 13 / 32

© www. cm. be Case Study: Hearing Instruments 1/10 Low-freq tone Hearing • Tonotopy of inner ear = Cochlea High-freq tone Hearing loss • Aging • Exposure to loud sounds Neural activity for low-freq tone Neural activitity for high-freq tone Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 14 / 32

Case Study: Hearing Instruments 2/10 DASP Challenges in hearing instruments (next slides) • • • Dynamic range compression Noise reduction Dereverberation Acoustic feedback cancellation Active noise control Etc. Technology Challenges in hearing instruments • Small form factor (cfr. user acceptance) • Low power: 1… 5 m. W (cfr. battery lifetime ≈ 1 week) • Low processing delay: 10 msec (cfr. synchronization with lip reading) Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 15 / 32

Case Study: Hearing Instruments 3/10 DASP Challenges: Dynamic range compression Dynamic range & audibility… Normal hearing Hearing impaired `signal dependent amplification’ subjects subjects Level 100 d. B Output Level (d. B) 100 d. B 100 d. B Input Level (d. B) Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 16 / 32

Case Study: Hearing Instruments 4/10 • However: Audibility does not imply intelligibility SNR • Hearing impaired subjects 20 d. B need 5. . 10 d. B larger signal -to-noise ratio (SNR) for speech understanding in noisy environments 0 d. B • Need for noise reduction (=speech enhancement) algorithms: 30 50 70 90 Hearing loss (d. B, 3 -freq-average) • State-of-the-art: monaural 2 -microphone adaptive noise reduction • Near future: binaural noise reduction (2 times 2 microphones) • Not-so-near future: multi-node noise reduction (see below) Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 17 / 32

Case Study: Hearing Instruments 5/10 DASP Challenges: Noise reduction & beamforming Multimicrophone ‘beamforming’, typically with 2 microphones e. g. ‘directional’ front microphone and ‘omnidirectional’ back microphone “filter-and-sum” the microphone signals Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction h p a r e t 4 3 18 / 32

Case Study: Hearing Instruments 6/10 Future: Multi-node noise reduction in sensor networks/Internet-of-Things 7 r e t p Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction ha 19 / 32

Case Study: Hearing Instruments 7/10 DASP Challenges: Dereverberation • Reverb = filtering effect (‘echo-ing’) of acoustic channel in between speaker and microphone(s) • Reverb has an impact on speech understanding • Dereverberation = undo filtering by acoustic channel (e. g. ‘inverse filtering’) Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 20 / 32

Case Study: Hearing Instruments 8/10 DASP Challenges: Acoustic feedback cancellation • Problem statement: Loudspeaker signal is fed back into microphone, then amplified and played back again • Closed loop system may become unstable (howling) • Similar to feedback problem in public address systems (for the musicians amongst you) Model F - Similar to echo cancellation in GSM handsets, Skype, … but more difficult due to signal correlation Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 5 r ha e t p 21 / 32

Case Study: Hearing Instruments 9/10 © S. Doclo/Univ. Oldenburg DASP Challenges: Active noise control (ANC) ANC to counteract noise leakage & occlusion effect, exploiting additional ‘internal’ reference microphone Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 6 r ha e t p 22 / 32

Case Study: Hearing Instruments 10/10 DASP Challenges: …piecing things together Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 23 / 32

Outline • Aims/scope • Contents • Case study: Hearing instruments • Lectures/Exercise Sessions/Exam Website/Questions Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 24 / 32

Lectures: 1 Intro (count as half) + 7 Lectures PS: Time budget = 7. 5*(2 hrs)*4 = 60 hrs Course Material: Slides – Use version 2019 -2020 ! – Download from DASP webpage homes. esat. kuleuven. be/~dspuser/dasp/ – Master copy available @ ESAT B 00. 10 (Ida Tassens) Course Prerequisite: DSP-CIS H 05 F 3 A/H 05 F 1 A (filter design, filter banks, optimal & adaptive filters) Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 25 / 32

Literature (general) (available in DSP-CIS library) • Simon Haykin `Adaptive Filter Theory’ (Prentice Hall 1996) • P. P. Vaidyanathan `Multirate Systems and Filter Banks’ (Prentice Hall 1993) Literature (specialized) (available in DSP-CIS library) • S. L. Gay & J. Benesty `Acoustic Signal Processing for Telecommunication’ (Kluwer 2000) • M. Kahrs & K. Brandenburg (Eds) `Applications of Digital Signal Processing to Audio and Acoustics’ (Kluwer 1998) • B. Gold & N. Morgan `Speech and Audio Signal Processing’ (Wiley 2000) Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 26 / 32

Exercise Sessions/Project Direction-of-arrival θ Acoustic source localization – – Direction-of-arrival estimation (‘MUSIC’ Algorithm) Noise reduction (‘Do. A informed beamforming’) Binaural synthesis and 3 D audio …all in a simulated set-up Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 27 / 32

Exercise Sessions/Project PS: groups of 2 • Runs over 4 weeks (non-consecutive) • Each week – 1 PC/Matlab session (supervised, 2. 5 hrs) – 2 ‘Homework’ sesions (unsupervised, 2*2. 5 hrs) PS: Time budget = 4*(2. 5 hrs+5 hrs) = 30 hrs • ‘Deliverables’ after week 2 & 4 • Grading: based on deliverables, evaluated during sessions • TAs: guiliano. bernardi@esat (English+Italian) randall. ali@esat (English+Spanish) . . be there ! Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 28 / 32

Exam • Oral exam, with preparation time • Open book • Grading 7 for question-1 7 for question-2 +6 for project ___ = 20 Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 29 / 32

Retake Exam • Oral exam, with preparation time • Open book • Grading 7 for question-1 7 for question-2 +6 for question-3 ___ = 20 (related to project work) Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 30 / 32

Website 1) TOLEDO 1) http: //homes. esat. kuleuven. be/~dspuser/dasp/ • • • Contact: guiliano. bernardi@esat Slides (use version 2018 !!) Schedule DSP-library FAQs (send questions to marc. moonen@esat) Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 31 / 32

Questions? 1) Ask teaching assistant (during exercises sessions) 2) E-mail questions to teaching assistant or marc. moonen@esat 3) Make appointment marc. moonen@esat ESAT Room B. 00. 14 Digital Audio Signal Processing Version 2019 -2020 Chapter-1: Introduction 32 / 32