DSP Everywhere Applications of DSP in Audio and

DSP Everywhere… Applications of DSP in Audio and Digital Communications Simon Doclo Dept. Elec. Engineering, K. U. Leuven simon. doclo@esat. kuleuven. ac. be www. esat. kuleuven. ac. be/~doclo/ DSP-II p. 1

Introduction • DSP in Digital Communications • Wireless systems: GSM, WLAN • 3 G-systems: UMTS, CDMA • Modems: Cable, ADSL, VDSL • Line echo cancellation • Satellite communications • Optical communication 14/12/01 Simon Doclo DSP Everywhere… p. 2

Introduction • DSP in Audio Applications • Audio and speech coding • Audio effects • Tele-conferencing • Voice-controlled systems • Hands-free telephony • Hearing aids / cochlear implants 14/12/01 Simon Doclo DSP Everywhere… p. 3

Introduction • Other applications • Image and video processing • Medical applications • Cryptography Anywhere (digital) signals are present, DSP-techniques are required! • Process control in chemical, pharmaceutical, energy plants • … 14/12/01 Simon Doclo DSP Everywhere… p. 4

Overview • Introduction • DSP in digital communications systems: – x. DSL-modems: modulation, equalisation • DSP in audio applications: – Hands-free communication: echo, noise and reverberation – Basic techniques: • Acoustic echo cancellation (AEC) • Multi-microphone beamforming – Application: hearing aids • Conclusion 14/12/01 Simon Doclo DSP Everywhere… p. 5

Telephone Line modems • High-speed data communication: – optical, cable, wireless, telephone line • Telephone Line Modems – voice-band modems : up to 56 kbits/sec in 0… 4 k. Hz band – ADSL modems : up to 6 Mbits/sec in 30 k. Hz… 1 MHz band – VDSL modems : up to 52 Mbits/sec in … 10 MHz band • Time to download 10 Mbyte-file: 14/12/01 Modem Time 56 Kbps voice-band modem 24 minutes 128 Kbps ISDN 10 minutes 6 Mbps ADSL 13 seconds 52 Mbps VDSL 1. 5 seconds Simon Doclo DSP Everywhere… p. 6

x. DSL Modems • ADSL : ‘Asymmetric Digital Subscriber Line’ • HDSL : ‘High Speed Digital Subscriber Line’ • VDSL : ‘Very High Speed Digital Subscriber Line’ …-1993: ADSL spurred by interest in video-on-demand (VOD) 1995 : ADSL/VOD interest decline 1996 : ADSL technology trials prove viability. 1997 -. . . : ADSL deployment, reoriented to data applications, as telco’s reaction to cable operators offering highspeed internet access with cable modems 2000 -… : VDSL 14/12/01 Simon Doclo DSP Everywhere… p. 7

x. DSL Modems Downstream Copper wire Central office Upstream Subscriber • Analog/digital telephone network: BW 3 k. Hz, SNR 35 d. B Shannon capacity • ADSL/VDSL: higher bandwidth, lower SNR + impairments • Bitrate depends on length of copper line Down Up Length Bandwidth ADSL 6 Mbps 640 Kbps 3 km 1. 1 MHz VDSL 52 Mbps 6. 4 Mbps 300 m 12 MHz vb. 14/12/01 Simon Doclo DSP Everywhere… p. 8

DMT Principles: IFFT/FFT-based modulation • Modulation-technique : DMT (Discrete Multitone) • Basic idea: – Decompose frequency into ‘tones’ (FFT/IFFT) – Assign bits according to SNR per tone SNR bits/toon frequentie – ADSL spec (=ANSI standard): • 256 tones, 512 -point (I)FFTs • carrier spacing fo= 4. 3215 k. Hz, basic sampling rate 2. 21 MHz (=512*4. 3215 k. Hz) – VDSL (=proposal): up to 4096 tones, same carrier spacing 14/12/01 Simon Doclo DSP Everywhere… p. 9

ADSL Spectrum • ADSL spectrum : 14/12/01 Simon Doclo DSP Everywhere… p. 10

Communication impairments (1) • Frequency-dependent channel attenuation introduces intersymbol interference (ISI) equalization • Coupling between wires in same or adjacent binders introduces crosstalk (XT) useful signal – Near-end Xtalk (NEXT) – Far-end Xtalk (FEXT) – Other systems (e. g. HPNA) NEXT FEXT • Radio Frequency Interference (RFI): e. g. AM broadcast, amateur radio • Noise: e. g. impulsive noise (=high bursts of short duration) • Echo: due to hybrid impedance mismatch echo cancellation Conclusion: Need advanced modulation, DSP, etc. ! 14/12/01 Simon Doclo DSP Everywhere… p. 11

Communication impairments (2) • ADSL channel attenuation, crosstalk, noise 14/12/01 Simon Doclo DSP Everywhere… p. 12

Modulation - Demodulation (1) • DMT-transmission block scheme: 4 -QAM IFFT 8 -QAM P / S Discrete equivalent channel S / P FFT FEQ Bitstream (freq domain) 14/12/01 Modulation (IFFT) Simon Doclo Time domain Demodulation Equalisation signal (FFT) DSP Everywhere… p. 13

Modulation - Demodulation (2) • Transmission: modulation is realized by means of 2 Npoint Inverse Discrete Fourier Transform (IFFT) (example N=4 ) real • Receiver: demodulation with inverse operation, i. e. FFT 14/12/01 Simon Doclo DSP Everywhere… p. 14

The Magic Prefix Trick (1) Additional feature : before transmission, a ‘prefix’ is added to each time-domain symbol, i. e. the last samples are copied and put up front : 14/12/01 Simon Doclo DSP Everywhere… p. 15

The Magic Prefix Trick (2) Prefix insertion : • in the receiver, the samples corresponding to the prefix are removed (=unused) : IFFT P / S Discrete equivalent channel S / P FFT FEQ 14/12/01 Simon Doclo DSP Everywhere… p. 16

The Magic Prefix Trick (3) • if channel impulse response has length L (= L non-zero taps) and ( is prefix length), then all ‘transient effects’ between symbols are confined to the prefix period : Tx-side Rx-side Channel ch(t) Tone 3 Tone 2 Tone 1 s(t) Tone 0 Prefix 14/12/01 From IFFT Simon Doclo * Tone 3 Tone 2 r(t) Tone 1 Tone 0 Guardband To FFT DSP Everywhere… p. 17

The Magic Prefix Trick (4) • Magic trick fails if , resulting in – inter-symbol-interference (ISI) = interference from previous symbol(s) (same carrier) – inter-carrier interference (ICI) = interference from other carriers • In the receiver, after removing the samples corresponding to the prefix, the i-th tone is observed, multiplied by a factor H(i. fo), i. e. the channel response for frequency f=i. fo • ‘Prefix trick’ is based on a linear convolution (filtering by channel impulse response) being turned into a circular convolution, which corresponds to component-wise multiplication in frequency domain easy equalization ! 14/12/01 Simon Doclo DSP Everywhere… p. 18

Overview • Introduction • DSP in digital communications systems: – x. DSL-modems: modulation, equalisation • DSP in audio applications: – Hands-free communication: echo, noise and reverberation – Basic techniques: • Acoustic echo cancellation (AEC) • Multi-microphone beamforming – Application: hearing aids • Conclusion 14/12/01 Simon Doclo DSP Everywhere… p. 19

Hands-free communication • Recorded microphone signals are corrupted by: • Far-end echoes acoustic echo cancellation • Acoustic background noise suppression • Room reverberation dereverberation • Application: hands-free telephony, hearing aids, voice control 14/12/01 Simon Doclo DSP Everywhere… p. 20

Signal model: some maths… • Multi-microphone signal enhancement algorithms: – Extract clean speech/audio signal from microphone recordings – Exploit spatial and frequency diversity between speech and noise • Microphone signals (m=1…M): unknown (=loudspeaker signal) • Output signal: compute filters g[k] : – echo cancellation: – noise reduction/dereverberation: • gm[k] cancels noise components • gm[k] focuses on speech s[k] 14/12/01 Simon Doclo DSP Everywhere… p. 21

Overview • Introduction • DSP in digital communications systems: – x. DSL-modems: modulation, equalisation • DSP in audio applications: – Hands-free communication: echo, noise and reverberation – Basic techniques: • Acoustic echo cancellation (AEC) • Multi-microphone beamforming – Application: hearing aids • Conclusion 14/12/01 Simon Doclo DSP Everywhere… p. 22

Acoustic echo cancellation (AEC) Suppress acoustic and line echo: – to guarantee normal conversation conditions : users do not like to hear a delayed and filtered version of their own voice – to prevent the closed-loop system from becoming unstable if amplification is too high 14/12/01 Simon Doclo DSP Everywhere… p. 23

Room Acoustics • Propagation of sound waves in an acoustic environment results in – signal attenuation – spectral distortion • The attenuation and distortion can be modeled quite well as a linear filtering operation • Non-linear distortion mainly stems from the loudspeakers. Its effect is typically of second order, therefore (often) not taken into account • The linear filter h[k] modeling the acoustic path between loudspeaker and microphone is represented by the acoustic impulse response 14/12/01 Simon Doclo DSP Everywhere… p. 24

Acoustic Impulse Response (1) Different parts: – dead time – direct path impulse and early reflections, which depend on the geometry of the room – an exponentially decaying tail called reverberation, coming from multiple reflections – For typical applications the impulse reponse is between 100 and 400 ms long several 100 to 1000 taps @ 8 -16 k. Hz memory requirement for circular buffers in DSP – Because people move around in the recording room, the acoustic impulse response is highly time-varying 14/12/01 Simon Doclo DSP Everywhere… p. 25

Acoustic Impulse Response (2) ESAT speech laboratory : Paleis voor Schone Kunsten : T 60 120 ms T 60 1500 ms Original speech signal : 14/12/01 Simon Doclo DSP Everywhere… p. 26

Acoustic Impulse Response : FIR or IIR ? • If the acoustic impulse response is modeled as – an FIR filter many hundreds to several thousands of filter taps are required – an IIR filter order can be reduced, but still several hundreds of filter coefficients are required (=‘bad’ model for acoustic impulse response) – Remark: IIR-filters good model for classical filters (LP, HP, BS) • hence FIR models are typically used in practice – as they are guaranteed to be stable – as adaptive filtering techniques are called for: • FIR adaptive filters are easier than IIR adaptive filters 14/12/01 Simon Doclo DSP Everywhere… p. 27

AEC based on Adaptive Filtering • Goal: Identify acoustic impulse response h[k] and subtract filtered loudspeaker signal from microphone signal • Thanks to the adaptivity – time-varying acoustics can be tracked – AEC is ‘self-learning’ – performance superior to performance of conventional techniques 14/12/01 Simon Doclo DSP Everywhere… p. 28

Adaptive Filtering Algorithms • Algorithm: 2 steps – Filter loudspeaker signal error signal indicates how close this signal is to recorded microphone signal – Update filter: update depends on error signal filtered signal desired signal 14/12/01 Simon Doclo DSP Everywhere… error signal p. 29

Normalized Least Mean Square (NLMS) Data filtering with Filter update Circular data buffer Filter coefficients L is the adaptive filter length, is the adaptation stepsize, is a regularization parameter and k is the discrete-time index 14/12/01 Simon Doclo DSP Everywhere… p. 30

Control Algorithm • ‘AEC is more than just an adaptive filter’ : – adaptive filter is supplemented with control software, which mainly controls the adaptation speed (e. g. no adaptation during double-talk) – In practice echo suppression is limited to 30 d. B due to time-variance, non-linearities, finite filterlength postprocessing (e. g. center-clipping) 14/12/01 Simon Doclo DSP Everywhere… p. 31

Real-time DSP Implementation (1) • AEC-implementation on DSP (lab equipment): – TMS 320 C 44 @ 50 MHz : data acquisition (ADC/DAC) – TMS 320 C 40 @ 50 MHz : acoustic echo cancellation (AEC) AEC 14/12/01 Simon Doclo ADC/DAC DSP Everywhere… p. 32

Real-time DSP Implementation (2) • Adaptive filtering part : several algorithms can be selected – NLMS : time-domain algorithm – PB-FDAF : frequency-domain algorithm (better performance) • Control software – double-talk detection – non-linear postprocessing algorithm • Variable sampling rate – Common sampling rates for speech applications: 8 k. Hz, 16 k. Hz – for audio applications: 22. 05 k. Hz, 44. 1 k. Hz, 48 k. Hz • Echo paths up to 325 ms can be modeled and tracked with the FDAF based on LMS at 8 k. Hz sampling frequency and 16 ms delay 14/12/01 Simon Doclo DSP Everywhere… p. 33

Real-time DSP Implementation (3) Execution times for the most important blocks of the DSP code were measured : N=768 FFT-size=128 fs=8000 Hz block 64 samples = 8 ms 14/12/01 Simon Doclo DSP Everywhere… p. 34

Demo Local speaker Output AEC without Double-talk Detection Far-end signal Near-end signal 14/12/01 Simon Doclo DSP Everywhere… p. 35

Overview • Introduction • DSP in digital communications systems: – x. DSL-modems: modulation, equalisation • DSP in audio applications: – Hands-free communication: echo, noise and reverberation – Basic techniques: • Acoustic echo cancellation (AEC) • Multi-microphone beamforming – Application: hearing aids • Conclusion 14/12/01 Simon Doclo DSP Everywhere… p. 36

Beamforming basics • Background/history: antenna array design for RADAR • Array elements are combined electronically such that: – array can be steered towards specific direction higher directivity – beam shaping is possible • Beamforming for hands-free communication : – focus beam on speech source(s) speech enhancement and dereverberation – put spatial nulls in direction of noise sources noise reduction • Classification: – fixed beamforming: data-independent fixed filters gm[k] e. g. delay-and-sum, weighted-sum, filter-and-sum – adaptive beamforming: data-dependent adaptive filters gm[k] e. g. LCMV-beamformer, Generalized Sidelobe Canceller 14/12/01 Simon Doclo DSP Everywhere… p. 37

Delay-and-sum beamforming (1) • Microphone signals are delayed and summed together array can be virtually steered to angle • Angular selectivity is obtained, based on constructive ( = ) and destructive ( ) interference • Uniform delay-and-sum beamforming implies – Uniform array equal inter-microphone distance – Uniformly distributed delays 14/12/01 Simon Doclo DSP Everywhere… p. 38

Delay-and-sum beamforming (2) • Spatial directivity pattern H( , ) for uniform DS-beamformer M=5 microphones d=3 cm inter-microphone distance =60 steering angle fs=16 k. Hz sampling frequency • H( , ) has sinc-like shape and is frequency-dependent 14/12/01 Simon Doclo DSP Everywhere… p. 39

Delay-and-sum beamforming (3) • For an ambiguity, called spatial aliasing, occurs. This is analogous to time-domain aliasing where now the spatial sampling (=d) is too large. M=5, =60 , fs=16 k. Hz, d=8 cm Spatial aliasing 14/12/01 Simon Doclo DSP Everywhere… p. 40

Filter-and-sum beamformer • Better directivity patterns than DS-beamformer are obtained with weighted-sum and filter-and-sum beamformers – e. g. Frequency-independent directivity pattern M=8 Logarithmic array L=50 =90 fs=8 k. Hz 14/12/01 Simon Doclo DSP Everywhere… p. 41

Adaptive beamforming • Adaptive filter-and-sum structure: – Minimize noise output power, while maintaining a chosen frequency response in look direction (and/or other linear constraints) – LCMV = Linearly Constrained Minimum Variance • minimize variance of output z[k] • in order to avoid desired signal to be distorted or cancelled out, J linear constraints are added 14/12/01 Simon Doclo DSP Everywhere… p. 42

Generalized Sidelobe Canceller (1) • GSC consists of three parts: – Fixed (delay-and-sum) beamformer, in order to achieve spatial alignment of speech source speech reference – Blocking matrix, placing spatial nulls in the direction of the speech source noise references – Multi-channel adaptive filter with delay Postproc 14/12/01 Simon Doclo DSP Everywhere… p. 43

Generalized Sidelobe Canceller (2) • Blocking matrix Ca : – creating maximum M-1 independent noise references by placing spatial nulls in look-direction – different possibilities: e. g. Griffiths-Jim, Walsh broadside • Problems of GSC: – impossible to reduce noise from look-direction – reverberation effects cause signal leakage in noise reference adaptive filter is only updated when no speech is present ! 14/12/01 Simon Doclo DSP Everywhere… p. 44

Overview • Introduction • DSP in digital communications systems: – x. DSL-modems: modulation, equalisation • DSP in audio applications: – Hands-free communication: echo, noise and reverberation – Basic techniques: • Acoustic echo cancellation (AEC) • Multi-microphone beamforming – Application: hearing aids • Conclusion 14/12/01 Simon Doclo DSP Everywhere… p. 45

Application: Hearing Aids (1) • Hearing problems are very common nowadays • Most of the users are dissatisfied with the performance of their hearing aid in noisy environments (cocktail party effect) increase speech intelligibility by reducing background noise • Traditional hearing aids: – one microphone, analog, limited signal processing – amplification of all incoming sound without distinction between different sound sources • Enabling technologies: – microphone miniaturisation integrate multiple microphones into one hearing aid – micro-electronics: size ASIC < 10 mm 2, low power consumption – advanced DSP techniques (noise reduction, feedback suppression) 14/12/01 Simon Doclo DSP Everywhere… p. 46

Application: Hearing Aids (2) • Improvement of speech intelligibility by reduction of background noise • BTE hearing aid with 2 (or more) closely-spaced microphones • GSC in switched mode: • Beamfomer : weights can be adapted during speech • Noise suppression (ANC) : only adaptation during noise • Speech detection : determine when speech is present 14/12/01 Simon Doclo DSP Everywhere… p. 47

Conclusion • DSP-techniques can be found in many everyday products: – audio applications: CD, Mini. Disc, hands-free telephony – communications: GSM, modems, WLAN – medical applications: hearing aids, cochlear implants • Implementation differences: – sampling rate, memory requirements, complexity • Basic techniques: – filters, filterbanks, FFT/IFFT frequency filtering – adaptive filters track changing systems – multi-sensor systems spatial filtering 14/12/01 Simon Doclo DSP Everywhere… p. 48