Speech recognition with amplitude and frequency modulations Implications

Speech recognition with amplitude and frequency modulations: Implications for cochlear implant design Fan-Gang Zeng Kaibao Nie Ginger Stickney Ying-Yee Kong Ashish Bhargave Hongbin Chen Michael Vongphoe Janice Chang • What’re AM and FM? • What are their perceptual roles? • Where to find it? • Implications?

What is fine structure? • Rosen’s definition: – Envelope (5 -50 Hz) – Periodicity (50 -500 Hz) – Fine structure (500 -10, 000 Hz) • Hilbert’s definition: – Temporal envelope – Fine structure Original AM Fine Structure FM

Little math • Flanagan (1980) “Parametric coding of speech spectra”. – Discard absolute phase: . – Discard relative phase (i. e. , frequency modulation):

Implementation • Combo of Dudley’s vocoder and Flanagan’s phase vocoder Zeng, Nie, Stickney et al. PNAS (2005)

Spectra: What does FM encode? Zeng, Nie, Stickney et al. PNAS (2005)

Sentence, speaker, and tone recognition Combo: Target: Masker: Zeng, Nie, Stickney et al. PNAS (2005)

Comparison with previous studies Shannon et al. 1995 Dorman et al. 1997 Zeng, Nie, Stickney et al. PNAS (2005) Zeng et al. 2005

Spectral resolution and noise type 30 -d. B SRT Zeng, Nie, Stickney et al. PNAS (2005)

Speech recognition in combined hearing HA+CI CI HA 10 -d. B SRT Kong, Stickney, and Zeng JASA (2005)

Frequency FM detection in CIs: Results Time Chen and Zeng JASA (2004)

Summary Using FM to improve auditory performance: – Speech cues are not redundant: FM complements AM in speech perception – FM is important for speech recognition with competing voice as maskers – FM is important for music and tonal language perception – FM is a slow version of fine structure that can be perceived and used to improve cochlear implant performance

Acknowledgements • • • NIH - NIDCD Chinese NSF Advanced Bionics Corp Cochlear Corp Medel Peter Assmann Ann Bradlow Keli Cao and CG Wei Larry Feth Ruth Litovsky Jones Ackland