How Does auditory perception organization works by Elvira

How Does auditory perception organization works ? by Elvira Perez and Georg Meyer Dept. Psychology, Liverpool University, UK Hoarse Meeting, Chrysler Ulm, Germany 28 th-30 th October, 2004

1. Introduction: Ears receive mixtures of sounds. We can tolerate surprisingly high levels of noise and still orientate our attention to whatever we want to attend. But. . . how the auditory system can do this so accurately?

1. Introduction: • Auditory scene analysis (Bregman, 1990) is a theoretical framework that aims to explain auditory perceptual organisation. • Basics: – Environment contains multiples objects • Decomposition into its constituent elements. • Grouping. • It proposes two grouping mechanisms: – 1. ‘Bottom-up’: Primitive cues (F 0, intensity, location) Grouping mechanism based on Gestalt principles. – 2. ‘Top-down’: Schema-based (speech pattern matching)

1. Introduction: • Primitive process (Gestalt. Koffka, 1935): – Similarity – Good continuation – Common fate – Disjoint locations – Closure

1. Introduction • Criticisms: Too simplistic. Whatever cannot be explained through the primitive processes, it is explained by the schema-based processes. • Primitive processes only work in the lab. • Sine-wave replicas of utterances (Remez et al. , 1992) – Phonetic principles of organization find a single speech stream, whereas auditory principles find several simultaneous whistles. – Grouping by phonetic rather than by simple auditory coherence.

3. Experiments (baseline): • The purpose of these studies is to explore how noise (a chirp) affects speech perception. • The stimulus used is a vowel-nasal syllable which is perceived as /en/ if presented in isolation but as /em/ if it is presented with a frequency modulated sine wave in the position where the second formant transition would be expected. • In the three experiments participants categorised the synthetic syllable heard as /em/ or /en/. • Direction, duration, and position of the chirp were the values manipulated.

3. Experiments Formant frequ. (Hz) vowel nasal 2700 2000 800 375 100 200 ms The perception of a nasal /n/ change to /m/ when adding a chirp between the vowel and nasal F 2

In 80% of the trials the participants heard the difference between up and down chirp. Experiment 1 Baseline/Direction vowel nasal 100 chirp up 200 ms chirp down

Experiment 2 Duration vowel nasal 100 200 ms

Experiment 3 Position vowel nasal 100 200 ms

5. Conclusions: Chirps from 4 ms to 20 ms duration and in a range of 1 k. Hz-2 k. Hz, independently of their direction, are apparently integrated into the speech signal and change the percept from /en/ to /em/. Subjects very clearly hear two objects, so that some scene analysis is taking place since the chirp is not integrated completely into the speech. Duplex perception with one ear. It seems that listeners can also discriminate the direction motion of the chirp when they focus their attention in the chirp and a more high level of auditory processing takes places (80% accuracy).

Mr. Background Noise • Do human listeners actively generate representation of background noise to improve speech recognition? • Hypothesis: Recognition performance should be highest if the spectral and temporal structure of interfering noise is regular so that a good noise model can be generated unpredictable noise.

Experiment 4 & 5 • Stimuli: chirps + /en/ • Ten subjects • The amplitude of the chirp vary (5 conditions: 0 d. B 8 d. B -14 d. B -20 d. B no-chirp) • Background noise (down chirps): – Quantity: Lots (170/20 ms) vs Few (19/20 ms) – Time of appearance: Regular vs Irregular • Categorization task 2 FC. • Threshold shifts.

Regular condition en en en Irregular condition en en en

Each point in the scatter is the mean threshold over all subjects for a give session. The solid lines show the Boltzmann fit (Eq. (1) for each individual subject in the fifth different conditions. All the fits have the same upper and lower asymptotes.

Exp. 4 rand/reg lots vs. few (t = -3. 34, df = 38, p = 0. 001). control vs. lots (t = -3. 34, df = 38, p = 0. 001). No effect between irregular and regular.

• Two aspects change from exp. 4 to 5: – Amplitude scale of the chirps (0 d. B -4 d. B -8 d. B 16 d. B no-chirp). – The conditions lots now includes 100/20’’ and before 170/20’’.

Exp. 5 rand/reg lots vs few (t = 2. 27, df = 38, p = 0. 05). control vs. lots (t = 3. 12, df = 38, p < 0. 05). No effect between irregular and regular.

5. Conclusions Only the amount of background noise seems to affect the performance of the recognition task. The regularity of the background noise seems an irrelevant cue to improve auditory stream segregation and therefore, speech perception. Counterintuitive phenomenon.

• Irrelevant sound effect (ISE) (Colle & Welsh, 1976) disrupts in serial recall. • The level of meaning (reverse vs forward speech), predictability of the sequence (random vs regular), and similarity (semantic or physical) of the IS to the target material, seems to have little impact in the focal task. (Jones et al. , 1990). • Changing state: The degree of variability or physical change within an auditory stream is the primary determinant of the degree of distrupion in the focal task.

Smooth change Frequency Abrupt change Time

Zoom in Top Down Up Bottom

Experiment 6 • Stimuli: Synthesised vowel-nasal + background FM tone + Chirps • Three blocks (200 trial each block): First Control, second Smooth or Abrupt (counterbalanced order) • Chirps: Four different frequencies: Up/Down/Top/Bottom • Five amplitudes: 0 d. B, -4 d. B, -8 d. B, -16 d. B, nochirp

Experiment 6 • • • Subjects: 42 Musicians vs Non musicians Female vs Male Nationality (27) Age(27. 7), languages spoken (3) Hearing issues (AP, RP, Tinnitus)

Results down vs up YES vs. smooth YES abrupt YES control vs No effect between smooth and abrupt. Tukey test p < 0. 001

Results Musicians vs. Non Musicians …no differences.

More analysis… • Take away the intermediate conditions – Results remain the same • Habituation – Differences between musicians and non musicians; overall, 10, 5, 3 first blocks, but not the first block. – Again control vs smooth/abrupt

6. Conclusions • It seems that listeners do not use pattern prediction as a cue for auditory perceptual organisation. • … or they do it extremely fast (3 ms), or is due to STM (pre-perceptual auditory storage) • Attention must be focused on an object (background noise) for a change in that object to detected (Rensink, et al, 1997) • … or we just ignore the information contained in the transitions for not being reliable.

Conflict area Formant frequ. (Hz) vowel 2700 2000 800 375 nasal

Ignore is not reliable Formant frequ. (Hz) vowel 2700 2000 800 375 nasal

Formant frequ. (Hz) 2700 vowel nasal 2000 800 375 2700 2000 800 375 vowel = /em/ nasal

• Which information are we taking in account? • Do we combine cues? • Do we measure the variances associated which each cue to test its reliability? • Maximum likelihood integrator • Antecedents: The nervous system seems to combine visual and haptic info similar to a MLE (Ernst, et al. 2002)

transition nasal noise

Conditions Vowel Transition Nasal e e ( ( ) ( n Full syllable n No transition n n ) No nasal ) No transition/nasal

Preliminary results

New Methodology • Until now Method of Constants: Several stimulus levels are chosen beforehand, and groups of observations are placed at each of these stimulus levels. The order of observations is randomized. A conventional method of estimation is used in fitting the psychometric function to the resulting data. • Adaptative procedures: The stimulus level on any one trial is determined by the preceding stimuli and responses. – Sequential experiment: The course of the experiment is dependent on the experimental data.

• Up-Down procedures or staircase method: The stimuli level (amplitude of the speech signal) is decreased after a positive response (or increase after a negative). • On each trial, the participant is required to give both a binary judgment (em/en/e) and a confident rating. The binary judgment are used to decide on the direction of change in the stimulus level, and the confident ratings are used to decide on the steps size (d. B). • Advantages: Most of the observations are placed at or near the 50% level

Conclusions • …In the next meeting.

Thank you