Effects of attention on auditory scene analysis Bob

Effects of attention on auditory scene analysis. Bob Carlyon MRC Cognition and Brain Sciences Unit, Cambridge, England With more than a little help from: Rhodri Cusack, Matt Davis, John Deeks, Jessica Foxton, Olaf Hauk, Antje Heinrich, Ingrid Johnsrude, Friedemann Pulvermuller, Yury Shtyrov, Sarah Thompson

Auditory scene analysis and attention In everyday life we listen to sounds such as speech in the presence of interfering sounds The auditory system has to: Determine which parts of the stimulus belong to the same source (cf. grouping or binding in vision) Deal with instances where portions of the target voice are masked I’ll describe two phenomena that reflect how the auditory system performs these tasks, & discuss the role of attention in each. Compare results between: Perform some task on the stimulus of interest throughout Perform a demanding competing task on another stimulus

Auditory Streaming Tone sequences can be perceived as either fused or segregated: Frequen cy Fused (galloping rhythm - “horse”) B A A Frequen cy Segregated (individual pulsing rhythms - “morse”) B A A Time “Morse” percepts increase with freq separation, rate, and build up over time

The “horse morse” task Our question: does build-up of streaming still occur if you’re not attending to the tones? Not our question: Is attention needed for all grouping processes?

Experiment 1 Baseline Condition “Horse morse” sequence presented for 20 sec to left ear (over headphones). Frequency separation between tones: 4, 6, 8, or 10 semitones Task: press button 1 or 2 depending on whether you hear “horse” or “morse”. Every time your percept changes, press the other button Two-task condition As above, except that during first 10 seconds a sequence of noise bursts presented to right ear. Each noise burst chosen at random to sound like it was either “approaching” or “departing” Task: respond “approach” or “depart” to each burst. After 10 sec, a message flashes on screen, noise bursts stop, and subject reverts to the “horse morse” task. One task condition Noise bursts are present, but subject ignores them and does “horse-morse” task throughout

Results When attention is diverted, build-up of streaming is reduced Or. . the act of switching attention to tones “resets” streaming

Attention & Streaming: Further findings Not due to switching tasks: if subjects do an orthogonal task on tones during 1 st half of sequence, and then start making streaming judgements, streaming has built up A similar effect can be obtained by requiring subjects to count backwards in 3 s silently, or perform a visual competing task

Attention & Streaming: Objective measure Build-up & attention effects can be measured objectively using task that can only be done when sequence heard as 1 stream (Thompson et al): Left ear: 13. 5 -sec ABA tone sequence, target early or late in sequence, delay on B tone: Right ear: Approach-depart noises throughout. Attend to tones throughout or Attend to noises for 6. 75 sec, then attend to tones

The continuity illusion (“CI”) When a sound is turned on and off briefly, and the silent gap is filled by an “inducing sound”, it can be heard as continuous. Allows us to perceive a sound as “whole” even when portions of it are masked. We’ve previously shown a neural correlate of the illusion using an EEG measure (“Mismatch Negativity”; Micheyl et al) Here we study the effects of CI on perception of speech sounds: Behavioral study (Carlyon et al, 2002, Acta Acustica) Neural (f. MRI) correlate (Heinrich et al, 2008, JCN) Effect of attention on neural correlate (Heinrich et al, brand new)

CI and vowel identification Performance in Illusion condition > Alternating condition Simultaneous: Alternating: FREQ--> Carlyon et al (2002) measured vowel identification under 3 main conditions: Illusion: TIME-->

CI and vowel identification: f. MRI study Main comparison is between “illusion” condition and a new condition, “illusion break”, in which formant level is increased & noise level decreased: Stimuli broadly similar to those in behavioural expt. Previous finding: primary auditory areas show more activation to stimuli with lots of onsets & offsets (Cusack, Carlyon, Johnsrude, & Epstein, 2001) We predicted an interaction: “Primary” areas more activated by “illusion break” “Speech sensitive” areas more activated by “illusion”

CI and vowel identification: f. MRI study “Illusion” & “Illusion break” conditions, plus 6 others designed to give speech or non-speech percepts: 8. 4 -sec sequences of 10 sounds. Each sequence= 1 condition 19 RH Subjects detect rare “soft” sounds Siemens 3 T Tim Trio system; head gradient coil Sparse imaging procedure

CI and vowel identification: f. MRI study 1) Define “Speech area” (Speech-Non Speech conditions) 2) Define “General Sound” area (Non Speech-Silence) 3) Compare activation of illusion vs illusion break conditions in the 2 areas Speech area: Bilateral posterior MTG General Sound area: Bilateral middle STG

CI and vowel identification: f. MRI study Speech area: Illusion> Illusion Break General Sound Area: Break > Illusion

Effect of attention on neural correlate of CI Repeat Heinrich et al study with 4 stimulus conditions (Vowel 75, Vowel 300, Illusion, Break). In all conditions the sounds were accompanied by Sequence of noise bursts in a remote frequency region Sequence of visual stimuli (oval balls) presented on a screen 3 Attention conditions: Listen for occasional attenuated sound in vowel-like sequence Detect an occasional visual deviant (“broken ball”) Detect occasional noise deviant (“depart sound”) Hypothesis: If CI depends on attention, then activation in speech region should be reduced when attention diverted in the “Illusion” condition, & more so than in Vowel 75 and Vowel 300 conditions

Effect of attention on neural correlate of CI Replicate location of Speech-sensitive region Attending to auditory or visual stimuli increased activity in auditory & visual sensory areas, respectively. Effects of attention on Vowel-Break and Illusion-Break Voxel with largest sound X attention interaction in each hemisphere, restricted to “speech” region from Heinrich et al (2008): RH: Attention affects illusory and “real” vowels similarly LH: Attention doesn’t affect illusory vowels …no evidence for affect of attention on the Continuity Illusion

Summary Attention has a strong effect on auditory streaming. Either: Diverting attention prevents build-up. Switching attention back to sequence “resets” streaming. Effects can be obtained with: Auditory, visual, or non-sensory competing tasks Subjective or performance-based measures No evidence for an effect of attention on continuity illusion When studying effects of attention on auditory scene analysis: Need to use a demanding competing task Effect of attention unlikely to be the same for all aspects of scene analysis (cf. harmonicity; Alain et al)