Eventreferent activation in the visual world Persistent activation

Event-referent activation in the visual world: Persistent activation is guided by both lexical and event representations Haley C. 1 University 1, 2 Dresang , 1, 3 Dickey , Michael Walsh & Tessa 1 Warren of Pittsburgh, 2 Center for the Neural Basis of Cognition, 3 VA Pittsburgh Healthcare System Figure 1: Sample visual display (for Example 1) Introduction Associatively-related competitor Atypical role-filler Comprehenders rapidly make use of lexically-specified and generalized event-related representations to guide predictive processing: Results • Lexically-specified verb-argument structure guides visual attention to both likely and unlikely potential arguments 1, 2 Typical role-filler • Generalized knowledge regarding event typicality activates likely Unrelated distractor upcoming event participants 3, 4 • Verb aspect activates event participants associated with distinct Gradient-competition models 6 predict that previously-activated information may not be (completely) suppressed, even in the face of incompatible bottom-up information. The current study examined the relative persistence of activation for referents favored by lexically-specified vs. generalized event representations. Example 2. Dative (recipient) verb conditions A= typical, B=atypical A. The girl scout wrote/had written a sweet note in crayon to the teacher after her visit last week. B. The girl scout wrote/had written a sweet note in crayon to the criminal after her visit last week. Example 3. Intransitive (location) verb conditions (from Ferretti et al. 5) A= typical, B=atypical A. The pitcher shielded/was shielding his eyes from the sun, out on the mound where he exhaled deeply. B. The pitcher shielded/was shielding his eyes from the sun, out on the deck where he exhaled deeply. Figure 2 A-B: Verb-centered window (marked with single underline in examples above) A: Methods (see Examples 1 -3). Each sentence was accompanied by four images: typical rolefiller (wrench), atypical role-filler (glue), associatively-related competitor (bike handle), and unrelated distractor (vase). 1 Proportion of fixations to Typical > Distractor Participants (n=32) heard sentences that varied in verbargument structure, event-related typicality and verb aspect 0. 8 0. 7 0. 6 0. 5 Stimuli were normed to ensure that images were accurately identified and that verbs were appropriately constraining in eliciting typical role-filler responses. “. . . was repairing the bike brakes …” “. . . repaired the bike brakes …” -400 -200 0 200 400 600 dat/ins preferred 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 Time Bins in ms (verb-centered) dat/ins simple loc preferred 0. 9 0. 8 0. 7 0. 5 “. . . was repairing the bike brakes …” “. . . repaired the bike brakes …” -400 -200 loc simple 0 200 400 600 dat/ins preferred Proportion of Fixations A: Typical Role-Filler Mentioned 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 Time Bins in ms (verb-centered) dat/ins simple loc preferred loc simple B: Atypical Role-Filler Mentioned 0. 7 0. 6 0. 5 “… the wrench …” 0. 4 0. 3 0. 2 -400 -200 0 200 400 typical role-filler fixations 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 Time Bins in ms (role-filler centered) atypical role-filler fixations unrelated distractor fixations • Aspect: Verb-type (dative/instrument vs. location) x Aspect interaction: Being in preferred (progressive) aspect resulted in more fixations to potential role-fillers, but only for intransitive-location verbs (β=-. 44, p=. 002; Figure 2 B) – replication of Ferretti et al. 5 Verb aspect may activate event participants associated with different temporalcausal stages of events. 5 (2) Persistent activation was found in the role-filler window for both atypical role-fillers (favored by lexically-specified representations) and typical role-fillers (favored by event-related representations), even when the opposite entity was explicitly mentioned. Two critical findings suggest that argument structure independently drives persistent activation of event referents: • People fixate the atypical role-filler (favored only by lexically-specified verb-argument representations) when the typical role-filler is mentioned. • The manipulation of argument structure further modulates preference to fixate unmentioned role-fillers. Conclusion 0. 5 0. 4 These results are consistent with gradient-competition models 6 and suggest that abstract, lexically-specified representations can have independent effects, dissociated from generalized event representations. 8 0. 3 0. 2 0 0 “… some glue …” 0. 1 • Verb-argument structure: Preference to fixate atypical role-fillers over distractors (β=1. 07, p<. 001; Figure 2 B) – lexically-specified verbargument information only information source favoring atypical>distractor fixations. (Though see paradoxical effect of verbargument structure: more fixations to atypical role-filler for adjunct intransitive-location verbs; β=-. 28, p<. 001, Figure 2 B). Role-fillers consistent with verb-argument representations are pre-activated upon hearing verbs. 1, 2 • Argument structure modulated effects, such that adjunct verbs (intransitive-location) elicited more fixations to the unmentioned rolefiller than argument verbs (dative/instrument; Figure 3 A: β=-. 73, p<. 001; Figure 3 B: β=-. 27, p=. 03). Verb aspect had no effects. 0. 6 0. 4 • Typicality: Preference to fixate typical role-fillers over distractors (β=1. 09, p<. 001; Figure 2 A), typical over atypical role-fillers (β=1. 09, p<. 001). Typical event participants are pre-activated upon hearing verbs. 3, 4 • Participants fixated the unmentioned role-filler more often than unrelated distractors when either atypical role-fillers (β=1. 15, p<. 001; Figure 3 A) or typical role-fillers were explicitly mentioned (β=1. 29, p<. 001; Figure 3 B). Persistent activation greater for typical role-filler. Figure 3 A-B: Role-filler window (marked with double underline in examples above) Linear mixed-effects models analyzed the distribution of fixations to images in: • Fixed-effect variables included time, argument structure (dative, instrument, intransitive verb types), aspect (temporally preferred: past perfect for datives, past progressive for instruments/locations; vs. simple past), and role-filler typicality (role-filler window only). Proportion of fixations to Atypical Role-Filler > Distractor, Verb-type x Aspect interaction 1 0. 9 0. 4 (1) a verb-centered time window, measuring predictive activation of expected/typical and unexpected/atypical rolefillers; and (2) a role-filler window (recipient/instrument/location), measuring persistent activation of the referents. B: Proportion of fixations to Typical Role-Filler > Distractor, Verb-type x Aspect interaction Proportion of fixations to Atypical > Distractor How might information activated by these knowledge sources persist during comprehension? Example 1. Instrument verb conditions (from Roland et al. 7) A= typical, B=atypical. A. Tim repaired/was repairing the bike brakes with nothing more than a wrench from his toolkit. B. Tim repaired/was repairing the bike brakes with nothing more than some glue from his toolkit. Proportion of Fixations temporal-causal event stages 5 (1) Replicated previous findings in the verb-centered window, indicating that these knowledge sources can pre-activate upcoming event referents. 1 -5 -400 -200 0 200 400 typical role-filler fixations 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 Time Bins in ms (role-filler centered) atypical role-filler fixations unrelated distractor fixations References: [1] Boland (2005); [2] Su (2013); [3] Mc. Rae & Matsuki (2009); [4] Metusalem et al. (2012); [5] Ferretti et al. (2007); [6] Kukona et al. (2014); [7] Roland et al (2012); [8] Warren et al. (2015)