Ling 411 18 The Cortical Representation of Verbs

Ling 411 – 18 The Cortical Representation of Verbs and Nouns

Schedule of Presentations Tu Apr 13 Th Apr 15 Tu Apr 20 Th Apr 22 Delclos Planum Temp Banneyer Categories Ruby Tso Writing Bosley Synesthesia Rasmussen 2 nd language Brown Lg&Thought Koby Music Mauvais LH-RH anat. Tsai Tones

The “Mental Lexicon” § A widely used term • Usually used by people who think of it as a symbol-based system – for example, a list of symbolic forms inside the brain somewhere • Introjection based on a familiar external object: the printed dictionary § (Clearly, a bad idea) § Connectionist perspective • Information corresponding to an external “lexical entry” is widely distributed, not in one place

Observation § Wernicke’s aphasics (posterior lesion) have more difficulty with nouns than verbs and Broca’s aphasics (anterior lesion) have more difficulty with verbs than nouns § Question: If the “mental lexicon” is indeed a widely distributed network, how do we interpret the apparent anterior/posterior differentiation between nouns and verbs? § Answer: cardinal nodes

Functional web for a noun showing mostly only cardinal nodes T Art C PR PA V Cardinal node for the whole functional web – in angular gyrus – likely location for many nouns – maybe in temporal lobe for some nouns

Functional web for a lexical item showing mostly only cardinal nodes Maybe there is also a lemma node T Art C P PA V T – Tactile C - Conceptual Art – Articulatory Image P – Phonological Image PA – Primary Auditory V – Visual

The lemma hypothesis § Lemma: term used by Willem Levelt, director of the Max Planck Institute for Psycholinguistics, Nijmegen, NL • And others § Lemma – cf. morpheme • But we have to allow also for complex morphemes (e. g. , compounds) § blackbird § Rice University § baseball team § Proposal: Lemma node as cardinal node of the functional word web

Lemmas Cf. morphemes § Three layers of language structure • • • Phonological Grammatical Semantic-Conceptual § With cardinal concept node § The grammatical layer • Intermediate between phonology and semantics • • Morphemes Complex morphemes (sometimes called lexemes) § hot dog § cold turkey Constructions § Units of the grammatical layer •

Three layers (strata) of language structure § Phonological • Demisyllables, phonological words § Lexico-grammatical • Lemmas, constructions • Lemmas: morphemes, complex morphemes § Conceptual • a. k. a. semological § Each stratum has its own structure

Phonology vis-à-vis Grammar § Analyzing sisters • • Phonological – two syllables § sis-ters Grammatical – two morphemes § sister-s § Same phonological form, different morphemes • • ring n. (e. g. , on a finger) ring v. (what a bell does) • • good bett- (as in bett-er) § Same morpheme, different phonological forms

Lexico-gramatical vis-à-vis conceptual § One lemma, different concepts • ring (n. ) § Jewelry on a finger § as in ‘ring of thieves’ § as in ring around the bathtub § as in boxing ring § One concept, different lemmas • 20 § twenty § score

Simple and complex lemmas: hotdog Concepts HOTDOG HOT DOG Complex lemma: hot dog Simple lemmas: hot, dog hot dog

More complex lemmas § Combinations of morphemes • • • sister-s hot dog cold turkey dog-ear-ed dog-day-s dog-star

Zhong Guo CENTRAL CHINA KINGDOM Complex lemma: zhongguo Simple lemmas: zhong, guo zhong guo

Coexisting Parallel Structures hamburger Hamburg ham cheese burger burg -er

Terminology: lexeme and lemma § Lexeme: • The information pertaining to a lexical item • Represented externally as a dictionary entry • Represented cortically as a functional web § Lexical functional web § Lemma: • The cardinal node of a lexical functional web

Lemma nodes § Connectionist perspective • Lemma node as cardinal node of the functional web for a lexical item (a. k. a lexical functional web) § Locations of lemma nodes • Hypothesis: Lemma nodes for different grammatical categories are located in different cortical areas § AG or MTG (or SMG? ) for noun lemmas § Frontal lobe for verb lemmas

Review of Recent Research 1. Research in support of discrete representations for nouns and verbs 2. Research in support of common representations for nouns and verbs 3. Proposed explanations for distinct representations for nouns and verbs in the mental lexicon 4. Evidence for/against different accounts

Mental representations of nouns and verbs § Aphasiology and experiments on neurologically intact individuals provide evidence for discrete representations of nouns and verb lemmas in the cortex • Nominal representations in the temporal lobe • or angular gyrus Verbal representations in the frontal lobe

Cortical representation of nouns and verbs: Types of Evidence § Dissociations are commonly observed in aphasic patients § Patients with frontal lobe damage tend to have trouble with verbs § Patients with posterior damage tend to have trouble with nouns § Semantic considerations: • Processes are controlled in frontal lobe • Objects are apprehended in temporal/parietal areas

Broca’s Aphasia § § Damage to frontal lobe Largely intact comprehension Nonfluent, agrammatic speech “Telegraphic speech” – • Abundance of content words (e. g. , nouns) • Lack of function words (e. g. prepositions) § Impaired verb processing • Bates, Chen, Tzeng, Li & Opie, 1991; Damasio & Tranel, 1993; Daniele, Giustolisi, Silveri, Colosimo & Gainotti, 1994; Lamb & Zhang, 2002; Shapiro & Caramazza, 2003

Wernicke’s Aphasia § Damage to temporal lobe and/or angular gyrus and/or SMG § Impaired comprehension § Fluent, grammatical, neologistic speech § Impaired noun processing • Bates et al. , 1991; Damasio & Tranel, 1993; Daniele et al. , 1994; Lamb & Zhang, 2002

Evidence from Aphasia: ROX § Patient ROX • severely agrammatic as a result of cortical atrophy § ROX demonstrated impaired retrieval and comprehension of verbs, but normal retrieval and comprehension of nouns § Conclusion: ROX’s verb impairment with relative sparing of nouns indicates that nouns and verbs are discretely represented Mc. Carthy, R. & Warrington, E. K. (1985). Category specificity in an agrammatic patient: The relative impairment of verb retrieval and comprehension. Neuropsychologia, 23, 709 -727.

More Evidence from aphasia § Zingeser & Berndt (1990) § 5 agrammatics, 5 anomics, 5 normals § Single word elicitation and connected speech tasks • unambiguous nouns and verbs • frequency and length were controlled § Conclusion: agrammatics have a selective verb deficit, whereas anomics have a selective noun deficit

Cross-linguistic Evidence from EEG § Pulvermüller et al. (1996; 1999) § German-speaking normal subjects § EEG recordings during a lexical decision task § The 30 Hz responses were stronger at recording sites close to the motor cortices in response to verbs, whereas they were stronger at posterior occipital sites over visual cortices in response to nouns

More Cross-linguistic Evidence § Miceli et al. (1984) § 5 agrammatics, 5 anomics and 10 normal controls (Italian speakers) § Object and action naming tasks § Agrammatics demonstrated a marked deficit in action naming; they were selectively impaired in processing verbs § Conclusion: nouns and verbs are distinct word class categories which are organized separately within the cortex

More Cross-linguistic Evidence § Miceli et al. (1988) § 7 subjects with lesions in various loci (Italian) § Conclusion: not only is the lexicon organized into grammatical classes of words, but grammatical class information is redundantly represented in both input and output webs

Cross-linguistic Evidence from Daniele et al. 1994 § Italian § Patient R. A. • • Atrophy of the posterior frontal lobes, including the posterior portion of the left inferior frontal gyrus Verb deficit • • Atrophy of the posterior left frontal lobe Verb deficit • • Atrophy of the left temporal lobe Noun deficit • posterior frontal lobe important in the production and comprehension of verbs, temporal lobe important in the production and comprehension of nouns § Patient G. G. § Patient G. P. § Conclusion: •

Contrary Evidence: Common Representations? Warburton et al. , 1996 PET study 9 neurologically intact subjects Noun and verb retrieval resulted in similar activation distributions § Verb retrieval generated significantly stronger activations than noun retrieval in temporal, parietal and premotor/prefrontal areas § This evidence, though not clearly indicative of an anatomical dissociation between nominal and verbal representations, demonstrates some sort of distinction between the processing of nouns and verbs § §

Flawed studies? § Wise et al. (1991) and Warburton et al. (1996) § A flaw in their methodology may account for the widely distributed activation • The researchers employed a verb generation task in which the subjects were asked to silently produce verbs that came to mind in response to auditorily presented nouns • Of course this would result in widely distributed brain regions being activated! • The brain is processing both nouns and verbs in this task § Any conclusions drawn from this task cannot be considered evidence against discrete nominal and verbal representations

Summary of Published Evidence on representations of nouns and verbs § Most prior research supports the hypothesis of discrete representations of nouns and verbs in the cortex

Proposed Explanatory Accounts § Syntactic/morphological • Shapiro & Caramazza, 2003; Shapiro, Pascual-Leone, Mottaghy, Gangitano & Caramazza, 2001 § Semantic-conceptual • Mc. Carthy & Warrington, 1985; Bates et al. , 1991; Daniele et al. , 1994; Pulvermüller et al. , 1996; Pulvermüller, et al. , 1999 § Lexical • Miceli et al. , 1984; Mc. Carthy & Warrington, 1985; Miceli et al. , 1988; Caramazza & Hillis, 1991; Damasio & Tranel, 1993; Daniele et al. , 1994; Hillis & Caramazza, 1995; Rapp & Caramazza, 1997; Lamb & Zhang, 2002; Laudanna et al. , 2002

Syntactic/morphological argument § Verbs are more closely associated with syntax than nouns • Verbs are more difficult to process than nouns because verbs are inextricably associated with syntax • Verbs can never “stand alone” in the same way nouns can because verbs cannot be dissociated from their arguments § Verbs carry more inflectional morphology than nouns (Bates et al. , 1991) § Perhaps the selective verb deficit of an agrammatic aphasic may be the cause of his/her agrammatism (Mc. Carthy & Warrington, 1985)

Semantic-conceptual accounts § Prototypical nouns represent objects and prototypical verbs represent actions § Lexical representations for actions are located close to the motor cortex (frontal lobe) because this is where motor planning occurs § Lexical representations for objects are located close to the sensory cortices (temporal lobe), presumably because objects are known primarily through perception § Query: What about abstract nouns and non-action verbs?

Divisions of Primary Motor and Somatic Areas Primary Somatosensory Area Leg Primary Motor Area Trunk Arm Hand Fingers Mouth Primary Auditory Area Primary Visual Area

Higher level motor areas Actions per. Formed by leg Actions performed by hand … by mouth Leg Primary Somatosensory Area Trunk Arm Hand Fingers Mouth Primary Auditory Area Primary Visual Area

Higher level motor areas Semantic information for (many) verbs Leg Primary Somatosensory Area Trunk Arm Hand Fingers Mouth Primary Auditory Area Primary Visual Area

Abstract nouns and verbs § A possible explanation: During language acquisition, the first nouns and verbs learned are prototypical object nouns and action verbs § The proximity principle dictates that nouns become represented in the temporal lobe, verbs in the frontal lobe § At a later stage in the learning process, when non-action verbs and abstract nouns are learned, they join the already well-entrenched networks dedicated to verb and noun processing located in the frontal and temporal lobes, respectively

Problem with purely semantic account § Fails to explain how subjects such as ROX (Mc. Carthy & Warrington, 1985) are able to mimic actions even though they are deficient in the comprehension and production of verbs. • The ability to mimic actions demonstrates that the meaning of verbs remains intact and unaffected by the neurological damage • Subjects like ROX provide support for lexical accounts of the noun-verb dissociation in that their deficit appears to be at the level of the lexeme rather than the conceptual representation § Lamb & Zhang (2010) provide evidence against this account • the target jiao-hua ‘to water flowers’ elicited the response *hua-shui (hua ‘flower’, shui ‘water (n. )’) • Basic meaning is preserved § Miceli et al. (1984): The agrammatics in this study produced semantically relevant nominalizations when naming actions

Lexical accounts § Lemma nodes of nouns and verbs are differently represented in the cortex § Independent of semantics

Lexical Account § The lexical (lemma) account may provide the best explanation for the noun-verb dissociation • (Miceli et al. , 1984; Mc. Carthy & Warrington, 1985; Miceli et al. , 1988; Caramazza & Hillis, 1991; Damasio & Tranel, 1993; Daniele et al. , 1994; Hillis & Caramazza, 1995; Rapp & Caramazza, 1997; Lamb & Zhang, 2002; Laudanna et al. , 2002) § Much of the evidence provided earlier supports the probability that cardinal nodes of verb lexemes are represented in the left frontal lobe, whereas those of nouns are represented in posterior cortex

Evidence from Damasio & Tranel (1993) § AN-1033 and Boswell • • impaired noun retrieval lesions in the left fronto-parietal operculum, posterior temporal region, inferior parietal lobule, and left anterior and middle temporal lobe § KJ-1360 • • impaired verb retrieval lesion in the left premotor cortex § Conclusion: there are relatively separate lexical systems for nouns and verbs – specifically, the left anterior and middle temporal cortices for nouns and the left frontal cortices for verbs Damasio, A. R. , & Tranel, D. (1993). Nouns and verbs are retrieved with differently distributed neural systems. Proceedings of the National Academy of Sciences of the United States of America, 90, 4957 -4960.

Evidence from Rapp & Caramazza 1997 § Patient P. B. S. • lesion in the left parietal, posterior frontal and temporal regions which include both Broca’s and Wernicke’s areas • produced a greater number of written nouns than spoken nouns • verb use was not normal in either modality, but the extent of the spoken verbal deficit was not easily determined because of the large number of neologisms produced in the place of verbs § Conclusion: the lexicon is organized so that neurological damage can selectively affect knowledge specific to grammatical class and modality Rapp, B. & Caramazza, A. (1997). The modality-specific organization of grammatical categories: Evidence from impaired spoken and written sentence production. Brain and Language, 56, 248 -286.

TMS Evidence from Shapiro et al. 2001 § Suppressed the excitability of a portion of the left prefrontal cortex in normal subjects through the use of repetitive transcranial magnetic stimulation (r. TMS) § Task: produce singular and plural forms of nouns or third-person singular and plural forms of verbs; then perform same task with pseudo-words § A delay in the subjects’ productions of verbs indicated that verbs are processed in the left prefrontal cortex, whereas nouns are not § A semantic explanation for the noun-verb dissociation does not fully account for the difference because both action and non-action verbs were utilized for this task § Also, delay in production of grammatical information even when pseudo-words used as verbs

Evidence from Chinese (Bates et al. 1991) § Chinese has virtually no inflectional morphology for nouns or verbs § Therefore, if the noun-verb dissociation occurs in the speech of Chinese-speaking Broca’s and Wernicke’s aphasics, the dissociation cannot be due to a difference in nominal and verbal morphology Bates, E. , Chen, S. , Tzeng, O. , Li, P. , & Opie, M. (1991). The noun-verb problem in Chinese aphasia. Brain and Language, 41, 203 -233.

Chinese Evidence (Bates et al. 1991) § 6 Broca’s aphasics, 7 Wernicke’s aphasics and 11 normal controls § Results indicated that Broca’s aphasics have a selective verb deficit while Wernicke’s aphasics have a selective noun deficit § The morphological account must be dismissed at this point

Study by Sylvia Chen § Sylvia Chen, a graduate student of Elizabeth Bates at UC San Diego § Subjects: • Ten Broca aphasics • § Reduced fluency and phrase length § Tendency to omit function words Ten Wernicke aphasics § Impaired comprehension § Fluent or hyperfluent speech § Marked word-finding difficulties § Semantic paraphasias

Nouns and Verbs in Chinese § Most nouns and verbs are disyllabic • Most morphemes are monosyllabic • Therefore, the nouns and verbs are compounds • Common types: V-N, N-N § Many have more than two syllables § Such compounds are learned as units • Just like complex lexemes in any language § Cf. hot dog, Zhong-guo

Chen’s experiment § Patients were tested in their ability to name • Pictures of objects (nominal compounds) • Pictures of common actions (verbal compounds) § All of the compounds have the form V-N • 13 verbal V-N compounds • 28 nominal V-N compounds

Chinese V-N Compounds § Disyllabic compounds of form V-N § Some are nouns, some are verbs § fei v. ‘to fly’ + ji n. ‘machine’ • feiji n. ‘airplane’ § chi v. ‘to eat’ + fan n. ‘rice’ • chifan v. ‘to have a meal’

The Experimental Task § 10 Broca’s aphasics, 10 Wernicke’s aphasics § Test with nominal compounds • Produce a word to describe a picture of an object § Test with verbal compounds • Produce a word to describe a picture of an action (Sylvia Chen – UCSD dissertation)

Typical Errors of Broca aphasics (for nominal compounds) Target Components Response fei-ji ‘airplane’ fei ‘to fly’ ji ‘machine’ ji wan-ju ‘toy’ wan ‘play’ ju ‘instrument’ ju shui-yi ‘pajamas’ shui ‘to sleep’ yi ‘clothes’ yi wei-qi ‘go (game)’ wei ‘to surround’ qi ‘chess’ qi-lu (lu ‘strategy’)

Summary of findings § Broca aphasics • Difficulty producing verbal components for both verbal and nominal compounds § Wernicke aphasics • Difficulty with noun components of verbal compounds • More general and varied difficulties with nominal compounds § Responses did not indicate that patients had trouble with the semantics of the target items § This despite the fact that these are compounds and are doubtless learned as units by speakers of Chinese

“Potency” of components of compounds 1. Semantic potency § Q: Do the meanings of the constituents have a bearing on the meaning of the composite? • understand • hot dog • blackboard • bluebird § A: Sometimes yes, sometimes no • Complex lexemes have a scale of transparency • § From transparent (bluebird) § To opaque (understand) Opaque lexemes are known as ‘idioms’

Potency of constituents of compunds 2. Grammatical potency § Q: Do the grammatical categories of the constituents of a compound have a bearing on properties of the composite? § Evidence for positive answer: • In Chinese aphasics with impaired verb access, nominal compounds are also affected if they have verbal components

Evidence for intact semantics § The subjects did not have difficulty with the semantics of the pictures, but only with the means of providing linguistic representations § Example: • Target: jiao-hua v. ‘to water flowers • • § jiao v. ‘to water’ + hua n. ‘flower’ Response: *hua-shui (shui n. ‘water’) Indicates that the (Broca’s aphasic) patient understood the meaning while failing to produce the verbal component of the standard compound Sylvia Chen (UCSD dissertation)

Errors of Broca aphasics (for nominal compounds) Target Components Response fei-ji ‘airplane’ fei ‘to fly’ ji ‘machine’ ji wan-ju ‘toy’ wan ‘play’ ju ‘instrument’ ju shui-yi ‘pajamas’ shui ‘to sleep’ yi ‘clothes’ yi wei-qi ‘go (game)’ wei ‘to surround’ qi ‘chess’ qi-lu (lu ‘strategy’)

Conclusions of Chen’s Experiment § Verb components and noun components are represented differently in the cortex § This differential representation of the components is independent of the representation of the compound as a whole, even for compounds that are wellestablished and frequently occurring (i. e. well entrenched)

Why is this interesting? § If a lexeme is learned as a unit, why should the components make a difference? • If lexemes are stored as units, the grammatical categories of their components shouldn’t matter § If it is a noun, why should a Broca aphasic have trouble with it? • Of course, we know from the experiment that it is because it has a verbal component § Moreover, some of these compounds are well -entrenched § How to explain?

Inferences from Chen’s Experiment § Verbal components of compounds are represented in the frontal lobe, even when they are components of nominal compounds (like ‘airplane’) that, as nouns, are presumably represented in the posterior cortex § The situation can only be understood in the context of a distributed network (rather than symbolic) representation of the linguistic information

Cortical representation of a compound Anterior Posterior fei-ji n. ‘airplane’ delay fei v. ‘fly’ ji n. ‘machine’

Cortical representation of a compound Anterior Posterior wan-ju n. ‘toy’ delay wan v. ‘play’ ju n. ‘thing’

Nodes for phonological recognition (presumably in Wernicke’s area) wan ‘to play’ demisyllables wan wa- -an

Or, maybe it works according to the new hypothesis deb [de-] ded det de- den del dek A maxicolumn (ca. 100 minicolumns) English example Divided into functional columns (Note that all respond to /de-/)

According to the new hypothesis wan 1 wan 2 [wa-] wai 1 wa- wang 1 wang 2 wai 2 A maxicolumn (ca. 100 minicolumns) Divided into functional columns (Note that all respond to /wa-/)

Nodes for phonological production (presumably in Broca’s area) wan ‘to play’ PPwan w- -an

Phonological nodes for wan ‘to play’ PPwan w- -an PRwan wa- Internal feedback nodes from PP to PR not shown -an

Add (cardinal) concept node for wan PLAY PRwan PPwan w- -an wa- -an

Add node for wan-ju ‘toy’ Anterior But this proposal looks too simple Posterior wan-ju PLAY ju PPwan w- PRwan -an wa- -an

Why wouldn’t it work instead like this? Anterior Posterior wan-ju PLAY ju PPwan Likely area of damage w- PRwan -an wa- -an

First try for wan-ju ‘toy’ Anterior Posterior We need to add lemma/morpheme nodes wan-ju PLAY ju PPwan Area of damage? w- PRwan -an wa- -an

Add lemma node for wan with links Lwan PRwan PPwan w- -an wa- -an

Add concept node for wan PLAY Lwan PRwan PPwan w- -an wa- -an

Add lemma node for wan-ju ‘toy’ Lwan-ju Anterior Posterior PLAY Lju Lwan PRwan PPwan Presumed area of damage w- -an wa- -an

The beauty of this account § Consistent with the lack of impairment of semantic connections § The node for the compound is unimpaired • Represents an object • Therefore, likely to be in posterior cortex § Consistent with patient’s ability to comprehend speech § Consistent with diagnosis of Broca’s aphasia § The trouble is just with production of the verbal component (lemma) of the compound

Conclusions § Nouns and verbs are discretely represented in the cortex § Cardinal lemma and concept nodes for nouns are represented in posterior regions (temporal and/or parietal) § Cardinal lemma and concept nodes for verbs are represented in anterior regions (frontal lobe) § The differences between the representations of nouns and verbs pertain to their lemma nodes as well as to their respective semantic-conceptual representations

Reexamine speech in Wernicke aphasia § Question: How to explain the bizarre speech output of Wernicke’s aphasics • Inability to form phonological images • • • § (already observed) Inability to monitor output § (already observed) But maybe this failure mainly their inability to process nouns (and adjectives? ) Add: damage to nearby conceptual area(s) § Test: analyze some examples of Wernicke’s aphasia speech

Erratic Speech in Wernicke’s Aphasia “I feel very well. My hearing, writing been doing well. Things that I couldn’t hear from. In other words, I used to be able to work cigarettes. I didn’t know how…. Chesterfeela, for 20 years I can write it. ” From Harold Goodglass Understanding Aphasia (1993)

Attempt to describe cookie theft picture Uh, well this is the. . . the /dodu/ of this. This and this. These things going in there like that. This is /sen/ things here. This one here, these two things here. And the other one here, back in this one, this one /gesh/ look at this one. Examiner: Yeah, what's happening there? (next slide)

Wernicke’s aphasia example cont’d I can't tell you what that is, but I know what it is, but I don't now where it is. But I don't know what's under. I know it's you couldn't say it's. . . I couldn't say what it is. I couldn't say what that is. This shu-- that should be right in here. That's very bad in there. Anyway, this one here, and that's it. This is the getting in here and that's the getting around here, and that's it. This is getting in here and that's the getting around here, this one and one with this one. And this one, and that's it, isn't it? I don't know what else you'd want.

end