Evaluating semantic similarity and sameness in studies of

  • Slides: 51
Download presentation
Evaluating semantic similarity and sameness in studies of polysemy and synonymy Jarno Raukko (U.

Evaluating semantic similarity and sameness in studies of polysemy and synonymy Jarno Raukko (U. Helsinki) 1

For a full version of the PPT, see handout distributed Oct 28, 2010. SKY

For a full version of the PPT, see handout distributed Oct 28, 2010. SKY WEBPAGE VERSION 2

Examples 1. Are thrifty and stingy synonyms? EXPECTED ANSWER: ”Well, not quite. ” 2.

Examples 1. Are thrifty and stingy synonyms? EXPECTED ANSWER: ”Well, not quite. ” 2. Are violin and fiddle synonyms? EXPECTED ANSWER: ”Well, almost. ” (SYNONYMY) 3. Does back have the same meaning in My back hurts and I came back? EXPECTED ANSWER: ”Not at all. Different. ” 4. Does back have the same meaning in I came back and I got it back? EXPECTED ANSWER: ”Well, almost. ” (POLYSEMY) 3

Relevance of semantic similarity (vs. difference) • In synonymy: you expect similarity for a

Relevance of semantic similarity (vs. difference) • In synonymy: you expect similarity for a pair/(set) of items to be of interest • In polysemy: primarily, you expect difference for a pair/(set) of items to be of interest; secondarily, you group items according to similarity and difference 4

Yet… SIMILARITY DIFFERENCE SYNONYMY POLYSEMY (DEFAULT) Synonymy is about similar meanings of different words.

Yet… SIMILARITY DIFFERENCE SYNONYMY POLYSEMY (DEFAULT) Synonymy is about similar meanings of different words. But you are interested in the differences between near-synonyms. Polysemy is about related (and therefore somehow similar) meanings of a word. (DEFAULT) Polysemy is about different (related) meanings of a word. 5

synonymy --- polysemy ? • Dirk Geeraerts tomorrow in Helsinki: ”The problem of synonymy

synonymy --- polysemy ? • Dirk Geeraerts tomorrow in Helsinki: ”The problem of synonymy and the problem of polysemy are essentially the same” • Dylan Glynn & Justyna Robinson (eds, in press) Polysemy and Synonymy. Corpus methods and applications in Cognitive Linguistics. Amsterdam: Benjamins. 6

synonymy WORD 1 WORD 2 If their semantic content is similar or the same,

synonymy WORD 1 WORD 2 If their semantic content is similar or the same, this is a case of synonymy. If their semantic content is (very) different, a researcher of synonymy ignores this case. 7

polysemy MEANING 1 OF WORD 1 MEANING 2 OF WORD 1 The starting point

polysemy MEANING 1 OF WORD 1 MEANING 2 OF WORD 1 The starting point is that Word 1 has at least 2 (different) meanings. If meanings 1 and 2 are very similar, this might be a case of vagueness. If meanings 1 and 2 are totally different (and not related semantically), this might be a case of homonymy. If meanings 1 and 2 are somewhat different but somehow relatable (or a bit similar), this is probably a case of polysemy. henceforth W = word M = meaning 8

scale of similarity: synonymy The meaning of W 1 and W 2 is… THE

scale of similarity: synonymy The meaning of W 1 and W 2 is… THE SAME ----------------- DIFFERENT perfect synonymy full synonymy near synonymy semi synonymy weak synonymy quasi synonymy NOT WORTH DISCUSSION 9

scale of similarity: polysemy The meaning of M 1 and M 2 (of W

scale of similarity: polysemy The meaning of M 1 and M 2 (of W 1) is… THE SAME ----------------- very DIFFERENT two vaguepolysemy homonymy instances ness (ambiguity) of the same meaning two instances of the same of different (yet related) meaning types 10

Main question • Is semantic similarity somehow different when we look at polysemy than

Main question • Is semantic similarity somehow different when we look at polysemy than we look at synonymy? 11

Differences so far • Which is the default, similarity or difference? • In synonymy,

Differences so far • Which is the default, similarity or difference? • In synonymy, we idealize on the extreme of the scale, but mainly look at the part of the scale which is (fairly) close to the extreme. • In polysemy, we operate pretty much on the whole scale, with focus on the middle. 12

synonymy --- polysemy ? • when you study synonymy, the polysemy of the items

synonymy --- polysemy ? • when you study synonymy, the polysemy of the items gets in the way – can you ever say “W 1 and W 2 are synonymous”? – should you always say “Mx of W 1 and My of W 2” are synonymous? • when you study polysemy, you often use synonyms to talk about meanings – “Are get ‘receive’ and get ‘arrive’ meanings of the same verb? ” 13

synonymy --- polysemy ? • Synonymy occurs when meaning is shared (but form differs)

synonymy --- polysemy ? • Synonymy occurs when meaning is shared (but form differs) • Polysemy occurs when form is shared (but meaning differs) • Synonymy is a relational lexical-semantic property that unites (parts of the semantic potential of) “accidentally” coinciding words – The forms of words involved in the synonymy relationship are arbitrary (although the relationships might be nonarbitrary, cf. Levin this morning) – The semantic value (that is shared) is motivating enough that two or more forms coincide on it – It is typical that one meaning can be expressed with two different words. 14

synonymy --- polysemy ? • Polysemy is a semantic property of one word at

synonymy --- polysemy ? • Polysemy is a semantic property of one word at a time that unites meanings. The relationship between them is motivated, but it is only sometimes predictable. – It is not accidental or arbitrary that words acquire polysemy. It is in their nature. : -) – It is typical for semantic value to be flexible, extended, and “multiplied”. – Polysemy is about categorization, both between words (W 1 covers a semantic territory) and within a word (M 1 and M 2 are categories too). • One form : One meaning • a principle that cognition may strive for / take as a default • synonymy breaks it • polysemy breaks it 15

synonymy --- polysemy ? • The role of co(n)text • You can evaluate synonymy

synonymy --- polysemy ? • The role of co(n)text • You can evaluate synonymy in identical co(n)texts: I like to play the fiddle in bars. I like to play the violin in bars. • Usually you evaluate polysemy in non-identical co(n)texts I got to Zabriskie Point. I got to a point in my life where… • But you can use identical co(n)texts as well. I got to be the last one. 16

evaluating • To study shades of semantic similarity, we need to evaluate it. •

evaluating • To study shades of semantic similarity, we need to evaluate it. • A corpus cannot tell us if two instances are semantically similar – It requires human judgement • The main use of evaluating in this paper: – How informants / test subjects / speakers evaluate the semantic similarity (or difference) of linguistic items in a more or less experimental setting (e. g. , similarity rating test) ≈ Data elicitation ≈ Population test 17

evaluating • quantitative: – Estimate the degree of synonymy (or semantic distance between two

evaluating • quantitative: – Estimate the degree of synonymy (or semantic distance between two meanings in polysemy) • qualitative: – Justify / explain / explicate the nature of / the reason for semantic similarity 18

evaluating takes place in real life as well • • synonymy (examples) – in

evaluating takes place in real life as well • • synonymy (examples) – in linguistic production, you e. g. estimate which of the near-synonyms might suit your needs best – in comprehension, you e. g. estimate whether near-synonyms that you have encountered refer to the same semantic value – in communication, when you negotiate meaning, you e. g. operate with synonymous alternatives polysemy (examples) – in production, you e. g. apply words to new contexts – in comprehension, you e. g. approximate meanings according to related meanings of the same word – jokes often exploit polysemy – polysemy may cause misunderstandings – in communication, when you negotiate meaning, you e. g. cross-check with polysemy of other words 19

(Back to experiments/elicitation. ) Expected difference between synonymy and polysemy, 1 • If an

(Back to experiments/elicitation. ) Expected difference between synonymy and polysemy, 1 • If an informant is asked to rate the semantic similarity/difference of two words, – the very fact that they are different words might cause her/him to presuppose that there is at least some semantic difference. – Therefore, rating two words ”semantically identical” requires a marked choice. – However, if the informant realizes that the researcher is after synonymy, then evaluating W 1 and W 2 as semantically similar is more likely. 20

Expected difference between synonymy and polysemy, 2 • If an informant is asked to

Expected difference between synonymy and polysemy, 2 • If an informant is asked to rate the semantic similarity/difference of two meanings of one word, – the very fact that they are uses/instances of the same word might cause her/him to presuppose that there is at least some semantic similarity. – Therefore, rating two words ”semantically totally/very different” requires a marked choice. – However, if the informant realizes that the researcher is after polysemy, then evaluating M 1 and M 2 as semantically different is more likely. 21

Factors that influence • In both cases (synonymy and polysemy) – it matters a

Factors that influence • In both cases (synonymy and polysemy) – it matters a great deal • Which test (type) we use • What the instructions (exact phrasings) are • Whethere is an example rating given by the researcher • What the selection of stimuli is • What the linguistic context of each stimulus is • Which types of cases have been placed in the beginning of the test (or, the order in general) 22

Factors that influence • Should we expect (total) consensus? • No. There will be

Factors that influence • Should we expect (total) consensus? • No. There will be subjective differences. • Why? • The nature of semantics: • Based on intersubjective convention • Based on negotiation and flexibility • Must allow for variability and variation 23

Examples from (more or less) experimental studies on synonymy and polysemy 24

Examples from (more or less) experimental studies on synonymy and polysemy 24

Whitten & al. 1979 (synonymy) • “Indicate the degree to which two words have

Whitten & al. 1979 (synonymy) • “Indicate the degree to which two words have the same meaning by writing a digit from 1 to 7. ” • 7 =excellent synonymy • 1 = poor synonymy • All 464 stimulus noun pairs were listed as synonyms in standard references. • The rated degree of synonymy ranged from 6. 79 to 2. 24. The median was 5. 08. • If placed within context of nonsynonym pairs, the ratings for the low end might have been higher. 25

Whitten & al. 1979 (synonymy) cont’d • Stimulus pairs at the high end: •

Whitten & al. 1979 (synonymy) cont’d • Stimulus pairs at the high end: • purchase – buy 6. 79 • lawyer – attorney 6. 78 • autumn – fall 6. 72 • penny – cent 6. 71 • taxi – cab 6. 71 • Stimulus pairs close to the median • college – university 5. 12 • output – yield 5. 10 • expert – authority 5. 09 • effort – attempt 5. 08 • servant – maid 5. 08 • soldier – warrior 5. 07 26

Whitten & al. 1979 (synonymy) cont’d • Stimulus pairs at the low end: •

Whitten & al. 1979 (synonymy) cont’d • Stimulus pairs at the low end: • thunder – clap • patient – invalid • visit – chat • suburb – neighborhood • needle – spike 2. 72 2. 55 2. 52 2. 34 2. 24 • Although instructions said that all stimuli are nouns, some of these are more common as verbs: buy, purchase, visit, chat • The polysemy is obvious in many cases: fall, authority, clap, patient, invalid 27

Whitten & al. 1979 (synonymy) cont’d • The main variable that they paid attention

Whitten & al. 1979 (synonymy) cont’d • The main variable that they paid attention to was the order of the two stimuli: ½ of the informants got “forward order”, ½ got “back order”. – In 1979 one of their main aims was to study the structurings of the mental lexicon and lexical access. – Example: purchase => buy 6. 72 buy => purchase 6. 86 – On average, perceived synonymy was affected by word order. – For 21 word pairs, the effect of the order was significant. 28

Whitten & al. 1979 (synonymy) cont’d • Some of the 21 word pairs where

Whitten & al. 1979 (synonymy) cont’d • Some of the 21 word pairs where the order played a significant role in the rating of the degree of synonymy: motive => reason quarter => fourth mission => task era => age appetite => hunger nectar => honey aborigine => native 6. 28 6. 24 5. 66 5. 80 5. 18 4. 94 4. 52 reason => motive fourth => quarter task => mission age => era hunger => appetite honey => nectar native => aborigine 5. 56 5. 00 4. 84 4. 60 4. 24 3. 68 3. 22 • Generalization: a more specific, more academic, and less polysemous word prompts a positive synonymy judgement more readily than vice versa. 29

Whitten & al. 1979 (synonymy) cont’d • Variance (between informants) – Mostly. 50– 1.

Whitten & al. 1979 (synonymy) cont’d • Variance (between informants) – Mostly. 50– 1. 20 at the end of 50 most synonymous • Exceptionally high variance at the high synonymy end: – murder => homicide 2. 75 (cf. homicide => murder 1. 03) – Mostly 2. 00– 3. 00 at the median of the scale • Exceptionally low variance: province => territory 1. 55 • Exceptionally high variance: congress => legislature 3. 79 – Mostly 2. 50– 4. 00 at the end of 50 least synonymous • That is, there was little consensus at the lower end of 30 the scale.

Raukko 1994 (polysemy) • “Decide whether the word get carries the same meaning or

Raukko 1994 (polysemy) • “Decide whether the word get carries the same meaning or two different meanings in the sentences. ” – 0 = the same meaning – 2 = somewhat different meaning – 4 = very different meaning (heuristic post hoc: 4 might mean homonymy; 0 would refer to two instances of the same meaning type; typical polysemy would be 1. . . 3) 31

Raukko 1994 (polysemy)(cont’d) • Data from my 1994 test, see handout. 32

Raukko 1994 (polysemy)(cont’d) • Data from my 1994 test, see handout. 32

Comparisons so far • Whitten & al. / synonymy – scale 1. . .

Comparisons so far • Whitten & al. / synonymy – scale 1. . . 7 (1 = very different meaning, 7 = same meaning) – synonymy ratings ranged 2. 24. . . 6. 79 – median 5. 08 (most pairs were viewed at least somewhat synonymous) • Raukko / polysemy – scale 0. . . 4 (0 = same meaning, 4 = very different meaning) – polysemy ratings ranged 0. 45. . . 3. 13 – average rating 1. 55, median 1. 34 (most pairs were viewed as having fairly similar but not identical meaning) 33

Comparisons so far • Whitten & al. / synonymy – informants saw synonymy where

Comparisons so far • Whitten & al. / synonymy – informants saw synonymy where they were supposed to • Raukko / polysemy – informants did not see large meaning difference for the most part => get is polysemous, not homonymous – they saw some similarities, some differences, as predicted => they saw polysemy • both – differing degrees of similarity were apparent – many ratings make sense, some don’t – method is useful but there are skewing effects and irreliability in several details of the setting 34

Conclusions so far • In both synonymy and polysemy studies, semantic intuitions vary. •

Conclusions so far • In both synonymy and polysemy studies, semantic intuitions vary. • In both synonymy and polysemy studies, finding a scale of semantic similarity is useful. – Cf. Sandra & Rice 1995: 125 • “[researchers of prepositional polysemy] cannot propose extremely fine-grained distinctions without bothering about empirical data” • “language users’ mental representation [. . . ] is [in fact] characterized by a high degree of granularity” 35

quantitative => qualitative • Whitten & al’s and Raukko’s similarity rating tests did not

quantitative => qualitative • Whitten & al’s and Raukko’s similarity rating tests did not include informants justifying and explaining their ratings. • E. g. , Liu (this symposium) reports tests with informants explaining their choices. • In Raukko’s study, qualitative results come from other types of tests – sorting test: (1) combine stimuli into categories, (2) give names to categories, etc. – production test: (1) produce examples of the use of polysemy, (2) explain links you find between them, etc. • Vanhatalo 2005 36

Vanhatalo 2005 (synonymy) • her Ph. D, The use of questionnaires in exploring synonymy

Vanhatalo 2005 (synonymy) • her Ph. D, The use of questionnaires in exploring synonymy • several types of tests – choose most likely components – rate components – choose better alternative (cf. Liu) – complete as sentences (only the word given) – define typical frames – spell out semantic differences 37

Vanhatalo 2005 (cont’d) • several factors investigated – 18 Finnish verbs of “nagging”, 17

Vanhatalo 2005 (cont’d) • several factors investigated – 18 Finnish verbs of “nagging”, 17 Estonian verbs of nagging • the gender and age of the portrayed speaker (the subject of “nag”) • the degree of irritation of the portrayed speaker and hearer • the volume of the vocal act – 2 -4 Finnish adjectives ‘important, central, crucial, significant’: open questions mainly 38

Vanhatalo 2005 (cont’d) • main results (Vanhatalo 2005: 40 -45): the questionnaire method –

Vanhatalo 2005 (cont’d) • main results (Vanhatalo 2005: 40 -45): the questionnaire method – helped to trace differences in the meaning and use of synonyms • many differences not documented before in dictionaries • sometimes consensus, sometimes deviation • useful especially for large groups of semantically similar words • (Vanhatalo did not use the method for placing synonyms on a scale of similarity) • both open questions and ratings should be used 39

Vanhatalo 2005 (cont’d) • main results (Vanhatalo) (cont’d) – helped to find differences between

Vanhatalo 2005 (cont’d) • main results (Vanhatalo) (cont’d) – helped to find differences between related words in Estonian and Finnish – sociodemographic variables caused fairly little variation • age and education affected a bit more than gender • answers critique 40

Vanhatalo 2005 (cont’d) • main results (Vanhatalo) (cont’d) – when both corpus method and

Vanhatalo 2005 (cont’d) • main results (Vanhatalo) (cont’d) – when both corpus method and questionnaire method were applicable, they yielded similar results • however, justification of results was different • questionnaire method dug up semantic properties that corpus method could not • in addition, can tackle low-frequency words – results of questionnaire method can be utilized in the production of electronic dictionaries 41

Other studies of synonymy that employ experimental techniques • • • Arppe & Järvikivi

Other studies of synonymy that employ experimental techniques • • • Arppe & Järvikivi 2002, 2007 Divjak & Gries 2008 Liu, in this symposium Oversteegen, in this symposium etc. 42

polysemy / qualitative • In experimental settings (e. g. , the sorting test): –

polysemy / qualitative • In experimental settings (e. g. , the sorting test): – An informant gives a name to a meaning type, a category within polysemy – An informant spells out the semantic link between two meanings – An informant draws a hierarchy between macrotypes and microtypes (more general and more specific meaning types) – An informant pinpoints at cases difficult to evaluate 43

And… • to conclude… 44

And… • to conclude… 44

Evaluating semantic similarity • Both synonymy and polysemy operate on the scale of semantic

Evaluating semantic similarity • Both synonymy and polysemy operate on the scale of semantic similarity vs. difference. • Knowing about the degree of similarity is one useful property of both. • The way to find out about it is to use elicitation/experiments. • There is deviation in informants’ ratings. • A simple explanation: informants use different criteria for evaluation. • Solutions: let them explicate the criteria. use multiple methods. 45

Synonymy vs. polysemy • Evaluating semantic similarity between the meanings of two separate words

Synonymy vs. polysemy • Evaluating semantic similarity between the meanings of two separate words (synonymy) is a matter of evaluating the match between two separate ”semantic events” – There should be mismatch, but there isn’t. • Evaluating semantic similarity/relatedness/ difference between the meanings of one word (polysemy) is a matter of comparing the applications of one single category. – There should be match between the semantic events. 46

Synonymy vs. polysemy • When you evaluate near-synonyms, you balance between (i) the ideal

Synonymy vs. polysemy • When you evaluate near-synonyms, you balance between (i) the ideal of what would constitute a perfect match and (ii) the nuances of the near-synonyms • When you evaluate meanings of a polysemous word, you balance between (i) the assumption that some meaning should be shared and (ii) the actual semantic profile of the uses 47

Synonymy vs. polysemy • In evaluating synonymy, the idealized equivalence can be taken from

Synonymy vs. polysemy • In evaluating synonymy, the idealized equivalence can be taken from the semantic description of either of the two words. • In evaluating polysemy, the common factor (”core meaning”, ”shared meaning”) may be hard to find, or become too abstract. Maybe the first task is easier? 48

General relevance • ”Insights in the equality or similarity of meaning may shed light

General relevance • ”Insights in the equality or similarity of meaning may shed light on meaning itself” (Oversteegen / SKY 2010, Helsinki) • The question of “identical meaning” is a crucial basis for e. g. typology and language comparisons: the problem of tertium comparationis – Cf. Haspelmath’s plenary on Saturday 49

References Arppe, Antti & Juhani Järvikivi 2007. Every method counts – Combining corpus-based and

References Arppe, Antti & Juhani Järvikivi 2007. Every method counts – Combining corpus-based and experimental evidence in the study of synonymy. Corpus Lingustics and Linguistic Theory 3: 2: 131 -159. Colombo, Lucia & Giovanni B. Flores d’Arcais 1984. The meaning of Dutch prepositions: a psycholinguistic study of polysemy. Linguistics 22: 51 -98. Divjak, Dagmar & Stefan Gries 2008: Clusters in the mind? Converging evidence from near-synonymy in Russian. The Mental Lexicon 3: 2: 188 -213. Geeraerts, Dirk – in this symposium Liu, Dilin – in this symposium Oversteegen, Eleonore – in this symposium Raukko, Jarno 2003. Polysemy as flexible meaning: experiments with English get and Finnish pitää. In Brigitte Nerlich & al (eds) Polysemy. Flexible patterns of meaning in mind and language. 161 -193. CONTINUED. . . 50

References Author’s cont’d contact Sandra, Dominiek & Sally Rice 1995. information Network analyses of

References Author’s cont’d contact Sandra, Dominiek & Sally Rice 1995. information Network analyses of prepositional meaning: mirroring whose mind – the • linguist’s or the language user’s? Cognitive Linguistics 6: 89 -130. Vanhatalo, Ulla 2005. Kyselytestit synonymian selvittämisessä (etc. ) [The • use of questionnaires in exploring synonymy, etc. ] Ph. D thesis, U-Helsinki. http: /ethesis. helsinki. fi/julkaisut/hum/ • suoma/vk/vanhatalo/kyselyte. pdf Whitten, William B. II, W: Newton Suter, and Michael L. Frank 1979. Bidirectional Synonym Ratings of 464 Noun Pairs. Journal of Verbal Learning and Verbal Behavior 18: 109 -127. e-mail: See handout and list of participants. home postal address See handout. affiliation Department of Modern Languages Metsätalo (Unioninkatu 40 B) FIN-00014 University of Helsinki Finland 51