Meaning Traditionally meaning in language has been studied
Meaning • Traditionally, meaning in language has been studied from three perspectives – The meaning of a text or discourse – The meanings of individual sentences or utterances – The meanings of individual words • We started in the middle, now we’ll look at the meanings of individual words. BİL 711 Natural Language Proessing 1
Word Meaning • We didn’t assume much about the meaning of words when we talked about sentence meanings – Verbs provided a template-like predicate argument structure – Nouns were practically meaningless constants • There has be more to it than that – The internal structure of words that determines where they can go and what they can do (syntagmatic) BİL 711 Natural Language Proessing 2
What’s a word? • Words? : Types, tokens, stems, roots, inflected forms? • Lexeme – An entry in a lexicon consisting of a pairing of a form with a single meaning representation • Lexicon - A collection of lexemes BİL 711 Natural Language Proessing 3
Lexical Semantics • The linguistic study of systematic meaning related structure of lexemes is called Lexical Semantics. • A lexeme is an individual entry in the lexicon. • A lexicon is meaning structure holding meaning relations of lexemes. • A lexeme may have different meanings. A lexeme’s meaning component is known as one of its senses. – Different senses of the lexeme duck. • an animal, to lower the head, . . . – Different senses of the lexeme yüz • face, to swim, to skin, the front of something, hundred, . . . BİL 711 Natural Language Proessing 4
Relations Among Lexemes and Their Senses • • • Homonymy Polysemy Snonymy Hypernym BİL 711 Natural Language Proessing 5
Homonymy • Homonymy is a relation holds between words have same form with unrelated meanings. – Bank -- financial institution, river bank • For this, we should create two senses of the lexeme bank. – Bat -- (wooden stick-like thing) vs (flying scary mammal thing) • The problematic part of understanding homonymy isn’t with the forms, it’s the meanings. – Causes ambiguity. – Nothing particularly important would happen to anything else in English if we used a different word for the little flying mammal things BİL 711 Natural Language Proessing 6
Polysemy • Polysemy is the phenomenon of multiple related meanings in a same lexeme. – Bank -- financial institution, blood bank • these senses are related. Are we going to create a single sense or two different senses? • While some banks furnish sperm only to married women, others are less restrictive – Serve • Which flights serve breakfast? • Does America West serve Philadelphia? • Does United serve breakfast and San Jose? • • Most non-rare words have multiple meanings The number of meanings is related to its frequency Verbs tend more to polysemy Distinguishing polysemy from homonymy isn’t always easy (or necessary) BİL 711 Natural Language Proessing 7
Synonymy • Synonymy is the phenomenon of two different lexemes having the same meaning. – Big and large – In fact, one of the senses of two lexemes are same. • There aren’t any true synonyms. – Two lexemes are synonyms if they can be successfully substituted for each other in all situations – What does successfully mean? • Preserves the meaning • But may not preserve the acceptability based on notions of politeness, slang, . . . • Example - Big and large? – That’s my big sister – That’s my large sister a big plane a large plane BİL 711 Natural Language Proessing 8
Hyponymy and Hypernym • Hyponymy: one lexeme denotes a subclass of the other lexeme. • The more specific lexeme is a hyponymy of the more general lexeme. • The more general lexeme is a hypernym of the more specific lexeme. • A hyponymy relation can be asserted between two lexemes when the meanings of the lexemes entail a subset relation – Since dogs are canids • Dog is a hyponym of canid and • Canid is a hypernym of dog – Car is a hyponymy of vehicle, vehicle is a hypernym of car. BİL 711 Natural Language Proessing 9
Ontology • The term ontology refers to a set of distinct objects resulting from analysis of a domain. • A taxonomy is a particular arrangements of the elements of an ontology into a tree-like class inclusion structure. • A lexicon holds different senses of lexemes together with other relations among lexemes. BİL 711 Natural Language Proessing 10
Lexical Resourses • There are lots of lexical resources available – Word lists – On-line dictionaries – Corpora • The most ambitious one is Word. Net – A database of lexical relations for English – Versions for other languages are under development BİL 711 Natural Language Proessing 11
Word. Net • Word. Net is widely used lexical database for English. • Web. Page: http: //www. cogsci. princeton. edu/~wn/ • It holds: – – The senses of the lexemes holds relations among nouns such as hypernym, hyponym, Member. Of, . . Holds relations among verbs such as hypernym, … Relations are held for each different senses of a lexeme. BİL 711 Natural Language Proessing 12
Word. Net Relations • Some of Word. Net Relations (for nouns) BİL 711 Natural Language Proessing 13
Word. Net Hierarchies • Hyponymy chains for the senses of the lexeme bass BİL 711 Natural Language Proessing 14
Word. Net - bass • The noun "bass" has 8 senses in Word. Net. 1. bass -- (the lowest part of the musical range) 2. bass, bass part -- (the lowest part in polyphonic music) 3. bass, basso -- (an adult male singer with the lowest voice) 4. sea bass, bass -- (the lean flesh of a saltwater fish of the family Serranidae) 5. freshwater bass, bass -- (any of various North American freshwater fish with lean flesh (especially of the genus Micropterus)) 6. bass, bass voice, basso -- (the lowest adult male singing voice) 7. bass -- (the member with the lowest range of a family of musical instruments) 8. bass -- (nontechnical name for any of numerous edible marine and freshwater spinyfinned fishes) • The adjective "bass" has 1 sense in Word. Net. 1. bass, deep -- (having or denoting a low vocal or instrumental range; "a deep voice"; "a bass voice is lower than a baritone voice"; "a bass clarinet") BİL 711 Natural Language Proessing 15
Word. Net –bass Hyponyms Results for "Hyponyms (. . . is a kind of this), full" search of noun "bass" 6 of 8 senses of bass Sense 2 bass, bass part -- (the lowest part in polyphonic music) => ground bass -- (a short melody in the bass that is constantly repeated) => thorough bass, basso continuo -- (a bass part written out in full and accompanied by figures for successive chords) => figured bass -- (a bass part in which the notes have numbers under them to indicate the chords to be played) Sense 4 sea bass, bass -- (the lean flesh of a saltwater fish of the family Serranidae) => striped bass, striper -- (caught along the Atlantic coast of the United States) Sense 5 freshwater bass, bass -- (any of various North American freshwater fish with lean flesh (especially of the genus Micropterus)) => largemouth bass -- (flesh of largemouth bass) => smallmouth bass -- (flesh of smallmouth bass) Sense 6 bass, bass voice, basso -- (the lowest adult male singing voice) => basso profundo -- (a very deep bass voice) Sense 7 bass -- (the member with the lowest range of a family of musical instruments) => bass fiddle, bass viol, bull fiddle, double bass, contrabass, string bass -- (largest and lowest member of the violin family) => bass guitar -- (the lowest six-stringed guitar) => bass horn, sousaphone, tuba -- (the lowest brass wind instrument) => euphonium -- (a bass horn (brass wind instrument) that is the tenor of the tuba family) => helicon, bombardon -- (a tuba that coils over the shoulder of the musician) => bombardon, bombard -- (a large shawm; the bass member of the shawm family) Sense 8 bass -- (nontechnical name for any of numerous edible marine and freshwater spiny-finned fishes) => freshwater bass -- (North American food and game fish) BİL 711 Natural Language Proessing 16
Word. Net – bass Synonyms Results for "Synonyms, ordered by estimated frequency" search of noun "bass" 8 senses of bass Sense 1 bass -- (the lowest part of the musical range) => low pitch, low frequency -- (a pitch that is perceived as below other pitches) Sense 2 bass, bass part -- (the lowest part in polyphonic music) => part, voice -- (the melody carried by a particular voice or instrument in polyphonic music; "he tried to sing the tenor part") Sense 3 bass, basso -- (an adult male singer with the lowest voice) => singer, vocalist, vocalizer, vocaliser -- (a person who sings) Sense 4 sea bass, bass -- (the lean flesh of a saltwater fish of the family Serranidae) => saltwater fish -- (flesh of fish from the sea used as food) Sense 5 freshwater bass, bass -- (any of various North American freshwater fish with lean flesh (especially of the genus Micropterus)) => freshwater fish -- (flesh of fish from fresh water used as food) Sense 6 bass, bass voice, basso -- (the lowest adult male singing voice) => singing voice -- (the musical quality of the voice while singing) Sense 7 bass -- (the member with the lowest range of a family of musical instruments) => musical instrument, instrument -- (any of various devices or contrivances that can be used to produce musical tones or sounds) Sense 8 bass -- (nontechnical name for any of numerous edible marine and freshwater spiny-finned fishes) => percoid fish, percoidean -- (any of numerous spiny-finned fishes of the order Perciformes) BİL 711 Natural Language Proessing 17
Internal Structure of Words • Paradigmatic relations connect lexemes together in particular ways but don’t say anything about what the meaning representation of a particular lexeme should consist of. • Various approaches have been followed to describe the semantics of lexemes. – – Thematic roles in predicate-bearing lexemes Selection restrictions on thematic roles Decompositional semantics of predicates Feature-structures for nouns BİL 711 Natural Language Proessing 18
Thematic Roles • Thematic roles provide a shallow semantic language for characterizing certain arguments of verbs. • For example: – Ali broke the glass. – Veli opened the door. • • Ali is Breaker and the glass is Broken. Thing of Breaking event; Veli is Opener and the door is Opened. Thing of Opening event. These are deep roles of arguments of events. Both of these events have actors which are doer of a volitional event, and things affected by this action. • A thematic role is a way of expressing this kind of commonality. • AGENT and THEME are thematic roles. BİL 711 Natural Language Proessing 19
Some Thematic Roles AGENT --The volitional causer of an event -- She broke the door EXPERIENCER -- The experiencer of an event -- Ali has a headache. FORCE -- The non-volitional causer of an event -- The wind blows it. THEME -- The participant most directly effected by an event -She broke the door. INSTRUMENT -- An instrument used in an event -He opened it with a knife. BENEFICIARY -- A beneficiary of an event -- I bought it for her. SOURCE -- The origin of the object of a transfer event -I flew from Rome. GOAL -- The destination of the object of a transfer event -I flew to Ankara. BİL 711 Natural Language Proessing 20
Thematic Roles (cont. ) • Takes some of the work away from the verbs. • It’s not the case that every verb is unique and has to completely specify how all of its arguments uniquely behave. • Provides a mechanism to organize semantic processing • It permits us to distinguish near surface-level semantics from deeper semantics BİL 711 Natural Language Proessing 21
Linking • Thematic roles, syntactic categories and their positions in larger syntactic structures are all intertwined in complicated ways. • For example… – AGENTS are often subjects – In a VP->V NP NP rule, the first NP is often a GOAL and the second a THEME BİL 711 Natural Language Proessing 22
Deeper Semantics • He melted her reserve with a husky-voiced paean to her eyes. • If we label the constituents He and her reserve as the Melter and Melted, then those labels lose any meaning they might have had. • If we make them Agent and Theme then we don’t have the same problems BİL 711 Natural Language Proessing 23
Selectional Restrictions • A selectional restriction augments thematic roles by allowing lexemes to place certain semantic restrictions on the lexemes and phrases can accompany them in a sentence. – I want to eat someplace near Bilkent. – Now we can say that eat is a predicate that has an AGENT and a THEME – And that the AGENT must be capable of eating and the THEME must be capable of being eaten • Each sense of a verb can be associated with selectional restrictions. – THY serves New. York. -- direct object (theme) is a place – THY serves breakfast. -- direct object (theme) is a meal. • We may use these selectional restrictions to disambiguate a sentence. BİL 711 Natural Language Proessing 24
As Logical Statements • For eat… • Eating(e) ^Agent(e, x)^ Theme(e, y)^Isa(y, Food) (adding in all the right quantifiers and lambdas) BİL 711 Natural Language Proessing 25
Word. Net • Use Word. Net hyponyms (type) to encode the selection restrictions BİL 711 Natural Language Proessing 26
Specificity of Restrictions • What can you say about THEME in each with respect to the verb? • Some will be high up in the Word. Net hierarchy, others not so high… • PROBLEMS • Unfortunately, verbs are polysemous and language is creative… – … ate glass on an empty stomach accompanied only by water and tea – you can’t eat gold for lunch if you’re hungry – … get it to try to eat Afghanistan BİL 711 Natural Language Proessing 27
Discovering the Restrictions • Instead of hand-coding the restrictions for each verb, can we discover a verb’s restrictions by using a corpus and Word. Net? 1. Parse sentences and find heads 2. Label thematic roles 3. Collect statistics on the co-occurrence of particular headwords with particular thematic roles 4. Use the Word. Net hypernym structure to find the most meaningful level to use as a restriction BİL 711 Natural Language Proessing 28
Motivation • Find the lowest (most specific) common ancestor that covers a significant number of the examples BİL 711 Natural Language Proessing 29
Word-Sense Disambiguation • Word sense disambiguation refers to the process of selecting the right sense for a word from among the senses that the word is known to have • Semantic selection restrictions can be used to disambiguate – Ambiguous arguments to unambiguous predicates – Ambiguous predicates with unambiguous arguments – Ambiguity all around BİL 711 Natural Language Proessing 30
Word-Sense Disambiguation • We can use selectional restrictions for disambiguation. – He cooked simple dishes. – He broke the dishes. • But sometimes, selectional restrictions will not be enough to disambiguate. – What kind of dishes do you recommend? -- we cannot know what sense is used. • There can be two lexemes (or more) with multiple senses. – They serve vegetarian dishes. • Selectional restrictions may block the finding of meaning. – If you want to kill Turkey, eat its banks. – Kafayı yedim. – These situations leave the system with no possible meanings, and they can indicate a metaphor. BİL 711 Natural Language Proessing 31
WSD and Selection Restrictions • Ambiguous arguments – Prepare a dish – Wash a dish • Ambiguous predicates – Serve Denver – Serve breakfast • Both – Serves vegetarian dishes BİL 711 Natural Language Proessing 32
WSD and Selection Restrictions • This approach is complementary to the compositional analysis approach. – You need a parse tree and some form of predicate-argument analysis derived from • The tree and its attachments • All the word senses coming up from the lexemes at the leaves of the tree • Ill-formed analyses are eliminated by noting any selection restriction violations BİL 711 Natural Language Proessing 33
Problems • As we saw last time, selection restrictions are violated all the time. • This doesn’t mean that the sentences are ill-formed or preferred less than others. • This approach needs some way of categorizing and dealing with the various ways that restrictions can be violated BİL 711 Natural Language Proessing 34
WSD Tags • What’s a tag? – A dictionary sense? • For example, for Word. Net an instance of “bass” in a text has 8 possible tags or labels (bass 1 through bass 8). BİL 711 Natural Language Proessing 35
Word. Net Bass The noun ``bass'' has 8 senses in Word. Net 1. 2. 3. 4. 5. 6. 7. 8. bass - (the lowest part of the musical range) bass, bass part - (the lowest part in polyphonic music) bass, basso - (an adult male singer with the lowest voice) sea bass, bass - (flesh of lean-fleshed saltwater fish of the family Serranidae) freshwater bass, bass - (any of various North American lean-fleshed freshwater fishes especially of the genus Micropterus) bass, bass voice, basso - (the lowest adult male singing voice) bass - (the member with the lowest range of a family of musical instruments) bass -(nontechnical name for any of numerous edible marine and freshwater spiny-finned fishes) BİL 711 Natural Language Proessing 36
Representations • Most supervised ML approaches require a very simple representation for the input training data. – Vectors of sets of feature/value pairs • I. e. files of comma-separated values • So our first task is to extract training data from a corpus with respect to a particular instance of a target word – This typically consists of a characterization of the window of text surrounding the target BİL 711 Natural Language Proessing 37
Representations • This is where ML and NLP intersect – If you stick to trivial surface features that are easy to extract from a text, then most of the work is in the ML system – If you decide to use features that require more analysis (say parse trees) then the ML part may be doing less work (relatively) if these features are truly informative BİL 711 Natural Language Proessing 38
Surface Representations • Collocational and co-occurrence information – Collocational • Encode features about the words that appear in specific positions to the right and left of the target word – Often limited to the words themselves as well as they’re part of speech – Co-occurrence • Features characterizing the words that occur anywhere in the window regardless of position – Typically limited to frequency counts BİL 711 Natural Language Proessing 39
Collocational • Position-specific information about the words in the window • guitar and bass player stand – [guitar, NN, and, CJC, player, NN, stand, VVB] – In other words, a vector consisting of – [position n word, position n part-of-speech…] BİL 711 Natural Language Proessing 40
Co-occurrence • Information about the words that occur within the window. • First derive a set of terms to place in the vector. • Then note how often each of those terms occurs in a given window. BİL 711 Natural Language Proessing 41
Classifiers • Once we cast the WSD problem as a classification problem, then all sorts of techniques are possible – Naïve Bayes (the right thing to try first) – Decision lists – Decision trees – Neural nets – Support vector machines – Nearest neighbor methods… BİL 711 Natural Language Proessing 42
Classifiers • The choice of technique, in part, depends on the set of features that have been used – Some techniques work better/worse with features with numerical values – Some techniques work better/worse with features that have large numbers of possible values • For example, the feature the word to the left has a fairly large number of possible values BİL 711 Natural Language Proessing 43
Statistical Word-Sense Disambiguation Where s is a vector of senses, V is the vector representation of the input By Bayesian rule By making independence assumption of meanings. This means that the result is the product of the probabilities of its individual features given that its sense BİL 711 Natural Language Proessing 44
Problems • Given these general ML approaches, how many classifiers do I need to perform WSD robustly – One for each ambiguous word in the language • How do you decide what set of tags/labels/senses to use for a given word? – Depends on the application BİL 711 Natural Language Proessing 45
- Slides: 45