NLP Module 1 Outline Introduction to Natural Language

NLP Module 1

Outline Introduction to Natural Language Processing Ø Morphology Ø Syntax Ø Semantics Ø Pragmatics Ø Discourse Ø

What is language? A way of communication between participants. Ø Participation can be either between: Ø v Human----human or v Human----computer

Types of languages Ø Natural languages v Also called: ü Informal language ü Unstructured language ü Non-regular language Ø Computer languages v Also called: ü Formal language ü Structured language ü Regular language

Natural languages Vs computer languages Ø Natural Language v Ambiguous v Context Sensitive v Informal v Descriptive v Unstructured v Uncontrolled >Computer language Non-ambiguous Context free Formal Prescriptive Structured Controlled

Natural language processing (NLP) Ø Natural language processing (NLP) refers to artificial intelligence methods of communicating with a computer in English or whatever language you may speak. v Ref[Frenzel, crash course, p. 145]

Branches of natural language processing

Branches of natural language processing Ø Natural Language Understanding (NLU) v Natural-language-understanding systems convert samples of natural human language into more formal representations that are easier for computer programs to manipulate. Ø Natural Language Generation (NLG) v Natural-language-generation systems convert information from computer databases into normaltext or sounding human language.

Natural language understanding (NLU) Ø Ø Difficult phase in NLP Consist of a number of sub-phases v Morphology: ü The study of basic elements (morphemes) of word is called morphology v Grammar: ü The system of rules for putting words together to form complete sentences and thoughts is called grammar ¨ Ref [Frenzel, crash course] ¨ Syntax Semantics ¨ v Context ü and pragmatics Refers to what people realy mean by what they say and write ¨ Ref [Frenzel, crash course]

Morphology

Morphology Definition Ø Ø The branch of linguistics that deals with how the words are constructed from the more basic units, called morphemes. Other definitions: v Morphology - is the study of the rules governing the formation of words. v Morphology — knowledge of the components of words (Jurafsky). meaningful

Morphology Basic Concepts v In linguistics, a morpheme is the smallest component of word, or other linguistic unit, that has semantic meaning. [Spencer, Pratt (1992). Morphological Theory. Oxford: Blackwell] [http: //en. wikipedia. org/wiki/Morphemes] Also v A morpheme is language. In a morphemes are and the other affixes. v ü defined as the minimal meaningful unit of a word like independently, we say that the in-, depend, -ent, and ly; depend is the root morphemes are, in this case, derivational [http: //en. wikipedia. org/wiki/Morphology_(linguistics)]

Morphology Basic Concepts (Cont. . ) v An affix is a morpheme that is attached to a word stem to form a new word. Affixes may be derivational, like English ness and pre-, or inflectional, like English plural -s and past tense -ed. [Gerdts, Donna B. (2003). "The morphosyntax of Halkomelem lexical suffixes". International Journal of American Linguistics 69 (4): 345– 356. doi: 10. 1086/382736] [http: //en. wikipedia. org/wiki/Affix] v In linguistics, intonation is variation of pitch while speaking. [http: //en. wikipedia. org/wiki/Intonation_(linguistics)] v he ü found it on the street. Here the rising pitch on street indicates that the question hinges on that word, on where he found it, not whether he found it.

Morphology Basic Concepts (Cont. . ) v In linguistics, a stem is a part of a word. The term is used with slightly different meanings. Usage 1: Friendships contains the stem friend, to which derivational suffix ship is attached to form new stem friendship, to which inflectional suffix s is attached (root of the word is not counted as stem) ü Usage 2: Friendships contains the stem word friendship to which inflectional suffix s is added ü

Types of morphology Inflectional morphology 2. Derivational morphology (word formation) 3. Compounding 1.

Inflectional Morphology Ø Inflectional morphology concerns the combination of stems and affixes where the resulting word: v Has the same word class as the original v Serves a grammatical/semantic purpose that is Different from the original ü But is nevertheless transparently related to the original ü

Inflectional Morphology Ø Intuitively, this involves creating different forms of a word, typically expressing grammatical information (e. g. tense, aspect, person, number, case, agreement, etc): v N plural -> N+s ü Baby -> babies

Derivational Morphology Ø Intuitively, this involves combining a bound morpheme (affix) with a word to create a new word: v A+ize -> V ü v [V [A legal ] +ize ] V + er -> N ü ü [N [V destroy ] +er ] [N [V compute ] +er ] +ize ] +ation ]

Compound Morphology Ø Intuitively, combining two words to make a third: v N+N -> N ü [N [ N table ] [N top ]] ü [N [ N student] [N film society ]] Ø In linguistics, a compound is a lexeme that consists of more than one stem.

Syntax

Syntax Ø Syntax concerns how words group together in larger chunks, namely phrases and sentences. v Ref[Frenzel, Crash Course] Ø Method of putting words in a specific order so they will have a correct form according to the language. v Ref[Frenzel, Crash Course]

Part-of-speech Words are traditionally grouped into equivalence classes called parts of speech (POS; Latin pars orationis) Ø Other names: Ø v word classes, v morphological classes, or v lexical tags v Ref[Jurafsky, page. 310]

Basic Language Units: 8 POS 1. 2. 3. 4. 5. 6. 7. 8. Nouns (person, place, thing, idea) Verbs (expresses action) Pronouns Adjectives Prepositions Adverbs Conjunctions Interjections

Part-Of-Speech Tagging Ø The process of marking up the words in a text as corresponding to a particular part of speech is called POS tagging. v Ø Ø Ref[Wikipedia: http: //en. wikipedia. org/wiki/Part-of-speech_tagging] Or the process of assigning part-of-speech to each word in a phrase, sentence or paragraph is called partof-speech tagging. POS is also called: grammatical tagging or v word-category disambiguation v

Tagset Ø Ø Ø Tagset is the set of tags from which the tagger is supposed to choose to attach to the relevant word Every tagger will be given a standard tagset A commonly used tagset consist of: v v v v N(Noun), V(Verb), ADJ(Adjective), ADV(Adverb), PREP(Preposition), CONJ(Conjunction) INTRJ(Interjection)

Part-Of-Speech Tagging Ø Example 1: v Kashif cuts an apple with a knife. N Kashif V cuts Det an N apple PREP with Det a N Knife

Part-Of-Speech Tagging Ø Example 2: v. A book was given to Marry by John Det a N book V (helping) was V given PREP to N Marry PREP by N John

Part-Of-Speech Tagging Ø Example 3: v The boy went home. Det the N boy V went N home

Part-Of-Speech Tagger Ø A Part-Of-Speech Tagger (POS Tagger) is a piece of software that reads text in some language and assigns parts of speech to each word (and other token), such as noun, verb, adjective, etc. v Ref [http: //nlp. stanford. edu/software/tagger. shtml]

Syntax analysis Ø Definition v In syntax analysis, the exact structure of an input sentence is determined by parser ü Ref[Frenzil, crash Course]

Syntactic Analyzer: Parser Ø Ø A piece of software that analyzes the input sentence syntactically is called parser Parser maps the words into a structure called parse tree. Parsing is analogous to the process of diagramming sentences The parser identifies: v v v Sentence (S) Noun Phrases (NP) Verb Phrases (VP) Prepositional Phrases (PP) Adjectival Phrase (Adj. P):

Syntactic Analyzer: Parser Ø Sentence (S) v S NP VP

Syntactic Analyzer: Parser Ø Noun Phrases (NP) v NP N ü [John] sleeps v NP PRON ü [He] sleeps v NP Det ü [A student] sleeps v NP NP ü PP [A student in the back] sleeps v NP Det ü N Adj N [A tall student] sleep

Syntactic Analyzer: Parser Ø Verb Phrase (VP): v VP V ü She [eats] v VP V ü She [eats mango] v VP V ü N PP She [gives apple in plate] v VP V ü NP She [eats an apple] v VP V ü N PP She [eats apple with spoon]

Syntactic Analyzer: Parser Ø Prepositional Phrase (PP): v PP PREP ü On table v PP PREP ü Ø N NP On the table Adjectival Phrase (Adj. P): v Adj. P Adj ü big v Adj. P Adv ü Very big Adj. P

Syntactic Analysis (Cont. . ) Ø Rules of syntax (grammar) specify the possible organization of words in sentences and allows us to determine sentence’s structure(s) v “John saw Mary with a telescope” John saw (Mary) with a telescope v John saw (Mary with a telescope) v

Syntactic Analysis – Parse Tree (Example 1) S NP N VP V NP Det N “Mary” “ate” “the” “apple”

Syntactic Analysis – Parse Tree (Example 2) Claudia sat on a stool

Syntactic Analysis – Parse Tree (Example 3) Ø John hit the ball.

Syntactic Analysis – Parse Tree (Example 4) Ø The cat sat on the mat.

Types of Parsing Ø Two types: v Top-down v Bottom up

Top-down Parsing Ø Ø A top-down parser searches for a parse tree by trying to build from the root node to the leaves. For example see figure 1. 1

Bottom-up Parsing In bottom-up parsing, a sentence is analyzed from left to right, determine all possible syntactic structures Ø A bottom-up parser builds a parse tree from its leaves to root node Ø The bottom-up parsing for the sentence “Sadia put the ball in the basket” is given in figure 1. 2 Ø

Bottom-up Parsing (Cont. . )

Semantics

Semantics Ø Semantics is a branch of linguistics dealing with the meaning of words, phrases and sentences v Ref[Kamil Wiśniewski, Aug. 12 th, angielski. info/linguistics/lexical-semantics. htm)] 2007, Semantics (http: //www. tlumaczenia-

Semantics is focused on the meaning of the words, phrases and sentences. Ø It is a very difficult process because sometimes a sentence is syntactically correct but semantically incorrect. Ø Consider the previous example 4 again Ø

Semantics Ø The mat sat on the cat. Ø (syntactically correct but semantically incorrect)

Semantics (Cont. . ) Ø It is concerned with the conceptual meaning and not the associative meaning Ø Conceptual meaning v Ø Tells us what a word in fact denotes, as for example Friday the 13 th is a day between Thursday the 12 th and Saturday the 14 th, and that is the conceptual meaning of the phrase Friday the 13 th. Associative meaning Yet, for many people the idea of the day “Friday 13 th” brings either thoughts of bad luck or misfortune or happiness. v What deals with associative meaning? v v Pragmatics deals with associative meaning.

Semantic Analysis Ø Meaning of a word, phrase or sentence can be analyzed from its different aspects: Semantic features (tense and singular/plural etc) v Semantic role (thematic role) (agent/patient and instrument) v ü v Ali killed asim with a knife (ali-agent, asim-patient, knife-instrument) Relationship between words e. g. Synonym and Antonym Sematic roles are very helpful for high level analysis in MT where it is important to know who is the agent and who is patient and whther or not any instrument is used in order to do the correct translation.

Relationship Between Words Ø Synonyms Are two words with very similar, almost identical meaning, such as buy and purchase, or cab and taxi. v In some cases however, although the meaning seems nearly identical there is a difference in the word usage or the level of formality and therefore the words can not always be substituted. v Ø Antonyms In this type of relationship between words two words have opposite meanings, the words such as male/female, old/new, interesting/boring are antonyms. v Gradable antonyms v ü v are opposites along a scale in that when someone says ‘I am not tall’ it does not necessarily mean ‘I am short’. Non-gradable antonyms ü do not present such flexibility: when we say ‘I am married’ the only antonym available in this sentence would be ‘I am single’.

Relationship Between Words (Cont. . ) Ø Hyponymy Sometimes the meaning of one word is included in the meaning of another, broader term. Then the relationship between words can be described as hyponym as in the case of words: vegetable and carrot. A carrot is necessarily a vegetable, therefore the meaning of the word vegetable is included in the word carrot, so carrot is a hyponym of vegetable. v Superordinate v ü ü In this relation the word vegetable is the superordinate (higher level term) of the word carrot. The relationship of hyponymy and superordination can be illustrated by the following diagram:

Relationship Between Words (Cont. . ) Ø Ø Homophones v are words which have different written forms, but the same pronunciation such as: right/write, to/too/two, bear/bare. Homophones are often mistaken for homonyms, but Homonyms v Words which have the same written or spoken forms and unrelated meanings, as for example: bat (flying creature) and bat (used in baseball), race (contest) and race (ethnic group). Polysemy (poly see me) v When a word has multiple related meanings then linguists speak of polysemy as with head for instance: head as a part of body; mind, or mental ability; a person in charge. Metonymy (met tou namy) v There exist a close connection of certain entities in everyday experience. The connection can be that of container-content, whole-part, or others. It is clearly visible in the following example ‘he drank the whole bottle’ when it is obvious that he did not drink the container, but the content of the bottle.

Reading/References Ø Ø Frenzel, Crash Course in Artificial Intelligence, Chapter No. 6. Yule G. 1996. The study of language. Cambridge: CUP. (http: //en. wikipedia. org/wiki/Semantics) Daniel Jurafsky, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition Kamil Wiśniewski, Aug. 12 th, 2007, Semantics (http: //www. tlumaczenia-angielski. info/linguistics/lexical-semantics. htm)

Pragmatics

Pragmatics: definition Ø Pragmatics v is a subfield of linguistics which studies the ways in which context contributes to meaning. ü Mey, Jacob L. (1993) Pragmatics: An Introduction. Oxford: Blackwell (2 nd ed. 2001).

Pragmatics: definition Ø Pragmatics – v The study of how language is used to accomplish goals ü (Juraffsky).

Pragmatics: definition Ø Pragmatics v is the study of the systematic relations between language and context. v Pragmatics study languages that is not directly spoken v The process in which a speaker implies and a listener infers i. e. v The speaker hints at or suggests a meaning and the listener assumes the correct intention

Pragmatics Ø Pragmatics focuses on discourse context, and so it includes studies of how discourses are structured, and how the listener manages to interpret a conversational partner in a conversation. v (Juraffsky) Ø It is an understanding between people to obey certain rules of interaction

Pragmatics Ø Pragmatics – v the analysis of language in terms of the situational context within which utterances are made, including the knowledge and beliefs of the speaker and the relation between speaker and listener

Pragmatics: examples Ø Can you lift that box? if you can't decide whether the speaker wants to discover how strong you are (a question) or v wants you to move the box (a request? ) v

Pragmatic competence Ø The ability to use language in a contextually appropriate fashion.

Units of Language – Subfields of Linguistics Manfred Pinkal (11 -10 -2006), Pragmatics, Universität des Saarlandes

Reading/References Ø Ø Yule G. 1996. The study of language. Cambridge: CUP. Wilson R. A. (Editor) 1999. The MIT encyclopedia of cognitive sciences. London: The MIT Press. Daniel Jurafsky, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition. Kamil Wiśniewski, July 12 th, 2007, Pragmatics. Retrieved from: http: //www. tlumaczenia-angielski. info/linguistics/pragmatics. htm, Retrieved date: Oct 16, 2010.

Discourse

Background to discourse Ø Ø Up to this point, we have focused primarily on language phenomena that operate at the word or sentence level. Of course, language does not normally consist of isolated, unrelated sentences, but instead of collocated, related groups of sentences. We refer to such a group of sentences as a discourse.

Definition Ø Ø Originally the word 'discourse' comes from Latin 'discursus' which denoted 'conversation, speech'. Definition 1: v "Discourse: a continuous stretch (an unbroken sequence of utterances) of (especially spoken) language larger than a sentence, often constituting a coherent unit such as a sermon (religious speech), argument, joke, or narrative (occurrence of an act or event)" v. Ref(Crystal 1992: 25).

Other definitions Ø Definition 2: v According to Cook (1990: 7) novels, as well as short conversations or groans might be equally rightfully named discourses. Ø Definition 3: v Discourse - The study of linguistic units larger than a single utterance v. Ref(Jurafsky, 1999)

Discourse unit Ø Ø In TBMT, the unit for analysis is a ‘discourse’ unit (DU) rather than a sentence. Definition of DU v. A DU is that part of the text which is between two successive discourse boundaries (DBs). Ø Discourse Boundary v DB, itself, is like a sentence boundary (e. g. '. ') across which there are no anaphoric and/or cataphoric reference links. Ø Ref(M. A. Khan, 2004)

Types of discourse unit Ø Mono-sentential Discourse v If a discourse consist of only one sentence then it is called mono-sentential discourse or simple discourse. v For example: ü Ø [Be patient on what you have in your hand. ] Poly-sentential Discourse v If a discourse consist of more than one sentences then it is called poly-sentential discourse or complex/ compound discourse.

Types of discourse unit (Cont. . ) Ø Example of poly-sentential discourse v[Los Angeles—A man who claims to be Angelina Jolie's former drug dealer says the star was high on cocaine when she took part in a TV interview back in 2000. v According to Franklin Meyer, the 25 -year-old actress visited him both before and after the interview with broadcaster Charlie Rose to buy cocaine. v 'She'd taken coke, ' he told Life & Style magazine. 'She did the show and then she came back over after. ‘ Ø Ref(The Daily News: 22 Oct, 2010) ]

Discourse analysts Ø A person performing discourse analysis is called discourse analyst.

Motivation/Significance of Discourse Analysis Ø Information extraction v Reference Ø Summarization v Text Ø coherence and reference resolution Conversational agents v Reference Ø resolution and text coherence Automatic essay grading v Text coherence

Motivation of Text segmentation Ø Information extractions v Some sections are more informative than others Ø Summarization v Includes Ø information from all sections Information retrieval v Retrieve particular information from the correct section

From where we can study a discourse? Ø A wide array of linguistic ‘texts’ are explored in the study of discourse. These might consist of: a conversation or a letter; v a speech, a memo or a report; v a broadcast, a newspaper article or an interview; v a lesson, a consultation or a confrontational (argumentation) encounter; v an advertisement, v a flyer or a piece of gossip. v

Discourse analysis also focuses on Cohesion Ø Coherence Ø References resolution Ø Ellipsis resolution and Ø Ambiguity resolution Ø v (will talk in coming lectures)

Reading/references Ø Ø Daniel Jurafsky, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition, 1999, Chapter No. 18. Kamil Wiśniewski, July 12 th, 2007, Discourse Analysis. Retrieved from: http: //www. tlumaczenia-angielski. info/linguistics/discourse. htm, Retrieved date: Oct 16, 2010. M. A. Khan, “ Text Based Machine Translation System”, Ph. D Thesis, 1995. The Daily News, “Jolie was high on cocaine during TV interview: Former drug dealer”, dated: 22 Oct, 2010. http: //dailymailnews. com/1010/22/Show. Biz/index. php? id=3 Ø M. A. Khan, “MACHINE TRANSLATION BEYOND SENTENCE BOUNDARIES ”, , In Proceedings of Workshop on Proofing Tools and Language Technologies, July 1 -2, Patras University, Greece. www. mabidkhan. com/. . . /Scientific%20 Khyber, %20 Vol%201, %202004. pdf