Natural Language Processing Lecture Notes 10 Chapter 14

Natural Language Processing Lecture Notes 10 Chapter 14 1

Big Transition • First we did words (morphology) • Then we looked at syntax • Now we’re moving on to meaning. 2

Semantics • What things mean • Approach: meaning representations which bridge the gap from linguistic forms to knowledge of the world • Serve the practical purposes of a program doing semantic processing 3

Semantic Processing • Representations that allow a system to – – Answer questions Determine truth Perform inference … 4

Types of Meaning Representations • • First-order predicate calculus Semantic networks Conceptual dependency Frame-based representations • See lecture for examples… 5

Verifiability • Does Spice Island serve vegetarian food? • Serves(spiceisland, vegetarianfood) • Verifiability: the system’s ability to compare representations to facts in memory 6

Current Focus in Class • Conventional meanings of words • Ignore context • Literal meaning 7

Ambiguity • I want to eat someplace that’s close to Pitt • Mary kissed her husband Joan did too • I baked the cake on the table • Old men and women go to the park • Every student ate a sandwich 8

Canonical Form • • • Does Spice Island have vegetarian dishes? Do they have vegetarian food at Spice Island? Are vegetarian dishes served at Spice Island? Does Spice Island serve vegetarian fare? Canonical form: inputs that mean the same thing should have the same meaning representations 9

Canonical Form • Simplifies reasoning • Makes representations more compact (fewer different representations) • BUT: makes semantic analysis harder – Need to figure out that “have” and “serve” mean the same thing in the previous examples; same for the various phrases for vegetarian food – BUT: can perform word sense disambiguation; use a single representation for all senses in a synset 10

Representational Schemes • We’re going to make use of First Order Predicate Calculus (FOPC) as our representational framework – Not because we think it’s perfect – All the alternatives turn out to be either too limiting or too complicated, or – They turn out to be notational variants 11

Knowledge Based Agents • Central component: knowledge base, or KB. • A set of sentences in a knowledge representation language • Generic Functions – TELL (add a fact to the knowledge base) – ASK (get next action based on info in KB) • Both often involve inference, which is? • Deriving new sentences from old 12

Fundamental Concepts of logical representation and reasoning • Information is represented in sentences, which must have correct syntax ( 1 + 2 ) * 7 = 21 vs. 2 ) + 7 = * ( 1 21 • The semantics of a sentence defines its truth with respect to each possible world • W is a model of S means that sentence S is true in world W • What do the following mean? – X |= Y – X entails Y – Y logically follows from X 13

Entailment • A |= B • In all worlds in which A is true, B must be true as well • All models of A are models of B • Whenever A is true, B must be true as well • A entails B • B logically follows from A 14

Inference KB |-i A Inference algorithm i can derive A from KB i derives A from KB i can derive A from KB A can be inferred from KB by i 15

Propositional Logic Syntax • Sentence -> Atomic. Sent | complex. Sent Atomic. Sent -> true|false| P, Q, R … Complex. Sent -> sentence | ( sentence ) | ( sentence ) | ( sentence ) [no predicate or function symbols] 16

Propositional Logic Sentences • If there is a pit at [1, 1], there is a breeze at [1, 0] P 11 B 10 • There is a breeze at [2, 2], if and only if there is a pit in the neighborhood B 22 ( P 21 P 23 P 12 P 32 ) • There is no breeze at [2, 2] B 22 17

Semantics of Prop Logic • In model-theoretic semantics, an interpretation assigns elements of the world to sentences, and defines the truth values of sentences • Propositional logic: easy! Assign T or F to each proposition symbol; then assign truth values to complex sentences in the obvious way 18

Propositional Logic • A ^ B is true if both A and B are true • A v B is true if one or both A and B are true • P Q equiv ~P v Q. Thus, P Q is false if P is true and Q is false. Otherwise, P Q is true. • ~A is true if A is false 19

Proofs • A derivation • A sequence of applications of (usually sound) rules of inference • Reasoning by Search • Example KB = A B, B C, D E, E F, D • Forward chaining: Add A, infer B, infer C • Backward chaining: F? E? D? Yes… • Sound but not complete inference procedures 20

Resolution • Resolution allows a sound and complete inference mechanism (search-based) using only one rule of inference • Resolution rule: – Given: P 1 P 2 P 3 … Pn, and P 1 Q 1 … Qm – Conclude: P 2 P 3 … Pn Q 1 … Qm Complementary literals P 1 and P 1 “cancel out” For your information only; resolution won’t be on exam 21

Resolution • Any complete search algorithm, applying only the resolution rule, can derive any conclusion entailed by any KB in propositional logic. • Refutation completeness: Given A, we cannot use resolution to generate the consequence A v B. But we can answer the question, is A v B true. I. e. , resolution can be used to confirm or refute a sentence Again, for your information only; will not be on the exam 22

Unsound (but useful) Inference • • Where there is smoke, there is fire Example KB: Fire Smoke, Smoke Abduction: conclude Fire Unsound: – Example KB 1: Fire Smoke, Dry. Ice Smoke, Smoke – Dry. Ice rather than Fire could be true 23

Propositional Logic FOPC B 11 (P 12 v P 21) B 23 (P 32 v P 23 v P 34 v P 43) … “Internal squares adjacent to pits are breezy”: All X Y (B(X, Y) ^ (X > 1) ^ (Y < 4) ^ (X < 4)) (P(X-1, Y) v P(X, Y-1) v P(X+1, Y) v (X, Y+1)) 24

FOPC Worlds • Rather than just T, F, now worlds contain: • Objects: the gold, the wumpus, people, ideas, … “the domain” • Predicates: holding, breezy, red, sisters • Functions: father. Of, color. Of, plus Ontological commitment 25

FOPC Syntax • Add variables and quantifiers to propositional logic 26

Sentence Atomic. Sentence | (Sentence Connective Sentence) | Quantifier Variable, … Sentence | ~Sentence Atomic. Sentence Predicate(Term, …) | Term = Term Function(Term, …) | Constant | Variable Connective | ^ | v | Quantifier all, exists Constant john, 1, … Variable A, B, C, X Predicate breezy, sunny, red Function father. Of, plus Knowledge engineering involves deciding what types of things Should be constants, predicates, and functions for your problem 27

Examples • Everyone likes chocolate – X (person(X) likes(X, chocolate)) • Someone likes chocolate – X (person(X) ^ likes(X, chocolate)) • Everyone likes chocolate unless they are allergic to it – X ((person(X) ^ allergic (X, chocolate)) likes(X, chocolate)) 28

Quantifiers • All X p(X) means that p holds for all elements in the domain • Exists X p(X) means that p holds for at least one element of the domain 29

Nesting of Variables Put quantifiers in front of likes(P, F) Assume the domain of discourse of P is the set of people Assume the domain of discourse of F is the set of foods 1. Everyone likes some kind of food 2. There is a kind of food that everyone likes 3. Someone likes all kinds of food 4. Every food has someone who likes it 30

Answers (DOD of P is people and of F is food) Everyone likes some kind of food All P Exists F likes(P, F) There is a kind of food that everyone likes Exists F All P likes(P, F) Someone likes all kinds of food Exists P All F likes(P, F) Every food has someone who likes it All F Exists P likes(P, F) 31

Answers, without Domain of Discourse Assumptions Everyone likes some kind of food All P (person(P) Exists F (food(F) and likes(P, F))) There is a kind of food that everyone likes Exists F (food(F) and (All P (person(P) likes(P, F)))) Someone likes all kinds of food Exists P (person(P) and (All F (food(F) likes(P, F)))) Every food has someone who likes it All F (food (F) Exists P (person(P) and likes(P, F))) 32

Interpretation • Specifies which objects, functions, and predicates are referred to by which constant symbols, function symbols, and predicate symbols. 33

Example • • • 3 people: John, Sally, Bill John is tall Sally and Bill are short John is Bill’s father Sally is Bill’s sister Interpretation 1 (others are possible): – – – “John”, “Sally”, and “Bill” as you think “person” {John, Sally, Bill} “short” {Sally, Bill} “tall” {John} “sister” {<Sally, Bill>} A 2 -ary predicate “father” {<Bill, John>} A 1 -ary function 34

Determining Truth Values of FOPC sentences • Connectives and negation are the same as in propositional logic 35

Example tall(father(bill)) ^ ~sister(sally, bill) • Assign meanings to terms: – “bill” Bill; “sally” Sally; “father(bill)” John • Assign truth values to atomic sentences – Tall(father(bill)) is T because John is in the set assigned to “tall” – ~sister(sally, bill) is F because <Sally, Bill> is in the set assigned to “sister” – So, sentence is false, because T ^ F is F 36

Determining Truth Values • Exist X tall(X) : true, because the set assigned to “tall” isn’t {} • All X short(X) : false, because there are objects that are not in the set assigned to “short” 37

Representational Schemes • What are the objects, predicates, and functions? 38

Choices: Functions vs Predicates • Rep-Scheme 1: tall(father. Of(bob)). • Rep-Scheme 2: Exists X (father. Of(bob, X) ^ tall(X) ^ (All Y (father. Of(bob, Y) X = Y))) • “father. Of” in both cases is assigned a set of 2 -tuples: {<b, bf>, <t, tf>, …} • But {<b, bf>, <t, tf>, <b, bff>, …} is possible if it is a predicate 39

Choices: Predicates versus Constants • Rep-Scheme 1: Let’s consider the world: D = {a, b, c, d, e}. red: {a, b, c}. pink: {d, e}. Some sentences that are satisfied by the intended interpretation: red(a). red(b). pink(d). ~(All X red(X)). All X (red(X) v pink(X)). But what if we want to say that pink is pretty? 40

Choices: Predicates versus Constants • • Rep-Scheme 2: The world: D = {a, b, c, d, e, red, pink} colorof: {<a, red>, <b, red>, <c, red>, <d, pink>, <e, pink>} pretty: {pink} primary: {red} Some sentences that are satisfied by the intended interpretation: color. Of(a, red). color. Of(b, red). color. Of(d, pink). ~(All X color. Of(X, red)). All X (color. Of(X, red) v color. Of(X, pink)). ***pretty(pink). primary(red). *** We have reified predicates pink and red: made them into objects 41

Inference with Quantifiers • Universal Instantiation: – Given X (person(X) likes(X, sun)) – Infer person(john) likes(john, sun) • Existential Instantiation: – Given x likes(x, chocolate) – Infer: likes(S 1, chocolate) – S 1 is a “Skolem Constant” that is not found anywhere else in the KB and refers to (one of) the individuals who likes sun. 42

FOPC • Allows for… – The analysis of truth conditions • Allows us to answer yes/no questions – Supports the use of variables • Allows us to answer questions through the use of variable binding – Supports inference • Allows us to answer questions that go beyond what we know explicitly 43

FOPC • This choice isn’t completely arbitrary or driven by the needs of practical applications • FOPC reflects the semantics of natural languages because it was designed that way by human beings • In particular… 44

Meaning Structure of Language • The semantics of human languages… – – Display a basic predicate-argument structure Make use of variables Make use of quantifiers Use a partially compositional semantics 45

Predicate-Argument Structure • Events, actions and relationships can be captured with representations that consist of predicates and arguments to those predicates. • Languages display a division of labor where some words and constituents function as predicates and some as arguments. 46

Example • Mary gave a list to John • Giving(Mary, John, List) • More precisely – Gave conveys a three-argument predicate – The first arg refers to the subject (the giver) – The second is the recipient, which is conveyed by the NP in the PP – The third argument refers to the thing given, conveyed by the direct object 47

Is this a good representation? • John gave Mary a book for Susan – Giving (john, mary, book, susan) • John gave Mary a book for Susan on Wednesday – Giving (john, mary, book, susan, wednesday) • John gave Mary a book for Susan on Wednesday in class – Giving (john, mary, book, susan, wednesday, in. Class) • John gave Mary a book for Susan on Wednesday in class after 2 pm – Giving (john, mary, book, susan, wednesday, in. Class, >2 pm) 48

Reified Representation • Exist b, e (ISA(e, giving) ^ agent(e, john) ^ beneficiary(e, sally) ^ patient(e, b) ^ ISA(b, book)) • “That happened on Sunday” • Add later (assuming S 2 is the skolem for e): – happened. On. Day(s 2, sunday) 49

Semantic Analysis • Semantic analysis is the process of taking in some linguistic input and assigning a meaning representation to it. – There a lot of different ways to do this that make more or less (or no) use of syntax – We’re going to start with the idea that syntax does matter • The compositional rule-to-rule approach 50

Compositional Analysis • Principle of Compositionality – The meaning of a whole is derived from the meanings of the parts • What parts? – The constituents of the syntactic parse of the input • What could it mean for a part to have a meaning? 51

Example • Ay. Caramba serves meat 52

Compositional Analysis 53

Augmented Rules • We’ll accomplish this by attaching semantic formation rules to our syntactic CFG rules • Abstractly • This should be read as the semantics we attach to A can be computed from some function applied to the semantics of A’s parts. 54

Example • Attachments • Easy parts… {Prop. Noun. sem} – NP -> Prop. Noun {Mass. Noun. sem} – NP -> Mass. Noun {Ay. Caramba} – Prop. Noun -> Ay. Caramba {MEAT} – Mass. Moun -> meat 55

Example • S -> NP VP • VP -> Verb NP • Verb -> serves • {VP. sem(NP. sem)} • {Verb. sem(NP. sem) • ? ? ? 56

Lambda Forms • A simple addition to FOPC – Take a FOPC sentence with variables in it that are to be bound. – Allow those variables to be bound by treating the lambda form as a function with formal arguments 57

Example 58

Example 59

Example 60

Example 61

Next Time • Read Chapters 14 and 15 62