BURC Bootstrapping Using Research Cyc By Kino Coursey

BURC: Bootstrapping Using Research. Cyc By Kino Coursey

Introduction to the Problem n n n Goal: To extend Cyc’s knowledge base using “relationships implied to be possible, normal or commonplace in the world” Prior work with Cyc knowledge entry has been manually oriented How will we collect commonsense without a body and manual labor…? Read, Parse, Mine! Proposal: Read text, Parse into a database, Extract relations between words, Propose hypothetical relations between concepts

Common Knowledge n Using an information channel model • Information the Sender considers the Receiver to already know • If the Sender does sends the info then … n n n Receiver will consider the Sender to ‘lack intelligence or experience’ (The sender is stupid). Receiver will believe the sender thinks the Receiver ‘lacks intelligence or experience’ (The sender thinks I’m stupid) Possibly the Sender is clarifying which among many possible common options they mean in this case • Since both parties know the information to send it would generate negative information content n Explains why it is hard to find common sense on the Internet!

Basic Analogy n n n The Shotgun approach to the Human Genome Extract millions of fragments then knit them back together by finding commonalities Will it work for the Human Menome?

What is Cyc? n n n “the world's largest and most complete general knowledge base and commonsense reasoning engine” Started in mid 1980’s (“should take only 10 years…. ”) Logic Based LISP oriented For Word. Net users, each Concept ≈ Synset Available from http: //www. opencyc. org http: //researchcyc. co m n Big (Research. Cyc v 0. 8) • • • n Constants Assertions Deduction 89, 379 968, 985 361, 185 Sample Collection Extents • • • English. Word 18, 007 Event 6, 050 Partially. Tangible 24, 387 • Microtheory 1, 688

Example of what Cyc currently knows about fingers Collection : Finger GAF Arg : 1 Mt : Universal. Vocabulary. Mt isa : Animal. Body. Part. Type genls : Digit-Anatomical. Part comment : "The collection of all digits of all Hands (q. v. ). Fingers are (typically) flexibly jointed and are necessary to enabling the hand (and its owner) to perform grasping and manipulation actions. " Mt : Base. KB defining. Mt : Animal. Physiology. Vocabulary. Mt Mt : Animal. Physiology. Mt proper. Physical. Part. Types : Fingernail Mt : Word. Net. Mapping. Mt (synonymous. External. Concept Finger Word. Net-Version 2_0 "N 05247839") (synonymous. External. Concept Finger Word. Net-1997 Version "N 04312497") GAF Arg : 2 Mt : Universal. Vocabulary. Mt (genls Little. Finger) (genls Index. Finger) (genls Thumb Finger) (genls Ring. Finger) (genls Middle. Finger) Mt : Human. Activities. Mt (body. Parts. Used-Type Typing Finger) Mt : Human. Social. Life. Mt (body. Parts. Used-Type Pointing. AFinger)

Example of what Cyc currently knows about fingers - 2 n Mt : Animal. Physiology. Mt -(conceptually. Related Fingernail Finger) (proper. Physical. Part. Types Hand Finger) (relation. All. Instance age Finger (Years. Duration 0 200)) (relation. All. Instance width. Of. Object Finger (Meter 0. 001 0. 2)) (relation. All. Instance height. Of. Object Finger (Meter 0. 001 0. 2)) (relation. All. Instance length. Of. Object Finger (Meter 0. 01 0. 5)) (relation. All. Instance mass. Of. Object Finger (Kilogram 0. 001 1)) GAF Arg : 3 Mt : Human. Physiology. Mt (relation. All. Exists anatomical. Parts Homo. Sapiens Finger) Mt : Vertebrate. Physiology. Mt (relation. All. Exists. Count physical. Parts Hand Finger 5) Mt : Universal. Vocabulary. Mt (relation. All. Only worn. On Ring-Jewelry Finger) Mt : Animal. Physiology. Mt (relation. Exists. All physical. Parts Hand Finger) GAF Arg : 4 Mt : General. English. Mt (denotation Finger-The. Word Count. Noun 0 Finger)

Bootstrapping with Research. Cyc n n n Cyc has vocabulary about objects in the world and relationships Cyc could still use more common relationships BURC uses what Cyc already has + lots of parsed text to create new Cyc entries for common relationships found in the text Lenat’s Bootstrap Hypothesis: once Cyc reaches a certain level/scale it can help in its own development and start using NLP to augment its knowledge base BURC should help test this hypothesis

The BURC Process From seeds…Hypothe-seed’s n n Use the link grammar parser for bulk parsing of text, primarily narratives based in ‘worlds like ours’. Other text styles could be included. Operates in two directions: • • Forward from text to Cyc. L Backwards from existing Cyc. L to the text to find new forward patterns

BURC Process - 2 n n Load the link fragments into a database (1 and 2 link fragments), and compute frequency of fragment occurrences. The database will be in a SQL format so multiple queries can be formed dynamically. Using Cyc knowledge as a starting point (the seeds), extract knowledge for use in Cyc: • Given a set of seed facts in Cyc, identify how those facts are represented as link fragments in the database • Generate conjectures as to new knowledge AND new knowledge extraction patterns using the fragment patterns.

BURC Process - 3 n Use Cyc knowledge directly to conjecture new statements: • Cyc has lexical knowledge, which can be used as templates against the DB to form new statements • For example, common adjectives applied to noun classes • Cyc knows “White. Color” and “Blouse” but does not know that white is a common blouse color, although it becomes apparent after reading some text n Optionally, gather supporting background statistics for hypothesis verification using other sources: • Perhaps Google desktop with a larger than fully parsed corpus • Perhaps check against answer extraction engines

Flow of Processing

KNEXT (KNowledge EXtraction from Text) n Deriving general world knowledge from texts and taxonomies: • http: //www. cs. rochester. edu/~schubert/projects/worldknowledge-mining. html • Lenhart K. Schubert and Matthew Tong, "Extracting and evaluating general world knowledge from the Brown Corpus", Proc. of the HLT-NAACL Workshop on Text Meaning, May 31, 2003, Edmonton, Alberta, pp. 7 -13. n n System extracts commonsense relationships from text Limited to the pre-parsed Penn Treebank Generated 117, 326 propositions (about 2 per sentence) About 60% judged reasonable by any given judge

KNEXT (Example) (BLANCHE KNEW 0 SOMETHING MUST BE CAUSING STANLEY 'S NEW, STRANGE BEHAVIOR BUT SHE NEVER ONCE CONNECTED IT WITH KITTI WALKER. ) A FEMALE-INDIVIDUAL MAY KNOW A PROPOSITION. SOMETHING MAY CAUSE A BEHAVIOR. A MALE-INDIVIDUAL MAY HAVE A BEHAVIOR CAN BE NEW. A BEHAVIOR CAN BE STRANGE. A FEMALE-INDIVIDUAL MAY CONNECT A THING-REFERRED-TO WITH A FEMALEINDIVIDUAL. ((: I (: I (: Q (: F (: Q (: P DET FEMALE-INDIVIDUAL) KNOW[V] (: Q DET PROPOS)) K SOMETHING[N]) CAUSE[V] (: Q THE BEHAVIOR[N])) DET MALE-INDIVIDUAL) HAVE[V] (: Q DET BEHAVIOR[N])) DET BEHAVIOR[N]) NEW[A]) DET BEHAVIOR[N]) STRANGE[A]) DET FEMALE-INDIVIDUAL) CONNECT[V] (: Q DET THING-REFERRED-TO) WITH[P] (: Q DET FEMALE-INDIVIDUAL))))

Other Extraction Pattern Research n n Towards Terascale Knowledge Acquisition (Pantel, Ravichandran and Hovy, 2004) Learning Surface Text Patterns for a Question Answering System (Ravichandran & Hovy, 2002) n Defined Pattern Precision P = Ca/Co Ca = total number of patterns with answer term present Co = Total number of patterns with any term present n DIRT – Discovery of Inference Rules from Text (Lin & Pantel, 2001)

Other Lexical Knowledge Research n n Verb. Ocean (Chklovski & Pantel): Collecting pairs and searching to verify relationships Lexical Acquisition via Constraint Solving (Pedersen & Chen): Acquiring syntactic and semantic classification rules of unknown words for LGP Information Extraction Using Link Grammar papers Automatic Meaning Discovery Using Google

Forward Mining Adjective Relations n n There are 1941 GAF’s on adj. Sem. Trans, the primary lexical adjective predicate Find applicable fragments and use definitions: • “Select * from LGPTable Where Num. Links=1 and Link 1='a' and Term 1 like '%. a' and Term 2 like '%. n‘ ” • Returns records [Term 1. a | Term 2. n] • Potentially test using either an internal or search engine based relevancy metric • Query Cyc for “(adj. Sem. Trans <term 1>-The. Word ? N Regular. Adj. Frame (? Pred : NOUN ? Val))” • Generate (plausible. Pred. Val. OFType <term 2> <? Pred> <? Val>) • Possibly generate parsing rule

Mining Adjective Knowledge Example n n n “white blouse” as factoid [white. a | blouse. n] Potentially test using an internal or search engine relevancy metric [GC=70400] (adj. Sem. Trans White-The. Word 11 Regular. Adj. Frame (main. Color. Of. Object : NOUN White. Color)) Hypothesis: (plausible. Pred. Value. Of. Type Blouse main. Color. Of. Object White. Color)

Mined Finger Descriptions 000010: (#$plausible. Pred. Value. Of. Type #$Finger #$feels. Sensation (#$Positive. Amount. Fn #$Level. Of. Soreness)) 000037: (#$plausible. Pred. Value. Of. Type #$Finger #$force. Capacity #$Strong) 000025: (#$plausible. Pred. Value. Of. Type #$Finger #$hardness. Of. Object #$Hard) 000037: (#$plausible. Pred. Value. Of. Type #$Finger #$hardness. Of. Object (#$Medium. To. Very. High. Amount. Fn #$Hardness)) 000002: (#$plausible. Pred. Value. Of. Type #$Finger #$has. Evaluative. Quantity (#$Medium. To. Very. High. Amount. Fn #$Goodness-Generic)) 000002: (#$plausible. Pred. Value. Of. Type #$Finger #$has. Physical. Attractiveness #$Good. Looking) 000047: (#$plausible. Pred. Value. Of. Type #$Finger #$isa (#$Left. Object. Of. Pair. Fn : REPLACE)) 000015: (#$plausible. Pred. Value. Of. Type #$Finger #$isa (#$Right. Object. Of. Pair. Fn : REPLACE)) 000155: (#$plausible. Pred. Value. Of. Type #$Finger #$length. Of. Object (#$Relative. Generic. Value. Fn #$length. Of. Object : REPLACE #$high. Amount. Of)) 000155: (#$plausible. Pred. Value. Of. Type #$Finger #$length. Of. Object (#$Relative. Generic. Value. Fn #$length. Of. Object : REPLACE #$high. To. Very. High. Amount. Of)) 000003: (#$plausible. Pred. Value. Of. Type #$Finger #$main. Color. Of. Object #$Black. Color) 000010: (#$plausible. Pred. Value. Of. Type #$Finger #$main. Color. Of. Object #$Light. Yellowish. Brown. Color) 000010: (#$plausible. Pred. Value. Of. Type #$Finger #$main. Color. Of. Object #$Moderate. Yellowish. Brown-Color) 000010: (#$plausible. Pred. Value. Of. Type #$Finger #$main. Color. Of. Object #$Sun. Tan-Flesh. Color) 000002: (#$plausible. Pred. Value. Of. Type #$Finger #$possessive. Relation #$Sudden. Change)

Mined Finger Descriptions 000006: (#$plausible. Pred. Value. Of. Type #$Finger #$possessive. Relation (#$High. Amount. Fn #$Speed)) 000094: (#$plausible. Pred. Value. Of. Type #$Finger #$rigidity. Of. Object (#$High. Amount. Fn #$Rigidity)) 000060: (#$plausible. Pred. Value. Of. Type #$Finger #$size. Parameter. Of. Object (#$Relative. Generic. Value. Fn #$size. Parameter. Of. Object : REPLACE #$high. Amount. Of)) 000052: (#$plausible. Pred. Value. Of. Type #$Finger #$size. Parameter. Of. Object (#$Relative. Generic. Value. Fn #$size. Parameter. Of. Object : REPLACE #$high. To. Very. High. Amount. Of)) 000060: (#$plausible. Pred. Value. Of. Type #$Finger #$size. Parameter. Of. Object (#$Relative. Generic. Value. Fn #$size. Parameter. Of. Object : REPLACE #$high. To. Very. High. Amount. Of)) 000285: (#$plausible. Pred. Value. Of. Type #$Finger #$size. Parameter. Of. Object (#$Relative. Generic. Value. Fn #$size. Parameter. Of. Object : REPLACE #$very. Low. To. Low. Amount. Of)) 000074: (#$plausible. Pred. Value. Of. Type #$Finger #$size. Parameter. Of. Object (#$Relative. Generic. Value. Fn #$size. Parameter. Of. Object : REPLACE #$very. Low. To. Low. Amount. Of)) 000029: (#$plausible. Pred. Value. Of. Type #$Finger #$speed. Of. Object-Underspecified (#$Low. Amount. Fn #$Speed)) 000138: (#$plausible. Pred. Value. Of. Type #$Finger #$surface. Feature. Of. Obj #$Slippery) 000074: (#$plausible. Pred. Value. Of. Type #$Finger #$temperature. Of. Object #$Warm) 000004: (#$plausible. Pred. Value. Of. Type #$Finger #$texture. Of. Object #$Rough) 000168: (#$plausible. Pred. Value. Of. Type #$Finger #$thickness. Of. Object (#$Relative. Generic. Value. Fn #$thickness. Of. Object : REPLACE #$high. Amount. Of)) 000168: (#$plausible. Pred. Value. Of. Type #$Finger #$thickness. Of. Object (#$Relative. Generic. Value. Fn #$thickness. Of. Object : REPLACE #$high. To. Very. High. Amount. Of)) 000182: (#$plausible. Pred. Value. Of. Type #$Finger #$wetness. Of. Object #$Wet)

Verb Semantic Filtering -1 Discovering what a finger can do… n A similar process can be used finding information based on verb semantic parsing frames n For each potential <NOUNWORD>-<VERB> pair query Cyc to find basic relationships using the verb semantic templates (#$and (#$denotation <NOUNWORD> ? NOUNTYPE ? N ? CYCTERM) (#$word. Forms ? WORD ? PRED ""<VERB>"") (#$speech. Part. Preds ? POS ? PRED) (#$sem. Trans. Pred. For. POS ? SEMTRANSPRED) (? SEMTRANSPRED ? WORD ? NUM ? FRAME ? TEMPLATE)) n Verify for each potential relationship (<SPRED> <VERTERM> <CYCTERM>) derivable from ? TEMPLATE that it makes sense in the ontology (#$and (#$arg 1 Isa <SPRED> ? VTYP) (#$arg 2 Isa <SPRED> ? CTYP) (#$genls <CYCTERM> ? CTYP) (#$genls <VERBTERM> ? VTYP) )

Verb Semantic Filtering -2 Templates of Movement… (verb. Sem. Trans Move-The. Word 0 Intransitive. Verb. Frame (and (isa : ACTION Movement. Event) (primary. Object. Moving : ACTION : SUBJECT))) (verb. Sem. Trans Move-The. Word 1 Intransitive. Verb. Frame (and (isa : ACTION Change. Of. Residence) (performed. By : ACTION : SUBJECT))) (verb. Sem. Trans Move-The. Word 2 Transitive. NPFrame (and (isa : ACTION Causing. Another. Objects. Translational. Motion) (object. Acted. On : ACTION : OBJECT) (done. By : ACTION : SUBJECT))) (arg 1 Isa performed. By Action) (arg 2 Isa performed. By Agent-Generic)

Verb Semantic Filtering - 3 n BURC can use Cyc’s knowledge of what things can perform what actions or have what attributes to filter out implausible relationships. (#$behavior. Capable. Of (#$behavior. Capable. Of n #$Finger #$Finger #$Finger #$ Causing. Another. Objects. Translational. Motion #$done. By) #$ Change. Of. Residence #$performed. By) #$Inspecting #$ performed. By) #$Movement- Translation. Event #$primary. Object. Moving) #$ Movement. Event #$primary. Object. Moving) #$ Pushing. An. Object #$provider. Of. Motive. Force) #$Sliding-Generic #$ object. Moving) #$Sliding-Generic #$ primary. Object. Moving) #$Slipping #$ object. Moving) #$Slipping #$ primary. Object. Moving) Cyc can help in its own knowledge entry process. 62% of generated hypothesis were filtered out using semantic role filtering.

The General Backwards Model n n Given some Cyc relation Pred(? X, ? Y) Create SQL search query • Lookup in Cyc lexical entries for X & Y LX, LY • Select * from LGPTable where Term 1="<LX>" and Term 3="<LY>“ • System returns records [LX | Link 1 | Term 2 | Link 2 | LY] (Freq) n Generate new hypothetical extraction patterns • Select * from LGPTable where Link 1="<L 1>" and Link 2="<L 2>" and Term 2="<T 2>“ • [* L 1 T 2 L 2 *] generate hypothetical record ( Pred |? S 1|? S 3 ) • Frequency information is propagated forward

Flow of Processing

Running the system n n Used a filtered set of the BNC (650 Meg of data) 5 parsers running in parallel for 70 hours generated 1. 91 Gig of output Reduced to 1 Gig of unique records with counts 783 Meg or 22 million fragments

Frequency of Fragments n n The distribution of fragments follow a smooth curve in log space Similar to zipf distribution for words, characters and n-grams

The Hunt for Common Fragments n n Forward mining was run over adjective links with more than one fragment and subject-verb with more than two links In both cases this was approximately the top 15% for each search class

Reductions

A source of potential knowledge n n n The various versions of Cyc have 10 to 20 assertions per constant BURC generates 14. 29 hypothetical assertions per constant Need to quantify the quality of BURC knowledge

Future Work -1 n Modify Cyc to utilize the extracted knowledge • Question generation (curiosity ? ) • Noticing exceptions n n Update parser and generate data in other knowledge formats (i. e. Open. Mind/Concept. Net) Generate better filtering methods for polysemous words in fragments Use synonyms and antonyms to expand hypothesis using Word. Net Examine effect of reporting the unusual instead of the usual

Future Work -2 n n Define admissibility criteria. How much evidence is necessary to consider a fact worthy of addition to the KB as commonplace? Determine performance relative to and in conjunction with volunteer commonsense knowledge entry projects. Create an interface for quick review of hypothesis by humans. Utilize knowledge and experience on the backwards miner

Can we ever be “Done” ? n n Explore definition of semantic coverage metrics for unmapped domains. The space of 2. 4 K of binary predicates applied to 85 K constants provides a 16 trillion combination search space, only a fraction of would be considered part of ‘common knowledge’.