Visual Semantics and Ontology of Eventive Verbs Minhua
Visual Semantics and Ontology of Eventive Verbs Minhua Eunice Ma and Paul Mc Kevitt School of Computing and Intelligent Systems Faculty of Engineering University of Ulster, Magee Derry/Londonderry, N. Ireland
Outline Background: CONFUCIUS Previous verb taxonomies Visual semantics & verb classes CONFUCIUS’ ontology of verbs Current status of implementation Relation to other work Conclusion & future work
Architecture of CONFUCIUS Natural language sentences Surface transformer Media allocator Prefabricated objects (knowledge base) Language knowledge mapping 3 D authoring tools, existing 3 D models & character models Visual knowledge (3 D graphic library) LCS lexicon Natural Word. Net Language Processing Text To Speech Sound effects semantic representations visual knowledge Animation engine Synchronising & fusion 3 D world with audio in VRML
NLP in CONFUCIUS Preprocessing Part-of-speech tagger Connexor FDG parser Syntactic parser Semanti c inferenc Word. Net e LCS database Disambiguati on FEATURES Morphologi cal parser Temporal Coreferen reasoning ce resolution Post-lexical Lexical temporal relations
Previous verb taxonomies Grammatical categorisation & valency Thematic roles (Fillmore, 1968; Jackendoff, 1990; Halliday, 1985; Dowty, 1991) Aspectual classes (Vendler, 1967; Stede, 1996) Semantic verb classes in Word. Net (Fellbaum, 1998) Levin’s (1993) verb classes Dimension of causation (Asher & Lascarides, 1995)
Grammatical categorisation & valency Subcategorisation description of verb categories in LDOCE (Longman Dictionary of Contemporary English) D – ditransitive I – intransitive L – linking verb with complement T 1 – transitive verb with NP object T 3 – transitive verb with infinitival clause as object
Grammatical categorisation & valency Subcategorisation description of verb categories in LDOCE (Longman Dictionary of Contemporary English) Syntactic valency Obligatory valency fillers (complements) e. g. subject, object Optional valency fillers (adjuncts) e. g. temporal, locational adjuncts Semantic valency (Leech, 1981)
Thematic roles Other names: theta-role, case role, deep grammatical function, transitivity role, valency role, case frame Extend syntactic analysis into semantic domain to capture roles of participants surface case (nominative, accusative) surface function (subject, object) Thematic roles (e. g. agent, patient/theme, instrument, source, goal, place) Classifying verbs based on thematic roles (Dixon, 1991)
Aspectual (temporal) classification Vendler’s (1967) verb classes activities: run, swim, sleep, cry statives: love, hate, know achievements: arrive, win, find, die accomplishments: build (a house), write (a book) Stede’s (1996) MOOSE ontology Formal ontologies DOLCE, SUMO, and CYC assume traditional aspectual (temporal) classification for events
Aspectual (temporal) classification Vendler’s (1967) verb classes Stede’s (1996) ontology of MOOSE situation state activity event protracted moment culmination transition activity protracted moment culmination Formal ontologies DOLCE, SUMO, and CYC assume traditional aspectual (temporal) classification for events
Aspectual (temporal) classification Vendler’s (1967) verb classes Stede’s (1996) ontology of MOOSE situation state activity event protracted moment culmination transition activity protracted moment culmination Formal ontologies DOLCE, SUMO, and CYC assume traditional aspectual (temporal) classification for events
Semantic verb classes in Word. Net Lexicographer file Contents verb. body grooming, dressing, bodily care verb. change size, temperature change verb. cognition thinking, judging, doubting verb. communication telling, asking, ordering, singing verb. competition fighting, athletic activities verb. consumption eating and drinking verb. contact touching, hitting, tying, digging verb. creation sewing, baking, painting verb. emotion feeling verb. motion walking, flying, swimming verb. perception seeing, hearing, feeling verb. possession buying, selling, owning verb. social political/social activities Taxonomic approach based on pure lexical semantics Reveal semantic organisation of lexicon in terms of lexical & semantic relations Top nodes of Word. Net’s verb file
Levin’s (1993) verb classes Theoretic ground -- semantic/syntactic correlations: verbs with similar meaning (identical LCSs in terms of specific meaning components) show same syntactic behaviors Verbs of motion Verbs of inherently directed motion: arrive, come, enter Leave verbs: leave, abandon, desert Manner of motion verbs: roll, run, sneak, waddle Verbs of motion using a vehicle: bike, drive, fly Chase verbs: chase, follow, track Accompany verbs: accompany, escort, guide Waltz verbs: clog, polka, waltz
Dimension of causation Asher and Lascarides’ (1995) dimension of causation-change causation and change are specified along four dimensions: locative, formal, matter, intentional cause locative loc-cause fml-cause mtr-cause sub of put sub of build sub of paint formal intent-cause sub of amuse matter loc-change obj of put intentional fml-change obj of build change intent-change mtr-change obj of amuse obj of paint
Visual semantics & verb classes Visual factors concerning verb categorisation Visual valency Somatotopic factors in visualisation Level-of-detail of visual information Verbs belonging to same class in the classification Visual “synonyms” Substitutable in same set of animation keyframes Visualisation of action verbs is effective evaluation of the classification
Visual valency Capacity of verb to take specific number and type of visual arguments in language visualisation (3 D animation) valency filler -- visual role 2 types of visual roles requiring different processes in visualisation human (biped articulated animate entity) object (inanimate entity) Visual valency overlaps with syntactic & semantic valency Visual modality requires more obligatory roles than surface grammar or semantics
Somatotopic effectors of action verbs Theoretical ground: execution/perception/visualisation of action verbs produced by same somatotopic effector activate same parts of cortex Distinguish facial expression (e. g. lip movement) & body posture (arm/leg/torso) in our ontological system facial expression – sing, laugh Leg – run, kick body posture Arm – wave, put torso – bow Further divisions like distinction between upper/lower arm, hands, & even fingers are possible
Level-Of-Detail (LOD) basic-level verbs & troponyms EVENT … go cause event level verbs … walk climb limp stride trot swagger Levels run jump manner level verbs jog romp skip bounce hop Verbs troponym level verbs Basic-level verb for. . . event level go, move atomic object manner level walk, jump human/non-atomic object troponym level limp, stride human
CONFUCIUS’ verb taxonomy 2. 2. 1. Action verbs 2. 2. 1. 1. One visual valency (the role is a human, (partial) movement) 2. 2. 1. 1. 1. Biped kinematics: arm actions (wave, scratch), leg actions (walk, jump, kick), torso actions (bow), combined actions (climb) 2. 2. 1. 1. 2. Facial expressions & lip movement, e. g. laugh, fear, say, sing, order 2. 2. 1. 2. Two visual valency (at least one role is human) 2. 2. 1. One human and one object (vt. or vi. +instrument) e. g. throw, push, kick, open, eat, drink, bake, trolley 2. 2. 1. 2. 2. Two humans, e. g. fight, chase, guide 2. 2. 1. 3. Visual valency ≥ 3 (at least one role is human) 2. 2. 1. 3. 1. Two humans and one object (inc. ditransitive verbs), e. g. give, show 2. 2. 1. 3. 2. One human and 2+ objects (vt. + object + implicit instr. /goal/theme) e. g. cut, write, butter, pocket, dig, cook 2. 2. 1. 4. Verbs without distinct visualisation when out of context: verbs of trying, helping, letting, creating/destroying 2. 2. 1. 5. High level behaviours (routine events), political and social activities e. g. interview, eat out (go to restaurant), go shopping
Text-to-Animation of single sentences Collision detection example (contact verbs: hit, collide, scratch, touch), no human role involved “The car collided with a wall. ” using Parallel. Graphics’ VRML extension--object-to-object collision non-speech sound effects H-Anim examples: action verbs 3 visual valency verbs “John gave Nancy a loaf of bread. ” “John put a cup of coffee on the table. ” H-Anim Site node locative tags of object (on_table tag for table object) 2 visual valency verbs “John pushed the door. ” “John ate the bread. ” “Nancy sat on the chair. ” For more demos, please visit http: //www. infm. ulst. ac. uk/~eunice/3 D_anim. html
Relation to other work Categorise verbs from visual semantics perspective Language visualisation in CONFUCIUS provides independent criteria for identifying classes of verbs sharing certain aspects of meaning, i. e. semantic/visual correlations Relation to Levin’s verb classes: “Carol cut the whole wheat bread. ” “Whole wheat bread cuts easily. ” 2. 2. 1. 3. 2, visual valency=3 2. 1. 2, visual valency=2 Verbs of cutting 1 to N “Nancy brought the book to John. ” Verbs of sending & carrying 2. 2. 1. 3. 1 “Nancy gave the book to John. ” Verbs of change of possession visual valency=3 N to 1
Conclusion & future work Categorise verbs from visual semantic perspective Provides independent criteria for identifying classes of verbs based on semantic/visual correlations Visual semantic analysis of eventive verbs revealed striking influences in taxonomic verb tree Various criteria ranging from visual valency, somatotopic effectors, to LOD are proposed Evaluation issues using automatic animation generation & psychological experiments Discourse level interpretation
- Slides: 22