fa buf cse lo Symbol Anchoring in a

fa buf @ cse lo Symbol Anchoring in a Grounded Layered Architecture with Integrated Reasoning Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo shapiro@cse. buffalo. edu S. C. Shapiro

fa buf @ cse lo Based on Stuart C. Shapiro & Haythem O. Ismail, “Anchoring in a grounded layered architecture with integrated reasoning, ” Robotics and Autonomous Systems 43, 2 -3 (May 2003) 97 -108. S. C. Shapiro

fa buf @ cse • • lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary S. C. Shapiro

fa buf @ cse lo Definition “Anchoring is the problem of connecting, inside an artificial system, symbols and sensor data that refer to the same physical objects in the external world. ” [Silvia Coradeschi & Alessandro Saffiotti] S. C. Shapiro

fa buf @ cse lo Embodied Cassie • A computational cognitive agent – Embodied in hardware – or Software-Simulated – Based on SNe. PS and GLAIR. S. C. Shapiro

fa buf @ cse lo Cassie, the FEVAHR (Foveal Extra. Vehicular Activity Helper-Retriever) S. C. Shapiro

fa buf @ cse S. C. Shapiro lo FEVAHR/Cassie in the Lab

fa buf @ cse lo Crystal Cassie First Person Perspective Views S. C. Shapiro

fa buf @ cse lo Patofil and Filopat from “The Trial, The Trail” A VR drama by Josephine Anstey et al. S. C. Shapiro

fa buf @ cse lo Delivery. Agent Using Byron Weber Becker’s Java version of Rich Pattis’ Karel the Robot S. C. Shapiro

fa buf @ cse S. C. Shapiro lo Magellan Pro. TM Mobile Robot from i. Robot

fa buf @ cse lo SNe. PS • Knowledge Representation and Reasoning – Intensional Representation – Propositions as Terms • SNIP: SNe. PS Inference Package – Specialized connectives and quantifiers • SNe. BR: SNe. PS Belief Revision • SNe. RE: SNe. PS Rational Engine (Acting Language) • Interface Languages – SNe. PSUL: Lisp-Like – SNe. PSLOG: Logic-Like – GATN for Fragments of English. S. C. Shapiro

fa buf @ cse lo Interaction with Cassie English (Statement, Question, Command) GATN (New Belief) [SNe. PS] Answer [SNIP] GATN (Current) Set of Beliefs [SNe. PS] Reasoning Parser Clarification Dialogue Looking in World Actions (Updated) Set of Beliefs [SNe. RE] [SNe. PS] Generator Reasoning English sentence expressing new belief answering question reporting actions S. C. Shapiro

fa buf @ cse lo Entities, Terms, Symbols, Objects • Cassie’s mental entity: a person named Stu • SNe. PS term: B 4 • Object in world: S. C. Shapiro

fa buf @ cse lo GLAIR Architecture Grounded Layered Architecture with Integrated Reasoning Knowledge Level Mind SNe. PS Perceptuo-Motor Level IP Sockets B o d y NL Sensory-Actuator Level Vision Sonar Wo rld S. C. Shapiro Proprioception Motion

fa buf @ cse lo Alignment KL SNe. PS term Body (PML/SAL) World S. C. Shapiro PML structure Object/Phenomenon Action

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo PML/SAL <Height, Width, Texture, . . . > S. C. Shapiro World Objects to Feature Tuples World

fa buf @ cse lo Feature Tuples to KL Terms KL PML/SAL Proper. Name(B 4, Stu) Alignment S. C. Shapiro <Height, Width, Texture, . . . >

fa buf @ cse lo Incomplete PML-Descriptions KL PML/SAL Height(B 4, B 12) <Height, nil, . . . > S. C. Shapiro

fa buf @ cse lo Unifying PML-Descriptions PML/SAL KL <x, nil, . . . > B 20 Isa B 30 <nil, y, nil, . . . > B 31 <nil, z, . . . > Prop B 6 S. C. Shapiro <x, y, z, . . . >

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo Deictic & Modality Registers for being situated in the world KL PML Terms denoting Cassie Addressee Current time Current state(s) of looking at x. . . S. C. Shapiro I You Now Vision. . .

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo KL Primitive Actions Aligned with PML/SAL Functions PML SAL World Find(B 4) M 2(B 4) <. . . > Modality Registers A KL symbol is a pivot coordinating different modalities. S. C. Shapiro

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo Aligning NOW using MTF Maximal Temporal Frame based on *NOW KL PML NOW Semi-lattice of times, all of which contain *NOW, any of which could be meant by “now” Finite---only conceptualized times of conceptualized states S. C. Shapiro

fa buf @ cse lo Moving NOW with MTF KL PML NOW Moves when Cassie acts, newly observes a state, or is informed of a new state. Always includes times of states in modality registers. S. C. Shapiro

fa buf @ cse lo Providing a Feel for Time ! time t 1 before ! duration after q t 2 KL PML hom NOW COUNT n 0 S. C. Shapiro

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo Aligning Lexemes/NL KL PML/SAL Proper. Name(B 4, Stu) Isa(B 4, person) npr ctgy “Stu” “person” ctgy num Grammar S. C. Shapiro World n sing NL

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse S. C. Shapiro lo Acting 1

fa buf @ cse lo Acting 2 Follow a red robot. I found a red robot. I am looking at a red robot. S. C. Shapiro

fa buf @ cse lo Acting 3 Follow a red robot. I found a red robot. I am looking at a red robot. I went to a red robot. I am near a red robot. I am following a red robot. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 1 Come here. I found you. I am looking at you. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 2 Come here. I found you. I am looking at you. I came to you, Stu. I am near you. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 3 Who am I? Your name is ‘Stu’ and you are a person. Who have you talked to? I am talking to you. Talk to Bill. I am talking to you, Bill. Come here. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 4 Come here. I found you. I am looking at you. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 5 Come here. I found you. I am looking at you. I came to you. I am near you. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 6 Who am I? Your name is ‘Bill’ and you are a person. Who are you? I am the FEVAHR and my name is ‘Cassie’. Who have you talked to? I talked to Stu and I am talking to you. S. C. Shapiro

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse • • lo Summary KL terms denote mental entities. KL terms aligned to PML structures. PML variables/registers contain KL terms. PML variables/registers/structures grounded in world via sensors & effectors. • PML attentional structures anchor symbols by changing only when motivated. S. C. Shapiro

fa buf @ cse lo Summary from Jackendoff Ray Jackendoff, Foundations of Language, Oxford University Press, 2002, Fig. 11. 1, p. 348. (Independent work) S. C. Shapiro

fa buf @ cse • • • lo For More Information Personnel Manual Tutorial Bibliography ftp’able SNe. PS source code etc. • http: //www. cse. buffalo. edu/sneps/ S. C. Shapiro