fa buf cse lo SymbolAnchoring in Cassie Stuart

fa buf @ cse lo Symbol-Anchoring in Cassie Stuart C. Shapiro and Haythem O. Ismail Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo {shapiro | hismail}@cse. buffalo. edu S. C. Shapiro

fa buf @ cse • • lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary S. C. Shapiro

fa buf @ cse lo Embodied Cassie • A computational cognitive agent – Embodied in hardware – or Software-Simulated – Based on SNe. PS and GLAIR. S. C. Shapiro

fa buf @ cse S. C. Shapiro lo Cassie, the FEVAHR

fa buf @ cse S. C. Shapiro lo FEVAHR/Cassie in the Lab

fa buf @ cse lo SNe. PS • Knowledge Representation and Reasoning – Intensional Representation – Propositions as Terms • SNIP: SNe. PS Inference Package – Specialized connectives and quantifiers • SNe. BR: SNe. PS Belief Revision • SNe. RE: SNe. PS Rational Engine • Interface Languages – SNe. PSUL: Lisp-Like – SNe. PSLOG: Logic-Like – GATN for Fragments of English. S. C. Shapiro

fa buf @ cse lo Interaction with Cassie English (Statement, Question, Command) GATN (New Belief) [SNe. PS] Answer [SNIP] GATN (Current) Set of Beliefs [SNe. PS] Reasoning Parser Clarification Dialogue Looking in World Actions (Updated) Set of Beliefs [SNe. RE] [SNe. PS] Generator Reasoning English sentence expressing new belief answering question reporting actions S. C. Shapiro

fa buf @ cse lo Entities, Terms, Symbols, Objects • Cassie’s mental entity: a person named Stu • SNe. PS term: B 4 • Object in world: S. C. Shapiro

fa buf @ cse lo GLAIR Architecture Grounded Layered Architecture with Integrated Reasoning Knowledge Level SNe. PS Perceptuo-Motor Level NL Sensory-Actuator Level Vision Sonar Proprioception S. C. Shapiro Motion

fa buf @ cse lo Alignment KL SNe. PS term Body (PML/SAL) World S. C. Shapiro PML structure Object/Phenomenon Action

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo World Objects to Feature Vectors PML/SAL <Height, Width, Texture, . . . > S. C. Shapiro World

fa buf @ cse lo Feature Vectors to KL Terms KL PML/SAL Proper. Name(B 4, Stu) Alignment S. C. Shapiro <Height, Width, Texture, . . . >

fa buf @ cse lo Incomplete Feature Vectors KL PML/SAL Height(B 4, B 12) <Height, nil, . . . > S. C. Shapiro

fa buf @ cse lo Unifying PML-Descriptions PML/SAL KL <x, nil, . . . > B 20 Isa B 30 <nil, y, nil, . . . > B 31 <nil, z, . . . > Prop B 6 S. C. Shapiro <x, y, z, . . . >

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo Deictic & Modality Registers KL PML Terms denoting Cassie Addressee Current time Current state(s) of looking at x. . . S. C. Shapiro I You Now Vision. . .

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo Primitive Actions Aligned with PML/SAL Functions KL PML Find(B 4) <. . . > Modality Registers S. C. Shapiro SAL World

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo Aligning NOW using MTF Maximal Temporal Frame based on *NOW KL PML NOW Semi-lattice of times, all of which contain *NOW, any of which could be meant by “now” Finite---only conceptualized times of conceptualized states S. C. Shapiro

fa buf @ cse lo Moving NOW with MTF KL PML NOW Moves when Cassie acts, newly observes a state, or is informed of a new state. Always includes times of states in modality registers. S. C. Shapiro

fa buf @ cse lo Providing a Feel for Time ! time t 1 before ! duration after q t 2 KL PML hom NOW COUNT n 0 S. C. Shapiro

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse lo Aligning Lexemes/NL KL PML/SAL Proper. Name(B 4, Stu) Isa(B 4, person) npr ctgy “Stu” “person” ctgy num Grammar S. C. Shapiro World n sing NL

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse S. C. Shapiro lo Acting 1

fa buf @ cse lo Acting 2 Follow a red robot. I found a red robot. I am looking at a red robot. S. C. Shapiro

fa buf @ cse lo Acting 3 Follow a red robot. I found a red robot. I am looking at a red robot. I went to a red robot. I am near a red robot. I am following a red robot. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 1 Come here. I found you. I am looking at you. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 2 Come here. I found you. I am looking at you. I came to you, Stu. I am near you. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 3 Who am I? Your name is ‘Stu’ and you are a person. Who have you talked to? I am talking to you. Talk to Bill. I am talking to you, Bill. Come here. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 4 Come here. I found you. I am looking at you. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 5 Come here. I found you. I am looking at you. I came to you. I am near you. S. C. Shapiro

fa buf @ cse lo Use of Indexicals 6 Who am I? Your name is ‘Bill’ and you are a person. Who are you? I am the FEVAHR and my name is ‘Cassie’. Who have you talked to? I talked to Stu and I am talking to you. S. C. Shapiro

fa buf @ cse • • S. C. Shapiro lo Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

fa buf @ cse • • lo Summary KL terms denote mental entities. KL terms aligned to PML structures. PML variables/registers contain KL terms. PML variables/registers/structures grounded in world via sensors & effectors. • PML attentional structures anchor symbols by changing only when motivated. S. C. Shapiro

fa buf @ cse • • • lo For More Information Personnel Manual Tutorial Bibliography ftp’able SNe. PS source code etc. • http: //www. cse. buffalo. edu/sneps/ S. C. Shapiro