alo uff b cse The GLAIR Architecture for
alo uff b @ cse The GLAIR Architecture for Cognitive Robotics Stuart C. Shapiro Department of Computer Science & Engineering and Center for Cognitive Science State University of New York at Buffalo shapiro@buffalo. edu
alo uff b @ cse GLAIR: Grounded Layered Architecture with Integrated Reasoning Focus of This Talk: Connecting Reasoning Mind with Acting/Sensing Body Cog. Rob 2010 S. C. Shapiro 2
alo uff b @ cse Motivations • Add acting and sensing to a reasoning agent. – First person reasoning; on-line acting & sensing. • Layers – Motivated by mind/body connections/distinctions. – Let same mind be plugged into different bodies. • Embodiment – Origin of beliefs in sensation & proprioception. – First-person privileged knowledge of own body. • Situatedness – Has a sense of where it is in the world. • Symbol grounding – In body-layer structures. – Symbol as pivot between various modalities. Cog. Rob 2010 S. C. Shapiro 3
alo uff b @ cse Outline Ø The Layers Symbol Grounding Putting it Together: Two examples Cog. Rob 2010 S. C. Shapiro 4
alo uff b @ cse GLAIR Architecture Mind Body KL Independent of lower-body implementation PMLa PMLb I/P s o c k e t s Dependent on lower-body implementation PMLc Proprioception Speech Hearing SAL Vision Motion Cog Rob 2010 S. C. Shapiro W O R L D 5
alo uff b @ cse Sensori-Actuator Layer • Sensor and effector controllers Cog. Rob 2010 S. C. Shapiro 6
alo uff b @ cse Perceptuo-Motor Layer • PMLa • PMLb • PMLc Cog. Rob 2010 S. C. Shapiro 7
alo uff b @ cse PMLc • Abstracts sensors & effectors • Body’s behavioral repertoire Cog. Rob 2010 S. C. Shapiro 8
alo uff b @ cse PMLb • Translation & Communication – Between PMLa & PMLc • Highest layer that knows body implementation Cog. Rob 2010 S. C. Shapiro 9
alo uff b @ cse PMLa • Grounds KL symbols – Perceptual structures – Implementation of primitive actions • Registers for Embodiment & Situatedness – Deictic Registers – Modality Registers Cog. Rob 2010 S. C. Shapiro 10
alo uff b @ cse • • • The Knowledge Layer Implemented in SNe. PS Agent’s Beliefs Representations of conceived of entities Semantic Memory Episodic Memory Quantified & conditional beliefs Plans for non-primitive acts Plans to achieve goals Beliefs re. preconditions & effects of acts Policies: Conditions for performing acts Self-knowledge Meta-knowledge Cog. Rob 2010 S. C. Shapiro 11
alo uff b @ cse SNe. PS • A KRR system • Every non-atomic expression is simultaneously – An expression of SNe. PS logic – An assertional frame – A propositional graph • Every SNe. PS expression is a term – Denoting a mental entity Cog. Rob 2010 S. C. Shapiro 12
alo uff b @ cse Ontology of Mental Entities • Entity – Proposition Agent can believe it or its negation Includes quantified & conditional beliefs – Act Agent can perform it – Policy Condition-act rule agent can adopt – Thing Other entities: individuals, properties, times, etc. Cog. Rob 2010 S. C. Shapiro 13
alo uff b @ cse Outline The Layers Ø Symbol Grounding Putting it Together: Two examples Cog. Rob 2010 S. C. Shapiro 14
alo uff b @ cse Entities, Terms, Symbols, Objects • Agent’s mental entity: a person named Stu • SNe. PS term: b 4 • Object in world: Cog. Rob 2010 S. C. Shapiro 15
alo uff b @ cse Alignment Mind (KL) KL term PML structure Body (PML/SAL) World Cog. Rob 2010 Object/Phenomenon S. C. Shapiro Action 16
alo uff b @ cse World Objects to Feature Tuples PML/SAL World <Height, Width, Texture, . . . > Cog. Rob 2010 S. C. Shapiro 17
alo uff b @ cse Feature Tuples to KL Terms KL PML/SAL Proper. Name(b 4, Stu) Alignment Cog. Rob 2010 S. C. Shapiro <Height, Width, Texture, . . . > 18
alo uff b @ cse Incomplete PML-Descriptions KL PML/SAL Height(b 4, b 12) <Height, nil, . . . > Cog. Rob 2010 S. C. Shapiro 19
alo uff b @ cse Unifying PML-Descriptions PML/SAL KL <x, nil, . . . > b 20 Isa b 30 <nil, y, nil, . . . > b 31 <nil, z, . . . > Prop <x, y, z, . . . > b 6 Cog. Rob 2010 S. C. Shapiro 20
alo uff b @ cse Deictic Registers For being situated in the world PML registers hold KL terms I term denoting agent YOU term denoting dialogue partner NOW term denoting current time Cog. Rob 2010 S. C. Shapiro 21
alo uff b @ cse Modality Registers For privileged first-person knowledge of what agent is doing Register for each modality, m, holds KL term denoting act that m is engaged in Cog. Rob 2010 S. C. Shapiro 22
alo uff b @ cse Outline The Layers Symbol Grounding Ø Putting it Together: Two examples Cog. Rob 2010 S. C. Shapiro 23
alo uff b @ cse Language-Mind-World-Mind PML/SAL KL m 76 b 30 Prop m 75 <green, nil> b 20 cassie b 6 <nil, robotic> (find<green, robotic> ) Isa m 2 WORLD VISION robbie Find a green robot. Cog. Rob 2010 S. C. Shapiro 24
alo uff b @ cse Building Episodic Memory KL m 1! event e 1 m 4! m 2! after before time t 1 event e 2 t 2 act agent time duration COUNT PML time agent act a 1 m 3! b 1 q a 2 I n 0 Cog. Rob 2010 NOW hom ACT S. C. Shapiro 25
alo uff b @ cse Summary • GLAIR is an architecture: – To connect mind and body • For symbol grounding – To provide first-person privileged knowledge of • The body • What it’s doing – To provide situatedness in the world • Via deictic registers Cog. Rob 2010 S. C. Shapiro 26
alo uff b @ cse Collaborators Past and present members of SNe. RG: The SNe. PS Research Group http: //www. cse. buffalo. edu/sneps/ CSE 719 S. C. Shapiro
- Slides: 27