fa buf cse lo Endowing Agents with a

fa buf @ cse lo Endowing Agents with a Personal Sense of Time Haythem O. Ismail & Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science State University of New York at Buffalo {hismail | shapiro}@cse. buffalo. edu S. C. Shapiro

fa buf @ cse • • • S. C. Shapiro lo Outline Introduction Subjective Time and Bodily Knowledge Time and External States Summary

fa buf @ cse • • • lo Uses of Time by a Cognitive Agent Reason about time Talk about time Reason and act in time Decide to act in timely fashion Remember acts and when done Sense current states Be informed of past/future states Understand & generate NL with correct tense & aspect. S. C. Shapiro

fa buf @ cse lo Cassie • A Computational Cognitive Agent • Based on SNe. PS – Logic-based – Network-based – Knowledge representation, reasoning, and acting. S. C. Shapiro

fa buf @ cse lo Embodied Cassie • A Cognitive Robot – Hardware – or Software-Simulated • Separate, but aligned – Body – Mind. S. C. Shapiro

fa buf @ cse lo GLAIR Architecture Grounded Layered Architecture with Integrated Reasoning Knowledge Level NL SNe. PS Perceptuo-Motor Level Sensory-Actuator Level Vision Sonar Motion Proprioception S. C. Shapiro

fa buf @ cse lo Symbol Grounding: Alignment lex ad e h s s a l c clas smo d green class ! find member lex ac tio n ct e j ob B 6 “Find the green robot. ” S. C. Shapiro lex robot

fa buf @ cse • • • S. C. Shapiro lo Outline Introduction Subjective Time and Bodily Knowledge Time and External States Summary

fa buf @ cse lo Deictic Center • • • S. C. Shapiro <I, YOU, NOW> Variables whose values are SNe. PS terms Aspects of embodiedness *I: SNe. PS term representing Cassie *YOU: person Cassie is talking with *NOW: current time.

fa buf @ cse lo Subjective Time • NOW contains SNe. PS term representing current time. • NOW moves when Cassie acts or perceives a change of state. S. C. Shapiro

fa buf @ cse lo Representation of Time before after ! after ? ? t n e ag ! time ? ? state ? ? ? ! before act B 1 n actio obje ct B 6 I lex find S. C. Shapiro NOW

fa buf @ cse lo t 1 NOW S. C. Shapiro Movement of Time v. 1 before ! after t 2 NOW before ! after t 3 NOW

fa buf @ cse lo The Pacemaker • PML process periodically increments variable COUNT. • *COUNT = some PML integer. • Reset to 0 when NOW moves. • Provides bodily “feel” of passing time. S. C. Shapiro

fa buf @ cse lo Quantizing Time Cannot conceptualize fine distinctions in time intervals. So quantize, e. g. into half orders of magnitude (Hobbs, 2000). S. C. Shapiro

fa buf @ cse lo Movement of Time with Pacemaker ! time t 1 before ! duration after q t 2 KL PML hom NOW COUNT n 0 S. C. Shapiro

fa buf @ cse • • • S. C. Shapiro lo Outline Introduction Subjective Time and Bodily Knowledge Time and External States Summary

fa buf @ cse lo Modality Variables • Similar to Deictic Center. • E. g. : VISION, AUDITION, WHEELS, ARMS • *VISION = Holds(Lookat(Cassie, Stu), t 3) – if vision currently occupied by looking at Stu – t 3 denotes the time during which Cassie will be looking at Stu – *NOW is during t 3 • Set at PML when bodily state starts/ceases. • One state may occupy multiple modalities. S. C. Shapiro

fa buf @ cse lo Knowing What You’re Doing • When NOW moves – For each modality variable v – s. t. *v = Holds(s, t) – Make *NOW a subinterval of t • So the agent believes it is now doing everything it is, in fact, doing. S. C. Shapiro

fa buf @ cse lo When you stop • When state s ceases – For each modality variable vi – s. t. *vi = Holds(s, ti) • Set vi to nil – Move NOW – Believe each ti is before *NOW. S. C. Shapiro

fa buf @ cse lo When you start • When state s starts – For each modality v that s occupies – set v to Holds(s, ti) – Move NOW. S. C. Shapiro

fa buf @ cse • • • S. C. Shapiro lo Outline Introduction Subjective Time and Bodily Knowledge Time and External States Summary

fa buf @ cse lo The Problem of the Fleeting Now How can you reason about “now” if it never stands still? S. C. Shapiro

fa buf @ cse lo Motivating Joke 9: 30: 00 AM (Door-to-Door Salesman): May I interest you in a brush? 9: 30: 02 AM (Homeowner): Not now. 9: 30: 03 AM (Salesman): Now? S. C. Shapiro

fa buf @ cse lo Fleeting Now Example 1 9: 15: 00: If the walk light is on now, cross the street. = If the walk light is on at 9: 15: 00, cross the street. 9: 15: 01: Turn to look at walk light. 9: 15: 02: The walk light is on at 9: 15: 02. Should you cross the street? Yes, but why? S. C. Shapiro

fa buf @ cse lo Fleeting Now Example 2 12: 15: 00: 12: 15: 02: 17: 00: 12: 19: 00: “Is John having lunch now? ” Agent walks to John’s office. Agent sees John at his desk, eating. Agent reports “yes”. Appropriate granularity. S. C. Shapiro

fa buf @ cse lo Fleeting Now Example 3 12: 15: 00: “Is John having lunch now? ” Agent knows John is at home without a phone. Agent contemplates driving to John’s home. Don’t bother---inappropriate granularity. S. C. Shapiro

fa buf @ cse lo The Vagueness of “now” I’m now giving a talk. I’m now teaching a course. I’m now visiting Houston. I’m now living in Buffalo. The agent is now walking to John’s office. The agent is now seeing if John is eating lunch. Multiple now’s at different granularities. S. C. Shapiro

fa buf @ cse lo NOW-MTF Maximal Temporal Frame based on *NOW Semi-lattice of times, all of which contain *NOW, any of which could be meant by “now” Finite---only conceptualized times of conceptualized states S. C. Shapiro

fa buf @ cse lo Moving NOW with MTF NOW S. C. Shapiro

fa buf @ cse lo Typical Durations “If the walk light is on now, cross the street. ” Relevant duration is typical duration of walk lights. “Is John having lunch now? ” Relevant duration is typical duration of lunch. Use quantized typical durations when updating NOW-MTFs. S. C. Shapiro

fa buf @ cse lo Using Appropriate Granularity Lunch time ? h c n Lu NOW Lu Yes! S. C. Shapiro nc h!

fa buf @ cse • • • S. C. Shapiro lo Outline Introduction Subjective Time and Bodily Knowledge Time and External States Summary

fa buf @ cse lo Summary • Distinguish body & mind, but align them. • Body (PML): – – – What people and things look like. Primitive and routine actions. Time intervals. Pacemaker: Feel for elapsing time. Deictic Center variables. Modality variables. • Mind: – Conceptualized people, things, actions, times, states. S. C. Shapiro

fa buf @ cse lo When Inquire about States Put them into MTF According to their typical duration. S. C. Shapiro

fa buf @ cse lo When NOW Moves • Use Pacemaker to measure old NOW. • Include current actions in MTF. • Include other states according to their typical durations. S. C. Shapiro

fa buf @ cse lo For More Information http: //www. cse. buffalo. edu/sneps/ S. C. Shapiro