SUBTLE Overview Situation Understanding Bot Through Language and

SUBTLE: Overview (Situation Understanding Bot Through Language and Environment) UMass Amherst, UMass Lowell, Penn, Stanford, Cornell, George Mason Mitch Marcus University of Pennsylvania

Our Guiding Vision Standing orders: clear the building: search for any bombs or weapons, find any injured persons and report them and their location so the medical team can find them, and tell any other persons to leave the building. 0: 00 Commander: Jr, I want you to search the room you are in first and when you go to leave head for the North end of the building, continue around East and we will meet up at the end of a long hallway. 0: 32 Jr: 10 -4 3: 13 Jr: I have cleared the first room and am proceeding North down a hallway. 3: 20 Commander: Mark that room as clear and make certain to check all the rooms in the hall. 3: 45 Jr: Room is marked. Proceeding to the next room. 3: 56 Commander: Wait. Shouldn’t you be going into the hallway? 4: 02 Jr: No, I went directly into the next room. 4: 05 Commander: Can you show me the map and mark your position. 4: 15 Jr: I am displaying the map, I am the yellow dot. 5: 23 Commander: Alright I see now, carry on. 10/16/2009 SUBTLE Year 2 Overview 2

Scientific Objective: Central Hypothesis Effective communications with autonomous bots in real-time situations requires that bots understand not only what is literally said, but also what is intended, the implicit meaning Example: Exchange between firefighters (Worcester Cold Storage Fire 1999): · P 2 (a) answers question (b) appears to give command · P 1’s implicit question: How can we find you? • Linguists call this the Question Under Discussion (QUD) · P 2 answers the QUD 10/16/2009 SUBTLE Year 2 Overview 3

Technical Approach: Key Ideas · We must develop techniques to analyze not only what sentence a speaker used, but also what that speaker’s implicit meaning. · The communication system must exploit the broad context of the environment. · The linguistic specification should incorporate formal models of language • To guarantee computationally efficient analysis methods • To facilitate study of the habitability and effectiveness of the result · The adequacy of specifications should be determined by empirical, corpus-based methods • Our research must proceed by collecting example corpora of interactions in increasingly complex (simulated) environments 10/16/2009 SUBTLE Year 2 Overview 4

Potential Breakthroughs · New frameworks and theories for • Robust automated understanding of linguistic intention • Linking linguistic intention to appropriate robot control · New machine learning algorithms for structured problems such as Natural Language · A formal computational specification of a significant subset of English · A testbed system for investigating HRI using natural language • Should ultimately enable military designers to develop powerful communication methods between bots and humans. 10/16/2009 SUBTLE Year 2 Overview 5

The SUBTLE team: senior participants · Natural Language Processing/Computer Science • Aravind Joshi, Penn • Mitch Marcus, Penn • David Smith, UMass Amherst (Research Scientist To join Y 3) • Fernando Peirera, Penn (On leave: Director of Research, Google) · Robotics/Electrical Engineering, Computer Science • George Pappas, Penn • Holly Yanco, UMass Lowell • Hadas Kress-Gazit, Cornell (Penn grad student Cornell) · Human Simulation & Graphics • Norm Badler, Penn • Jan Allbeck, George Mason (Penn grad student George Mason) · Machine Learning: Andrew Mc. Callum, UMass Amherst · Linguistics: Chris Potts, Stanford · Current Grad Students: 8 10/16/2009 SUBTLE Year 2 Overview 6

People Actively Involved Year 2 · Faculty • • · Grad Students Norm Badler Aravind Joshi Mitch Marcus Andrew Mc. Callum George Pappas Chris Potts Holly Yanco • • · New Ph. Ds • Jan Allbeck • Hadas Kress-Gazit · Undergraduates · Limited involvement: • Fernando Pereira · Newly Involved • David Smith • Florian Schwarz 10/16/2009 Chris Czyzewicz Munjal Desai Kuzman Ganchev Dan Hestand Karl Schultz Qiuye Zhao Pengfei Huang Dan Brooks • • Victoria Schwanda HCI, Cornell Jessica Ouyang Dan Keller Phil Kovac SUBTLE Year 2 Overview 7

Framework – Year 1 Review Natural Language Processing Linguistics Robotics Parsing Graphics/ Human Simulation Parse tree, indices, semantic tags Semantics Machine Learning World Model Underspecified predicate logic Pragmatics Joint I nferen Parameterized Action Representations 10/16/2009 ce SUBTLE Year 2 Overview Linear Temporal Logic 8

Year 1: Primary Accomplishments · SUBTLE architecture intensively reworked · Key components built and tested • LTL →FSA transducer • PAR (Procedural Action Representation) for Prag. Bot · “Prag. Bot 1” corpus collection tool built, initial corpus collected, NLP analyzer built · New results: Probabilistic pragmatics meets probabilistic inference 10/16/2009 SUBTLE Year 2 Overview 9

Evolution of Framework Natural Language Processing Linguistics Robotics Parsing Graphics/ Human Simulation Parse tree, indices, semantic tags Semantics Machine Learning World Model Underspecified predicate logic Pragmatics Joint I nferen Parameterized Action Representations 10/16/2009 ce SUBTLE Year 2 Overview Linear Temporal PAR + LTL Logic 10

Evolution of Framework Natural Language Processing Linguistics Robotics Joint Inference Parsing Graphics/ Human Simulation Parse tree, indices, semantic tags Semantics Machine Learning World Model Underspecified predicate logic Pragmatics PAR + LTL 10/16/2009 SUBTLE Year 2 Overview 11

Evolution of Framework Natural Language Processing Linguistics Robotics Joint Inference Parsing Graphics/ Human Simulation Parse tree, indices, semantic tags Semantics Machine Learning World Model Underspecified predicate logic Pragmatics PAR + LTL 10/16/2009 SUBTLE Year 2 Overview 12

Accomplishments to date I · Ph. D dissertations completed • Hadas Kress-Gazit, Transforming high level tasks to low level controllers, Dec 2008 (Asst Prof, Cornell) — Uses Linear Temporal Logic Model Checking to compile complex constraints and requirements into real-time hybrid robot controller. • Jan Allbeck, Creating 3 D Animated Human Behaviors for Virtual Worlds, June 2009 (Asst Prof, George Mason) — Extended Parameterized Action Representation (PAR) for robust, multi-agent setting. Developed extensive authoring tools. · LTL controller is now outputting PAR representations to drive virtual animation for corpus collection, system development · PAR simulation handles multi-agent setting needed for team exercises of Years 4 -5. 10/16/2009 SUBTLE Year 2 Overview 13

LTL Example: “Find Nemo” · Task spec in English • “Nemo can only be in Regions 1, 3, 5 and 8. Look for Nemo and if you find him, turn your video camera on and stay where you are. If he disappears again, turn the camera off and resume the search. ” (There are 12 regions which define the robot propositions {r 1, …, r 12}). · Fragment encodes possible moves that enforce: • if R sees Nemo, stay in that region at next step with camera on. Otherwise, camera off. - Next ¨ - Always ◊ - Eventually 10/16/2009 SUBTLE Year 2 Overview 14

The resulting controller can control virtual robot & actual robot Talk 5 10/16/2009 SUBTLE Year 2 Overview 15

CAROSA: Authoring PARs for NL Predicates, for multiple agents, … Talk 6 10/16/2009 SUBTLE Year 2 Overview 16

Accomplishments II · A new formal account of Grice’s maxims, a foundation of linguistic pragmatics · Embedded and tested in a computation implementation of a rich account of formal linguistic pragmatics • In communicating — Be truthful — Be relevant — Be brief • Encoding (for the moment in Markov Logic): 10/16/2009 Assert(p) => True(p) =>Assert(p). //Truthful Assert(p) => Qud(q) ^ About(p, q)). 10 (Qud(q) ^ About(p, q)) => Assert(p) //Relevance Assert(p) => !Commander. Believe(p). 10 !Assert(p) => Commander. Believe(p) //Brief SUBTLE Year 2 Overview 17

Short term application: Response Relevance · Pragmatics of Relevance, Brevity for Bot’s responses formalized within Probabilistic Markov Logic // 1. General description of what it means to have completed the task. Report(Task. Complete) <=> (FORALL x (Relevant(x) => Found(x))). // 2. If you find something, report it. 5 Found(x) => Report(x) // 3. Report only relevant things. 10 Report(x) => Relevant(x) 10 !Relevant(x) => !Report(x) // 4. If you can report that the task is complete, report nothing else. 5 Report(Task. Complete) => ((y != Task. Complete) => !Report(y)) 10/16/2009 SUBTLE Year 2 Overview Talk 3 18

Pragmatics & Action → Pragmatics in Action Talk 4 10/16/2009 SUBTLE Year 2 Overview 19

Integration and State Summarization 10/16/2009 SUBTLE Year 2 Overview 20

Other Accomplishments · Last Year: Initial Prag. Bot web-based interface constructed for corpus collection • 50 Human-Human corpus of collaborative interactions in simple isomorphic environment captured & analyzed · New framework & toolkit for joint inference about to be released • Allows efficient machine inference of probabilistic models across very large, complex knowledge bases • Joint Segmentation & Coreference of research paper citations: 1295 mentions, 134 entities, 36487 tokens • Compare with Markov Logic Networks (Alchemy) — ~25% reduction in error (segmentation & coref) — 3 -20 x faster · … 10/16/2009 SUBTLE Year 2 Overview 21

First Year Review Report – 10/2008 Develop a common simulation and experimentation scenario…which is demanding and sophisticated…. You must get busy on building the corpus from a major search and rescue simulation/experiment/game…. Build a robot language community…. Start to visit Do. D labs and establish the groundwork for technology transfer. Continue to assume that future…bots will have better…capabilities than current bots, yet whenever possible use both current bots and simulated future bots to test algorithms and tools. Focus on developing how the world view (situation awareness) will influence the language and understanding of the bot. While the first… 2 -3 years is focusing…mostly on the bot receiving and understanding information, the overall goal…is to provide a framework for dialog among a team of several bots and several humans … Don’t lose sight of this important goal while performing the preliminary work. Develop more robust (human-based) performance metrics and base your models and goals on these. 10/16/2009 SUBTLE Year 2 Overview 22

First Year Review Report – 10/2008 1. Develop a common simulation and experimentation scenario…which is demanding and sophisticated…. 2. You must get busy on building the corpus from a major search and rescue simulation/experiment/game…. 3. Build a robot language community…. 4. Start to visit Do. D labs and establish the groundwork for technology transfer. 10/16/2009 SUBTLE Year 2 Overview 23

The Prag. Bot I Data Collection Environment During Year 1: · Prag. Bot data collection interface implemented • • Two players in world with cards (each can hold 3). Goal: pick suit, find 6 -in-a-row · Prag. Bot symmetric corpus of 50 dialogues collected · NL analyzer built to map corpus to PAR · Guidance: “Develop a common simulation and experimentation scenario … which is demanding and sophisticated…. ”, “build the corpus from a major search and rescue simulation/…/game…. 10/16/2009 SUBTLE Year 2 Overview 24

Example data: Cooperation · · · · % pragbot_chat_log_2007. 11. 01 AD at 19. 48. 20 EDT. txt Player 2: i have 4 H Player 1: I want it! Player 1: where is it? Player 2: should i leave it for you somewhere? Player 1: sure Player 1: where are you? Player 2: okay, where are you? Player 1: I'm near the top Player 2: i'm left side. Player 1: next to the gap near the middle Player 2: i'll leave the card in the upper left corner. Player 1: awesome 10/16/2009 SUBTLE Year 2 Overview 25

Talk 2 Last Year’s Annual Review · First Annual Review: 3 Oct 2008 – UPenn • “Develop a common simulation and experimentation scenario … which is demanding and sophisticated…. ” • Response: Pragbot 2. 0 10/16/2009 SUBTLE Year 2 Overview 26

Prag. Bot 2 scenarios are our targets · The Prag. Bot 2 world provides a simple yet relevant world for scenarios for testing humanrobot communication using language · Limits in world align with limits of current robot perception · One extension: We will allow Jr to pick up objects in our test scenarios. • Will simulate in Robot demo 10/16/2009 SUBTLE Year 2 Overview 27

Workshop on Situated Understanding · First Year Review: “Build a robot language community …. ” · When: two full days July 23 -24, 2009 · Where: Institute for Research in Cognitive Science, Penn · Who: 26 participants, including many members of SUBTLE advisory committee · Workshop Focus: • Human-Robot Communication through NL and other contexts requiring recognition of goals and intentions — Interactive fiction — Embodied agents — Social avatars — Avatars in games involving language communication —… 10/16/2009 SUBTLE Year 2 Overview 28

Workshop Talks · · Four talks by SUBTLE participants Dialog as Planning with Knowledge and Sensing • · Towards a Robotic Architecture for Natural Spoken Human-Robot Dialogues • · William Schuler (OSU) Engagement and Deixis for a Humanoid Robot • · Matthew Stone (Rutgers) A Simple Computational Model of Interactive Language Comprehension • · James Allen (Rochester) Models and Skills for Understanding Communicative Intentions • · Matthias Scheutz (Indiana) & Kathleen Eberhard (Notre Dame) Intention Situated in Collaborative Dialogue • · Ron Petrick & Mark Steedman (Edinburgh) Candy Sidner (BAE) From Events to Natural Language in the Interactive Fiction System Curveship • Nick Montfort (MIT) 10/16/2009 SUBTLE Year 2 Overview 29

Connections to Do. D labs – Year 2 · “Start to visit Do. D labs and establish the groundwork for technology transfer. ” · 2007—Visit NRL Talk: The generality of pragmatic inference: Message enrichment in multi- agent interactions (Potts) · 5/09 HRI Workshop & Group Meeting sponsored by ARL’s Advanced Decision Architectures (ADA) Collaborative Technology (Yanco) · 5/09—Visit ARL Talk: SUBTLE (Marcus) · 7/09—Situated Understanding Workshop (All) · 7/09—Kick-off meeting for new ONR MURI & Talk (Potts) 10/16/2009 SUBTLE Year 2 Overview 30

Behind plan: Two areas • Corpus Collection • Implementation of Pragbot 2 more difficult than expected • • • 3 D implementation ambitious, given that game must completely download as webapp Initial game completed and limited data now collected, but scenario requires tuning. Connection of Pragmatics to Language • • • Alchemy system for Markov Logic good for small examples but Alchemy can’t handle rich highly interconnected logical models Expected: that’s why we were betting on Mc. Callum’s Factorie But we didn’t expect the limitations to hit so soon Factorie beta due within a week or two! Talk 7 10/16/2009 SUBTLE Year 2 Overview 31

Summary of progress · Crucial Steps Forward • Architecture evolving as components connect —LTL PAR —Pragmatics Robot’s expectations • Pragbot 2 functioning • Two grad students to strong faculty positions, with continued involvement (1 undergrad to HCI Ph. D program) • Situated Understanding Workshop building community · Where behind • Just beginning to gather data • Factorie is crucial for progress · Pieces about to come together 10/16/2009 SUBTLE Year 2 Overview 32

Schedule for the Day… 8: 35 8: 45 9: 30 10: 10 10: 50 11: 15 11: 55 12: 30 p. m. 1: 10 1: 50 2: 05 2: 45 3: 00 4: 30 Introduction SUBTLE: Overview Pragbot 2. 0: Moving 'Pragbot' Language Interactions Toward More Realistic Situations The interplay of linguistic & contextual inferences Break Integration and State Summarization Lunch Meaning to motion: Transforming specifications to provablycorrect control Integrating Linear Temporal Logic and a Parameterized Action Representation & Creating 3 D Animated Human Behaviors for Virtual Worlds Break Joint Inference for the NLP Pipeline: Probabilistic Programming and the FACTORIE System SUBTLE Year 3 plans Advisory Committee Meeting Advisory Committee Initial Report Review Adjourns 10/16/2009 SUBTLE Year 2 Overview Joe Myers Mitch Marcus Chris Czyzewicz/Norm Badler Chris Potts Dan Hestand / Munjal Desai /Holly Yanco Hadas Kress-Gazit/George Pappas Jan Allbeck/Norm Badler David Smith/Andrew Mc. Callum/Karl Schultz Mitch Marcus 33

BACKUP 10/16/2009 SUBTLE Year 2 Overview

Components built and tested · PAR implemented and tested for SUBTLE world · High level autonomous actions added to robot • Search, Find, Follow, Snapshot · World Model Ontology DB and API implemented · LTL stress-tested in Urban Challenge enviroment • Connection from Pseudo-English to LTL to Robot tested 10/16/2009 SUBTLE Year 2 Overview 35