CALO Learning Overview AIC Machine Learning Discussion Group

CALO Learning Overview AIC Machine Learning Discussion Group 26 October 2004 with material shamelessly pilfered from previous presentations by: • Tom Dietterich/Leslie Kaelbling (Transfer Learning) • Colin Evans (Task Setup) • Lynn Voss (Task Discussion) • David Martin (Task Fulfillment)

The Learning Picture (Terminology) labeled training set annotated corpus execution traces … algorithm input device input meta-learning algorithm naïve Bayes maximum entropy C 4. 5 k-means learning by being told … learned device output (test) instance current state document corpus prior knowledge … Bayesian network HMM decision tree information extractor procedure clusterer … predicted categories ranked lists facts/relations social networks clusters …

ML Today: “Engineered” Learning data set algorithm input device input learning algorithm learned device human engineers features, invents algorithms, runs experiments to find the best performance on (static) data sets device output

The Vision: Learning in the Wild ENVIRONMENT algorithm input device input learning algorithm learned device system decides when to learn, what to learn, and how to learn, and adapts itself through interaction with the environment device output

The Vision Behind the Vision: Robust, Enduring Systems transfer learning CALO immediately performs Task B better CALO learns to perform Task A CALO learns to perform Task B faster

Example 1: Transfer of Learned Facts Task A: Meeting Planning Task B: Purchasing Who should attend budget meeting for Project X? Who can approve purchases on Project X? Learned on Task A Financial officers should attend budget meetings Learned on Task B Stephen Q. is financial officer for Project X Stephen Q. should attend budget meeting Transfer Learning, Tom Dietterich & Leslie Kaelbling Financial officers can approve purchases Stephen Q. can approve purchases

Example 2: Transfer of Learned Subprocedures Task A: Purchasing Computers Task B: Purchasing Books Tradeoff Specs, Price, Availability Computer Meets Specs Computer Specs: • CPU speed • Memory size • Disk size Availability Shipping Cost Availability: • Discontinued • Back ordered • Delivery date Transfer Learning, Tom Dietterich & Leslie Kaelbling Tradeoff Specs, Price, Availability Book Meets Specs Book Specs: • Title • Author • Binding Availability Shipping Cost Availability: • Out of print • Back ordered • Delivery date

Example 3: Transfer of Learned Ontology Task A: Tenure review in university Task B: Command control in Air Force Leader Leader Leader Leader Member Member Member Member Member Member Organization is a hierarchy of groups Each group has a team leader and team members The members of all groups except the lowest are the team leaders of subgroups Note: Domain facts and procedures do NOT transfer: Tenure dossier flows up hierarchy Transfer Learning, Tom Dietterich & Leslie Kaelbling Orders flow down hierarchy

Example 4: Transfer of Learned Feature Relevance Task A: Routing Complaints Task B: Meeting Scheduling Job title determines job responsibilities Carpenter: framing, installing cabinets Drywaller: taping, sealing, texturing Painter: masking, painting Contractor: scheduling, project planning “Chief Evangelist” might be able to substitute for “Evangelist” in meeting These inferences can be made without even knowing what “sealing” or “Evangelist” mean Transfer Learning, Tom Dietterich & Leslie Kaelbling

CALO Organization • Technology Focus Centers (TFCs) • • • Reasoning & Action (RA) Cyber Awareness (CA) Physical Awareness (PA) Multi-Modal Dialogue (MMD) Learning (L) • Scenarios (Year 1) • meeting scheduling • meeting understanding • laptop purchase

Functional Columns (Year 2) • Task Setup • recognize implications of starting a new task • information harvesting, scheduling setup, dossier preparation • Task Discussion • integrate results of interaction between humans and CALOs into task management • meeting understanding • Task Fulfillment • support user in performing tasks • scheduling, procurement

Task Setup: Information Harvesting Task Setup, Colin Evans

Learning in Task Setup: Information Harvesting Component Input Device Sem. Ex desktop contents extraction rules facts about and relations between documents, email, etc. DEX home pages information extractor (CRF) information about people, including contact information, areas of expertise, and social groups Activity Clusterer Algorithm bidirectional clustering emailbox Output activity (task) groupings of email

Task Setup: Scheduling Setup Task Setup, Colin Evans

Learning in Task Setup: Scheduling Setup Component Algorithm Input Device Output Speech. Act email type message classifier Activity Classifier email activity classifier activity of message email. ME email information message extractor Task Manager advice email type meeting constraints policies for determining meeting constraints (e. g. , start time constraints, participants)

Task Setup: Dossier Preparation Task Setup, Colin Evans

Learning in Task Setup: Dossier Preparation Component Algorithm Task. Tracer Device desktop logging operations Naive Bayes Activity Classifier Input desktop operation task (activity) classifier email message activity (task) classifier Output relations between tasks (activities) and desktop operations activity (task) of message

Task Discussion: Meeting Room Us e hea r w/ d s et a er eo St er am C T R A a Bo rd SM e Fra m Task Discussion, Lynn Voss Us he er w ad / se t e Us he er ad w/ se t am am • Stereo Camera • (IR - Blue Eyes Camera) • Array Microphones • All attached around a user’s laptop Fr Fr Frame includes: e CAMEO

Task Discussion: Architecture Participant List CAMEO Whiteboard’s Panoramic Stereo MPEG encoder Camera Meeting Dossier Supporting Docs Topic Agenda SMART Board Digital Ink Frame Close Talking Speech IRIS Data Store Task Setup Instrumented Raw Text Notes & Data Power Point Capture Meeting Recorder Architecture NTP Meeting Playback System Face Tracker Body Tracker Face Recog. 3 D-Gesture Activity Recog. Video & Array Microphone Classifiers Speaker localization Head, eye, gaze tracker Affect Recog. Object Recog. Charter Handwriting 2 D Gesture Digital Ink Recognizers Multi-parser Tracking Data Integrator OAA Facilitator Task Discussion, Lynn Voss Meeting Room Audio Server OOV End Pointer Prosody Agent Suite Transcription Dialogue Manager Suite FSDB MS Project Agent Meeting Record / MOKB Meeting Room IRIS Data Store Purchase Request Tracking Data Integrator & Audio Server Offline Analysis Suite Agenda Topics Phases Action Items Roles Tasks Milestones MSBITS Rough Sum. MS Project File Meeting Browser User Feedback Loop OOV Words

Learning in Task Discussion: Project Plan Capture Component Agent Sphinx Algorithm multimodal learning Input Device speech, gestures, writing Output words (complete model) instruction chart types 2 D gesture recognizer (written) symbols speech recognizer speaker ID handwriting recognizer words

Learning in Task Discussion: Physical Awareness Component CAMEO Body. Tracker Frame Algorithm Input Device Output face recognizer person ID coarse activity recognizer sitting, standing, walking, … articulated tracker participant movement 3 D gesture recognizer pointing, orientation object recognizer objects affect recognizer affect speaker localizer speaker location speaker detection speaker ID high-level activity recognizer

Learning in Task Discussion: Meeting Awareness Component Algorithm Meeting Analysis Suite Input Device Output topic tracker topic shifts agenda tracker agenda segments action item identifier action items decision identifier decisions meeting phase segmenter meeting phases role tracker participant role acoustic model speech transcript

Task Discussion: Meeting Record Content • Raw Streams • raw audio, raw video, whiteboard strokes, text notes, PPT presentations • Low Level Events • • • out-of-vocabulary words participant locations with torso and body positions participant activities (coarse) who spoke to whom recognized affects recognized words & symbols on the whiteboard word transcripts new participants new chart types new 2 D & 3 D gestures

Task Discussion: Meeting Record Content • High Level Events • • • project plan (task names, durations, milestones) participants, including entrance/exit when each agenda item was discussed topics/subtopics and relevance to agenda action items, including responsible parties, deadlines decisions and proposers; alternative proposals and reasons for/against • participant roles (participator, observer, presenter) • meeting phases (introductions, discussions, briefings, presentations)

Task Fulfillment: Scheduling Formulate Scheduling Request (Task Setup) Relax Scheduling Request Get User Selections and/or Confirmations Gather Information Update Calendars, Send Notifications Prepare Schedule Candidates Send Reminders Task Fulfillment, David Martin

Learning in Task Fulfillment: Scheduling Component Algorithm Input Device PLIANT SVM Task Manager advice policies for scheduling, relaxation, reminder Auto. Minder reinforcement learning reminder strategy procedural learner memorybased learner schedule ranker Output value (cost) revision procedure case-based learning scheduling procedure

Task Fulfillment: Purchasing Select Type of Item Learn Vendors Add Vendors Wrap Vendors Choose Vendors & Define Requirements Task Fulfillment, David Martin Get Quotes Relax Query Get User Selections and/or Confirmation Refine Purchase Procedure Execute Purchase Procedure

Learning in Task Fulfillment: Purchasing Component Algorithm Input Device product Know. It Output vendor sites product ontologies Fetch learning by being told Web site wrapper product information model of new source Prometheus Mediator LOQR C 4. 5 unsatisfiable query decision rules relaxed query Tailor learning by being told revised procedure Task Manager advice policies for procurement

The CALO Test Main Claim: CALO performs well and, through learning, performs even better. • The Test • • AP-style exam Administered regularly throughout the year Must show general improvement overall. Only learning in the wild counts.

The CALO Test 0 O 2 1 O L A C 2. 3 O L A C 2. Test Score L A C 2. improvement due to engineering improvement due to learning 0 O L A C 3. total improvement due to engineering and learning

Situated Learning CALO is a cognitive assistant. • Task Manager (the heartbeat of CALO) • controls what CALO does • situation assessment • workflow management • Knowledge Machine/Query-Update Manager • what CALO knows • CALO ontology

Situated Learning CALO is deployed in the office environment. • IRIS • suite of integrated desktop applications • ontology-driven architecture • provides instrumentation and automation facilities

Learning Issues CALO is not (yet) a robust, enduring system. • • • much in-the-wild learning is not truly online concept drift/shift is not addressed disparate sources are not coordinated new tasks require human engineering ontology changes require lobotomies learning is component-specific