Learning and Exploiting Context in Agents About Context

Learning and Exploiting Context in Agents • • • About Context Learning Domains with Content Trying it out Integrating Learning and Inference Learning Context Bruce Edmonds, AAMAS 2002 (Talk Outline)

Context in Human Cognition • • • Bruce Edmonds, AAMAS 2002 language memory concepts and categorization affect social cognition (probably) reasoning (About Context)

Context In AI John Mc. Carthy (1971), Generality in Artificial Intelligence “p is true in context i” asserted in context c p r q i Bruce Edmonds, AAMAS 2002 s c (About Context)

Context In ML Main purposes: • to maintain learning when there is a hidden/unexpected change in context • to apply learning gained in one context to different context • to utilise already known information about contexts to improve learning Bruce Edmonds, AAMAS 2002 (About Context)

The Problem 3 choices: • Use Global Approaches – But inference and learning can be hard • Specify all the contextual information – Can be onerous • Learn Contextual Information with Content – Need an algorithm Bruce Edmonds, AAMAS 2002 (About Context)

Bruce Edmonds, AAMAS 2002 (About Context)

Transfer of knowledge from learning to application context Is (only) possible when: 1. some of the possible factors influencing an outcome are separable in a practical way 2. foreground features and others can be usefully distinguished 3. background factors are capable of being recognized later 4. world is regular enough for – such models to be learnable – such learnt models to be useful when applied Bruce Edmonds, AAMAS 2002 (About Context)

Fuzzy Domain & Crisp Content Bruce Edmonds, AAMAS 2002 (About Context)

Coincident Clusters of Domain&Content make a Context M 1 M 2 Abstract to a context Bruce Edmonds, AAMAS 2002 (About Context)

An Evolutionary Algorithm D 3. 7 2. 1 p 0. 9 2. 2 Some Space of Characteristics Bruce Edmonds, AAMAS 2002 (Learning Domain & Content)

Comparison in an Artificial Stock Market Environment: • Traders (n context, n straight GP) • 1 Market maker: prices and deals: 5 stocks • Traders buy and sell shares at current market price, but do not have to do so • Traders have memories, can observe actions of others, index, etc. • Modelling ‘arms-race’ • Actions change environment Bruce Edmonds, AAMAS 2002 (Trying it out)

Total Assets in a Typical Run Black=context, White= non-context Bruce Edmonds, AAMAS 2002 (Trying it out)

Total Assets of Context Traders – Total Assets of Normal Traders, scaled by standard deviation of assets (7 agents of each type, 9 runs) (Bold=average, Light= scaled difference for one run) Bruce Edmonds, AAMAS 2002 (Trying it out)

Average for 10 runs with 3 traders of each type Bruce Edmonds, AAMAS 2002 (Trying it out)

Snapshot of model domains in one trader Volatility - past 5 periods 950 900 850 800 750 700 750 Bruce Edmonds, AAMAS 2002 850 950 Volume - past 5 periods (Trying it out)

The model contents in snapshot Bruce Edmonds, AAMAS 2002 (Trying it out)

The Problems of Under- and Over. Determination 1. Under-determination • Neither nor can be inferred Choose a more specific context 2. Over determination • Both and can be inferred Choose a less specific context Bruce Edmonds, AAMAS 2002 (Integrating Learning and Inference)

Universal learn and infer loop repeat learn and/or up update beliefs deduce intentions, plans and actions until finished Bruce Edmonds, AAMAS 2002 (Integrating Learning and Inference)

Learn and infer loop using context repeat recognise/learn/choose context, c induce/update beliefs in c deduce predictions/conclusions in c until predictions are possible, consistent and actions/plans can be determined plan & act (starting from c) until finished Bruce Edmonds, AAMAS 2002 (Integrating Learning and Inference)

Recap - Clusters of Domain&Content make a Context M 1 M 2 Abstract to a context Bruce Edmonds, AAMAS 2002 (Integrating Learning and Inference)

Heuristics for Learning Context (I) • Formation: if there is a cluster of similar domains then create a context • Abstraction: if contexts share models with the same domain, abstract them to super context • Specialisation: If restricting the domain allows more models, create a subcontext. Bruce Edmonds, AAMAS 2002 (Integrating Learning and Inference)

Heuristics for Learning Context (II) • Content Correction: If only a few models are in error then remove or correct them • Content Addition: If a model has the same domain as an existing context, then add it Bruce Edmonds, AAMAS 2002 (Integrating Learning and Inference)

Heuristics for Learning Context (III) • Context Restriction: If most models are in error, exclude that situation from context • Context Expansion: If most models work in new conditions, then expand the context • Context Removal: If a context has few models or a tiny domain, forget the context Bruce Edmonds, AAMAS 2002 (Integrating Learning and Inference)

Bruce Edmonds bruce. edmonds. name Centre for Policy Modelling cfpm. org Context Home Page www. context. umcs. maine. edu Bruce Edmonds, AAMAS 2002 The End!