Cognitive architectures Soar Lotzi Blni EEL 5708 Cognitive

Cognitive architectures. Soar. Lotzi Bölöni EEL 5708

Cognitive science • Cognitive science is usually defined as the scientific study either of mind. • Highly interdisciplinary: it is said to consist of, take part in, and collaborate with: – Psychology (especially cognitive psychology), – Linguistics, – Neuroscience, – Artificial intelligence (neural network research in particular), – Philosophy EEL 5708

Unified Theories of Cognition • Is a theory which attempts to unify all theories of the mind in a single framework. • Allen Newell (1990) proposed that the current state-of-the-art in experimental psychology could now support such theories, based on years of accumulated results. • To assert a unified theory of cognition, one must propose mechanisms by which the results of these human cognitive experiments can be reproduced. The codification and simulation of these mechanisms is tantamount to designing an architecture for general intelligence. EEL 5708

Psychological Validity as an Issue in Cognitive Architectures • Does the architecture make any attempt to model aspects of human behavior? • The answer is not always either easy or straightforward. • Some research in cognitive architectures is concerned with modeling the methods by which humans solve problems. • Another approach is to try to develop architectures which behave intelligently without regard to the psychological plausibility of the method by which the behavior is achieved. • Still yet another approach is to claim that intelligence can not be achieved without modeling the architecture of the brain first, and then determining the methods which will produce the desired behavior. • Power Law of Learning: – “the logarithm of the reaction time for a particular task decreases linearly with the logarithm of the number of practice trials taken. ” – With more practice at a task, people seem to always be getting faster. – However, the rate of learning decreases the more practice one has. EEL 5708

Cognitive architectures • “Architecture” – is a key word for this domain • In computer science, the architecture of a system is a fixed structure that provides a system which can be programmed. • In cognitive science, the term refers to the architecture of the mind: a fixed structure underlying the flexible domain of cognitive processing. – Cognitive architectures – for humans – Architectures for intelligent agents – for agents – The ambition of SOAR is to be both: a basis for both human and artificial cognition. EEL 5708

Examples of cognitive architectures Subsumption Architecture (Brooks) Heterogeneous Asynchronous Architecture (Gat) Plan-then-compile Architectures (Theo) Planning and Learning Architecture (PRODIGY) Modular-Integrated Architecture (ICARUS) Adaptive Intelligent Systems (AIS) A Meta-reasoning Architecture for 'X' (MAX) A Basic Integrated Agent (Homer) Problem-Space Architecture (Soar) Situated Action + Planned Action (Teton) Real-Time, Decision-Theoretic Architecture (RALPHMEA) The Entropy Reduction Engine (ERE) EEL 5708

Soar EEL 5708

Short Long history • • Started in 1983 a group led by Allen Newell 22 years of history. 8 versions. Probably about 300 papers It is still under active development. Academic center: University of Michigan Commercial arm: Soar. Tech Inc. EEL 5708

Philosophy (in their own words) • The Soar project is an attempt to develop and apply a unified theory of human and artificial intelligence. • The core of the effort is the architecture – the fixed base of tightly coupled mechanisms – underlying intelligent behavior. • This architecture then forms the basis for wideranging investigations into basic intelligent capabilities – such as problem solving, planning learning, knowledge representation, natural language, perception and robotics. • This is a true cognitive-science enterprise, where human and artificial evidence and criteria are constantly intermingled in service of progress in both areas. EEL 5708

Components 1. 2. 3. 4. 5. 6. 7. 8. Problem Spaces Long-Term Memory Attribute-Value Representation Preference Memory Decision Procedure Perceptual-Motor Subsystems Goal-Directed Behavior Chunking-Based Learning EEL 5708

Problem spaces • Represents all tasks as collections of problem spaces • Problem space: – States + operators that manipulate states EEL 5708

Long term memory • Soar's long term memory is a production system based on Ops 5. Productions: – have a set of conditions, which are patterns to be matched to working memory, – a set of actions to perform when the production fires. – Conditions can match to all the current goals, problem spaces, states and operators on the context stack in working memory. • Actions can only add elements to preference memory. These elements are attribute/value pairs for some object and a preference, which indicates a (lack of) desire to add this element to working memory. • Soar performs no conflict resolution between competing productions- all productions which match the current working memory fire. EEL 5708

(Let us remember…) Productions (defrule mammal (animal ? name) (warm-blooded ? name) (not (lays-eggs ? name)) => (assert (mammal ? name)) (printout t ? name " is a mammal" crlf)) • The problem with a production system is the efficiency of matching a number of conditions against the knowledgebase. • The RETE algorithm EEL 5708

Working memory • Soar's working memory consists of a set of (Object ^attribute value) elements. Value may be a symbolic constant, a number, a string, or an Object. The context stack consists of the context objects currently in working memory: all goals, problem spaces, states, and operators. All context elements are attached to a goal, and all goals except the top goal point to a supergoal, imposing a linear order on the context stack. • There must be some chain of elements from a context object to every element in working memory. If this chain is broken the element is removed from working memory. • The Decision Cycle can examine and modify the entire context stack. The preference memory determines which elements enter and leave working memory. EEL 5708

Decision cycle • Soar works in a loop called decision cycle: – Elaboration – Decision – Repeat • Elaboration: – all productions which match the current working memory fire. All productions fire in parallel. The elaboration phase runs to Quiescence (until no more productions fire). • Decision – examines any preferences put into preference memory – chooses the next problem space, state, operator or goal EEL 5708

Impasse • If there is not enough information (or the information is contradictory) for the decision phase to choose the next slot value, then an impasse results. There are four types of impasses: 1. When two are more elements have equal preference, then there is a "tie impasse". 2. When no preferences are in working memory, this causes a "nochange impasse“ 3. When the only preferences in working memory are rejected by other preferences, then there is a "reject impasse". 4. A "conflict impasse" results when preferences claim that two or more elements are each better choices then the others. • Impasses occur when there is a lack of applicable knowledge in the current problem space, so they, in a sense, signal that problem solving needs to take place. This problem solving proceeds in the form of an automatically generated subgoal. EEL 5708

Perceptual-Motor System • The perceptual subsystem consist of independant modules for each input channel. They run asynchronously with respect to each other and to the remainder of the architecture. The perceptual modules deliver data directly into working memory whenever it is available. • The motor subsystem consist of independant modules for each output channel. They run asynchronously with respect to each other and to the remainder of the architecture. The motor modules accept commands from working memory and execute them. Their progress can then be monitored through sensors that are fed back into the system via the perception subsystem. • All perceptual and motor behavior is mediated through working memory. • Encoding and decoding production are used to convert between highlevel structures use by the cognitive system, and the low-level structures used by the perceptual and motor subsystems. • These productions are the same as regular Soar production except that they match on perceptual and motor working memory elements and are independant of context (problem space, state, operator). • This autonomy from context is critical because it allows the decision procedure to proceed without waiting for quiescense, which may not occur in a rapidly changing environment. EEL 5708

Chunking-based Learning • Psychological phenomena of chunking: – The association of chunks (expressions or symbols) into a new, single chunk. – In Soar, chunking collapses the results of an impasse into a production which can then be fired if the same, or similar situation occurs again, thus avoiding the impasse. – This leads directly to Soar's ability to move from problematic to routine behavior. – Since the learning mechanism creates new knowledge in the same form as the rest of the system's knowledge (i. e. productions), the uniformity of the representation is maintained by the learning mechanism. • Because chunking is based on the results of a resolved impasse, it is an experience • based learning technique. • Since this dependency analysis uses only those working memory elements that were used in impasse resolution, chunking generalizes implicitly. The situation that leads to a chunk does not need to be reproduced exactly for the chunk to fire later; only those elements which led directly to the chunk are necessary. • The architecture directs the creation of a chunk whenever an impasse is resolved. Thus, chunking is both a universal and reflexive process. Additionally, the chunking mechanism is fixed and impenetrable and can not be improved by learning. • Finally, chunking can be viewed as a caching mechanism. EEL 5708

New additions to SOAR • • Reinforcement learning Episodic memory Working memory activation Emotion modeling EEL 5708

Programming SOAR • Programming at the knowledge level – Ideally, adjustments and enhancements of behavior should be possible by simply conveying the appropriate knowledge to the system. – One would hardly call this “programming” – But we don’t know how to do it. • Programming as a problem space system – Augmentation in the store of program spaces. – It is primarily a modelling activity • Programming as a symbol level system – Essentially a production system… • … and when everything fails (BL) – Bring the big hammer and hack around the problems in a procedural way. • Anything missing? EEL 5708