Global Brain Simulations Wodzisaw Duch Google Duch Department

Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering, Nanyang Technological University (NTU), Singapore

Attention-Based Artificial Cognitive Control Understanding System (ABACCUS) Large EU integrated project (>150 pp), with 9 participants: • • King’s College London (John G. Taylor, coordinator), UK Centre for Brain & Cognitive Development, Berkbeck College, University of London, UK Cognition and Brain Sciences Unit, Medical Research Council, UK Robotics and Embedded Systems, Technical University of Munich, G Institute of Neurophysiology and Pathophysiology, Universitätsklinikum Hamburg-Eppendorf, G Institute of Computer Science, Foundation for Research and Technology – Hellas, Heraklion, Crete, GR National Center for Scientific Research “Demokritos”, Athens, GR • Dipartimento di Informatica, Sistemistica, Telematica, Universita di Genova, I • Dep. of Informatics, Nicholaus Copernicus University, Torun, PL

Motivation AI and CI has not been able to create decent human-computer interfaces, solve problems in computer vision, natural language understanding, cognitive search and data mining, or even reasoning in theorem proving. Practical: humanized, cognitive computer applications require a brainlike architecture (either software or hardware) to deal with such problems efficiently; it is at the center of cognitive robotics. Science: understand how high-level cognition arises from low-level interactions between neurons, build powerful research tool; to understand complex systems is to be able to build them. Computer speeds have just reached brain power (about 1016 binop/s), but computers are far from brain’s complexity and processing style.

Scheme of the brain. . . High-level sketch of the brain structures, with connections based on different types of neurotransmiters marked in different colors.

ABACCUS Goals • Assumption: gross neuroanatomical brain structure is critical for its function, therefore it should be preserved. • To demonstrate how fusion of the appropriate brain-based models, guided by the overall architecture of the brain, and by its developmental learning stages, can help attain high-level cognitive processing capabilities. • Show basic language understanding and reasoning abilities for direct human-machine communication, at the level of a pre-school child, mimicking solutions used by the human brain. • Develop an attention control systems for focusing in sensory surveillance tasks, and for image searching. • Development of control structures for autonomous machines. • Create its own goals in an autonomous fashion. • Founded on neuro-scientific understanding of attention and the sensory and motor systems it controls, development in children, simplified modeling, computer power.

Special hardware? • Many have proposed the construction of brain-like computers, frequently using special hardware. • Connection Machines from Thinking Machines, Inc. (D. Hills, 1987) was almost successful, but never become massively parallel. • CAM Brain (ATR Kyoto) – failed attempt to evolve the large-scale cellular neural network; to evolve one must know the function. • Needed: elements based on spiking biological neurons and the layered 2 -D anatomy of mammalian cerebral cortex. • ALAVLSI (EU Consortium), a general architecture for perceptual attention and learning based on neuromorphic VLSI technology. Coherent motion + speech categorization, project ends in 2005.

Other attempts? Artificial Development (www. ad. com) is building CCortex™, a complete 20 -billion neuron simulation of the Human Cortex and peripheral systems, on a cluster of 500 computers - the largest neural network created to date. Artificial Development plans to deliver a wide range of commercial products based on artificial versions of the human brain that will enhance business relationships globally. Rather unlikely? The Ersatz Brain Project – James Anderson (Brown University), based on modeling of intermediate level cerebral cortex structures - cortical columns of various sizes (mini ~102, plain ~104, and hypercolumns ~105). Nof. N, Network of Networks approximation, 2 D BSB network.

Nomads G. Edelman (Neurosciences Institute) & collaborators, created a series of Darwin automata, brain-based devices, “physical devices whose behavior is controlled by a simulated nervous system”. (i) The device must engage in a behavioral task. (ii) The device’s behavior must be controlled by a simulated nervous system having a design that reflects the brain’s architecture and dynamics. (iii) The device’s behavior is modified by a reward or value system that signals the salience of environmental cues to its nervous system. (iv) The device must be situated in the real world. Darwin VII consists of: a mobile base equipped with a CCD camera and IR sensor for vision, microphones for hearing, conductivity sensors for taste, and effectors for movement of its base, of its head, and of a gripping manipulator having one degree-of-freedom; 53 K mean firing +phase neurons, 1. 7 M synapses, 28 brain areas.

Sketch of the ABACCUS system Computational Platform, Simulation Environment and Integration Neuroscience and Development Vision Speech Tactile Memory System Motor Control Reasoning System Feedback Attention Control system Value Maps Drive and Intrinsic reward system Atomization system Learning of PFC goals Working Memory Action/Object reward system Rough sketch of the ABACCUS system, based on simplified spiking neurons.

Natural perception Spectrogram of speech: hearing a sentence.

Spiking vs. mean field Linked Pools Brain: 1011 Neurons A C B Mean-Field Model: Networks of Spiking Neurons Neuron Pools Pool neuron 1 neuron 2 M Neurons Pool Activity: Integrate and Fire Model: neuro n 1 2 3 M spikes

Synaptic Dynamics Synapses Spike Soma EPSP, IPSP Spike

Primary objective 1 n To develop linguistic powers of ABACCUS system. n Use training of single words by associating their representations to internal representations of objects and actions. n Use pair-wise associations to learn word pairs (like ‘kick ball’), extend syntactically and functionally the use of function words. n Working memory modules, with associated phonological coding, will be created and fused in the language component, both for speech understanding and generation. n Extension for abstract concepts by tagging the associated words to clusters of concrete object/action representations.

Primary objective 2 n To create a high-level cognitive system, able to solve problems requiring reasoning, thinking, imagination and creativity. n Based on the basic control concept of a forward model, acting as a predictor and working under attention control. Forward models include various semantic and goal networks. n n Sequences of activations of representations will be learnt and thereby used to achieve goals. Various of these forward models will be present and branch into each other during the running process by means of lateral connections. New routes will occur allowing complex goals to be achieved, or even new, previously unrecognized goals, to be arrived at (required for creativity).

Example of action internal models Rough sketch of the ACTION subnetwork. Sensory information enters the supplementary motor area (SMA) (1) cortico-thalamo-cortical (the ‘short’ loop); (2) cortico-striatal-GPi-thalamocortical (the ‘long’ loop); (3) cortico-(STN GPe)-GPi-thalamo -cortical (‘first indirect’ loop); (4) cortico-striatal-(STN GPe)-GPithalamo-cortical (the ‘second indirect’ loop).

Primary objective 3 n Extract a simplified architecture for the attention control system. n Should involve both sensory and motor control, especially joint sensory-motor control, systems whose creation is guided by neuroscientific knowledge about the brain. n Approach based on infant development to train ABACCUS incrementally on the computer platform. n Use neuroscientific data to guide the architecture of a large-scale neural simulation of the relevant components of the human brain. n Use feed-forward/feedback training simulations based on the simplified brain architecture following developmental processes, using the sensor and response signals of the robotic embodiment.

Primary objective 4 n Developing the ability to learn novel objects, both as stimuli and as associated reward values. n Use reward/penalty feedback training with attention control, with associated value maps constructed to learn to encode the values of stimuli and responses to them in the environment. n Temporal sequence or schemata (as object/action sequences as well as the associated rewards involved) will be constructed to function as predictors and so support reasoning, along with automatic response learning. n Neural codes of visual, auditory and tactile concepts will be learnt in a feedforward manner, without attention, as will codes for motor responses controlling the embodied robot.

Primary objective 5 Develop a robotic embodiment; this involves: n Specification of robots available for ABACCUS control. n Development of sensor and response systems in the robot. Sensory data mappings to the central computational platform. n n n An environmental task domain in which ABACCUS can develop its concepts and cognitive powers. Simulation platform for the developmental training (on a Beowulf cluster or similar powerful computer system). n Specifications for integration of various models on the central computational platform. n Compatibility of computational languages used by partners.

Attention Control n Attention control systems will be achieved by use of the early flow of activity from low-level cortex rapidly to prefrontal sites. n This initially uncoded activity will be used to bias feedback activity sent to parietal and/or temporal lobe, so as to bias the attention feedback from there down to the activated inputs encoded in the object representations in the temporal lobe. n This feedback loop will be trained by causal Hebbian learning, with the attention feedback represented by contrast gain amplification, so by sigma-pi network quadratic nodes, onto neurons in hierarchically lower modules; the goal representations will be learnt as part of this processing. n The manner in which a dual-control stereo camera can be used most effectively will be explored by modeling the manner in which eye and attention control are separable.

Drives, Value Maps & Emotions n Drives and reflexes will be enumerated and modeled as separate systems to be used as a launching pad for more controlled developmental sensory representations and responses. n The rewards system will also be developed as a separate component, based on the learning of reward/error prediction. n ABACCUS will be trained by reward error to develop value maps for objects and responses as goals, to bias learning of feedback attention to filter out responses in complex environments. The effect of the reward signal will be checked to determine if there is any modification of the earlier goal representations learnt under attention control. n Value maps associated to rewards attached to sequences of stimuli (schemata) will give emotional bias in the cognitive decision-making it can perform.

Short-term working memory n Two sorts of working memory are presently known: buffer or slave working memories, of a fixed and limited persistence, and longer lasting prefrontal activities of duration determined by the importance and difficulty of the goal they represent. n Buffer memory is fed by the semantic representations, whose activity they extend, and thereby allow the development of attended state estimators in the various modalities for more general use across other cortical regions, especially goal sites. n Buffer sites will be modeled by suitable recurrent circuits coupled to the semantic maps from which they receive input. n Rehearsal of the working memory will be extended to various transformations that are also known to occur in goal sites, such as deletion, comparison and related functions.

Concept Understanding and Abstraction n n High–level concepts will be created by use of a hierarchy of networks, each being more specific in its features to which it is responsive than the next lower one. The understanding based on such ontology allows to ‘understand’ various levels of concepts, by the associated super- or subordinate concept levels activated by trained connectivity at the levels of specificity allowed by the number of hierarchical layers. Abstraction will arise through creation of specific connections of a new word representation, describing the concept, to its various more concrete word and object/action representatives. Word representations will be linked to emotion state descriptors, that are connected to the associated reward value map representations of the associated system state. For example the word ‘anger’ would be associated with the underlying state arising from blockage, by the environment, of the gaining of a goal.

Basic tasks n 2 D domain, enclosed within four walls with obstacles on the floor and 2 D objects on the walls, of varying shapes (square, triangle, circle) and colors (red, green, blue), emitting various tones. Objects are potential goals, to be learnt and responded to. n Perceptions/Concepts: To develop internal representations of the 2 D objects, based on shape/color feature maps created by learning, after relevant clustering on the feature map activations, to learn a representation for the objects. n Actions/Goals: learn to touch/manipulate the 2 D objects based on stimulus-action rewards and fused representation as a goal; to attentionally bias lower order perceptual representations just learnt (using dopamine-based reward learning).

Intermediate tasks n 2 D domain with walls, with 3 D objects as potential goals; objects with simple shapes (sphere, box, ring), colors, emitting various sounds, some in motion (to help develop tracking). n Perception/Concepts/Goals: To develop internal representations of the 3 D objects, based on a set of their 2 D views, clustered over time (by use of binocular representations of depth). Certain examples of moving stimuli will be created so as to be learnt. n Actions/rewards: Motor responses to move specific objects to certain places, such as to stack an object on top of another, or place one object inside another (object fusion observed developing in both chimps and human infants). n Language: Word maps for actions and objects concepts will be created, using associations of the word maps to the relevant object maps; attention will be important in this learning process.

Advanced task domain n Perception/Concepts/Goals: complex 3 D objects present with various features in various modalities, including animal shapes emitting sounds and smells. The internal concept representations will be extended to hierarchies of concepts, so leading to abstraction processes (where more abstract, basic level concepts are activated, such as that for ‘animal’ by specific animal inputs, so leading to spread of activity to related concepts and to associated reasoning potentialities). n Actions: with added complex tasks requiring reasoning to achieve fast/accurate responses: manipulating objects, putting objects into slots/matching shapes, Tower of Hanoi (involving virtual movements of objects – translations and stacking – at a fixed set of different sites), Incident Detection, etc. AI-based approaches to such tasks exist (cf. SOAR), but ABACCUS will be based on the flexible posterior representation transformation, goal creating and resolving powers of a neural system.

More on advanced task domain Language powers will be expanded so that ABACCUS can be talked to, using limited length word strings (initially of length two or three). These strings will be understood in terms of the associated object/action maps and responses made by the system, either as actions or action sequences or suitably meaningful speech responses. Reasoning and creation of new types of responses will be done by use of lateral spreading in memory to achieve new trajectories in the schemata space. The ‘Incident Detection & Tracking Scenario’ involves a robot moving in the internal environment, although with novel stimuli and obstacle layout. This includes ramps and stationary or slow moving obstacles. Candidate events of interest in this paradigm are: a) Moving entity detection and classification (humans, animals). b) Static environment variation detection (unexpected objects). c) Abnormal auditory events. d) Abnormal visual events.

Work packages Summary of ABACCUS workpackages.

Summary ABACCUS is the most ambitious project formulated so far, an attempt to simulate functions of most brain structures. High order functions (including conscious-like behavior? ) should result from elementary interactions in a system with proper brain-like architecture. Neuroscience and developmental science integrated in one computational model, useful for behavioral experiments and computational psychiatry. Wish us good luck. . .