Cognitive Systems in the FP 6 IST Programme

Co. Sy Cognitive Systems for Cognitive Assistants Henrik I Christensen Kungl Tekniska Högskolan J.

GOAL Advance the Science of Mind TARGET DOMAIN Human-like autonomous physical (robotic) systems PRACTICAL

CORE IDEA Self-modifying architecture (reactive, deliberative, reflexive) CRITICAL ISSUES Multiplicity of representations Recognition of

Co. Sy Cognitive Systems for Cognitive Assistants Develop the Fundamentals of Cognitive Systems Ask

JAST Joint-Action Science and Technology Harold Bekkering Institute for Cognition & Information Alois Knoll

GOAL Extend cognitive systems to multi-agent scenarios PRACTICAL VALIDATION Two autonomous robotic agents co-operating

CORE IDEA Dialogue supported perception, reasoning, & action: cross-modal associations CRITICAL ISSUES Detection and

JAST Joint-Action Science and Technology Co-operating Cognitive Systems Dynamical Models Think: Shared Understanding http:

Robot. Cub Robotic Open-Architecture for Cognition, Understanding, and Behaviours Giulio Sandini DIST, University of

Robot. Cub Robotic Open-Architecture for Cognition, Understanding, and Behaviours Rodney Brooks MIT Gordon Cheng

GOAL 54 degree of freedom cognitive humanoid robot PRACTICAL VALIDATION Hand-eye coordination Grasp and

CORE IDEA Cognition emerges through exploratory learning and co-development through embodied physical interaction CORE

Robot. Cub Robotic Open-Architecture for Cognition, Understanding, and Behaviours Cognition through Co-development Exploration, Manipulation,

COSPAL Cognitive Systems using Perception-Action Learning Gösta Granlund Linköping University Josef Kittler University of

GOAL New architecture that combines Symbolic Reasoning & Artificial Neural Networks PRACTICAL VALIDATION Emulate

CORE IDEA Action precedes perception: Associations of perception and action are triggered by the

COSPAL Cognitive Systems using Perception-Action Learning Combination of Symbolic Reasoning & Neural Networks Think:

MACS Multi-Sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances

GOAL New architecture for mobile goal-directed autonomous manipulation PRACTICAL VALIDATION Mobile manipulator robot using

CORE IDEA Use affordance to couple perception, action, and reasoning CORE IDEA Perception of

SPARK Spatio-temporal Patterns for Action-oriented Perception in Roving Robots Paolo Arena University of Catania

GOAL New biologically-inspired architecture for action-oriented perception PRACTICAL VALIDATION Robot guidance system TARGET DOMAIN

CORE IDEA Perception is a spatio-temporal pattern formation process which directly influences the associated

SPARK Spatio-temporal Patterns for Action-oriented Perception in Roving Robots Perception-Action as Emergent Spatio-Temporal Patterns

Mind RACES from Reactive to Anticipatory Cognitive Embodied Systems Rino Falcone ISTC-CNR Ana Almeida

GOAL Investigate different anticipatory cognitive mechanisms and architectures PRACTICAL VALIDATION Simulations Robotic platforms TARGET

CORE IDEA Anticipation is a key component of cognition CRITICAL ISSUES ‘Anticipatory emotions’ Performance

Mind RACES from Reactive to Anticipatory Cognitive Embodied Systems Comparative Analysis & Development of

GNOSYS An Abstraction Architecture for Cognitive Agents John G. Taylor King’s College London Christos

GOAL Complete conceptual architecture for cognitive agents PRACTICAL VALIDATION Patrolling, incident detection, and tracking

CORE IDEA Abstraction creates concept hierarchies Reasoning then makes inferences for virtual actions CRITICAL

GNOSYS An Abstraction Architecture for Cognitive Agents Conceptual Architecture Think: Abstraction, Concepts, Reasoning www.

Based on limited information Co. Sy JAST Architecture Robotic Exploration Robotic Manipulation Incremental Learning

Based on limited information Self-Modifying Architecture Co. Sy JAST Self-Organization Symbolic Representations Neural Networks

Project THINK … Co. Sy Integration JAST Shared Understanding Robot. Cub Open Cognitive Humanoid

Slides: 48

Download presentation

Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Co. Sy Cognitive Systems for Cognitive Assistants Henrik I Christensen Kungl Tekniska Högskolan J. Kevin O’Regan CNRS, University of Paris V Jeremy Watts & Aaron Sloman University of Birmingham Hans Uszkoreit & Geert-Jan Kruijff DFKI Ales Leonardis University of Ljubljana Bernt Schiele Technical University of Darmstadt Bernhard Nebel & Wolfram Burgard Alfred Ludvig University of Freiburg www. Cognitive. Systems. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GOAL Advance the Science of Mind TARGET DOMAIN Human-like autonomous physical (robotic) systems PRACTICAL VALIDATION Explorer robot Playmate robot Philosopher agent e. g. machines to help house-bound people who can’t rely on others FOCUS Integration of capabilities - perception - action - representations - interaction - reflection. . . Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

CORE IDEA Self-modifying architecture (reactive, deliberative, reflexive) CRITICAL ISSUES Multiplicity of representations Recognition of affordances Coping with uncertainty Introspection CORE IDEA Combine many capabilities: Deployed concurrently Interacting asynchronously Developing over time CRITICAL ISSUES Life-long open-ended incremental learning Balance between innate and acquired knowledge Communication: language & representations Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Co. Sy Cognitive Systems for Cognitive Assistants Develop the Fundamentals of Cognitive Systems Ask and Answer the Big Questions! Think: Integration www. Cognitive. Systems. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

JAST Joint-Action Science and Technology Harold Bekkering Institute for Cognition & Information Alois Knoll Technical University of Munich Ivan Toni Centre for Cognitive Neuroimaging Radboud University Nijmegen (KUN) Jan-Peter de Ruiter Max Planck Institute for Psycholinguistics Nijmegen Astros Chatziastros Max Planck Institute for Biological Cybernetics Tubingen Nikolaos Uzunoglu Institute of Communication & Computer Systems, Athens Ellen Gurman Bard University of Edinburgh Estela Bicho Universidade do Minho http: //212. 205. 85. 69/jast/index. htm Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GOAL Extend cognitive systems to multi-agent scenarios PRACTICAL VALIDATION Two autonomous robotic agents co-operating on a complex construction task Joint-Action Science and Technology TARGET DOMAIN Robot-robot and human-robot co-operative joint action FOCUS Exploitation of shared beliefs, intentions, and goals Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

CORE IDEA Dialogue supported perception, reasoning, & action: cross-modal associations CRITICAL ISSUES Detection and repair of performance errors (derived from both self and partner) Joint-Action CORE IDEA Self-organizing systems (non-linear dynamical systems model of cognition) Science and Technology Inspired by brain imaging data CRITICAL ISSUES Imitation Real-time adaptive planning Verbal and non-verbal communication Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

JAST Joint-Action Science and Technology Co-operating Cognitive Systems Dynamical Models Think: Shared Understanding http: //212. 205. 85. 69/jast/index. htm Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Robot. Cub Robotic Open-Architecture for Cognition, Understanding, and Behaviours Giulio Sandini DIST, University of Genova Kerstin Dautenhahn University of Hertfordshire Paolo Dario Scuola Superiore Sant’Anna, Pisa Rolf Pfeiffer University of Zurich Claes von Hofsten University of Uppsala Darwin Caldwell University of Salford Jose Santos Victor Instituto Superior Tecnico, Lisbon Auke Billard Ecole Polytechnique Federale de Lausanne Luciano Fadiga University of Ferrara Francesco Becchi Tele. Robot S. r. l. Emilio Bizzi European Brain Research Institute www. Robot. Cub. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Robot. Cub Robotic Open-Architecture for Cognition, Understanding, and Behaviours Rodney Brooks MIT Gordon Cheng ATR Yasuo Kuniyoshi University of Tokyo Juergen Konczak University of Minnesota Hideki Kozima NICT (CRL) www. Robot. Cub. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GOAL 54 degree of freedom cognitive humanoid robot PRACTICAL VALIDATION Hand-eye coordination Grasp and manipulate lightweight objects Crawl on all fours and sit up TARGET DOMAIN Open research platform: humanoid robot size & shape of a 2 year old child FOCUS Co-development of cognitive skills through exploration and manipulation Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

CORE IDEA Cognition emerges through exploratory learning and co-development through embodied physical interaction CORE IDEA Initially deals with immediate events CRITICAL ISSUES Phylogenic configuration vs Ontogenic development Increasingly acquires a predictive capability CRITICAL ISSUES Exploration Manipulation Imitation Gestural Communication Cognitive Systems in the FP 6 IST Programme t Anticipation / Planning / Deliberation / Prediction IST 2004, The Hague, 16 th November 2004

Robot. Cub Robotic Open-Architecture for Cognition, Understanding, and Behaviours Cognition through Co-development Exploration, Manipulation, Gesture, & Imitation Think: Open Cognitive Humanoid Systems www. Robot. Cub. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

COSPAL Cognitive Systems using Perception-Action Learning Gösta Granlund Linköping University Josef Kittler University of Surrey Gerald Sommer Christian-Albrechts-University of Kiel Vaclav Hlavac Czech Technical University in Prague www. cospal. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GOAL New architecture that combines Symbolic Reasoning & Artificial Neural Networks PRACTICAL VALIDATION Emulate capability of small children to solve manipulation puzzles TARGET DOMAIN Robotic exploration, Manipulation, & Problem solving FOCUS System architecture able to semi-autonomously & incrementally build up knowledge and solution strategies Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

CORE IDEA Action precedes perception: Associations of perception and action are triggered by the system’s own actions CRITICAL ISSUES Initial bootstrapping Incremental learning Attention Exploration and adaptation CORE IDEA Bi-directional interface between - Perception-Action module - Symbolic module CRITICAL ISSUES Robustness Generalizability Scalability Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

COSPAL Cognitive Systems using Perception-Action Learning Combination of Symbolic Reasoning & Neural Networks Think: Action-dependent Perception www. cospal. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

MACS Multi-Sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances Erich Rome Fraunhofer AIS Erol Sahin Middle Eastern Technical University, Ankara Lucas Paletta Joanneum Research George Dorffner ÖFAI, Vienna Patrick Doherty Linköping University www. macs-eu. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GOAL New architecture for mobile goal-directed autonomous manipulation PRACTICAL VALIDATION Mobile manipulator robot using vision, 3 -D laser scanner, 2 Do. F arm & magnetic gripper TARGET DOMAIN Mobile manipulation robotics FOCUS Affordance: ‘a resource or support that the environment offers an agent for action’, which the agent can directly perceive and employ (e. g. drivability of a track, graspability of an object) Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

CORE IDEA Use affordance to couple perception, action, and reasoning CORE IDEA Perception of affordances implies interpretation based on capabilities for physical action and current goals CRITICAL ISSUES Control based on action-oriented interpretation (affordance-based control) Acquired by learning CRITICAL ISSUES Architecture Perception Representation Learning Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

MACS Multi-Sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances Affordance-based integration of perception, action, and reasoning Think: Affordances www. macs-eu. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

SPARK Spatio-temporal Patterns for Action-oriented Perception in Roving Robots Paolo Arena University of Catania Holk Cruse University of Bielefeld Ákos Zarándy ANALOGIC Ángel Rodrígues Vázquez ANAFOCUS Manuel Velarde IP-UCM, Universidad Computense, Madrid Barbara Webb University of Edinburgh www. spark. diees. unict. it Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GOAL New biologically-inspired architecture for action-oriented perception PRACTICAL VALIDATION Robot guidance system TARGET DOMAIN Mobile robotics FOCUS Non-linear dynamical systems theory Multi-modal sensors Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

CORE IDEA Perception is a spatio-temporal pattern formation process which directly influences the associated motor behaviour CORE IDEA Dynamically emergent patterns CRITICAL ISSUES Integration of sensor stimuli Concise iconic representation Spatio-temporal array computer (SPARC) CRITICAL ISSUES Synergetics Non-linear dynamical systems theory Emergent computation Dynamic pattern formation Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

SPARK Spatio-temporal Patterns for Action-oriented Perception in Roving Robots Perception-Action as Emergent Spatio-Temporal Patterns Spatio-temporal Array Computer (SPARC) Think: Non-Linear Dynamical Systems www. spark. diees. unict. it Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Mind RACES from Reactive to Anticipatory Cognitive Embodied Systems Rino Falcone ISTC-CNR Ana Almeida e Paiva Instituto Superior Tecnico Georg Dorffner OFAI Christian Balkenius Lund University Joachim Hoffmann University of Wuerzburg Boicho Kokinov New Bulgarian University Fabio Adezio Noze s. r. l. Juergen Schmidhuber Scuola Universitaria Professionale della Svizzera Italiana www. mindraces. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GOAL Investigate different anticipatory cognitive mechanisms and architectures PRACTICAL VALIDATION Simulations Robotic platforms TARGET DOMAIN Embodied systems FOCUS Anticipation Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

CORE IDEA Anticipation is a key component of cognition CRITICAL ISSUES ‘Anticipatory emotions’ Performance metrics CORE IDEA Comparative analysis of models of anticipation, and development of models CRITICAL ISSUES Expectation-driven attention, monitoring, and control Goal-directed behaviour, pro-active planning, and prediction by analogy Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Mind RACES from Reactive to Anticipatory Cognitive Embodied Systems Comparative Analysis & Development of Anticipatory Systems Think: Anticipation www. mindraces. org Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GNOSYS An Abstraction Architecture for Cognitive Agents John G. Taylor King’s College London Christos Emmanouilidis Zenon S. A. Hanspeter Mallot University of Tuebingen Pietro Morasso DIST, University of Genova Panos Trahanias Foundation for Research and Technology - Hellas www. ics. forth. gr/gnosys Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GOAL Complete conceptual architecture for cognitive agents PRACTICAL VALIDATION Patrolling, incident detection, and tracking in unknown outdoor environments TARGET DOMAIN Robot navigation & surveillance FOCUS Complete integration of the perception-knowledge acquisitionabstraction-reasoning- action generation cycle Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

CORE IDEA Abstraction creates concept hierarchies Reasoning then makes inferences for virtual actions CRITICAL ISSUES Inspired by human cognitive processes CORE IDEA Action selection based on greatest reward Goals-oriented model to fuse user-derived and agent-derived tasks/goals CRITICAL ISSUES Attention (complexity, priority, novelty, goals) Reinforcement learning Actions modify current knowledge through perception Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

GNOSYS An Abstraction Architecture for Cognitive Agents Conceptual Architecture Think: Abstraction, Concepts, Reasoning www. ics. forth. gr/gnosys Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Based on limited information Co. Sy JAST Architecture Robotic Exploration Robotic Manipulation Incremental Learning Attention Affordances Language/ Gesture Multi-Agent Robot. Cub COSPAL MACS SPARK Mind RACES GNOSYS Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Based on limited information Self-Modifying Architecture Co. Sy JAST Self-Organization Symbolic Representations Neural Networks Dynamical Systems Cognitive Systems in the FP 6 IST Programme Robot. Cub COSPAL MACS SPARK Mind RACES GNOSYS IST 2004, The Hague, 16 th November 2004

Project THINK … Co. Sy Integration JAST Shared Understanding Robot. Cub Open Cognitive Humanoid System COSPAL Action-dependent Perception MACS Affordances SPARK Emergent Spatio-temporal Patterns Mind RACES Anticipation GNOSYS Abstraction, Concepts, Reasoning Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004

Cognitive Systems in the FP 6 IST Programme IST 2004, The Hague, 16 th November 2004