ORION Projectteam Monique THONNAT INRIA Sophia Antipolis Creation

ORION Project-team Monique THONNAT INRIA Sophia Antipolis Creation: July 1995 Multidisciplinary team: artificial intelligence, software engineering, computer vision Evaluation May 2006

Contents n Team Presentation n Research Directions n Cognitive Vision 2002 -2006 n Reusable Systems 2002 -2006 n Objectives for the next Period Evaluation May 2006 Orion 2

Team presentation (May 2006) 4 Research Scientists: François Bremond (CR 1 Inria) Sabine Moisan (CR 1 Inria, HDR) Annie Ressouche (CR 1 Inria) (team leader) Monique Thonnat (DR 1 Inria) 1 External Collaborator: Jean-Paul Rigault (Prof. UNSA Inria secondment) 4 Temporary Engineers: Etienne Corvee, Ruihua Ma, Valery Valentin, Thinh Van Vu 7 Ph. D Students: Bui Binh, Bernard Boulay, Naoufel Kayati, Le Thi Lan, Mohamed Becha Kaaniche, Vincent Martin, Marcos Zuniga Evaluation May 2006 Orion 3

Research directions Objective: Intelligent Reusable Systems for Cognitive Vision: n Interpretation of static images n Video understanding Reusable Systems: n Program Supervision n LAMA Software platform Evaluation May 2006 Orion 4

Orion team positioning Cognitive Vision: n n Image interpretation (ECVision European network on cognitive vision, EUCognition) vs. computer vision (INRIA Cog. B) Video understanding (USC Los Angeles, Georgia Tech. Atlanta, Univ. Central Florida, NUCK Taiwan, Univ. Kingston UK, INRIA Prima) Reusable Systems: n Program supervision: e. g. , scheduling (ASPEN and CASPER at JPL), image processing (Hermès at Univ. Caen, Ex. TI at IRIT)… n Platform approach: e. g. , ontology management (Protegé at Stanford), frameworks for multi agents (Aglets, Jade, Oasis at LIP 6), distributed object community (Oasis at INRIA Sophia)… Evaluation May 2006 Orion 5

Cognitive Vision : Image Interpretation 2002 -2006 Objective: semantic interpretation of static 2 D images n n n Recognition of object categories (versus individuals) Recognition of scenes involving several objects with spatial reasoning Intelligent management of image processing programs Towards a cognitive vision platform Evaluation May 2006 Orion 6

Cognitive Vision : Image Interpretation 2002 -2006 Scientific achievements: n Knowledge acquisition: n A visual concept ontology with 144 spatial, color and texture concepts [MVA 04] n n Learning: n Visual concept detectors [IVC 06] n Image segmentation parameters [ICVSa 06] Cognitive vision platform n Architecture [ICVS 03] n Object class recognition algorithm [CIVR 05] Evaluation May 2006 Orion 7

Cognitive Vision: Image Interpretation 2002 -2006 Self Assessment: n Strong points: n Visual concept ontology as user-friendly intermediate layer between image processing and application domain n n Automatic building of the visual concept detectors Still open issues: n Learning for image segmentation n Temporal visual concept ontology Evaluation May 2006 Orion 8

Cognitive Vision: Video Understanding 2002 -2006 Objective: n Real time recognition of interesting behaviors How? n Data captured by video surveillance cameras n Original video understanding approach mixing: n computer vision: 4 D analysis (3 D + temporal analysis) n artificial intelligence: a priori knowledge (scenario, environment) n software engineering: reusable VSIP platform Evaluation May 2006 Orion 9

Cognitive Vision: Video Understanding 2002 -2006 Objective: Interpretation of videos from pixels to alarms Segmentation Classification Tracking Scenario Recognition Alarms access to forbidden area 3 D scene model Scenario models Evaluation May 2006 Orion A priori Knowledge 10

Cognitive Vision: Video Understanding 2002 -2006 Scientific achievements: n n n Multi-sensor video understanding: n 2 to 4 video cameras overlapping or not [IDSS 03, JASP 05] n Video cameras + optical cells + contact sensors [AVSS 05]… Learning: n parameter tuning[MVAa 06] n frequent temporal scenarios models [ICVSb 06] Temporal scenario: n a new real time recognition algorithm [IJCAI 03, ICVS 03] n a new representation language [MVAb 06, ECAI 02, KES 02] Evaluation May 2006 Orion 11

Cognitive Vision: Video Understanding 2002 -2006 Industrial impact: n Strong impact in visual surveillance (metro station, bank agency, building access control, onboard train, airport) n 4 European projects (ADVISOR, AVITRACK, SERKET, CARETAKER) n 5 industrial contracts with RATP, ALSTOM, SNCF, Credit Agricole, STMicroelectronics n n 2 transfer activities with BULL (Paris), VIGITEC (Brussels) Creation of a start-up Keeneo July 2005 (8 persons) for industrialization and exploitation of VSIP library. Evaluation May 2006 Orion 12

Cognitive Vision: Video Understanding 2002 -2006 Intelligent video surveillance of Bank agencies Evaluation May 2006 Orion 13

Cognitive Vision: Video Understanding 2002 -2006 n “Unloading Global Operation” Toulouse - 3 rd June 2004 14

Cognitive Vision: Video Understanding 2002 -2006 Airport Apron Monitoring “Unloading Operation” European AVITRACK project Toulouse - 3 rd June 2004 15

Cognitive Vision: Video Understanding 2002 -2006 Self Assessment: n Strong points: n Video understanding approach: real time, effective techniques used by external academic and industrial teams n Launch of an evaluation competition for video surveillance algorithms (ETISEO) with currently 25 international teams n Still open issues: n Learning n Multi sensor Evaluation May 2006 Orion 16

Reusable Systems: Program Supervision Reusable Systems: original approach for the reuse of programs with program supervision techniques Program supervision: Automate the (re)configuration and execution of programs n selection, scheduling, execution, and control of results Knowledge-based approach: knowledge modeling, planning techniques, …. . Evaluation May 2006 Orion 17

Reusable Systems: LAMA Platform Reusable Systems: Reuse of tools to design knowledgebased systems (KBS) LAMA Software Platform: provide generic components and tools raise new issues, to be abstracted into new components Problem Solving KBS Virtuous Circle Evaluation May 2006 Set of toolkits to facilitate design and evolution of KBS elements: n engines, GUI, knowledge languages, learning and verification facilities… Software Engineering approach: genericity, frameworks, objects and components Orion 18

Reusable Systems: LAMA Platform LAMA Designer Expert Java graphic library for GUIs Compilers/verifiers generators for knowledge description languages Blocks Verification library for knowledge bases Program Supervision Framework for engine design & Object knowledge Recognition representation support and Model task specific layers Calibration Evaluation May 2006 Orion Task dedicated GUI Task dedicated Language with compiler & KB verification Task dedicated Engine KBS Knowledge Base User 19

Reusable Systems: Program Supervision 2002 -2006 Scientific achievements: n Improvement of the Pegase engine (Pegase+) n n Distributed program supervision n n Multithreading, extensions to the YAKL language [ECAI 02] Supervision Web server, multi-agent techniques, interoperability Pegase/Java/agents [TC 06] Cooperation with image and video understanding n n Object recognition task using program supervision [ICTAI 03] Interoperability with VSIP: program supervision for video understanding [ICVSc 06] Evaluation May 2006 Orion 20

Reusable Systems: LAMA Platform 2002 -2006 Scientific achievements: n Enforcing LAMA safe usage n n Verification of LAMA component extensions relying on Model Checking approach [Informatica 01, SEFM 04] Encompassing new tasks n n Classification and object recognition in images: new engine and new knowledge representation language [ICTAI 03] Model calibration in hydraulics: new engine/language (Ph. D codirected with INPT and CEMAGREF) [KES 03, JH 05] Evaluation May 2006 Orion 21

Reusable Systems: Self Assessment Strong points: n Real time performance (Pegase+ and video) n n LAMA genericity at work n n n Different tasks (supervision, classification, calibration) in various application domains (hydraulics, biology, astronomy, video surveillance…) Shorter development time and safer code Reuse of concepts as well as code n n Using program supervision costs less than 5% of overall processing time Several variants of a task sharing common concepts Extensibility and commitment to Standards Evaluation May 2006 Orion 22

Objectives for the next period 1/5 Creation of a new INRIA project-team PULSAR Perception Understanding and Learning Systems for Activity Recognition Theme: Cog. C Multimedia data: interpretation and man-machine interaction Multidisciplinary team: artificial intelligence, software engineering, computer vision Objective: n Research on Cognitive Systems for Activity Recognition n Focus on spatiotemporal activities of physical objects n From sensor output to high level interpretation Evaluation May 2006 Orion 23

Objectives for the next period 2/5 PULSAR Scientific objectives: Two research axes: n Scene Understanding for Activity Recognition n Generic Components for Activity Recognition PULSAR Applications: n Safety/security (e. g. intelligent surveillance) n Healthcare (e. g. assistance to the elderly) Evaluation May 2006 Orion 24

Objectives for the next period 3/5 PULSAR: Scene Understanding for Activity Recognition n Perception: multi-sensors, finer descriptors n Understanding: uncertainty, 4 D coherency, ontology for AR n Learning: parameter setting, event detector, activity models, program supervision KB (risky objective) Evaluation May 2006 Orion 25

Objectives for the next period 4/5 PULSAR Generic Components for Activity Recognition From LAMA Platform to AR platform: n Model extensions: n n n User-friendliness and safeness of use: n n n modeling time and scenarios handling uncertainty theory and tools for component frameworks scalability of verification methods Architecture improvement: n n n parallelization, distribution, concurrence real time response domain specific software and graphical interface plugging Evaluation May 2006 Orion 26

Objectives for the next period 5/5 Short term objectives: Scene Understanding for Activity Recognition n Perception: gesture analysis n Understanding: n n ontology-based activity recognition n uncertainty management Learning: primitive event detectors learning Generic Components for Activity Recognition n Model of time and scenarios n Internal concurrency and distributed architecture Evaluation May 2006 Orion 27