Pattern Recognition and Image Analysis Group RFAI Document

Pattern Recognition and Image Analysis Group (RFAI) Document (Image) Analysis related work Laboratory of Computer Science (LI) François Rabelais University Tours city, France

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 6. 7. Graph matching and embedding 2

Tours city Paris Tours - 137 046 people (2009) - 204 km southwest of Paris - Region « centre - Indre et loire » - 1 h 20 from Paris by high speed train - Direct train connection to Charles de Gaulle -Orly airport in 2 h 00 3

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 6. 7. Graph matching and embedding 4

François-Rabelais University, les deux lions / Portalis François Rabelais University Faculties Art & human sciences, economy, business & management, health, information and technology Students 21 207 (2 500 foreign students) Teachers 1 300 Support staff 1000 Laboratories 40 Place 5 François Rabelais i. e. a famous French writer of XV° Century 5

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 6. 7. Graph matching and embedding 6

School of Engineering Polytech - 12 schools in France (Grenoble, Lille, Marseille, Montpellier, Nantes, Nice. Sophia, Paris-UPMC, Paris ORSAY, Savoie, Orléans, Tours, Clermont-Ferrand) - 12 000 students -720 students - 5 departments (with Labs) Urban Planning CITERES Mechanics LMR Electronics LMP Computer Science Embedded computing LI 7

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 8

Laboratory of Computer Science 77 people, 5 research groups (2009) Pattern Recognition and Image Analysis Data Bases and Natural Language Processing Visual Data Mining and Biomimetic Algorithms Scheduling and Control Handicap and New Technologies 9

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 10

Pattern Recognition and Image Analysis (RFAI) (1) Medical Imaging - Image segmentation (ultrasound, MRI) - Video analysis, 3 D reconstruction Document Image Analysis - Layout analysis & document recognition - OCR, word spotting & signature verification - Symbol recognition & spotting - Content based Image Retrieval - Camera based Recognition - Graph matching and embedding Machine learning for time series prediction 11

Professors Pattern Recognition and Image Analysis (RFAI) (2) Ph. D Romuald Boné Alireza Alaei Pascal Makris Hubert Cardot Sabine Barrat Julien Olivier Jean-Yves Ramel Thierry Brouard Nicolas Ragot Mathieu Delalandre Partha Roy Muzzamil Luqman Nicolas Sidere Romain Raveaux Gilles Verley 12

Ph. D Students & engineers Pattern Recognition and Image Analysis (RFAI) (3) Fareed Ahmed Anh Khoi Ngo ho Ahmed Ben Salah The Anh Pham Aymen Cherif Cyrille Faucheux Frédéric Rayar 13

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 14

Partnership contracts Scholarships Local government projects National projects International projects 2003 -2006 Navidomass 2006 -2009 EPEIRES 2004 -2007 BVH 2004 -today ATOS 2005 -today PIVOAN 2008 -2009 HEC 2005 -2011 SNECMA 2008 -2011 AAP 2010 -2011 VIED 2010 -2013 Bnf 2010 -2013 Digidoc 2011 -2014 Google 2011 -2012 ISRC 2011 -2012 DOD 2011 -2015 Indo. French 2012 -2015 SPD 2012 -2015 Madonne Projects & partners outline (1) DIA people plot 20 16 12 8 4 0 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 15

Projects & partners outline (2) National projects People Institutes Length Funding ACI MADONNE 2003 -2006 55 8 2 years 110 k€ ANR Navidomass 2006 -2009 40 7 3 years 443 k€ Technovision ÉPEIRES 2004 -2007 30 7 3 years 100 k€ ANR Digidoc 2011 -2014 18 7 3 years 866 k€ Institut de Recherche en Informatique et Systèmes Aléatoires (Rennes) Laboratoire d'Informatique de Traitement de l'Information (Rouen) Laboratoire d’informatique image et interaction (La Rochelle) Centre de Recherche en Informatique de Paris 5 (Paris) Laboratoire Lorrain de Recherche en Informatique et ses Applications (Nancy) Laboratoire Informatique (Tours) Centre d’Etude Supérieures de la Renaissance (Tours) Laboratoire d'Info. Rmatique en Image et Systèmes d'information (Lyon) Laboratoire Bordelais de Recherche en Informatique (Bordeaux) 16

Projects & partners outline (3) Local government projects (i. e. projets region centre) Length Funding Institutes People PIVOAN 3 1 1 year 33 k€ AAP 3 1 1 year 38 k€ M. Luqman T. H. Pham So famous ! Centre des études supérieures de la renaissance – bibliothèque virtuelle humaniste Maison des Sciences de l'Homme Ph. D Scholarships R. J. Qureshi Partnership contracts Higher Education Commission (HEC) of Pakistan Vietnam International Education Development (VIED) international high-technology group in aerospace, defense and security Bibliothèque Nationale de France - portail Gallica Bilateral program People Institutes Length Funding Indo. French 3 2 3 year 70 k€ Digitalisation company capturing, automatically processing, and managing all company’s incoming documents Atos Origin is a leading international IT services provider for business solutions 17

Projects & partners outline (4) Computer Vision Center Document Analysis Group Barcelona - Spain “J. Llados, E. Valveny” Dept. of Computer Science and IS Osaka Prefecture University Osaka - Japan “K. Kise” Indian Statistical Institute Kolkata - India “U. Pal” Computational Intelligence Laboratory Athens - Greece “B. Gatos” 18

Camera based Recognition CBIR Graph matching & embedding Symbol recognition & spotting OCR, word spotting & signature verification Layout analysis & document recognition Projects & partners outline (5) Madonne 2003 -2006 Navidomass 2006 -2009 EPEIRES 2004 -2007 BVH 2005 -today ATOS 2005 -2009 PIVOAN 2008 -2009 HEC 2005 -2011 SNECMA 2008 -2011 AAP 2010 -2011 VIED 2010 -2013 Bnf 2010 -2013 Digidoc 2011 -2014 Google 2011 -2012 ISRC 2011 -2012 DOD 2011 -2015 Indo. French 2012 -2015 SPD 2012 -2015 19

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 20

Layout analysis & document recognition “AGORA (1)” People Jean-Yves Ramel Funding CESR partnership, Madonne, PIVOAN Starting 2005 Ref J. Y. Ramel and al. User-driven Page Layout Analysis of historical printed Books. IJDAR, 2007. (1) Text/graphics separation Foreground map: adaptive binarization [Saul 2000] with connected component labeling, text/graphics separation is done in terms of size of connected components (2) Line/word segmentation 1. Background map: statistical distribution of white and black pixel on horizontal and vertical scanline 2. Fusion: word segmentation (i. e. connected components grouping) is done in terms of thresholding on the background map.

Layout analysis & document recognition “AGORA (2)” People Jean-Yves Ramel (3) Interactive system (i. e. user driven analysis) Funding CESR partnership, Madonne, PIVOAN Starting 2005 Ref J. Y. Ramel and al. User-driven Page Layout Analysis of historical printed Books. IJDAR, 2007. Vertical position average = 0, 46 std deviation = 0, 41 Horizontal position average = 0, 51 std deviation = 0, 07 (4) Results, since 2005: 300 books (50 000 pages) http: //www. bvh. univ-tours. fr/ 22

Layout analysis & document recognition “Document image characterization (1)” People Nicholas Journet Funding CESR partnership & Madonne project Starting 2006 Ref N. Journet and al. Document Image Characterization Using a Multiresolution Analysis of the Texture: Application to Old Documents. IJDAR, 2008. (1) Descriptor based on five features: (1) main direction Directional (2) isotropy rose (3) standard deviation Spatial (4) ink/paper transition (5) white spaces separating the collateral elements 23

Layout analysis & document recognition “Document image characterization (2)” People Nicholas Journet Funding CESR partnership & Madonne project Starting 2006 Ref N. Journet and al. Document Image Characterization Using a Multiresolution Analysis of the Texture: Application to Old Documents. IJDAR, 2008. (2) Segmentation: -features are extracted at four different resolution (4 5 = 20 features) - features are then processed with the clustering algorithm CLARA (Clustering LARge Applications) [Kaufman 1990] to achieve automatic segmentation in text/graphics/background 24

Layout analysis & document recognition “Document image characterization (3)” People Nicholas Journet Funding CESR partnership & Madonne project Starting 2006 Ref (3) Indexing applied on two different problems -Layout retrieval, distance is based on a contingency table [Younes 2004] -Graphics retrieval, distance based on a dissimilarity function N. Journet and al. Document Image Characterization Using a Multiresolution Analysis of the Texture: Application to Old Documents. IJDAR, 2008. Handmade dataset I (400 images) Handmade dataset II (400 images) 25

Layout analysis & document recognition “Cognitive digitalization” People A. K. Ngo ho, N. Ragot, J. Y. Ramel Funding Digidoc project Starting 10/2011 Ref Na Topic: Incremental and interactive learning for document image, application for intelligent cognitive scanning of old documents. Problematic: - Estimate the scan parameters according to usage, past experience. - Improve the scan parameters for a document during the scanning. - Detect the default settings for a document, a collection, a work. 26

Layout analysis & document recognition “Document classification” Form People Mathieu Delalandre Funding DOD project Starting 12/2011 Ref Na Publicity Topic: Recognition of administrative forms for companies Problematic: - high variability “ 600 to 800 classes” - binary images at 300 dpi - time constraint: to 1, 5 s per image - commercial systems can’t outperform a 60% recognition rate Goals: 1. To gain in robustness (set of adapted and robust specialists) 2. To gain in flexibility (self learning, content adaptation) Free letter Acknowledge reply 27

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 28

OCR, word spotting and signature verification “Robust OCR-I (1)” People Key idea : improving OCR robustness by using similar technics as those used for handwriting recognition: - Hidden Markov Models without explicit segmentation - Adapting a polyfont OCR to specificities of pages (fonts/noise) Kamel Ait-Mohand Funding Navidomass project Starting 2006 Ref K. Ait-Mohand al. Structure Adaptation of HMM Applied to OCR. ICPR, 2010. (1) Feature extraction is based on a sliding window and Ho. G features (no word/character segmentation) Sliding window initial model Structure adaptation training New model (2) HMM classification and training - HMM characters models are learnt on a synthetic dataset (numerous fonts, degradations possible, no limits in the number of samples per character) = > polyfont OCR system - Each character model can be adapted to a specific font/book using only few lines of transcriptions. The HMM model is adapted at the structure level (number of states) and at the parameter level (Gaussian MAP adaptation). 29

OCR, word spotting and signature verification “Robust OCR-I (2)” People Kamel Ait-Mohand Experiments done using 100 fonts with the degradation model of Baird Funding Navidomass project Starting 2006 Ref K. Ait-Mohand al. Structure Adaptation of HMM Applied to OCR. ICPR, 2010. Training Testing Degradation models Baird thresholding 12 sparse pixels Font size Resolution 300 dpi Fonts Image sets (lines) 70 30 10000 15000 Commerc Polyfont ial OCR Average (30 fonts) blurred Adapted 88. 72% 91. 70% 97. 37% 69. 21% 98. 33% 98. 68% 44. 78% 85. 46% 98. 09% 52. 73% 39. 73% 74. 89% 30

OCR, word spotting and signature verification “Robust OCR-II” People J. Y. Ramel, N. Ragot, S. Barrat Funding SDP Starting 10/20012 Ref Na Topic: digitalization and indexing of a military document database for retired pay Problematic: - large amount of data (800 000 applications every 3 years) - large heterogeneity: from XIX° century “middle” to today, handwritten and typographic documents, different languages, no common layout, different colors, etc. 31

OCR, word spotting and signature verification “Performances prediction/control of OCR” People Ahmed Ben Salah, K. Ait -Mohand Funding BNF Starting 10/2010 Ref Na Problematic: control and cost reduction of the digitization service to know which collection/document/part of document is OCRisable and at which quality üSelect only adequate documents to be sent to the private service provider in charge of the digitization and OCRisation • Studies of relationships between meta-data information (date, format, …) and OCR results => difficult without deep analysis of the pages • Characterization of image content with SIFT+LBP; regression towards OCR results Real OCR Result 0 -70% 70 -80% 80 -85% 85 -90% 90 -100% Good (≥ 88%) 0% 0% 6, 1% 23, 11% 77, 12% Undecidable (82%-88%) 5, 5% 16% 53, 43% 59, 11% 20, 34% Bad (<82%) 94, 55% 84% 40, 47% 17, 78% 2, 54% Perfomance expected üControl of OCR quality assessed by the service provider • Detection of text zones forgotten by OCR using correct detection performed (contextual information) => in progress • Verification of OCR result by matching with image (=> in a near futur) 32

OCR, word spotting and signature verification “Semi-automatic transcription (1)” People Jean-Yves Ramel, Frédéric Rayar Topic: user driven transcription of character in historical books Funding CESR partnership, PIVOAN, Google Starting 2008 Ref S. Hocquet and al. Analyse de classes de formes pour la transcription de textes imprimés anciens. CIFED, 2010 (1) Segmentation process based on Agora Standardized output (e. g. Alto) (3) Transcription & Typography studies (2) Clustering process Finer description of shapes Features extraction and selection 33

OCR, word spotting and signature verification “Semi-automatic transcription (2)” People Jean-Yves Ramel, Frédéric Rayar Funding CESR partnership, PIVOAN, Google Experiments, The Vésales book Pages Starting 2008 Ref connected components Custers (i. e. Classes) 40 000 S. Hocquet and al. Analyse de classes de formes pour la transcription de textes imprimés anciens. CIFED, 2010 150 1 062 081 10% 90% > 10 occurrences < 10 occurrences 93% of the text 7% of the text 0. 5% (top 200) 89. 5% 85 % of text 8% of text Reasons is noise spot character on verso touching characters split character 34

OCR, word spotting and signature verification “Word Spotting (1)” People Topic: Word Retrieval in Historical Documents P. P. Roy, J. Y. Ramel, F. Rayar Funding AAP , Renon Starting 10/2010 Ref P. P. Roy and al. An efficient coarse-to-fine indexing technique for fast text retrieval in historical documents", DAS 2012. Text Extraction AGORA Transcription by codebook Manuscripts Codebook of Primitives Query word Sequence of primitives Primitive. String Matching Word Detection 35

OCR, word spotting and signature verification “Word Spotting (2)” People P. P. Roy, J. Y. Ramel, F. Rayar Funding AAP , Renon Starting 10/2010 Ref P. P. Roy, F. Rayar and J. Y. Ramel. An efficient coarse-to-fine indexing technique for fast text retrieval in historical documents", DAS 2012. Topic: Word Retrieval in Historical Documents The codebook is created using a clustering algorithm by template matching of similarity Overcoming of segmentation problems are solved by the Water reservoir method. Query word is thus converted into a string of primitives. Approximate string matching algorithm is used for string matching Tests done from - 24 pages -corresponding to 57324 primitives -clustered in 183 representative primitives -P/R computed with 20 query word images 36

OCR, word spotting and signature verification “Multlingual Word Spotting” People N. Ragot, J. Y. Ramel, U. Pal Funding IFCPAR Starting 04/2012 Ref Na Topic: Robust multilingual word spotting: Problematic: - Query by text/image - Partial matching allowed (for occlusion, special characters) - Matching in two steps : global (shape context) / local (HMM) 37

OCR, word spotting and signature verification “Online signature verification (1)” People Nicolas Ragot Funding ATOS project Starting 2005 Ref N. Ragot and al. Study of Temporal Variability in On-Line Signature Verification. ICPR, 2008 Problematic: to evaluate impact of temporality (i. e. time evolution) on signature, for performance evaluation of signature verification algorithms. (1) Database acquisition signers 18 sessions 12 signatures/ session total signatures 10 mean time interval 2160 total duration 2 weeks 25 months Training loop if rough differences (based on length, duration, speed) Enrollment “ 5 signatures” Final acquisition “ 5 signatures” 38

OCR, word spotting and signature verification “Online signature verification (2)” People Nicolas Ragot Funding ATOS project Starting 2005 Ref N. Ragot and al. Study of Temporal Variability in On-Line Signature Verification. ICPR, 2008 variation (2) Statistical analysis (2. 1) Global i. e. without temporal variability Speed variation Length variation Duration correlation yes no stable (2. 1) With temporal variability Total duration per signer/ session Total length per signer/ session (3) Performance evaluation Authentication (i. e. recognition) algorithm based on a Coarse to fine approach - Coarse step on “basic” features (length, duration) - Fine step based on DTW Proposed dataset Dataset without temporality 39

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 40

Symbol recognition & spotting “Vectorization and Gb. R (1)” People Funding Starting Ref Jean-Yves Ramel (1) Contour detection, chaining and polygonalisation [Wall 1984] Na J. Y. Ramel. A Structural Representation for Understanding Line-Drawing Images. IJDAR, 2000. (2) Quadrilateral building (2. 1. ) Matching (2. 2. ) Sorting (2. 3. ) Merging 41

Symbol recognition & spotting “Vectorization and Gb. R (2)” People Funding Starting Ref Jean-Yves Ramel (3) Graph based representation Na J. Y. Ramel. A Structural Representation for Understanding Line-Drawing Images. IJDAR, 2000. (3) Pros and cons Cons - lost of connectivity Cons - parasite quadrilaterals Pro - better representation of filled & crossed areas

Symbol recognition & spotting “Generation of synthetic documents (1)” People Funding Starting Ref Mathieu Delalandre Key idea To use a same background layer with different symbol layers Na M. Delalandre and. Generation of Synthetic Documents for Performance Evaluation of Symbol Recognition & Spotting Systems. IJDAR, 2010. Graphical documents are composed of two layers (1) Constraint model M 1 C 1 M 2 C 2 M 3 M 4 C 3 C 4 L c 1 c 2 symbol model loaded symbol bounding box and control point alignment p L 1 p 1 θ 1 p 2 L 2 43 θ 2

Symbol recognition & spotting “Generation of synthetic documents (2)” People Funding Starting Ref Mathieu Delalandre Na M. Delalandre and. Generation of Synthetic Documents for Performance Evaluation of Symbol Recognition & Spotting Systems. IJDAR, 2010. (2) Building engine and user interaction (2) run (1) edit (3) display Symbol Models Building Engine Background Image 44

Symbol recognition & spotting “Generation of synthetic documents (3)” People Funding Starting Ref Mathieu Delalandre (3) Datasets Na M. Delalandre and. Generation of Synthetic Documents for Performance Evaluation of Symbol Recognition & Spotting Systems. IJDAR, 2010. (4) Performance evaluation - Goal is to evaluate variability impact of produced datasets on spotting system(s) - Experiments have been done from the spotting system of R. Qureshi Background sets Mean localization results

Symbol recognition & spotting “Graph scoring for symbol spotting (1)” People Rashid Qureshi Funding HEC scholarship (1) Graph based representation: based on the Jean-Yves Ramel’s work Starting 2005 Ref R. Qureshi and al. Spotting Symbols in Line Drawing Images Using Graph Representations. GREC, 2008. (2) Seeds detection in graph: a set of scoring functions is computed from all nodes and edges Scoring functions Edges PE 1 parallel segments PE 2 Ù junctions PE 3 comparable length segments Nodes PN 2 PN 3 2 -3 connection short length segments (3) Score propagation: based on a shortest path algorithm, a global score is normalized from individual score of edge/node 46

Symbol recognition & spotting “Graph scoring for symbol spotting (2)” People Rashid Qureshi Funding HEC scholarship (4) Results & performance evaluation 1 Starting 2005 R. Qureshi and al. Spotting Symbols in Line Drawing Images Using Graph Representations. GREC, 2008. 0 SESYD dataset Precision Ref Recall 47

Symbol recognition & spotting “Bayesian based system for symbol spotting (1)” People Muzzamil Luqman Funding HEC scholarship Starting 2008 Ref M. M. Luqman. A Content Spotting System for Line Drawing Graphic Document Images. ICPR, 2010. (1) Representation phase: used the graph based representation of Jean. Yves Ramel (2) Description phase: approach based on attributes (of nodes and edges) (3) Learning and classification phases base on Bayesian network (3. 1. ) Discretization step: based on the Akaike Information Criterion (3. 2. ) Learning step: - network topology is done from a genetic algorithm - parameters conditional probabilities is done from a maximum likelihood estimation (3. 3. ) Classification step: 48

Symbol recognition & spotting “Bayesian based system for symbol spotting (2)” Funding HEC scholarship (4) Performance evaluation at recognition level ISRC 2003 dataset Starting 2008 M. M. Luqman. A Content Spotting System for Line Drawing Graphic Document Images. ICPR, 2010. Clean level 1 level 2 level 3 Binary degrade Hand drawn Scalability clean yes Yes 100% no no 99% 96% 93% 92% no no 98% 94% 92% 91% no no 91% 77% 71% 69% no no 98% 95% 93% 92% Hand drawn Binary degrade (5) Improvements of Rashid Qureshi’s results SESYD dataset Precision Ref Scaling Muzzamil Luqman Rotation People 49 Recall

Symbol recognition & spotting “Graph Embedding” People N. Sidere, J. Y. Ramel Topic: Topological Graph Embedding Funding Navidomass Starting 2007 Ref (1) A lexicon is generated from the network of non-isomorphic graphs N. Sidère et al. Vector Representation of Graphs : Application to the Classification of Symbols and Letters. ICDAR 2009. (2) The embedding is based on occurrences of the patterns 50

Symbol recognition & spotting “International Symbol Recognition Contest 2011” (1) Workshop GREC 2011 Funding Support of TC 10 Contest starting March 2011 Starting 2011 Training datasets 6 th of April 2011 Ref E. Valveny and al. Report on the Symbol Recognition and Spotting Contest. GRECLNCS 2012. Call of participation 2 sd of May 2011 Final datasets 25 th of July People M. Delalandre, R. Raveaux Recognition Tests Localization Tests Contest slot 25 th July - 1 st August 2011 http: //iapr-tc 10. univ-lr. fr/index. php/symbol-contest-2011 id domain models symbols distortion Training #1 -#7 Technical 36 -150 16650 Rotation, Scaling, Kanungo, Context Final #1 -#4 Technical 36 -150 16800 Rotation, Scaling, Kanungo, Context id domain models Training #8 -#15 Electrical Architectural 16 -21 40 835 None, Kanungo Final #5 -#12 Electrical Architectural 16 -21 120 3463 None, Kanungo images symbols distortion

Symbol recognition & spotting “International Symbol Recognition Contest 2011” (2) People The participant: Nayef, N. & Breuel, T. On the Use of Geometric Matching for Both: Isolated Symbol Recognition and Symbol Spotting Workshop on Graphics Recognition (GREC), 2011 M. Delalandre, R. Raveaux Funding Support of TC 10 Starting 2011 Ref E. Valveny and al. Report on the Symbol Recognition and Spotting Contest. GRECLNCS 2012. set name models Recognition Tests Localization Tests Connected components filtering noise recognition rate Final #1 50 Kanungo 94. 76% Final #3 150 Kanungo 85. 88% Final #4 36 Context 96. 22% Contour detection and sampling Geometric matching groundtruth set name domain models noise precision / recall Final #5 Architectural 16 None 0. 62 / 0. 99 Final #6 Architectural 16 Kanungo 0. 64 / 0. 98 Final #9 Electrical 21 None 0. 37 / 0. 56 Final #10 Electrical 21 Kanungo 0. 44 / 0. 63 results intersection 52

Symbol recognition & spotting “Performance characterization of symbol localization (1)” People Mathieu Delalandre Open problem Funding Na Starting 2008 M. Delalandre and al. A Performance Characterization Algorithm for Symbol Localization. GREC, 2010. The characterization method must do some rejection, ways to solve are. . . 1. To define and apply manual threshold (bad. . . ) 2. To propose a method for adaptative tthresholding, how to do ? Key idea, characterization method based on context Groundtruth, gravity centers, contours p 2 p 1 Result points detection rate Ref how to make the difference between segmentation errors of background with segmentation errors of objects Highest probabilities p 3 Lowest probabilities probability error 53

Symbol recognition & spotting “Performance characterization of symbol localization (2)” Mathieu Delalandre (1) The method Funding Na Starting 2008 M. Delalandre and al. A Performance Characterization Algorithm for Symbol Localization. GREC, 2010. 3. 1 Localization comparison (1. 2) Probability scores are computed from a groundtruth point gi and the result point r, considering the neighboring groundtruth point gj. Final result is computed considering all the groundtruth using a probability score function 3. 2 Probability scores 3. 4 Transform function 1 g si ε 0 score error (ε) 1 1 single (Ts) detection rates 3. 3 Matching algorithm (1. 4) Transform function make results context independent, making difference with self-matching of groundtruth, to achieve coherent comparison of methods on different datasets 0 Results (1. 1) Localization comparison moves results from Euclidean to a scale space, to deal with the scale invariance (1. 3) Matching algorithm looks for statistical distribution of single, miss, merge and split cases, in an decreasing order of precision using a bipartite list. single detections Ts Ref Groundtruth alarm (Tf) multiple (Tm) ε detection rate People 0 0 probability error score error 54 1

Symbol recognition & spotting “Performance characterization of symbol localization (3)” 0. 60 Mathieu Delalandre Funding Na floorplans i(1) = 0. 496 0. 40 Starting 2008 i(ε) M. Delalandre and al. A Performance Characterization Algorithm for Symbol Localization. GREC, 2010. 0. 20 0. 00 0. 10 0. 20 0. 30 0. 40 0. 50 0. 60 0. 70 0. 80 0. 90 SESYD dataset Drawing level Symbol level 1, 00 -0. 20 floorplans score error (ε) diagrams Ref electrical diagrams i(1) = 0. 529 (2) Results obtained from the Rashid Qureshi’s system Setting backgrounds Dataset images 5 models 16 100 symbols 2521 5 models 17 100 symbols 1340 0. 50 0. 40 i(ε) People floorplans 0. 30 electrical diagrams 0. 20 0. 10 0. 00 score error (ε) 0. 10

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 56

Content based Image Retrieval “Robust key points detection for document image retrieval” People The Anh Pham Funding VEID scholarship Starting 10/2010 Ref Na The work focuses on robustness-keypoint detection, robustness in DIA including: 2 D and illumination, noises, artifacts, blurred. Keypoints detection takes part of the general features extraction, some typical features used as key points and their characteristics are: Level of detail (primitive) Robustness Regions Like-blob points Edges, contours Corners, salient points Patterns Semantic So, some ideas of new robust keypoint detection may be: - Detecting robust features first, then extracting salient points, or - Using robust methods (i. e Machine learning, Model- based, Parametric-based) to extract salient points, or - Combination of above methods 57

Content based Image Retrieval “Logo recognition and spotting” People Mathieu Delalandre Funding DOD project Starting 04/2012 Ref Na Problematic - document classification can be supported by logo recognition - a meta engine will manage the decision rules - spotting will depend of a previous stage of page segmentation Page segmentation Document classification Logo spotting Logo recognition Meta engine - binary images at 300 dpi - time constraint: to 1, 5 s per image High variability and scalability of logos Logo are rich graphical parts, 300 dpi and binarization could result in a high level of degradation Some logo recognition case looks like OCR only 58

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 6. 7. Graph matching and embedding 59

Camera based recognition “Robust OCR for video text recognition” People Thierry brouard Funding SNECMA project Starting 2008 Ref International patent Problematic -Automatic routing of input letters by digitization and OCR of input documents received from customers -Response time < 3 s, Recognition rate > 80%, Precision Equal to 100%, Java environment on mutualized servers Approach based on cognitive vision and knowledge based systems (blackboard & mathematical theory of evidence), to achieve robust segmentation & OCR 60

Camera based recognition “Real time logos recognition in urban environment” People Mathieu Delalandre Funding Na (JSPS grant) Starting 2010 Ref Na video a set of images Problematic - Logo detection from video capture using some handled interactions, to display context based information (tourist check points, bus stop, meal, etc. ). - Hard points are the real time constraints and the complexity of the recognition task. First goal of the project is to support the real time recognition. We start from the hypothesis than logo appear in a static way in video. We propose to achieve an automatic control/selection of image capture to reduce the amount of data to processing charge frequency, resolution parameters Workflow management size frame Video capture Frame stack stored frame Frame selection tag/remove motion data Detection/ Recognition 3 D Mapping overlapping Gyro access 61

Talk workplan 1. 2. 3. 4. 5. 6. Tours city François-Rabelais University, les deux lions / Portalis School of Engineering Polytech’Tours Laboratory of Computer Science RFAI group DIA related work 6. 1. Projects & partners outline 6. 2. Layout analysis and document recognition 6. 3. OCR, word spotting and signature verification 6. 4. Symbol recognition & spotting 6. 5. Content Based Image Retrieval 6. 6. Camera based recognition 62