Text Detection in Images and Video Outline n

Text Detection in Images and Video

Outline n n n n Background Video Demo (Google translate, Huayu. Navi, Camcard) Optical character recognition (OCR) ICDAR Competition Part I: Caption detection in video Part II: Scene text detection Conclusion

Background n n n Detecting text and caption from videos is important and in great demand for video retrieval, annotation, indexing, and content analysis. Provide high level semantic information such as program name, speaker name, speech content, sports scores, date, time, location, and so forth. Scene text detection attempts to extract textual information from images/videos in more natural settings.

Possible Applications Annotation, tagging, indexing, search n Translation n Navigation n Book cover recognition n License plate recognition n Computerized aid for visually impaired n Automatic geocoding of businesses n

Demo Video Google translate n Huayu. Navi n Camcard n

OCR http: //code. google. com/p/tesseract-ocr n Tesseract OCR (pptx file) n

ICDAR Competitions International Conference on Document Analysis and Recognition n Competitions http: //www. icdar 2011. org/EN/column/colum n 26. shtml n Robust Reading Competition Challenge 2: Reading Text in Scene Images http: //robustreading. opendfki. de/wiki/Scene Text n

Text Localization Task

Word Recognition Task

Part I: Caption Detection n Text From Corners: A Novel Approach to Detect Text and Caption in Videos, IEEE Transactions on Image Processing, 2011.

Existing Methods n Most existing approaches can be generally classified into three categories 1. 2. 3. texture based methods connected component based methods edge based methods

Corner Features n Three-fold advantages of corner points Corners are frequent and essential patterns in text regions. n The distributions of corner points in text regions are usually orderly. n generates more flexible and efficient criteria, under which the margin between text and nontext regions in the feature space is discriminative. n

Corner Extraction

Feature Description (1/2) n morphology dilation on the binary corner image n Corner points are dense and usually regularly placed in a horizontal string. n The text can be effectively detected by figuring out the shape properties of the formed regions.

Feature Description (2/2) n Five region properties : n n n Area -> Ra Saturation -> Rs Orientation -> Ro aspect ratio -> Ras position -> Rc bounding box : smallest rectangular that completely encloses the corner points formed regions.

Area n The area of a region is defined as the number of foreground pixels in the region enclosed by a rectangle bounding box.

Saturation n Saturation specifies the proportion of the foreground pixels in the bounding box that also belong to the region, which can be calculated by

Orientation n Orientation is defined as the angle between the xaxis and the major axis of the ellipse

Aspect Ratio and Position n n Aspect Ratio: Aspect Ratio of a bounding box is defined as the ratio of its width to its height. Position: We describe the position of a region with its centroid.

Language-independent

Part II: Scene Text Detection Paper 1: Detecting text in natural scenes with stroke width transform, CVPR 2010. n Paper 2: How salient is scene text? Proceedings of the 2012 10 th IAPR International Workshop on Document Analysis Systems, Pages 317 -321. n 3: Stroke Filter n

Paper 1 n Detecting text in natural scenes with stroke width transform, CVPR 2010.

Detected Text Area in Natural Scenes

The Extraction Process

Stroke Width Transform (SWT)

Flowchart

Performance Evaluation

Paper 2 n How Salient is Scene Text?

How Salient is Scene Text? n Comparing the performance of four attention models in scene text detection: Torralba’s saliency map (color) n Torralba’s saliency map (Intensity) n Harel’s GBVS n Zhang’s Fast Saliency n Itti’s saliency map (N 2 P 2 CI) n

Input and Ground Truth

Results

Performance Comparison

Stroke Filter (1/3) n Original

Results

Stroke Filter (2/3) n Improved

Stroke Filter (3/3) n Fast

Conclusions Caption detection is easier due to its nature of origin. n Robust scene text detection is more challenging due to unconstrained environments. n Detecting Chinese text in natural images is still an open problem. n