Textual Information Access for the Visually Impaired Ramani

Textual Information Access for the Visually Impaired Ramani Duraiswami

Project • Approximately 1 million blind and 5 million visually impaired people in the US – Comparable fractions of the population elsewhere • Their Goal: Lead an independent productive life – To do this they need to understand a significant amount of textual information from the environment • Examples – – Signs on streets, supermarkets/groceries, buildings Labels on medicines, products Instructions on equipment, computers Newspapers, magazines • Our goal: Provide them with devices to read text – Computer vision, Pattern recognition (OCR), computational audio, and wearable computers to help them.

Restricted initial goal: magazine reader • Tool targeted to people with some vision • Use optical character recognition (OCR) software – OCR works with flat, high resolution scanned text. • Read out the recognized text with text-to-speech (TTS) software • Goal: read text with a video camera, OCR software, portable/wearable computer, headphones • Problems – Text is on a surface that is mostly planar or curved – Warped by perspective projection to the camera – Camera images are low resolution (640 x 480) – OCR uses scanned images at 600 to 2400 dpi

System Design • For good OCR – Characters must be 30 -60 pixels tall – Characters must not be skewed • Camera captures initial image of page • Using initial images – Estimate font size and page structure – Determine scanning pattern • Zoom into pieces of the image – Register each zoomed piece with the big picture – Capture image at optimum focus and zoom • To make sure pieces are not too small – Use super-resolution techniques • Mosaic pieces together and dewarp

Super-resolution • Convert a sequence of many (different) coarsely sampled images into a high resolution image.