Advanced Crosswalk Detection for the Bionic Eyeglass Mihly

Advanced Crosswalk Detection for the Bionic Eyeglass Mihály Radványi Balázs Varga Kristóf Karacs Pázmány Péter Catholic University Berkeley, 2010.

Summary • • • Bionic Camera/Eyeglass The task Difficulties Algorithmic description Results Further plans

Bionic Camera/Eyeglass • Visual info audio MULTIMODAL • Three situations – home • banknote, color recognition, etc. – office • pictograms, displays, etc. – street • escalator’s direction • route number-, crosswalk detection, etc. • Autonomous crash avoidance

The task • Crosswalk recognition / detection – Based on road marks • dark – bright – alternating • parallel – Mobile device • navigation fast decision • Low consumption ( ~1 W) • High computing power (TOps/s) visual microprocessor

Difficulties • shadows – tricky – disturbing • • light conditions traffic missing marks others categorize!

Different approaches • Matlab simulation – Image Processing and Mat. CNN toolbox – 2 nd generation • Mean shift color segmentation • On-chip – Bi-i visual processor • 128 x 128 pixel • Still image, video • 10 fps (1 st generation)

Manual design - flow • Goal: clearest zebra lines Adaptive threshold Contrast mask Input image AND Color mask Mean shift RECALL CNN templates Supposed zebra lines

Asphalt detection - introduction • foreground-background segmentation – • foreground (ROI): asphalt + stripes mean shift segmentation – – – Similar to K-means • arbitrary number of clusters! Iterative, nonparametric YCb. Cr color space Input image (Y – Cb – Cr) Select i data points Calculate position of a mode* (Iterative scheme on a Probability Density Function using a moving Gaussian window) No *dense region of the feature space Put pix. with similar color into same class Density gradient map with modes ( ) Ar e P clu ix all ste els re d? Yes Biggest class results asphalt

Asphalt detection - segmentation Steps of segmentation 1. Preprocessing • band-pass color filtering 2. Foreground-background segmentation • mean shift 3. Post processing • morphological operations • eliminates false positive/false negative pixels Input Result of preprocessing Classes of mean shift Masked foreground Result of post processing

Manual design - again • Goal: clearest zebra lines Adaptive threshold Contrast mask Input image AND Color mask Mean shift RECALL CNN templates Supposed zebra lines

Manual design – flow No. 2 Space-variant Localization of huge patches Supposed zebra lines LOGDIF EDGE Decision A h 600 400 h’ 200 A’ where Scalar output

Results - simulation

Results • Enlarged dataset Crosswalk detected Crosswalk non detected Crosswalk 33 7 No Crosswalk 2 41 • 89. 2% performed well False positive (2. 4%)

The prototype • Nokia N 95 + Bi-i + Wi. Fi • Eye-Ris

Further plans • Tracking – Correlations between frames • Online test with blind users(done…) • Background estimation through image fusion

Thanks for your attention!