License Plate Recognition LPR Autogate System April 14

License Plate Recognition (LPR) Auto-gate System April 14 th, 2009 Danny Choi (President and CEO) Jaehoon Shin (CMO and CFO) Wook Sun Shim (CTO and COO)

Agenda • • • Introduction Motivation System Overview Hardware – Metal Detector – Microcontroller Communication Module LPR System – Camera Module – License Plate Locator – Letter Recognition Finances Timeline Final Thoughts Questions and Answers

Introduction License Plate Recognition System Convert the license plate number to ASCII 245 AFC License Plate uniquely identifies vehicle. Various police forces Electronic toll Collection Pay per use roads Monitoring traffic activity

Motivations

Motivations Auto Theft ◦ On an average day in B. C. 37 vehicles are stolen 62 vehicles are broken into One third occurred in the private residences ◦ Underground parking lots with Auto gate “Stop until the gate close” No physical barrier !

Auto-gate Parking Lot ◦ Must carry key entry devices Could be lost, stolen, or no battery ◦ RF signals can be stolen RF (radio frequency) remote control analyzer Following after vehicles Walk-ins No Physical barrier !

Motivations LPR Auto-gate system: Virtual Barrier ◦ The system recognizes the vehicle and checks for verification ◦ Following-after vehicles or walk-ins are controlled ◦ Alerting a trespassing and save the image for further crime investigation and enforcement.

Motivations Conveniences ◦ Eliminating the use of key entry device. ◦ Simply “come and enter” No need to wait for the gate to close! ◦ Registering for guest vehicles: Type in the plate number

System Overview Block Diagram Metal Detector LPR System Microcontroller Board (LCD & LED)

Logic Diagram Red LED & Alarm Start N Vehicle? Y Capture & Image Processing Y Green? N Red LED Y Green LED N Verified?

Hardware

Hardware – Metal Detector The Concept of Mutual Inductance Main Components ◦ TL 072 (Dual Operational Amplifier), NE 555 (Timer), a couple of Inductors

Hardware – Metal Detector Schematic Diagram

Hardware - Microcontroller (Atmel ATmega 168) ◦ High density of various features (ADC, PWM, LCD driver, and etc) ◦ GNU Complier Collection (GCC) supported Voltage Regulator (L 7805) ◦ 9 V battery to 5 V LCD ◦ 2 x 24 characters display ◦ HD 44780 protocol Outputs (piezoelectric buzzer and LEDs)

Alternatives Vehicle detection ◦ Road tube, Magnetometer sensor, Weigh-In-Motion (WIM) , Piezoelectric sensor, and etc Microcontroller Atmel ATmega 168 Zilog Z 86 E 3416 VSC Speed 20 MHz 16 MHz Memory 16 K Ram 1 Kb 512 b I/O pin 23 24 ADC Yes No Package DIP-28 Price $4 -5 $11

Communication Module RS 232 serial I/O port ◦ Cheaper and Simpler solution than USB! Use CSerial source code Baud Rate 115200 Data Bits 8 Parity None Stop Bits 1 Flow Control None

LPR System

LPR System Brain of the overall system. Open. CV – Computer Vision library by Intel. Module: ◦ ◦ ◦ Camera Module License Plate Locator (Skew Correction Module) Letter Recognition Module Verification Module

LPR System

LPR System – Camera Module Grab a frame from a webcam Refresh the frame every 0. 5 seconds If signal comes in from microcontroller, save current frame.

LPR System – License Plate Locator

LPR System – License Plate Locator Original: Dilation: Erosion:

LPR System – License Plate Locator Original: Dilation: Erosion after Dilating the image:

LPR System – License Plate Locator Grey Scaled: Dilation:

LPR System – License Plate Locator Dilation: Erosion after Dilation:

LPR System – License Plate Locator Subtract: Threshold and Noise Filter

LPR System – License Plate Locator

LPR System – License Plate Locator

LPR System – Skew Correction Canny Edge Detection

LPR System – Skew Correction

LPR System – Skew Correction Not integrated with the overall system. ◦ Not every input image had detectable edges. ◦ Assumption: the camera is in its fixed position, almost perfectly parallel to the ground. ◦ System can also recognize the letters from license plate with tilt angle of ~10° Other Possible Solutions: ◦ Hough Transform ◦ Radon Transform

LPR System – Letter Recognition Localized Invert Paint Binary out Image margins Plate

LPR System – Letter Recognition Cropped plate

LPR System – Letter Recognition Character Segmentation

LPR System – Letter Recognition Character Segmentation Template Matching !

LPR System – Letter Recognition

LPR System – Letter Recognition Ratio is not valid Not a character !

LPR System – Letter Recognition Template Matching ~ ~ I, O, Q, U, Y, Z

LPR System – Letter Recognition Identification … SIX

LPR System – Letter Recognition Identification … B!

LPR System – Letter Recognition 616 BBL _

Finances Equipment Estimated Cost Actual Cost Camera $100 $0 Microcontroller & development kit $300 $20 Metal detector $50 $0 Miscellaneous (LCD, cable, case, alarm, etc) $50 $80 Total $500 $100

Finances Equipment Market price Project Cost Camera $150 $0 Microcontroller & development kit $20 Metal detector $800 $0 Miscellaneous (LCD, cable, case, alarm, etc) $400 $80 $1370 $100 Total

Timeline – Milestone Chart

Timeline – Gantt Chart

Final Thoughts We learned: ◦ ◦ ◦ Team dynamics Digital Image Processing techniques Programming in C/C++ Circuitry Microcontroller Challenges ◦ A group of three members ◦ Localization

Final Thoughts Possible Future Works ◦ ◦ ◦ Improve skew correction Implement with actual gate (or motor) Finer resolution camera User Interface More accurate LPR algorithm “The complexity of each of these subsections of the program determines the accuracy of the system” (en. wikipedia. org/wiki/Automatic_number_plate_recog nition)

Acknowledgement Dr. Parvaneh Saeedi Jun Kim Steve Whitmore Patrick Leung, P. Eng Jamie Westell Jason Lee

References Insurance Corporation of British Columbia (ICBC). 2008. Insurance Corporation of British Columbia (ICBC). 2007. Stop Auto Crime in Multi-Residential and Mixed-Use Buildings. Auto Crime and Fraud. Mia Dauvergne. Statistics Canada. 2008. Motor Vehicles Theft in Canada, 2007. Statistics Canada. 2008. Motor Vehicle Theft. G. Baridski and A. Kaehler. 2008. Learning Open. CV: Computer Vision with the Open. CV Library. C. WU, L. C. ON, C. H. WENG, T. S. KUAN, K. NG. August 2005. A Macao License Plate Recognition System.

Questions?