Adarza Technologies Bacteria Classification Assistant ENSC 440 Presentation

Adarza Technologies Bacteria Classification Assistant ENSC 440 Presentation April 19 th, 2005 Members: Edward Goh Linda Wu Josna Rao Geeda Ip

Agenda for Today • • • • Introduction System Overview Data Acquisition Image Processing Data Filtering User Interface Testing Procedure Cost Analysis Future Features Conclusion Acknowledgements Q&A Demonstration © Adarza Technologies 2005

Company Profile • Adarza Technologies – Edward Goh (Chief Executive Officer) – Linda Wu (Chief Operations Officer and Chief Financial Officer) – Josna Rao (Chief Technology Officer) – Geeda Ip (Chief Information Officer) © Adarza Technologies 2005

Introduction • The Bacteria Classification Assistant – Helps user identify bacteria – Assists with training new technicians • Technologically Relevant – microbiology field – Applications – ID microscopic objects © Adarza Technologies 2005

System Overview • . NET framework with Open Computer Vision Library • Microsoft Access - Database • Windows Forms - GUI • Accepts JPEG images from hardware • Returns subset of database • Classification Criteria: – – Colour Size Boundary Shape Colony Shape © Adarza Technologies 2005

BCA Block Diagram © Adarza Technologies 2005

Data Acquisition • Digital Camera Microscope • Camera - Canon Powershot G 5 with Scopetronix Max. View Plus accessories for the microscope • Microscope - Olympus VMT fitted with 1 X, 2 X objectices © Adarza Technologies 2005

Image Processing Agenda • Preprocessing Modules – Edge sequence – Centre Finding • CC Processing Modules – Size – Colour – Boundary Shape – Colony Shape © Adarza Technologies 2005

Pre-processing • Features that are not specifically classification criteria • Edge detection • Edge sequencing • Centre detection © Adarza Technologies 2005

Edge Processing • Returns image of colony in white, on black background • Method: – Split image into red, green, and blue planes – Remove noise – Find edges in each plane by calculating gradient and setting threshold – Adjust threshold until contour closes – Floodfill colony white and background black © Adarza Technologies 2005

Original Image Blue Edge Green Edge © Adarza Technologies 2005 Final Edge Red Edge

Edge Examples Streptococcus sanguis © Adarza Technologies 2005

Edge Examples Chryseomonas luteola © Adarza Technologies 2005

Edge Examples Candida albicans © Adarza Technologies 2005

Edge Examples Neisseria gonorrhoeae • Transparent Edge • No distinct colour difference © Adarza Technologies 2005

Edge Sequencing • Detects contours • Sorts and orders the points from contours • Returns an Open. CV sequence object • Used to determine boundary shape and colony shape © Adarza Technologies 2005

Centre Finding Module • Centroid equations • Returns cv. Point • Relies on edge image • Centre used for boundary and colony shape © Adarza Technologies 2005

Centre Samples © Adarza Technologies 2005

Size Module • Goal: 4 Categories – Tiny, Small, Medium, Large • Comparison based on 2 D area • Uses library function to count non-zero pixels • Requires monochrome version of sample and calibration images • Ratio of calibration and sample area taken © Adarza Technologies 2005

Size Samples © Adarza Technologies 2005

Colour Module • Goal: 7 Categories – Non-pigmented: White, Gray, White-Yellow-Gray, Red – Pigmented: Blue, Green, Yellow, Purple • Returns two values: primary and secondary – Primary: most likely – Secondary: for borderline cases • Method: analyze image in Hue-Saturation. Brightness channels – Search for local maxima • Some examples © Adarza Technologies 2005

Hue-Saturation-Brightness • Hue: 0° ~ 360° • Saturation: 0% ~ 100% • Brightness: 0% ~ 100% © Adarza Technologies 2005

Colour Module Method • Image separated into 3 planes Hue Plane Saturation Plane © Adarza Technologies 2005 Value Plane

Colour Module Method • Each plane analyzed with mask 100 Hue Plane 184 Saturation Plane © Adarza Technologies 2005 239 Value Plane

Colour Module • Pigmented colonies: © Adarza Technologies 2005

Colour Examples • Yellow Colony © Adarza Technologies 2005

Colour Module • Non-pigmented Colonies White Gray White. Yellow. Gray Red - Transparent © Adarza Technologies 2005

Colour Example • White Colony © Adarza Technologies 2005

Colour Module • Test Cases 1. Strongly pigmented colonies – Ex: Chryseomonas luteola 2. Very white colonies – Ex: Stomatococcus mucilaginosis 3. Borderline cases – Relies on secondary colour for identification – Relax contraints © Adarza Technologies 2005

Colony Shape Module • Goal: Classify colony as Circular or Irregular • Method: – Histogram of Radial Lengths • Circular: – Single large peak • Irregular: – Multiple peaks of lower height © Adarza Technologies 2005

Colony Shape Module • Histogram Analysis: – Locate max peak in histogram – Evaluate average height of surrounding columns – Use threshold to determine if height great enough to be circular Number of edge points Length of radial vector © Adarza Technologies 2005

Colony Shape Module Circular Irregular © Adarza Technologies 2005

Colony Shape Module • Test Cases: 1. Colonies of different sizes: – Small number of edge points: Passed(correct classifier returned) – Large number of edge points: Passed 2. Different edge cases: – Washout edges: Passed (always irregular) © Adarza Technologies 2005

Boundary Shape Module • Goal: 4 Categories – Smooth, Undulating, or Finger-like • Method of edge angles – Histogram of Edge Angles – Difference between tangent angle and normal angle • Smooth – 0° • Undulating – Some at 0° • Fingerlike – Widely varying from 0° to 90° © Adarza Technologies 2005

Boundary Shape Module • Histogram Analysis: – – Evaluate average height of bins around 0° Evaluate max peak height and location Evaluate average height of bins around 90° Combination of 4 major thresholds 10% Number of edge points © Adarza Technologies 2005 0° 90 ° Edge Angle

Boundary Shape Module Smooth Undulating Finger-like © Adarza Technologies 2005

Boundary Shape Module • Reliability: – Borderline cases: • Certain colonies have characteristics of more than one shape type • Edge may also be unreliable, perhaps due to blurry colony – Solution: • Peaks closer to thresholds in number and angle are given two shape classifiers • Both classifiers used to filter results from database © Adarza Technologies 2005

Boundary Shape Module • Test Cases: 1. Colonies of different sizes: Passed 2. Different edge cases: – Washout edges: Passed (always finger-like) – Blurry edges: Two shape classifiers returned in 85% of tests 3. Colonies of different shapes: – Circular: Passed – Irregular: Passed – Highly irregular (length more than twice width): – Leads to some unreliability, often classified as fingerlike © Adarza Technologies 2005

Database • Introduction • Technology – Microsoft Access 2000 – ADO. NET • Considerations in design: – – Normalization Size Portability Adaptability © Adarza Technologies 2005

Database Structure Overview © Adarza Technologies 2005

Bacteria Picture Table • Where the main data for each record is stored. • Can store reference or sample entries • BCAVERSION field © Adarza Technologies 2005

Boundary Shape Table • Structure: • Sample Data: © Adarza Technologies 2005

Colony Shape Table • Structure: • Sample Data: © Adarza Technologies 2005

Colour Table • Structure: • Sample Data: © Adarza Technologies 2005

Size Table • Structure: • Sample Data: © Adarza Technologies 2005

Photo and Calibration Tables • Photo Table – Each entry in the Photo Table points to a bacteria picture file on disk. – Each entry also points to an entry in the Calibration Table. – Pictures from the same batch would point to same Calibration entry. • Calibration Table – Each entry in the Calibration Table points to a calibration picture file on disk. – Each entry also points to an entry in the Lighting Condition Table. • Lighting Condition Table – Describes the lighting condition under which the batch of pictures were taken. © Adarza Technologies 2005

How Files are Stored on Disk • Top Level – Where the BCA program is located. • Database Level – Where the Access database (BCA. mdb) is located. • Data Level – Where all the pictures are located. © Adarza Technologies 2005

Goals of the GUI • Collect all necessary input from user. • Guide user through sample image analytical process. • Present information from the database in a user-friendly manner. • Allow user to easily search, view, and modify database. © Adarza Technologies 2005

Startup Screen • Choice 1: Analyze sample image • Choice 2: View/Modify database ©

Browsing Through Database © Adarza Technologies 2005

Modified information appears in blue © Adarza Technologies 2005

Warning before navigating away © Adarza Technologies 2005

Warning before deletion © Adarza Technologies 2005

After Deletion © Adarza Technologies 2005

Searching and Setting Filters • Values between criteria are AND-ed • Values within criteria are OR-ed © Adarza Technologies 2005

Before Filters are Set • Check marks indicate which filters to set. • Check box items are in black. • Once we click “Set Filters”… © Adarza Technologies 2005

After Filters are Set • Check box items appear in blue. • Number of filtered records are reflected in the title bar: • Can undo filters or add/remove additional filters. © Adarza Technologies 2005

After Filters are Set • Selecting/deselecting additional filters will toggle the appropriate filter buttons. © Adarza Technologies 2005

Analyze Sample Walk-Through • Get the calibration image file. • Get a point inside the calibration image. • Get the sample image file. • Get a crop of a single bacterial colony. • Get a rectangle within the colony. • Next… © Adarza Technologies 2005

Analyze Sample Walk-Through © Adarza Technologies 2005

Add to Database • Automatically fill in calculated values. • Notifies user if required information is missing. © Adarza Technologies 2005

Testing Procedure • Individual Module testing – Setting threshold levels – 100% accuracy on non-border cases – At least 90% accuracy on border cases • Consider both primary and secondary choices • Integrated testing – Multiple testing of reference set – Searching for consistency – Compromise between computer and human perception © Adarza Technologies 2005

Cost Analysis Item Estimated Market Price (CDN) $100. 00 Research expenses (photocopying, printing, etc. ) Our Estimated Cost $100. 00 Our Real Cost $10. 25 $127. 98 Textbooks on C++ and Visual Studio Developmental tools and software · MS Visual Studio. NET Academic Edition 2003 $135. 00 $0. 00 · Matlab Student Version with image acquisition, image processing, and fuzzy logic toolboxes. $177. 00 $0. 00 · Paint Shop Pro 9. 0 $144. 99 $0. 00 $12. 00 · CVS Advanced Sony Cyber-shot 3. 2 DSC-P 72 mega-pixel digital camera $300. 00 $0. 00 Microscope by C&A Scientific $135. 00 $0. 00 Lab Petri Dishes 100 x 15 mm (20/Pk) $13. 50 $0. 00 Travel expense for consultation meetings $15. 00 $7. 75 Website $15. 00 $18. 11 $24. 04 Consultation Lunch Contingencies Total: $20. 00 $0. 00 $1, 055. 49 $150. 00 $200. 13 © Adarza Technologies 2005

Future Features • Image Processing Features – Active contours – Additional classification criteria • User Interface Features – Advanced search capabilities – Remove extra input from user – Results ordered by probability © Adarza Technologies 2005

Conclusion • Proof of concept complete • Further development is possible © Adarza Technologies

Acknowledgements • Dr. Mirza Faisel Beg, Assistant Professor of the School of Engineering Science • Mr. David Chan, Senior Lab Technician @ BCCDC • Dr. Swee Han Goh, Associate Professor @ BCCDC © Adarza Technologies 2005