AUTOMATIC DETECTION AND CLASSIFICATION OF DENTAL RESTORATIONS IN

AUTOMATIC DETECTION AND CLASSIFICATION OF DENTAL RESTORATIONS IN PANORAMIC RADIOGRAPHS Talia Yeshua 1. 7. 19 In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

Talia Yeshua, Jerusalem College of Technology, Dept. of Applied Physics, Israel Ya’akov Mandelbaum, Jerusalem College of Technology, Dept. of Applied Physics, Israel Ragda Aslan-Abdallah, Hadassah University Hospital Dept. of Oral Medicine, Israel

Chen Nadler, Hadassah University Hospital Dept. of Oral Medicine, Israel Levana Zemour, Jerusalem College of Technology, Dept. of Computer Science, Israel Laureen Cohen, Jerusalem College of Technology, Dept. of Computer Science, Israel

Daniel Kabla, Jerusalem College of Technology, Dept. of Applied Physics, Israel Ori Gleisner, Jerusalem College of Technology, Dept. of Applied Physics, Israel Isaac Leichter, Jerusalem College of Technology, Dept. of Applied Physics, Israel

Today’s Overview 1 • A panoramic image of the oral cavity includes plentiful information. 2 • Automatically mapping of the dental restorations is necessary. 3 • Computer vision and machine learning can be used for extracting this information. In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

Overview of Topic • • Background Purpose Dataset Technology Results Benefits Summary In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

BACKGROUND • A panoramic image is an external dental scan X-ray of the whole oral cavity. • Minimal discomfort • Low radiation Panoramic image cone-beam CT imaging intra-oral radiograph In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

BACKGROUND • In order to extract useful information from a panoramic image, a careful review by a dentist with the proper expertise is required. • Currently, however, a radiologic informative report is not regularly designed. In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

PURPOSE • To develop a prototype for an automatic radiologic report that will extract the information from the panoramic image. • The first stage is to develop a machine-learning artificial intelligence system for automatic detection and classification of dental restorations in the oral cavity. In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

DATASET • 63 anonymized panoramic images. • The panoramic images contained 300 dental restorations such as fillings, crowns, root canal treatments and implants. In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

DATASET The dental restorations were classified into nine categories. Group number Type of restoration Acronyms 1 Crown (only) CRW 2 Root Canal Treatment with Core RCT-CO 3 Root Canal Treatment with Core & Crown RCT-CO-CRW 4 Dental Implant (only) DI 5 Dental Implant with Abutment DI-A 6 Dental Implant with Crown DI-CRW 7 Amalgam Filling AF 8 Composite Filling CF 9 Connected restorations MULTI In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

DATASET An experienced dentist evaluated the images and marked manually on each image all the existing dental restorations. A panoramic image with the dentist’s annotations of dental restorations In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

TECHNOLOGY • The algorithm was developed in the MATLAB programming environment. • The algorithm is based on computer vision and machine learning. 1 st step - Detect the restoration 2 nd step - Classify the restoration In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

TECHNOLOGY 1 st step -Detect the restoration First, the region of interest (ROI), including the maxilla and mandible, was automatically cropped. In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

TECHNOLOGY 1 st step -Detect the restoration Then , the restorations were detected, using a gray-level threshold. Ø Brighter areas are represented by higher gray-level values. Ø The gray level values of most restorations are higher than those of the teeth and the surrounding tissues. Pixels with different brightness Gray-level values of these pixels In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

TECHNOLOGY 1 st step -Detect the restoration Adaptive Threshold accommodates large variability of brightness in the image. Detection by global threshold The original image Detection by adaptive threshold In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

TECHNOLOGY 1 st step -Detect the restoration Morphological operators were used to improve the detection The original image Initial detection following erosion, dilation, opening and closing In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

TECHNOLOGY 2 st step - Classify the restoration • Each dental restoration is characterized by a unique shape and gray level distribution. • 20 numerical features describing the shape and texture were calculated for each restoration. In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

TECHNOLOGY 2 st step - Classify the restoration • 3 categories of false detection, were added in order to automatically classify the false marks in the detection stage. In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

TECHNOLOGY 2 st step - Classify the restoration Supervised machine learning analyzed the information and automatically classified the restoration into categories. Train data Dataset Test data Create a classification algorithm by machine learning Analysis of the classification successes rate In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

TECHNOLOGY 2 st step - Classify the restoration 5 fold cross validation training A cross-validation method was applied for testing the performance of the algorithm. In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

RESULTS 1 st step -Detect the restoration The algorithm automatically detected 95% of the existing restorations in the panoramicimages in the dataset 100% Series 1 100% 91% 100% 88% 92% 80% 60% 40% 20% 0% CRW RCT DI AF CF Success rate of detection for various categories of restorations In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

RESULTS 2 st step - Classify the restoration The algorithm classified successfully 92% of the detected regions 91% 93% 87% 94% UL TI To er. L ap ot h e En Ot ame he l r. F al se CF 90% Ov M W -C R DI DI 96% AF Series 1 100% -A 93% DI RW -C CO TRC T- RC CR CO 89% W 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 100% Success rate of classification for various categories of the detected findings In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

BENEFITS • • • Minimal discomfort Low-cost Low radiation One image includes all the information Information can be extracted from routine images In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

Summary • Panoramic images include essential information regarding the oral cavity. • To date, a radiologic report is not committed to the image. • Computer vision and machine learning tools can detect the dental restorations and classify them into categories. • The obtained information serves as an initial step toward automatic detection of oral pathologies in a panoramic image. In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,

QUESTIONS? In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem,