Online Preprocessing of Gesture Sign Using Background Substructure

Online Preprocessing of Gesture Sign Using Background Substructure and Edge Detection Algorithms By Farah F. Alkhalid University of Technology-Iraq Control and Systems Engineering 12/16/2021 1

Outlines • • • Abstract. Problem Statement. Proposed System. Canny Edge Detection Algorithm. Background Subtractor and Threshold Algorithm. Two Algorithms Python Program. Proposed System Results. Data Set Used in Training. Deep Learning Model. Test and train Program. Test and train output. Conclusion. 12/16/2021 2

Abstract • One of the important missions is to communicate with mute and deaf people, and listen to what they are need. The gesture sign is the first solution we can use to make connection between us, but this way has rules and basics, we should respect and be informed with, at the same time not all people can find the need to know how translate sign language, for this reason we proposed a model to translate sign language but before use the model we should make preprocessing image in order to get accurate classification, in this paper, background subtraction and edge detection are used together to present image with high edge details and only gesture sign which is the region of interest, this work done using Open. CV in python 3. 12/16/2021 3

Problem Statement • Since all datasets of sign language are captured with empty background, the focus will be on the Region of Interest (ROI), which refers to gesture sign. Therefore, to make the training model successful it is important to provide gesture signs with an empty background. 12/16/2021 4

Problem Statement • Hand motions are perceived from a video arrangement. To perceive these signals from a live video sequence, at first it is needed to take out the locale hand alone evacuating all the undesirable segments in the video arrangement. In this manner, the whole issue could be illuminated utilizing two stages: • Canny edge detection. • Background subtractor and Threshold. 12/16/2021 5

Proposed System 12/16/2021 6

Canny Edge Detection Algorithm • The procedure of edge detection includes recognizing sharp edges in the picture and creating a paired picture as they appear. Commonly, the procedure draws white lines on a dark foundation to demonstrate those edges. 12/16/2021 7

Canny Edge Detection Algorithm 12/16/2021 8

Canny Edge Detection Algorithm 12/16/2021 9

Background Subtractor and Threshold Algorithm • The background subtraction strategy performs well for situations where there is a need to distinguish moving items in a static scene. • As the name demonstrates, this calculation works by recognizing the foundation and subtracting it from the present edge to acquire the forefront 12/16/2021 10

Background Subtractor and Threshold Algorithm 12/16/2021 11

Two Algorithms Python Program Open. CV Library Background Subtractor Canny Edge Video Capture Block Image Show End Capturing by using “q” key 12/16/2021 12

Background Subtractor and Threshold Algorithm 12/16/2021 13

Proposed System Results 12/16/2021 14

Data Set Used in Training • A new dataset consists of 54, 049 images of Ar. SL alphabets performed by more than 40 people for 32 standard Arabic signs and alphabets. The number of images per class differs from one class to another. Sample image of all Arabic Language Signs is also attached. The CSV file contains the Label of each corresponding Arabic Sign Language Image based on the image file name. 12/16/2021 15

Deep Learning Model 12/16/2021 16

Test Program Test generation Apply to Model 12/16/2021 17

Test and train 12/16/2021 18

Conclusion • The gestures presented in diverse backgrounds have to be accurately processed and segmented so that it can be classified precisely by the hand gesture recognition system. This study compares the performance of the proposed Image Segmentation Algorithm with a standard Canny's Edge Detection Algorithm and Background Subtraction Algorithm. These two techniques have worked together to submit a new image with a gesture sign with a high edge to represent the pause of the sign. The two algorithms are free, fast, and supports all operating systems. The image will enter to the sign recognition model using deep learning which is the next step of this research in order to dispose of the unwanted features, the results are satisfying and have details that are needed to be fed to the next recognition model, where the testing accuracy is =85. 71% when using the two algorithms 12/16/2021 19

Thank you. . Questions? ? 12/16/2021 20