Using deep machine learning to conduct objectbased identification

Using deep machine learning to conduct object-based identification and motion detection on safeguards video surveillance Yonggang Cui 1, Zoe N. Gastelum 2, Ray Ren 1, Michael R. Smith 2, Yuewei Lin 1, Maikael A. Thomas 2, Shinjae Yoo 1, Warren Stern 1 1 Brookhaven 2 Sandia National Laboratory, Upton, USA National Laboratories, Albuquerque, USA BNL-209172 -2018 -COPR

2 Review of Surveillance Videos q A labor-intensive task for inspectors with time constraints q Software is used to automate the task q Scene-change-based algorithms are available today, but with limitations Source: www. iaea. org Ø High false-alarm rate q Software can perform better with advanced features: Ø Object recognition/classification Ø Object-based motion detection BNL-209172 -2018 -COPR Source: www. emcbc. doe. gov/

3 Use Deep Machine Learning to Solve the Problem q Advantages of deep machine learning Ø Unique in solving problems with multi-dimensional features/parameters Ø Automatic data processing q Challenges in application Ø Ø q Complicated settings in nuclear facilities Limited domain data set for training machine learning algorithms Time constraints in image/video review (inspection) Limited computational resources in field Development path Ø Started addressing object detection task Ø More complicated tasks will be addressed later q Our solution to object detection task Ø Construct a deep machine learning neural network for object detection and classification Ø Use a transfer learning technique to train the network BNL-209172 -2018 -COPR

4 Deep Machine Learning Neural Network q A neural network uses networked nodes to simulate human brain. q Its function is determined by weighted information propagation and transfer function of nodes. q A network can have millions of parameters and needs huge data set to determine their values (training). Source: Wikipedia Input Layers BNL-209172 -2018 -COPR Hidden Layers Output Layer

5 YOLO Neural Network q You Only Look Once (YOLO) only needs to process an image once to perform detection Ø A single neural network predicts bounding boxes and class probabilities Ø Can be optimized end-to-end directly on detection performance Ø Less computation, execution on PC with GPU card q Image-based algorithm suitable for safeguards surveillance camera data BNL-209172 -2018 -COPR Method SSD 321 R-FCN SSD 513 FPN FRCN Retina. Net-50 -500 Retina. Net-101 -800 YOLOv 3 -416 YOLOv 3 -608 m. AP-50 45. 4 51. 9 50. 4 59. 1 50. 9 53. 1 57. 5 55. 3 57. 9 Time 61 85 125 172 73 90 198 29 51 • m. AP: mean average precision • FPS: frames per second • Reference: Joseph Redmon, Ali Farhadi, “YOLOv 3: An Incremental Improvement”, CVPR 2016, ar. Xiv: 1804. 02767.

6 Transfer Learning Solves the issue of insufficient domain training data q Re-training requires much less computational resource. Some trainings were done on a laptop in this project. q Pre-training Accuracy assessment public domain image set domain test data Results Source: wikipedia Large data set (1+ millions) Transfer parameters Replace the layers Re-training domain training data Small data set (a few hundreds) BNL-209172 -2018 -COPR

7 Test Facilities and Object Classes BNL-209172 -2018 -COPR

8 Prediction Results – Example Images Once object detection is implemented, object-based motion detection can be done easily. BNL-209172 -2018 -COPR

9 Quantitative Results Precision Recall Class: drum AP: 78. 16% Precision Class: white-container AP: 85. 42% Precision Class: yellow-box AP: 92. 3% Recall q Precision and recall values vary slightly for different objects of interest. q Optimal threshold setting in algorithm gives balanced performance. q The threshold can be tuned for a specific object. BNL-209172 -2018 -COPR

10 Summary We proposed to use deep machine learning technique to improve the image-review process of video surveillance in international nuclear safeguards. q We adapted the YOLO neural network for this application and demonstrated its usability in laboratory and operational environments. q The preliminary results show good performance of the algorithm. q BNL-209172 -2018 -COPR

11 Acknowledgement We thank the U. S. Department of Energy, National Nuclear Security Administration, Office of International Nuclear Safeguards (NA-241) as the primary sponsor of the project. q We also thank the following staff members at BNL and SNL for consultation and facility operation. q Ø BNL: Joe Carbonaro, Bob Mc. Nair, Glen Todzia, Edward Richards Ø SNL: Mary Arnhart, Phillip Kay, Donalk Hanson Questions ? BNL-209172 -2018 -COPR