Comparison of complex background subtraction algorithms using a

Comparison of complex background subtraction algorithms using a fixed camera Geoffrey Samuel Ph. D Researcher Intelligent Systems and Robotics Research Group (ISR) Creative Technologies University of Portsmouth

Comparison of complex background subtraction algorithms using a fixed camera Intro � Background subtraction is a important and vital step for computers to understand interpreter a real-world scene � It allows a computer to ignore a background so to concentrate on a foreground object Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Hypothesis � Each background subtraction algorithm will have its advantages and disadvantages, and that looking and comparing these with a real-world situation, it would be possible to pick one algorithm or a method of combining algorithms to produce a algorithm capable of balancing speed with quality. Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera The Goal � Test and evaluate the quality and speed of existing background subtraction algorithms on a complex background with different everyday motions, and to compare the results with those of the extracted “Ground Truth” Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Complex Background Static Background: Background does not contain any secondary “unwanted” motion. Controlled environment. Complex Background: Background contains secondary “unwanted” motion such as the winds effect on trees or blinds. Real-world data. Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Synthetic Test Data Advantages: • Automatically got the “Ground Truth”. • More control over each test clip. Disadvantages: • Manual frame by frame “Ground Truth” extraction. • Added artefacts from the Chroma keying and compositing. Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera The Experiment � To Create a set of synthetic data with the “Ground Truth” � To test different motions with each background subtraction algorithm � To Compare the results of each algorithm with that of the “Ground Truth” Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera The Motions � 7 everyday motions were chosen: �Drinking �Jogging �Picking up wallet �Scratching head �Sitting down �Standing up �Walking � Each motion started on the left of the screen and concluded on the right. Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Creating the test scenarios Green Screen with actor Back Ground Final Composite Geoffrey Samuel “Ground Truth” www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera The Algorithms Back Plate Difference │framei – backplate│>Ts 50 Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera The Algorithms Frame Difference │framei – framei-1│>Ts 50 Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera The Algorithms Approximate median (x = ( framei - framei-1 – framei-2. . . framei-n ) > Ts ) → {background += 1} → {background -= 1} Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera The Algorithms Mixture of Gaussians frame(it = μ) = Σi=1 ωi, t. ț(μ, o) k Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Measuring the Quality Compare the Per-Pixel value of each frame with the “Ground Truth” (0, 576) (0, 0) Geoffrey Samuel (768, 576) (768, 0) (0, 576) (0, 0) (768, 576) (768, 0) www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Results - Quality Test Motions Backplate Difference Frame Difference Approximate Median Mixture of Gaussian % of image # of pixels Drinking 90. 78% 401577. 3019 82. 12% 363282. 5031 89. 52% 396024. 7107 83. 78% 370625. 2327 Jogging 88. 24% 390349. 3529 88. 88% 393194. 9412 92. 14% 407602. 3824 88. 20% 390146. 7941 Picking up Wallet 91. 26% 403717. 114 88. 22% 390256. 9035 83. 40% 368940. 5088 90. 19% 398979. 9737 Scratch head 88. 18% 390065. 7255 84. 87% 375422. 2549 90. 56% 400599. 9216 86. 15% 381117. 049 Sitting down 88. 51% 391528. 6796 80. 07% 354204. 932 82. 28% 363994. 2039 81. 68% 361327. 3981 Standing up 89. 40% 395491. 6311 83. 82% 370787. 165 80. 99% 358290. 4563 83. 78% 370631. 6893 Walking 88. 47% 391373. 5094 89. 81% 397309. 3396 94. 22% 416820. 1321 90. 01% 398195. 3396 Most correct pixels Geoffrey Samuel Most incorrect pixels www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Results - Quality Percent of correctly identified pixels Average Percent of correctly identified pixels per frame 100. 00% 95. 00% 90. 00% Backplate Difference 85. 00% Frame Difference Approximate Median 80. 00% Mixture of Gaussian 75. 00% 70. 00% Drinking Jogging Picking up Wallet Scratch head Sitting down Standing up Walking Test Motions Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Results - Speed Test Motions Backplate Difference Frame Difference Approximate Median (Average of 100 times) Mixture of Gaussian Drinking 0. 0507 0. 0004 0. 3301 10. 6954 Jogging 0. 0507 0. 0025 0. 0691 10. 8219 Picking up Wallet 0. 0492 0. 0819 0. 0730 12. 2895 Scratch head 0. 0450 0. 0850 0. 0718 10. 6132 Sitting down 0. 0420 0. 0692 0. 0662 10. 8503 Standing up 0. 0416 0. 0747 0. 0529 12. 7196 Walking 0. 0319 0. 0129 0. 0541 10. 5202 “Fastest” Algorithm Geoffrey Samuel “Slowest “Algorithm www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Results - Speed Average processing time per frame in Seconds (run 100 times) Average time to process per frame 14. 0000 12. 0000 10. 0000 8. 0000 Backplate Difference Frame Difference 6. 0000 Approximate Median Mixture of Gaussian 4. 0000 2. 0000 0. 0000 Drinking Jogging Picking up Wallet Scratch head Sitting down Standing up Walking Test Motions Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Results - Speed. . . now ignoring the Mixture of Gaussian speed results Average processing time per frame in Seconds (run 100 times) Average time to process per frame 0. 3500 0. 3000 0. 2500 0. 2000 Backplate Difference 0. 1500 Frame Difference Approximate Median 0. 1000 0. 0500 0. 0000 Drinking Jogging Picking up Wallet Scratch head Sitting down Standing up Walking Test Motions Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Conclusion � Backplate difference was the fastest and produce the highest results in 4 out of 7 tests. � Frame difference was the ONLY algorithm to correctly remove the complex background, but couldn't correctly identify the foreground element. Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Conclusion Frame Difference : Correctly Removed Complex Background Incorrectly Removed inside of Subject Backplate Difference : Correctly Identified Subject Incorrectly kept Complex Background Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Taking it further A new method that incorporated both the speed of updating to remove the background and yet the knowledge of the background to properly extract the wanted foreground element. Theory Framework idea: ƒ Frame Difference Geoffrey Samuel Backplate Difference Complex background removed www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Where can this lead? � Application of this technology could be used in: �Games �Surveillance �Mesh reconstruction and silhouette extraction �Various computer vision tasks Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Any Questions? Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Acknowledgments � UK Engineering and Physical Science Research Council � Seth Benton for his Matlab code Geoffrey Samuel www. Geoff. Samuel. com

Comparison of complex background subtraction algorithms using a fixed camera Thank you for your time � Geoffrey. Samuel@Port. ac. uk � www. Geoff. Samuel. com Geoffrey Samuel www. Geoff. Samuel. com