Introduction to image and video processing using Matlab

Introduction to image and video processing using Matlab Dr. Heyfa Ammar Slide

Outline 1. 2. 3. 4. 5. 6. Object detection in a static environment using image processing toolbox How to document/publish our code Read and display a video Detection of moving objects using different techniques: foreground detection, motion estimation. Escape behavior detection Face detection and tracking Slide 2

What is digital image processing (DIP)? A DIP is essentially: l Information extraction from an image l Quality enhancement of an image Slide 3

What is a Digital Image? A digital image is a representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels l Slide 4

What is Matlab? l l l High level language Interactive development environment Suitable for: • • Numerical computation Data analysis and visualization Algorithm development and programming … Slide 5

Example 1: Object appearance detection l l l Goal: explore images to detect a new object appearing in a static environment Tool : Image processing toolbox of Matlab Approach: • • Develop using static images Document with a published report Slide 6

Example 1: Object appearance detection (cont. ) Steps 1. Read the images : Background image and the studied image 2. Convert to the gray scale 3. Compute the images difference to extract the new appeared object 4. Remove noise using a morphological operation 5. Get some statistics about the detected region Slide 7

Mathematical morphology l Used on binary and graylevel images for many purposes such as: • • l Noise removal from background Holes filling Filtering out some content … Requirements • • Structural element : disk, square, rectangle, … Appropriate morphological operation: • • • Erosion Dilation Opening* Closing* … Slide 8

Mathematical morphology : Erosion Accommodate the entire structuring elements when its origin is on the border of the original set A Origin of B visits every element of A At each location of the origin of B, if B is completely contained in A, then the location is a member of the new set, otherwise it is not a member of the new set. Slide 9

Mathematical morphology (cont. ) l Slide 10

VIDEO PROCESSING Computer vision toolbox of Matlab Slide 11

Image processing (IP) and computer vision (CV) l l l IP and CV are similar, but CV typically solves a more complicated problem. IP manipulates images, and possibly extracts some basic information from the image. CV extends IP to “interpret” or “understand” a real-world scene, often to make a decision. Image processing Remove noise Adjust contrast …. Computer vision Detect Identify Classify Recognize Track … Interpretation Pedestrian Car Accident … Slide 12

Before examples in Matlab … § “Vision” package with for computer-vision related capabilities § System Objects : Class of Matlab objects with required methods - step, reset, release, clone, … % initialize system objects reader = vision. Video. File. Reader (‘video. mpg’); display= vision. Video. Player(); % processing while ~is. Done(reader) Rule: Initialize then step. . . image=step(reader); step(display, image); end Slide 13

Example 2 : Read and display a video Some instructions in Matlab… Slide 14

Example 3 : People detection l Algorithm details • • Detects unoccluded people in the upright position Uses histogram of oriented gradients (Ho. G) features and trained Support Vector Machine (SVM) classifier Slide 15

Example 4: Object detection using Gaussian Mixture Models l l Goal : Foreground detection using GMM Algorithm • • Each pixel is modeled as a mixture of Gaussians to determine whether or not a pixel is part of the background Label connected foreground pixels Slide 16

Example 4: Object detection using Gaussian Mixture Models l l Goal : Foreground detection using GMM Algorithm • • Each pixel is modeled as a mixture of Gaussians to determine whether or not a pixel is part of the background Label connected foreground pixels Slide 17

Example 5: Object detection using Gaussian Mixture Models l l Goal : Foreground detection using GMM Algorithm • • • Each pixel is modeled as a mixture of Gaussians to determine whether or not a pixel is part of the background Label connected foreground pixels Reject small blobs Slide 18

Example 6: Motion estimation l l Goal: explore a video to track objects based on their motion Algorithm • • Optical flow estimation Filter out motion vectors with small magnitudes Slide 19

Example 7: Escape behavior detection l l Goal: Detect an escape behavior based on the motion vectors Algorithm • • • Optical flow estimation Calculate the sum of the pixels velocities in each image of the video Detection through thresholding Slide 20

Face detection and tracking l l Goal: detect a single or multiple faces and track them Algorithm • • • Cascade object detector Features detection KLT algorithm for tracking Slide 21