Object Detection Object Recognition Object Detection Object Identification
Object Detection
Object Recognition Object Detection Object Identification Where is Rania? Where is a Face? Is there a face in the image? Who is it? Is it Ahmed or Hassan?
a challenge: object perception – Object detection – Object segmentation – Object recognition • Typical systems require human-prepared training data; can we use autonomous experimentation?
a challenge: object perception – Object detection – Object segmentation – Object recognition Fruit detection • Typical systems require human-prepared training data; can we use autonomous experimentation?
a challenge: object perception – Object detection – Object segmentation – Object recognition Fruit segmentation • Typical systems require human-prepared training data; can we use autonomous experimentation?
a challenge: object perception – Object detection – Object segmentation – Object recognition Fruit recognition • Typical systems require human-prepared training data – can’t adapt to new situations autonomously
Object Detection • Find the location of an object if it appear in an image – Does the object appear? – Where is it?
Face Detector
Face detection ? +1 face features x classify F(x) • We slide a window over the image • Extract features for each window • Classify each window into face/non-face -1 not face y
Classification + + + - • Examples are points in Rn • Positives are separated from negatives by the hyperplane w + w - • y=sign(w. Tx-b) + - - -
Classification + + + - • • + w - + - y=F(x, ) • - parameters of F, e. g. =(w, b) - - x Rn - data points P(x) - distribution of the data y(x) - true value of y for each x F - decision function: • We want F that makes few mistakes
Loss function + + POSSIBLE CANCER + + - • Our decision may have severe implications • L(y(x), F(x, )) - loss function + w - How much we pay for predicting F(x, ), when the true value is y(x) + ABSOLUTELY NO RISK- OF CANCER - • Classification error: • Hinge loss
Face Detection – basic scheme Face examples Classification Result Off-line training Classifier Feature vector (x 1, x 2 , …, xn) Feature Extraction Non-face examples Pixel pattern Search for faces at different resolutions and locations
14
Feature Engineering or Feature Learning? • In Vision: SIFT, HOG, pixels, sparse coding, RBM, autoencoder, LCC, scattering net (Mallat), deep conv net (discriminative feature learning), etc. • In NLP: N-gram, hashing, XXX, YYY, ZZZ etc. • In Speech: MFCC’s, PLPs, SPLICE (for noise robustness), autoencoder, scattering spectra, learned mapping from filterbank to MFCCs, DNN, etc. 15
Training and Testing Training Set Train Classifier False Positive Labeled Test Set Correct Sensitivity Classify
Learning Components Start with small initial regions Expand into one of four directions Extract new components from images Train SVM classifiers Choose best expansion according to error bound of SVMs
Some Examples
What Types of Problems Fit (not fit) Deep Learning (some conjectures) “Perceptual” AI e. g. : Image/video recognition Speech/text understanding Sequential data with temporal structure (stock market prediction? ) Non-obvious data representations Deep learning already shows tremendous benefits “Data matching” e. g. : Malware detection(ICASSP-2013) movie recommender, speaker/language detection? Easy data representation e. g. , histogram of events, user-watched movies, etc. Deep learning may not win over standard machine learning
Window-based models Generate and score candidates • Scans the detector at multiple locations and scales face/non-face Classifier
Haar-features (1) • The difference between pixels’ sum of the white and black areas
Haar-features (2) • Capture the face symmetry
Haar-features (3) Type A Four types of haar features Can be extracted at any location with any scale! A 24 x 24 detection window
Integral image (1) Sum of pixel values in the blue area Example: 2 1 2 3 4 3 3 2 1 2 2 3 4 2 1 1 1 2 Image Time complexity? 2 3 5 8 12 15 5 8 11 16 22 28 9 14 18 24 31 39 Integral image
Integral image (2) 1 3 a c 2 4 b d a = sum(1) b = sum(1+2) c = sum(1+3) d = sum(1+2+3+4) Sum(4) = ? d + a – b – c Four-point calculation! A, B: 2 rectangles => 6 -point C: 3 rectangles => 8 -point D: 4 rectangles => 9 -point
Feature selection • A weak classifier h f 1 f 2 f 1 > θ (a threshold) => Face! f 2 ≤ θ (a threshold) => Not a Face! h= 1 if fi > θ 0 otherwise
Feature selection • Idea: Combining several weak classifiers to generate a strong classifier α 1 ~performance of the weak classifier on the training set α 2 α 3 …… αT α 1 h 1+ α 2 h 2 + α 3 h 3 + … + αTh. T >< Tthresold weak classifier (feature, threshold) h 1 = 1 or 0
K Nearest Neighbors + + - + + +o - - + - + - - - • Memorize all training data • Find K closest points to the query • The neighbors vote for the label: Vote(+)=2 Vote(–)=1
K-Nearest Neighbors (silhouettes) Kristen Grauman, Gregory Shakhnarovich, and Trevor Darrell, Virtual Visual Hulls: Example-Based 3 D Shape Inference from Silhouettes
K-Nearest Neighbors Silhouettes from other views 3 D Visual hull Kristen Grauman, Gregory Shakhnarovich, and Trevor Darrell, Virtual Visual Hulls: Example-Based 3 D Shape Inference from Silhouettes
Support vector machines + m ar - gi + + w n + + + - - - + - + - - - • Simple decision • Good classification • Good generalization
Support vector machines + + + w + - + + - - - + - + - - - Support vectors:
Slides by Pete Barnum Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR 05
centered diagonal uncentered cubic-corrected Slides by Pete Barnum Sobel Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR 05
Neural Network based FD Cited from “Neural network based face detection”, by Henry A. Rowley, Ph. D. thesis, CMU, May 1999 EE 465: Introduction to Digital Image Processing Copyright Xin 35
Vehicle Detection • Intelligent vehicles aim at improving the driving safety by machine vision techniques 36
Chain Code
Chain Code
This example shows that the Chain Code is independent of Location, Starting Point and orientation
Segmentation • Segmentation – Roughly speaking, segmentation is to partition the images into meaningful parts that are relatively homogenous in certain sense
Segmentation by Fitting a Model • One view of segmentation is to group pixels (tokens, etc. ) belong together because they conform to some model – In many cases, explicit models are available, such as a line – Also in an image a line may consist of pixels that are not connected or even close to each other
Canny and Hough Together
Hough Transform • It locates straight lines • It locates straight line intervals • It locates circles • It locates algebraic curves • It locates arbitrary specific shapes in an image – But you pay progressively for complexity of shapes by time and memory usage
Hough Transform for circles * You need three parameters to describe a circle ** * * Vote space is three dimensional
First Parameterization of Hough Transform for lines
Hough Transform – cont. • Straight line case – Consider a single isolated edge point (xi, yi) • There an infinite number of lines that could pass through the points – Each of these lines can be characterized by some particular equation
Line detection • Mathematical model of a line: y Y = mx + n Y 1=m x 1+n Y 2=m x 2+n P(x 1, y 1) P(x 2, y 2) x YN=m x. N+n
Image and Parameter Spaces y Y = mx + n Y = m’x + n’ slope Y 1= m x 1+ n Y 2=m x 2+ n m’ n’ x Image Space YN =m x N + n m n intercept Parameter Space Line in Img. Space ~ Point in Param. Space
Looking at it backwards … Parameter space Y 1= m x 1+ n Can be re-written as: Fix (-x 1, y 1), Vary (m, n) - Line m’ slope n’ m n intercept n = -x 1 m + Y 1
Least squares line fitting • Data: (x 1, y 1), …, (xn, yn) • Line equation: yi = m xi + b • Find (m, b) to minimize y=mx+b (xi, yi) Matlab: p = A y; Modified from S. Lazebnik
Hough Transform – cont. • Hough transform algorithm 1. Find all of the desired feature points in the image 2. For each feature point For each possibility i in the accumulator that passes through the feature point Increment that position in the accumulator 3. Find local maxima in the accumulator 4. If desired, map each maximum in the accumulator back to image space
HT for Circles • Extend HT to other shapes that can be expressed parametrically • Circle, fixed radius r, centre (a, b) – – (x 1 -a)2 + (x 2 -b)2 = r 2 accumulator array must be 3 D unless circle radius, r is known re-arrange equation so x 1 is subject and x 2 is the variable – for every point on circle edge (x, y) plot range of (x 1, x 2) for a given r
Hough Transform – cont. Here the radius is fixed
Hough circle Fitting
Hough circle Fitting
- Slides: 55