Convex Quadratic Programming for Object Location Hao Jiang

Convex Quadratic Programming for Object Location Hao Jiang, Mark S. Drew and Ze-Nian Li School of Computing Science Simon Fraser University

Introduction v Object localization is an important task in computer vision Template 2

Object localization and labeling v Object localization can be formulated as labeling problems. Site p Neighboring relation c(p, fp) Label fp (q-p) (fq-fp) q fq Template c(q, fq) v Consistent labeling Ø Find small cost label assignment. Ø Enforce labeling consistency of neighboring sites. 3 Target Object

Previous methods for consistent labeling v Consistent labeling, in general form, is NP-hard. v Polynomial time schemes exist for special cases Ø Dynamic Programming (DP). Ø Max-flow [Ishikawa 2000, Roy 98]. v Approximation schemes Ø Greedy (local searching) methods q Relaxation labeling (RL) [Rosenfeld 76]. q Iterated Conditional Modes (ICM) [Besag 86]. Need good initialization and easily trapped in local minimum. Ø Global searching methods q Graduated Non-Convexity (GNC) [Blake & Zisserman 87] q Belief Propagation (BP) [Pearl 88, Weiss 2001]. q Graph Cut (GC) [Boykov & Zabih 2001]. 4

Focus of the research v Previous methods become slow as the number of labels goes to the order of several thousand. (# of labels can be in the order of thousand or bigger) Many vision problems such as object matching, large scale motion, tracking etc will benefit from the solver. (# of labels in the order of hundred) Consistent labeling 5 Hard to solve Well solved The focus of the research

The trick of the proposed scheme v In stead of working on the original label space, we represent the label set with a small number basis labels. v We convert the hard problem into a sequence of simpler problems built using only the basis labels. v In this way, the size of the approximation problem is largely decoupled from the original label set. v Each sub-problem is a convex problem and can be globally optimized. v A successive relaxation implementation is used to zero in the target. 6

The non-linear optimization problem p (q-p) c(p, fp) q fp (fq-fp) fq c(q, fq) v The labeling problem can be solved by optimizing: 7

Convex relaxation v To convert the labeling cost term into linear functions, for each site, we define a basis label set. Ø Each label can then be represented as a linear combination of the basis labels. Ø The cost of the label is approximated by the linear combination of the costs of these basis labels. c(s, t) fs = a * J 1 + b * J 2 c(s, fs) = a * c(s, J 1) + b * c(s, J 2) a+b=1 J 1 fs t J 2 8

Convex relaxation (Cont’) c(s, j 1)xs, j + c(s, j 2) xs, j + c(s, j 3) xs, j + c(s, j 4) xs, j + c(s, j 5) xs, j 1 2 3 4 xs, j 1+ xs, j 2+ xs, j 3+ xs, j 4+ xs, j 5 =1 j 2 c(s, j 2) j 3 c(s, j 1) j 1 c(s, j 3) 5 j 5 c(s, j 4) c(s, j 5) j 4 j 1 xs, j + j 2 xs, j + j 3 xs, j + j 4 xs, j + j 5 xs, j 1 2 3 4 5 v The L 2 norm smoothness terms do not need additional conversions. 9

Convex quadratic program (CQP) We have the convex quadratic program: 10

In some cases, the CQP is exact v If x are binary numbers, the mixed integer program is exactly equivalent to the original problem. v If c(s, t) is convex over t for each s 2 S, the CQP is equivalent to the continuous extension of the original problem. c(s, t) Continuous extension t t Feasible solution 11

In general it is an approximation v For general problems, the CQP approximates each labeling cost surface with the 3 D lower convex hull. Label cost c(s, t) t =(x, y) y x The convexified surfaces are much simpler than the original ones. 12

The CQP can be greatly simplified v The most compact basis labels correspond to the lower convex hull vertices. # of original label: 400 # of basis label: 20 v This shows a way to simplify the label set in labeling. v We can safely discard many labels without worrying about the problems met in previous methods. 13

Successive convexification To improve the approximation we use the successive convexification (SC) scheme as follows: 14

Experimental results Matching Random Dots 15

Experimental results (Cont’) Matching Random Dots 16

Experimental results (Cont’) Matching Leaf 17

Experimental results (Cont’) Matching Face 18

Experimental results (Cont’) Matching Hand 19

Conclusion v Set out an object localization method which can deal with textureless objects in strong background clutter. v Propose a successive convex quadratic method v CQP large decouples the number of target image points with the size of the convex program – It searches the whole target image quickly. v Successive convexification is studied to improve the result recursively. v Successive convexification is a general method can be applied to any convex regularization problem. 20

Thank you! 21

In summery v Label space is approximated with small number of basis labels. v Original problem is converted to a sequence of much easier convex programs and solved by successive relaxation. Matching Costs Lower Convex Hull Template Target Image Convexification Trust Region Shrinking v The size of the convex program is largely decoupled with the number of candidate labels 22