Distributed Algorithms for Camera Sensor Networks Ren Vidal
Distributed Algorithms for Camera Sensor Networks René Vidal Center for Imaging Science Johns Hopkins University
Image and Video Analysis Today 24 hrs/min 30 million cameras 4 bn hrs/week Tasks Application Areas End Users Detection, Segmentation, Motion Estimation, Stabilization, Registration, Categorization National Security Recreation Governments Consumer
Camera Networks • Classical approach – One camera wired to one computer – Multiple cameras wired to a central processing unit – Processing than by a human operator or by a central computer • Problems – Flexibility: Wiring makes it hard to deploy new cameras – Robustness: central node failure = entire system failure – Scaling: processing and bandwidth requirements do not scale well
Camera Sensor Networks • Motes – Small, wireless devices – battery powered – limited memory and computing power • Applications – Surveillance – Environmental monitoring – Smart homes Question Can we deploy vision algorithms on camera sensor networks?
Challenges to Computer Vision Algorithms Traditional computer vision algorithms Existing algorithms are centralized: all images are sent to one node for processing Computer Vision • Sensor networks have limited resources – Limited processing power and memory – Slow wireless channel – Nodes can have limited communication range Challenge Traditional computer vision algorithms require resources not available in a camera sensor network
Challenges to Sensor Network Algorithms Traditional distributed algorithms for sensor networks Existing algorithms have been designed for processing simple scalar measurements Sensor Networks • In computer vision applications – Measurements (images) are high-dimensional – Measurements are corrupted by noise, outliers – Estimates are non-Euclidean (e. g. rotations) Challenge Traditional sensor network algorithms cannot be directly used for computer vision applications
Toward Distributed Computer Vision Algorithms • Centralized algorithms • Considerable complexity Computer Vision Sensor Networks • Distributed algorithms • Only simple problems Our goal • Develop distributed computer vision algorithms that – Are efficient: local processing + short communications – Converge to the centralized solution – Can handle outliers, packet losses, data on manifolds
Outline I. Review of existing distributed algorithms – Spanning tree algorithm – Belief propagation algorithm – Consensus algorithms II. Distributed linear computer vision algorithms – Triangulation and pose estimation – Distributed Principal Component Analysis – Affine structure from motion III. Distributed nonlinear computer vision algorithms – Pose estimation – Camera localization – Distributed face recognition IV. Generalized consensus algorithms – Extension to measurements with outliers – Extension to handle packet losses
Part I: Review of Existing Algorithms
Existing Distributed Algorithms: Notation • Assume each node has an initial scalar measurement and a state • Represent the network with communication graph Nodes Links Goal Estimate states from data
Existing Distributed Algorithms: Spanning Tree • The simplest algorithm one can imagine Spanning tree algorithm 1. 2. 3. 4. Find a spanning tree for the network Send values from leaves to nodes The root computes desired quantity Propagate the result down the tree Problems Discovering and maintaining a spanning tree might be difficult
Existing Distributed Algorithms: Belief Propagation • Find estimates using more complex statistical models Markov graphical model Estimates Measurements Unary potentials Binary potentials
Existing Distributed Algorithms: Belief Propagation • Find estimates using more complex statistical models Markov graphical model Belief propagation • Pass messages between neighboring nodes • Upon convergence, each node gets the marginal Problem • Convergence guaranteed only for some types of graphs
Existing Distributed Algorithms: Consensus Particular case: averaging consensus Minimize sum of square differences Gradient descent Mean is preserved Global minimum at consensus
Existing Distributed Algorithms: Consensus Convergence All states converge to the average when the graph is connected
Existing Distributed Algorithms: Comparison Spanning Tree General cost functions Belief Propagation Consensus Yes/No Averaging, min/max General graphs No Acyclic Some cyclic Connected Changes in topology No ? Yes Our research Extend consensus algorithms for solving problems beyond averaging and min/max
Generalized Consensus Algorithms • Linear problems: least squares and PCA – R. Tron and R. Vidal. “Distributed Computer Vision Algorithms Through Distributed Averaging. ” IEEE Conference on Computer VIsion and Pattern Recognition, pages 57 - 63, 2011 • Nonlinear problems: consensus on manifolds – R. Tron, R. Vidal, and A. Terzis. “Distributed Pose Averaging in Camera Sensor Networks via Consensus on SE(3)”. International Conference on Distributed Smart Cameras, 2008 • Applications in computer vision – – – R. Tron and R. Vidal. “Distributed Face Recognition via Consensus on SE(3). ” Workshop on Omnidirectional Vision, European Conference on Computer Vision, 2008 E. Elhamifar and R. Vidal. “Distributed Calibration of Camera Sensor Networks”. International Conference on Distributed Smart Cameras, 2009 R. Tron and R. Vidal. “Distributed Image-Based 3 -D Localization of Camera Sensor Networks”. IEEE Conference on Decision and Control, 2009 • Nonlinear problems: robustness to outliers – J. Li, E. Elhamifar, I-J. Wang and R. Vidal. “Consensus with Robustness to Outliers via Distributed Optimization. ” Conference on Decision and Control, 2010 • Robustness to packet losses – Y. Chen, R. Tron, A. Terzis and R. Vidal. “Corrective Consensus: Converging to the Exact Average. ” IEEE Conference on Decision and Control, 2010
Part II: Distributed Linear Computer Vision Algorithms
What can we do with just averaging? • Solve the simplest problems – Linear systems – Singular Value Decompositions Linear triangulation Linear pose recovery Principal Component Analysis Make algorithms based on linear algebraic operations distributed Subspace segmentation (GPCA) Affine Structure from Motion R. Tron and R. Vidal. Distributed Computer Vision Algorithms Through Distributed Averaging. IEEE Conference on Computer VIsion and Pattern Recognition, pages 57 - 63, 2011.
Example: Linear Triangulation Data Images of a point from known perspective cameras Goal Find the 3 -D coordinates of the point R. Tron and R. Vidal. Distributed Computer Vision Algorithms Through Distributed Averaging. IEEE Conference on Computer VIsion and Pattern Recognition, pages 57 - 63, 2011.
How to get a distributed algorithm… Centralized Least Squares Distributed Least Squares R. Tron and R. Vidal. Distributed Computer Vision Algorithms Through Distributed Averaging. IEEE Conference on Computer VIsion and Pattern Recognition, pages 57 - 63, 2011.
Triangulation results • Network of 5 cameras with ring topology Camera 4 Camera 2 Camera 3 Camera 1 Camera 5 R. Tron and R. Vidal. Distributed Computer Vision Algorithms Through Distributed Averaging. IEEE Conference on Computer VIsion and Pattern Recognition, pages 57 - 63, 2011.
Triangulation results Reconstruction from local neighborhood Camera 5 1 2 3 4 R. Tron and R. Vidal. Distributed Computer Vision Algorithms Through Distributed Averaging. IEEE Conference on Computer VIsion and Pattern Recognition, pages 57 - 63, 2011.
Triangulation results All local estimates Iteration 13 10 1 4 7 R. Tron and R. Vidal. Distributed Computer Vision Algorithms Through Distributed Averaging. IEEE Conference on Computer VIsion and Pattern Recognition, pages 57 - 63, 2011.
Triangulation results R. Tron and R. Vidal. Distributed Computer Vision Algorithms Through Distributed Averaging. IEEE Conference on Computer VIsion and Pattern Recognition, pages 57 - 63, 2011.
Part III: Distributed Nonlinear Computer Vision Algorithms
Non-linear problems • Use a camera network to locate an object of interest Goals Localize the object Localize the cameras Calibrate the cameras
Pose estimation • Assume each camera measures the pose of an object • Our goal is to average all the measurement to get a globally consistent estimate • How to average rotations? Riemannian distance Frèchet mean Tron, Vidal, Terzis - “Distributed pose averaging in camera networks via consensus on SE(3)” International Conference on Distributed Smart Cameras 2008
Pose estimation • Direct extension of consensus to manifolds Cost function Update equations Tron, Vidal, Terzis - “Distributed pose averaging in camera networks via consensus on SE(3)” in International Conference on Distributed Smart Cameras 2008
Pose estimation Convergence • Can be applied to most Riemannian manifolds! • Converges to a consensus configuration, – This requires the measurements to be sufficiently close – Conditions depend on the curvature of the manifold • not always equivalent to the global Frèchet mean – We provide an additional algorithm to compute it Tron, Vidal, Terzis - “Distributed pose averaging in camera networks via consensus on SE(3)” in International Conference on Distributed Smart Cameras 2008
Application to Face Recognition • Each node uses appearance-based methods (eigenfaces, tensorfaces) to estimate the pose of a face • Nodes use distributed averaging to find global pose estimate • Weizmann database: 27 people, 5 poses (17 degrees apart), 4 illuminations, 3 expressions R. Tron and R. Vidal. “Distributed Face Recognition via Consensus on SE(3). ” Workshop on Omnidirectional Vision, October 2008
Application to Face Recognition • Average recognition rate and pose error using Tensorfaces R. Tron and R. Vidal. “Distributed Face Recognition via Consensus on SE(3). ” Workshop on Omnidirectional Vision, October 2008
Part IV: Generalized Consensus Algorithms
Generalized Consensus Algorithms Any convex robust loss function Lp loss Huber loss ‘Fair loss’ Particular case: Averaging J. Li, E. Elhamifar, I-J Wang, and R. Vidal. Consensus with Robustness to Outliers via Distributed Optimization. IEEE Conference on Decision and Control, December 2010.
Generalized Consensus Algorithms More General nt The Generalized Consensus Problem ale v i u q E Distributed Dual method of multipliers J. Li, E. Elhamifar, I-J Wang, and R. Vidal. Consensus with Robustness to Outliers via Distributed Optimization. IEEE Conference on Decision and Control, December 2010.
Conclusion Consensus On manifolds Linear triangulation Pose averaging Linear pose recovery Camera localization PCA Subspace segmentation Affine Sf. M
Acknowledgements • Funding • Students – NSF CNS-0834470 “Distributed Sensing via Robust Consensus on Manifolds” – – Roberto Tron Yin Chen Ehsan Elhamifar Jixin Li • Collaborators – Andreas Terzis (JHU) – I-Jeng Wang (APL) Our website http: //vision. jhu. edu
Further reading… Surveys and tutorials • R. Tron and R. Vidal, “Distributed Algorithms for Camera Sensor Networks. ” IEEE Signal Processing Magazine, 2011 • R. Radke, “A Survey of Distributed Computer Vision Algorithms. ” Handbook of Ambient Intelligence and Smart Environments, 2009 • Soro, Heinzelman, “A Survey of Visual Sensor Networks. ” Advances in Multimedia, 2009
- Slides: 38