Multiple View Geometry Ilan Shimshoni Dept of Management

Multiple View Geometry Ilan Shimshoni Dept of Management Information Systems University of Haifa ishimshoni@mis. haifa. ac. il

Examples of Applications • Visual robot navigation • 3 D reconstruction from video • Was there a goal in a game from two video cameras?

What do we know? • • Camera parameters Camera position with respect to the scene Relative camera position (known or constrained) Objects in the scene Their positions Characteristics of objects in the scene Nothing!!!

• The more we know the easier it is. • The more we need to know the harder it is to perform the experiment. • Example stereo reconstruction – Camera parameters known – Relative camera positions known and constrained • Goal: reconstruct scene. Find for each point in the first image its corresponding point in the second image. • How can we make the task easier?

• Another example: scene reconstruction from a video sequence – Camera parameters not known – Camera positions not known – Much easier for the user • Something in between: Scene reconstruction from a set of images taken by a plane (geodesic dept) – Position of plane known quite accurately – High quality camera which has been accurately calibrated

How do we solve problems? • Example: Pose estimation • The algorithm is given: – A model of a 3 D object – The geometric imaging parameters of the camera – An image of the object • Goal: – Calculate accurately the position of the camera with respect to the object

Applications • A robot would like to grab an object with its gripper using a camera which is positioned on the robot • A robot would like to know where it is. The object is the room. • A satellite would like to know its orientation in space from an image it took of the stars. The object is the universe

• The input:

Steps in developing an algorithm • Define f(X, x, ) • Find a method to compute such that f(X, x, ) = 0 • What should we do if there is measurement noise in x? • How do we match x to X? • How do we deal with incorrect matches (outliers)? • Build a full system.

Criminisi

MIT city scanning project • • • Produce image hemispheres Localize hemispheres using GPS Match hemispheres using vision methods Find accurate positions of cameras Reconstruct buildings Reconstruct fine details

3 D reconstruction from image sequence

Cipolla 2008

Elections November 2008