3 D Projections Barel Mashiach Sean Man instructor

3 D Projections Barel Mashiach, Sean Man instructor: Hagit Hel-Or

Overview • Pin-hole model • From 3 D to 2 D • Camera projection • Homogeneous coordinates • Camera calibration • Vanishing points & lines • Perspective cues • Distortions

Pin-Hole camera model Y Z X

Pin-Hole camera model Y image plane Z X optical center

Pin-Hole camera model • images by Abelardo Morell

Overview • Pin-hole model • From 3 D to 2 D • Camera projection • Homogeneous coordinates • Camera calibration • Vanishing points & lines • Perspective cues • Distortions

Loss of information - Parallelism • images from 1 X. com

Loss of information - Length • images from 1 X. com

Loss of information - Angles • images from 1 X. com

Projection properties • Many-to-one: any points along same ray map to same point in image • Points → points • Lines → lines (collinearity is preserved) – But line through focal point projects to a point • Planes → planes – But plane through focal point projects to line • Conic → conic – But not has to be the same conic shape

Kubrick - One-Point Perspective (1928 -1999)

Overview • Pin-hole model • From 3 D to 2 D • Camera projection • Homogeneous coordinates • Camera calibration • Vanishing points & lines • Perspective cues • Distortions

Modeling projection y f z x • To compute the projection P’ of a scene point P, form the visual ray connecting P to the camera center O and find where it intersects the image plane. • All scene points that lie on this visual ray have the same projection in the image

Modeling projection

Modeling projection y f z x • The coordinate system • The optical center (O) is at the origin • The image plane is parallel to xy-plane or perpendicular to the z-axis, which is the optical axis

Modeling projection y f z x • Projection equations • Derived using similar triangles:

Modeling projection P P’ O

Focal Length comparison

From World to Camera coordinates Y Z X world coordinates

From Camera to Image Y image coordinates Z X y x camera coordinates

Sensor Sizes comparison

Homogeneous Coordinates - reminder • Homogeneous Coordinates are used for mapping �� +1 Cartesian coordinates from �� to �� - an increase of one dimension. • from cartesian to homogenous: • from homogenous to cartesian:

Homogeneous Coordinates - properties • Facilitate performing transformations: affine, projective, homography etc. • Scale invariance: no meaning for any �� non-zero scaling factor. • e. g. (1, 2, 3) ≡ (2, 4, 6) ≡ (3, 6, 9). . . are all the same.

Homogeneous Coordinates - properties Image plane

Coordinates transformations • From World to Camera: called Extrinsic Transformation, denoted by the rotation-matrix R and translation vector t. • From Camera to Image: called Intrinsic Transformation, denoted by the matrix K. • together the transformations create the Camera Projection:

Extrinsic Transformation • in non-homogeneous coordinates:

From World to Camera coordinates Y Z X world coordinates

Extrinsic Transformation • in homogeneous coordinates:

Intrinsic transformation O

Intrinsic transformation - Principal Point • Principal point (p): point where principal axis intersects the image plane • Camera coordinate system: origin of the image is at the principal point • Image coordinate system: origin is in the corner • How to go from camera coordinate system to image coordinate system? O

Intrinsic transformation - Principal Point

Intrinsic transformation - Pixel Coordinates • mx pixels per meter in horizontal direction • my pixels per meter in vertical direction

Intrinsic transformation - Pixel Coordinates

Camera transformation 2 D point (3 x 1) Camera to pixel coord. trans. matrix (3 x 3) Perspective projection matrix (3 x 4) World to camera coord. trans. matrix (4 x 4) 3 D point (4 x 1)

Overview • Pin-hole model • From 3 D to 2 D • Camera projection • Homogeneous coordinates • Camera calibration • Vanishing points & lines • Perspective cues • Distortions

Camera calibration

Camera calibration • Given n points with known 3 D coordinates Xi and known image projections xi, we can estimate the camera parameters. Xi xi

Camera calibration

Overview • Pin-hole model • From 3 D to 2 D • Camera projection • Homogeneous coordinates • Camera calibration • Vanishing points & lines • Perspective cues • Distortions

The projection plane • Why do we need homogeneous coordinates? • • Represent points at infinity, homographies, perspective projection, multi-view relationships. What is the geometric intuition? • A point in the image is a ray in projective space.

The projection plane • Each point (��, ��) on the plane is represented by a ray (� • All points on the ray are equivalent: (��, 1) ≡ (�� Image plane

The projection plane Image plane

The projection plane • A line in the image correspond to a plane of rays through the origin. • • All the rays (��, ��) satisfying: �� + � suppose we deal with normalized homogeneous coordinates, we can refer to it as: �� + In vector notation:

The projection plane • we notice that plane. is orthogonal to the corresponding Image plane

Point and line duality • Finding the line joining two points and : and is orthogonal to both and • Finding the intersection point of two lines and : is orthogonal to both Image plane

Vanishing points vanishing point • images from 1 X. com

Vanishing points • Vanishing point: a projection of a point at infinity (ideal point). • Each direction in space has its own vanishing point. • • All lines going in that direction converge at that point. What is the vanishing point of lines parallel to the image plane?

Vanishing points • images from 1 X. com

Vanishing points e e g ma I n pla Vanishing point Ideal point for a ground plane Camera center C Line on the ground plane

Vanishing points • Properties of vanishing points: • Parallel lines have the same vanishing point V. • V is the projection of their intersection (ideal point). • V corresponds to a ray from C to these lines. • Image may have more than one vanishing point. • There is only one vanishing point for each group of parallel lines. ge a Im e n pla Camera center C Vanishing point Line on the ground plane Ideal point for a ground plane

Vanishing Lines • Any set of parallel lines on a plane define a vanishing point, and the union of all these vanishing points is the vanishing line. Different planes defines different vanishing lines.

Vanishing Lines • In particular, all vanishing points of lines parallel to the ground plane are located on a “vanishing line” (horizon line). Note that different planes define different vanishing lines. Camera center C The ground plane

Vanishing Lines • Horizon: vanishing line of the ground plane below) the horizon • • All points at the same height as the camera project to the horizon Provides way of comparing height of objects Camera center C • Points higher (resp. lower) than the camera project above (resp. The ground plane

Computing vanishing points (from lines) • An improvement: use more than two lines and compute the “closest” point of intersection using Least Square approach.

Computing vanishing line Is this parachutist higher or lower than the person taking this picture?

Computing vanishing line Horizon line Vanishing point