Dynamic View Morphing performs view interpolation of dynamic

Dynamic View Morphing • performs view interpolation of dynamic scenes

Expanded Theory • orthography • methods for finding camera-to-camera transformation • virtual camera not restricted to line connecting original cameras • “weak rectification” is sufficient for physical correctness • appearance of straight-line motion without camera-to-camera transformation

motion from time=0 to time=1, as seen through A

For Orthographic Projection physically correct straight-line motion (because motion vectors aligned) constant-velocity motion (because motion vectors identical)

For Perspective Projection • IF first make image planes parallel to: – motion of object, and – each other • THEN orthographic results apply • condition above is “weak rectification”

A time = 0 B time = 1 camera views related by fundamental matrix F

time = 1 time = 0 A B camera views still related by same fundamental matrix F

A time = 0 B time = 1

A B each object W has its own fundamental matrix FW

Camera-to-camera transformation • denoted TAB • once known, view interpolations portray “constant velocity” motion • potential for model building

Finding TAB • can be determined from fundamental matrices for two distinct objects • can be determined from four conjugate directions • can be approximated from two conjugate directions

A B each object W has its own fundamental matrix FW

Environment Map • “environment map” or “panoramic mosaic” or “plenoptic function”: all the light that reaches a given point in space at an instant in time

Environment Map Morphing • view morphing for environment maps time=0. 0 time=0. 4 time=1. 0

Environment Map Morphing • (STEP 1) find fundamental matrix • (STEP 2) “strongly rectify” the views that is, make TBA = a b c 0 1 0 0 0 1 then notice that, for any point in space, camera A and camera B will give the same y and z coordinates

Environment Map Morphing • (STEP 3) project environment map onto “image cylinder” (a. k. a “pipe”) this is the cylinder y 2 + z 2 = 1 • (STEP 4) interpolate conjugate points and morph

z = 1 “image plane” y 2 + z 2 = 1 “image cylinder”

Interpolating Augmented Views

Benefits • placing synthetic object over real object – segmentation – point correspondences – camera-to-camera transformation – added realism: moving parts, shadows, transparency, don’t morph synthetic object – can also use real object views instead of a synthetic object

Benefits • automation – by matching edges, computer can place model automatically – all previous benefits become automated • scenario visualization – combine synthetic objects with real scenes to create new scenarios

cylinder y 2 + z 2 = 1

TBA x after applying TBA B A and B = A

Outline • layering; static scenes, improvement • orthography • generalization of math for view morphing • making objects appear to follow line • Tab and how to find

Layering

Underlying Mathematics • “weak” rectification: image planes parallel • virtual movement not restricted to line

Orthography • long-distance photography • no prewarps needed! (physical correctness) • straight-line motion by aligning directions

Preconditions/Output

Appearance of Straight-line Motion

Orthographic Projection physically correct straight-line motion constant-velocity motion A B

TBA x B = A

t=1 t=0 B took this view A took this view after applying TBA A and B

A B physically correct straight-line motion constant-velocity motion