RANSACAssisted Display Model Reconstruction for Projective Display Patrick

RANSAC-Assisted Display Model Reconstruction for Projective Display Patrick Quirk, Tyler Johnson, Rick Skarbez, Herman Towles, Florian Gyarfas, Henry Fuchs Department of Computer Science University of North Carolina at Chapel Hill Emerging Display Technologies 2006 – March 26, 2006

Multi-Projector Display 2 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Perspectively Correct P 3 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Perspectively-correct Rendering P matrices Intrinsics and extrinsics of each projector Viewer’s location Display Surface Model New Method for Display Surface Estimation! 4 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Rendering Methods Rendering Algorithm 1 -pass (homography) 2 -pass (projective tex. ) Display Shape Planar surface Complex shaped surface References Raskar 99, Yang 01, Sukthankar 01, Chen 02, Steele 02, Wallace 04 5 Raskar 98, Yang-Welch 01, Brown 02, Cotting 04 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

1 st Pass A Primer: 2 -Pass Rendering Geometry: application defined Viewpoint: user’s position Result: “ideal image” 2 nd Pass Geometry: screen surface • Use “ideal image” as projected texture Viewpoint: projector’s position Result: projector’s image Display Surface Viewer Projector 6 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Display Surface Estimation Display Surface Model Generation 3 D Stereo Reconstruction 3 D Point Cloud 7 Tessellated Mesh RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Artifacts with Tessellated Meshes Inaccurate corner representations Sampling issue Texture mapping distortion on planar surfaces Reconstruction errors Holes in surface model Meshing errors 8 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Visual Artifacts 9 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display P

Rooms Are Piecewise Planar New display model estimation method Fit planes to point cloud Convert planes to polygons for rendering Advantage Corners can be accurately estimated (intersection of planes) Noiseless models eliminate texture distortion 10 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

RANSAC Plane Fitting Random Sample Consensus Designed to work with many outliers Finds largest set of inliers Code from Peter Kovesi, Univ. Western Australia Hypothesis plane from 3 random points Finds plane with most inliers User specified fit tolerance Fits least-squares plane to these inliers 11 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Method Fit. Plane( C ) Fit one plane P to point cloud C using RANSAC Remove P inlier points from cloud C’ = C – Pinliers Recursively loop using outliers from this pass as input for next plane 12 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

RANSAC Plane Fitting 13 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane To Polygons Task: What planes intersect? Reduce complexity to a 2 D problem Establish a “floor” plane orthogonal to the “wall” planes Project wall points onto this plane Casts problem as a line intersection problem 14 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Cast into a 2 D Task Intersecting Line (Plane) Pairs Red – Orange – Green – Cyan – Magenta – Black – Blue 15 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Results Less than 2 -3% dimensional error Re-projection: Std. Dev. < 0. 5 pixels 16 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Results P V Improved corner matching (< 2 pixels) Lack of texture distortion in application scenery 17 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Flight. Gear Demo 18 P P RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Installations UNC Lab 3 walls with columns in each corner UNC lobby and I/ITSEC 2005 2 -wall corner Naval Research Lab 3 walls with a column in one corner 19 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

SUMMARY: New Surface Model Generation Method Recursively extract planes from 3 D point cloud with RANSAC-assisted algorithm Removes outliers easily Fits noisy data well Convert Planes to Polygons for 2 -pass rendering Results in simple, accurate model Much less distortion than with tessellated mesh 20 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Future Work Use optimal triangulation algorithm Rather than DLT; less noise in data Generalized Planes to Polygon solution Extract other surface functions Applicability of cylinders, quadrics, etc. Apply techniques to create larger virtual environments Continuous calibration/refinement during operation 21 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Thank You Funding Office of Naval Research Award N 00014 -03 -1 -0589 • DARWARS Training Superiority program • VIRTE – Virtual Technologies and Environments program 22 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

23 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Planes to Polygons 1. Compute intersection line between two planes Align this with Y axis 2. Project down all points to get floorplan Line segments, based on range of inliers 3. Find closest 2 D line intersections Determines which planes to intersect 4. Intersect the planes, make height uniform 5. Truncate non-intersecting planes (edges) 24 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Room Coordinate System Motivation Can easily estimate position of viewer Auto-alignment performed Incorporated into Planes to Polygons algorithm Chosen to be intersection of planes Also have tool to change coordinate system Extends to a tracked user 25 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

3 D Stereo Reconstruction 1. Display feature set with projector(s) Checkerboard, Gray-coded stripes, Gaussian blobs, etc. 2. Image with stereo camera pair 3. Extract, identify, match feature points 4. Triangulate corresponding points Model represented by 3 D point cloud! 26 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane 1 27 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane 2 28 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane 3 29 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane 4 30 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display