Outline Realtime StructuredLight Range Scanning of Moving Objects
Outline • Real-time Structured-Light Range Scanning of Moving Objects 11/27/2020 Computer Vision 1
Generating 3 D Models 11/27/2020 Computer Vision 2
Epipolar Constraints 11/27/2020 Computer Vision 3
Stereopsis • Difficulties – Establishing dense correspondence 11/27/2020 Computer Vision 4
Triangulation Object Laser Camera • Project laser stripe onto object 11/27/2020 Computer Vision 5
Triangulation Object Laser (x, y) Camera • Depth from ray-plane triangulation 11/27/2020 Computer Vision 6
Triangulation • Faster acquisition: project multiple stripes • Correspondence problem: which stripe is which? 11/27/2020 Computer Vision 7
Continuum of Triangulation Methods Multi-stripe Multi-frame Single-stripe Single-frame Slow, robust 11/27/2020 Fast, fragile Computer Vision 8
One-Shot Active 3 D Shape Acquisition • M. Proesmans, et. al. , ICPR, 1996. 11/27/2020 Computer Vision 9
One-Shot Active 3 D Shape Acquisition – cont. 11/27/2020 Computer Vision 10
One-Shot Active 3 D Shape Acquisition – cont. 11/27/2020 Computer Vision 11
One-Shot Active 3 D Shape Acquisition – cont. 11/27/2020 Computer Vision 12
One-Shot Active 3 D Shape Acquisition – cont. 11/27/2020 Computer Vision 13
One-Shot Active 3 D Shape Acquisition – cont. 11/27/2020 Computer Vision 14
One-Shot Active 3 D Shape Acquisition – cont. 11/27/2020 Computer Vision 15
Shape. Snatcher • A system was developed that can change any digital camera into a 3 D one 11/27/2020 Computer Vision 16
Shape. Snatcher 11/27/2020 Computer Vision 17
Shape. Snatcher 11/27/2020 Computer Vision 18
Fast 3 D Scan Technologies 11/27/2020 Computer Vision 19
Fast 3 D Scan Technologies 11/27/2020 Computer Vision 20
A 4 Vision. com 11/27/2020 Computer Vision 21
Time-Coded Light Patterns • Assign each stripe a unique illumination code over time [Posdamer 82] Time Space 11/27/2020 Computer Vision 22
Codes for Moving Scenes • Assign time codes to stripe boundaries • Perform frame-to-frame tracking of corresponding boundaries – Propagate illumination history Illumination history = (WB), (BW), (WB) [Hall-Holt & Rusinkiewicz, ICCV 2001] 11/27/2020 Computer Vision Code 23
Stripe Boundaries 11/27/2020 Computer Vision 24
Designing a Code • Want many “features” to track: lots of black/white edges at each frame • Try to minimize ghosts – WW or BB “boundaries” that can’t be seen directly 11/27/2020 Computer Vision 25
Designing a Code 0000 1101 1010 1111 0010 0101 1000 1011 0110 0001 0100 1001 1110 0011 [Hall-Holt & Rusinkiewicz, ICCV 2001] 11/27/2020 Computer Vision 26
Implementation • Pipeline: Project Code Capture Images Find Boundaries Match Boundaries Decode Compute Range • DLP projector illuminates scene @ 60 Hz. • Synchronized NTSC camera captures video • Pipeline returns range images @ 60 Hz. 11/27/2020 Computer Vision 27
Projected Patterns 11/27/2020 Computer Vision 28
Results 11/27/2020 Computer Vision 29
Desired Features • • • Low noise Guaranteed high accuracy High speed Low cost Automatic operation No holes 11/27/2020 Computer Vision 30
3 D Model Acquisition Pipeline 3 D Scanner 11/27/2020 Computer Vision 31
3 D Model Acquisition Pipeline 3 D Scanner View Planning 11/27/2020 Computer Vision 32
3 D Model Acquisition Pipeline 3 D Scanner View Planning 11/27/2020 Alignment Computer Vision 33
3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Merging 11/27/2020 Computer Vision 34
3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Done? Merging 11/27/2020 Computer Vision 35
3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Done? Merging Display 11/27/2020 Computer Vision 36
3 D Model Acquisition Difficulties • Much (often most) time spent on “last 20%” • Pipeline not optimized for hole-filling • Not sufficient just to speed up scanner – must design pipeline for fast feedback 11/27/2020 Computer Vision 37
Real-Time 3 D Model Acquisition 11/27/2020 Computer Vision 38
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Human Done? Merging Display 11/27/2020 Computer Vision 39
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner Alignment View Planning Challenge: Real Time Done? Merging Display 11/27/2020 Computer Vision 40
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Done? Alignment Part I: Structured-Light Triangulation Merging Display 11/27/2020 Computer Vision 41
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Part II: Fast ICP Done? Merging Display 11/27/2020 Computer Vision 42
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Part III: Voxel Grid Done? Merging Display 11/27/2020 Computer Vision 43
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Part II: Fast ICP Done? Merging Display 11/27/2020 Computer Vision 44
Aligning 3 D Data • ICP (Iterative Closest Points): for each point on one scan, minimize distance to closest point on other scan… 11/27/2020 Computer Vision 46
Aligning 3 D Data • … and iterate to find alignment – Iterated Closest Points (ICP) [Besl & Mc. Kay 92] 11/27/2020 Computer Vision 47
ICP in the Real-Time Pipeline • Potential problem with ICP: local minima – In this pipeline, scans close together – Very likely to converge to correct (global) minimum • Basic ICP algorithm too slow (~ seconds) – Point-to-plane minimization – Projection-based matching – With these tweaks, running time ~ milliseconds [Rusinkiewicz & Levoy, 3 DIM 2001] 11/27/2020 Computer Vision 48
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Part III: Voxel Grid Done? Merging Display 11/27/2020 Computer Vision 49
Merging and Rendering • Goal: visualize the model well enough to be able to see holes • Cannot display all the scanned data – accumulates linearly with time • Standard high-quality merging methods: processing time ~ 1 minute per scan 11/27/2020 Computer Vision 50
Merging and Rendering 11/27/2020 Computer Vision 51
Merging and Rendering 11/27/2020 Computer Vision 52
Merging and Rendering 11/27/2020 Computer Vision 53
Merging and Rendering + 11/27/2020 Computer Vision 54
Merging and Rendering • Point rendering, using accumulated normals for lighting 11/27/2020 Computer Vision 55
Example: Photograph 18 cm. 11/27/2020 Computer Vision 56
Result 11/27/2020 Computer Vision 57
Postprocessing • Real-time display – Quality/speed tradeoff – Goal: let user evaluate coverage, fill holes • Offline postprocessing for high-quality models – Global registration – High-quality merging (e. g. , using VRIP [Curless 96]) 11/27/2020 Computer Vision 58
Postprocessed Model 11/27/2020 Computer Vision 59
Comparison with Commercial 3 D Scanner 11/27/2020 Computer Vision 60
- Slides: 59