RealTime 3 D Model Acquisition Szymon Rusinkiewicz Olaf
- Slides: 45
Real-Time 3 D Model Acquisition Szymon Rusinkiewicz Olaf Hall-Holt Marc Levoy Princeton University Stanford University
3 D Scanning
Possible Research Goals • Low noise • Guaranteed high accuracy • High speed • Low cost • Automatic operation • No holes
3 D Model Acquisition Pipeline 3 D Scanner
3 D Model Acquisition Pipeline 3 D Scanner View Planning
3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment
3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Merging
3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Done? Merging
3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Done? Merging Display
3 D Model Acquisition Difficulties • Much (often most) time spent on “last 20%” • Pipeline not optimized for hole-filling • Not sufficient just to speed up scanner – must design pipeline for fast feedback
Real-Time 3 D Model Acquisition
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Human Done? Merging Display
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner Alignment View Planning Challenge: Real Time Done? Merging Display
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Done? Alignment Part I: Structured-Light Triangulation Display Merging
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Part II: Fast ICP Done? Merging Display
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Part III: Voxel Grid Done? Merging Display
Triangulation Object Laser Camera • Project laser stripe onto object
Triangulation Object Laser (x, y) Camera • Depth from ray-plane triangulation
Triangulation • Faster acquisition: project multiple stripes • Correspondence problem: which stripe is which?
Continuum of Triangulation Methods Multi-stripe Multi-frame Single-stripe Slow, robust Single-frame Fast, fragile
Time-Coded Light Patterns • Assign each stripe a unique illumination code over time [Posdamer 82] Time Space
Codes for Moving Scenes • Assign time codes to stripe boundaries • Perform frame-to-frame tracking of corresponding boundaries – Propagate illumination history Illumination history = (WB), (BW), (WB) [Hall-Holt & Rusinkiewicz, ICCV 2001] Code
Designing a Code • Want many “features” to track: lots of black/white edges at each frame • Try to minimize ghosts – WW or BB “boundaries” that can’t be seen directly
Designing a Code 0000 1101 1010 1111 0010 0101 1000 1011 0110 0001 0100 1110 0111 1100 1001 0011 [Hall-Holt & Rusinkiewicz, ICCV 2001]
Implementation • Pipeline: Project Code Capture Images Find Boundaries Match Boundaries Decode • DLP projector illuminates scene @ 60 Hz. • Synchronized NTSC camera captures video • Pipeline returns range images @ 60 Hz. Compute Range
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Part II: Fast ICP Done? Merging Display
Aligning 3 D Data • ICP (Iterative Closest Points): for each point on one scan, minimize distance to closest point on other scan…
Aligning 3 D Data • … and iterate to find alignment – Iterated Closest Points (ICP) [Besl & Mc. Kay 92]
ICP in the Real-Time Pipeline • Potential problem with ICP: local minima – In this pipeline, scans close together – Very likely to converge to correct (global) minimum • Basic ICP algorithm too slow (~ seconds) – Point-to-plane minimization – Projection-based matching – With these tweaks, running time ~ milliseconds [Rusinkiewicz & Levoy, 3 DIM 2001]
Real-Time 3 D Model Acquisition Pipeline 3 D Scanner View Planning Alignment Part III: Voxel Grid Done? Merging Display
Merging and Rendering • Goal: visualize the model well enough to be able to see holes • Cannot display all the scanned data – accumulates linearly with time • Standard high-quality merging methods: processing time ~ 1 minute per scan
Merging and Rendering
Merging and Rendering
Merging and Rendering
Merging and Rendering +
Merging and Rendering • Point rendering, using accumulated normals for lighting
Example: Photograph 18 cm.
Result
Postprocessing • Real-time display – Quality/speed tradeoff – Goal: let user evaluate coverage, fill holes • Offline postprocessing for high-quality models – Global registration – High-quality merging (e. g. , using VRIP [Curless 96])
Postprocessed Model
Recapturing Alignment
Summary • 3 D model acquisition pipeline optimized for obtaining complete, hole-free models • Use human’s time most efficiently • Pieces of pipeline selected for real-time use: – Structured-light scanner for moving objects – Fast ICP variant – Simple grid-based merging, point rendering
Limitations • Prototype noisier than commercial systems – Could be made equivalent with careful engineering – Ultimate limitations on quality: focus, texture • Scan-to-scan ICP not perfect alignment drift – Due to noise, miscalibration, degenerate geometry – Reduced, but not eliminated, by “anchor scans” – Possibly combine ICP with separate trackers
Future Work • Faster scanning – Better stripe boundary tracking – Multiple cameras, projectors – High-speed cameras, projectors • Application in different contexts – Cart- or shoulder-mounted for digitizing rooms – Infrared for imperceptibility
Acknowledgments • Collaborators: – Li-Wei He – James Davis – Lucas Pereira – Sean Anderson • Sponsors: – Sony – Intel – Interval
- Szymon rusinkiewicz
- Acquisition vs learning examples
- Połączenie śrubowe dociskowe
- Szymon bielecki
- Szymon jakubczak
- Szymon grabowski pł
- Simon grabowski
- Professor olaf wendler
- Olaf krings
- Ivko magister
- Olaf zielke
- Erwin olaf biografie
- Olaf schneider rechtsanwalt bonn
- Olaf heinicke
- Business model canvas exam questions
- Olaf lobermeier
- Olaf medenbach
- Student information system st olaf
- St olaf nmr
- Olaf blanke out of body
- Handballverband westfalen
- Olaf diegel
- Nnsip
- Olaf d
- Bbwe
- 247remote
- Real-time system definition
- Realtime aps software
- Push notification firebase
- Realtime streaming protocol
- Ecurisa
- Forrester real time interaction management
- Lightning realtime
- Simple online and realtime tracking
- Visual rendering
- Real time operating system concepts
- Realtime communications
- Realtime it
- Realtime it
- Realtime it
- Asiaplus realtime
- Cac realtime
- Realtime forex
- Eva rov
- Rendering realtime compositing
- Realtime big data