A Unified MultiSurface MultiResolution Workspace with CameraBased Scanning

A Unified Multi-Surface, Multi-Resolution Workspace with Camera-Based Scanning and Projector. Based Illumination Tyler Johnson and Henry Fuchs University of North Carolina – Chapel Hill IPT/EGVE July 18, 2007 - Weimar, Germany

Multi-Projector Display 2 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Previous Work 3 • Projected desktop • Interaction with virtual documents • Camera-based scanning Digital Desk [Wellner ’ 93] • Multi-resolution workspaces Focus Plus Context Displays [Baudisch ’ 01], Escritoire [Ashdown ’ 03] • Multi-surface projected workspace [Ashdown ’ 04] T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Workspace Configuration 4 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Calibration Cameras pre-calibrated Project structured light to determine Projector calibration Workspace geometry • RANSAC-based plane-fitting. [Quirk EDT ’ 06] Also, display structure light with LCD Calibrate as a projector Encoded Structured Light 5 Workspace Walls LCD Panel T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Two-Pass Rendering Determine image warping to remove distortion caused by surface geometry Display Surface Projector Viewer 6 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Single vs Multi-Surface Content Two methods of display Single vs. multi-surface 7 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Single Surface Content User selects region where the window should appear Compute P-Matrix from 3 D->2 D corner correspondences • Choose arbitrary COP Two-pass rendering determines what to project 8 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Multi-Surface Content Same as for single surface, except that COP is user viewpoint Viewpoint can be obtained from separate tracking system 9 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Projector-Based Illumination Projection of imagery onto certain objects is often undesirable Simple interface allows imagery to be replaced with illumination 10 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Camera-Based Scanning User selects copy area with selection window Selection window rendered as a single-surface window 11 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Camera-Based Scanning Remove perspective distortion using two -pass rendering Selection window P-matrix Camera 12 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Video 13 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Contributions Two-pass rendering as a unifying framework for digital workspaces Image generation • Rectified single surface imagery • Perspectively correct multi-surface imagery Camera-based scanning • Removal of perspective distortion LCD panel as a projector Simplifies calibration/rendering 14 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Future Work Automatic object illumination Camera-based detection of objects occluding projection Improved scanning resolution Trigger optical zoom, multi-camera superresolution etc. Continuous calibration Online calibration monitoring and refinement 15 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Thank You Funding support: ONR N 00014 -03 -1 -0589 DARWARS Training Superiority program VIRTE – Virtual Technologies and Environments program 16 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”