panoramic viewfinder patrick baudisch desney tan drew steedly
panoramic viewfinder patrick baudisch desney tan, drew steedly, eric rudolph, matt uyttendaele, chris pal, and richard szeliski microsoft research OZCHI 2005 Preview: overall area covered so far Cropping frame: this area will survive cropping Viewfinder: what the camera sees right now
summary preview cropping frame ultra portable PC + web cam viewfinder
stitching photos into a panorama from photos [Teodosio & Bender, 1993] from video [Irani and Anandan, 1998]
stitching: allows taking wide-angle pictures
. . . getting hi-res photos
but sometimes it goes wrong…
but sometimes it goes wrong… • missing content (65% respondents) • tricky because of perspective projections • ghosting (88% respondents) • hard to detect • stitching failed (38% respondents) • lack of overlap, of texture, of focus • when users notice flaws it is too late
panoramic viewfinder
user interface Preview: overall area covered so far Cropping frame: this area will survive cropping Viewfinder: what the camera sees right now
walkthrough: goal building 115 building 114
walkthrough
post-processing • restitching • upload photo to PC • restitch using our high-quality offline stitcher maximum image quality
post-processing • auto cropping to rectangular shape • • • all desired content is preserved cropping frame guarantees this or manual cropping
live demo…
dealing with stitching failure • if user moves camera too quickly (or lack of texture) • viewfinder remains stationary • but continues to update • turns red, error sound • to fix the problem • pan camera back halfways inside panorama • and take an additional shot • or zoom out • alert ends when next match is found
the real-time cropping frame. . .
…is the key component 1. shoot the desired scene elements 2. fill the bounding box around these elements cropping frame tells users when they are done it is the focus of the user’s attention (traditional: viewfinder is focus of attention)
interaction • requirements • • • goal: make sure we have enough content too much content is ok resulting design: control cropping frame indirectly by adding content • • • no resize handles! users can focus on pointing camera interaction transfers to consumer camera
what it does not do • for artistic reasons, user may want to crop • cannot crop panorama while shooting • crop offline, with all other photos
related work
related work • stitch arbitrary order and arrangement [Szeliski & Shum ‘ 97] • real-time [Peleg and Herman, 1997]
our focus • user interface • preview to help users create successful panorama • match the interaction model of existing digital cameras allow port to existing camera interaction mechanisms
stitch assist canon powershot S 230 only horizontal HP Photosmart R 707 stitch in camera layback mode still need to retake panorama we made 4 -way assist can’t fill area take content in any order can add content post-hoc
related work on interaction tiltable interfaces [Rekimoto 1996] chameleon [Fitzmaurice 1993] peep-hole displays [Yee 2003] paintable interfaces [Baudisch 1998]
implementation
computing panorama • • written in C, frontend GDI+ and Direct. X 9 feature-based stitching [Brown et al 92] • uses same libraries as off-line stitcher 1. 2. 3. 4. • extracting multi-scale oriented patches matching with previous image estimating camera orientation warping image Difference: keep existing panorama unchanged • • benefit: faster (~ constant time) drawback: more accumulation error
capture modes • “video mode” • accumulates calibration error (Sawney et al. ’ 98) • “auto mode” • snap picture only when necessary • “manual mode” • best control, avoid motion blur
computing cropping frame • naive: O(n 4), but we do it in ~ O(n 2) • step 1: downsample image by factor 4 • step 2: for each pixel • compute hor and vert spans • step 3: for each pixel • upper bound = hor span x vert span • if (upper bound < current max) move on • else traverse down length of horizontal span starting at this pixel
future work & conclusion
future: user study. . . • as soon as we have a more reliable stitcher • consider mocking-up stitcher to simulate 100% reliability
• Losing track breaks the flow • Even with perfect stitcher there will be stitching failures • We need to be able to lose track without breaking the flow
code editing vs. stitching 1. Batch compiler 1. Offline stitcher 2. Forced syntax 2. Panoramic viewfinder 3. Curly underlines 3. Asynchronous stitcher
Panorama: largest sub panorama created so far Last frame added to the panorama 2 1 Viewfinder: what the camera sees right now Burst buffer: # pictures taken, but not processed Recycle buffer: # of frames not yet matched
future: form factor…
future • start thinking about using cameras in mobile phones as pointing devices
conclusions • one step closer to making process of taking panoramic pictures similar to process of taking normal photos
read more & try out patrickbaudisch. com/projects
END
- Slides: 40