Virtual Contact Sheets Wolfgang Richter Kiryong Ha Alok

Crowdsourcing the News [1] “When every cell phone can record video and take pictures,

CNN: Caught Red Handed [2] I’m a sneaky thief. http: //www. pdl. cmu. edu/

Problem Older, uploaded Cloud Recent, not uploaded http: //www. pdl. cmu. edu/ 4 Wolfgang

Fundamental Mobile Challenges • Real-time sync not possible: How far can we degrade the

Just-in-Time Search 1. Check virtual contact sheet • Collection of fidelity-reduced images in the

Near Real-Time Sync (1) Search hit (2) Retrieve http: //www. pdl. cmu. edu/ Cloud

Contact Sheet Analogy http: //www. pdl. cmu. edu/ 8 Wolfgang Richter © September 20

Evaluating Feasibility [3] Benchmark Face Detection, RGB Histogram, Brightness Filter Varying Resolution and JPEG

Energy Savings 10 images = 5 seconds call time (Droid) http: //www. pdl. cmu.

Bandwidth Savings Orders of magnitude savings in bandwidth. http: //www. pdl. cmu. edu/ 11

What about Search Accuracy? Face Detection No one-size-fits-all sweet spot. http: //www. pdl. cmu.

Conclusion & Future Work • Searches retain meaning • No one-size-fits-all, but • Orders

Further Reading [1] TED. http: //www. ted. com/talks/paul_lewis_crowdsourcing_the_news. html. [2] CNN. http: //www. cnn.

Full Fidelity Average photo size: 374 KB 12, 000+ Flickr dataset http: //www. pdl.

Reduce Resolution Average photo size: 221 KB 12, 000+ Flickr dataset http: //www. pdl.

JPEG Quality Factor Average photo size: 6 KB 12, 000+ Flickr dataset http: //www.

Bandwidth Savings Table Resolution 100 95 75 50 25 5 640 x 480 221

Slides: 18

Download presentation

Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes, Lin Zhong*, Mahadev Satyanarayanan PARALLEL DATA LABORATORY Carnegie Mellon University *Rice University

Crowdsourcing the News [1] “When every cell phone can record video and take pictures, everyone is a potential news source. ” Who killed Ian Tomlinson? http: //www. pdl. cmu. edu/ 2 Wolfgang Richter © September 20

CNN: Caught Red Handed [2] I’m a sneaky thief. http: //www. pdl. cmu. edu/ 3 Wolfgang Richter © September 20

Problem Older, uploaded Cloud Recent, not uploaded http: //www. pdl. cmu. edu/ 4 Wolfgang Richter © September 20

Fundamental Mobile Challenges • Real-time sync not possible: How far can we degrade the • Energy objects in alife search before it • Infinite battery desired loses meaning? • Bandwidth • Increasingly metered and charged http: //www. pdl. cmu. edu/ 5 Wolfgang Richter © September 20

Just-in-Time Search 1. Check virtual contact sheet • Collection of fidelity-reduced images in the cloud 2. Pull fidelity image on hit • Search full fidelity; present full fidelity results to searcher http: //www. pdl. cmu. edu/ 6 Wolfgang Richter © September 20

Near Real-Time Sync (1) Search hit (2) Retrieve http: //www. pdl. cmu. edu/ Cloud 7 Wolfgang Richter © September 20

Contact Sheet Analogy http: //www. pdl. cmu. edu/ 8 Wolfgang Richter © September 20

Evaluating Feasibility [3] Benchmark Face Detection, RGB Histogram, Brightness Filter Varying Resolution and JPEG Quality Factor Using http: //www. pdl. cmu. edu/ 9 Wolfgang Richter © September 20

Bandwidth Savings Orders of magnitude savings in bandwidth. http: //www. pdl. cmu. edu/ 11 Wolfgang Richter © September 20

What about Search Accuracy? Face Detection No one-size-fits-all sweet spot. http: //www. pdl. cmu. edu/ 12 Wolfgang Richter © September 20

Conclusion & Future Work • Searches retain meaning • No one-size-fits-all, but • Orders of magnitude bandwidth savings • Significant energy savings • More search primitives? • Perceptual hashing • Face recognition • Texture Filters • Better fidelity reduction? • Keep faces at high fidelity http: //www. pdl. cmu. edu/ 13 Wolfgang Richter © September 20

Further Reading [1] TED. http: //www. ted. com/talks/paul_lewis_crowdsourcing_the_news. html. [2] CNN. http: //www. cnn. com/2010/CRIME/08/24/new. jersey. theft. photo/index. html? hpt=C 1. [3] Simple. CV. http: //simplecv. org/. Jan Harkes, Mahadev Satyanarayanan, Ardalan Amiri Sani, and Benjamin Gilbert. A Contact Sheet Approach to Searching Untagged Images on Smartphones. CMU Technical Report CMU-CS-11 -132. Ardalan Amiri Sani, Wolfgang Richter, Xuan Bao, Trevor Narayan, Mahadev Satyanarayanan, Lin Zhong, and Romit Roy Choudhury. Opportunistic Content Search of Smartphones. Rice University Technical Report TR 0627 -2011. Xuan Bao, Trevor Narayan, Ardalan Amiri Sani, Wolfgang Richter, Romit Roy Choudhury, Lin Zhong, and Mahadev Satyanarayanan. The Case for Context-Aware Compression. Hot. Mobile 2011. Jason Flinn and Mahadev Satyanarayanan. Energy-aware Adaptation for Mobile Applications. SOSP 1999. Y. Nimmagadda, K. Kumar, and Y. H. Lu. Energy-efficient image compression in mobile devices for wireless transmission. ICME 2009. http: //www. pdl. cmu. edu/ 14 Wolfgang Richter © September 20

Full Fidelity Average photo size: 374 KB 12, 000+ Flickr dataset http: //www. pdl. cmu. edu/ 15 Wolfgang Richter © September 20

Reduce Resolution Average photo size: 221 KB 12, 000+ Flickr dataset http: //www. pdl. cmu. edu/ 16 Wolfgang Richter © September 20

JPEG Quality Factor Average photo size: 6 KB 12, 000+ Flickr dataset http: //www. pdl. cmu. edu/ 17 Wolfgang Richter © September 20

Bandwidth Savings Table Resolution 100 95 75 50 25 5 640 x 480 221 KB 114 KB 47 KB 31 KB 19 KB 6 KB 320 x 240 61 KB 33 KB 14 KB 10 KB 6 KB 2 KB 160 x 120 17 KB 10 KB 5 KB 3 KB 2 KB 826 B 80 x 60 5 KB 3 KB 2 KB 1 KB 892 B 460 B 40 x 30 2 KB 1 KB 725 B 588 B 484 B 345 B 20 x 20 1000 B 727 B 501 B 436 B 385 B 316 B http: //www. pdl. cmu. edu/ 18 Wolfgang Richter © September 20