Creating Better Thumbnails Chris Waclawik Project Motivation Thumbnails

Creating Better Thumbnails Chris Waclawik

Project Motivation • Thumbnails used to quickly select a specific image from a set (when lacking appropriate metadata), but… • Larger sets require smaller, less detailed thumbnails.

Thumbnail effectiveness Can you find the picture with the dog in it?

Problem Statement • How can thumbnails be made more effective?

Show only the salient parts

Two subproblems • How do we determine the salient portions of the image? • Knowing the above, how should we crop the image?

Determining Image Saliency • F. Liu, M. Gleicher, Region enhanced scaleinvariant saliency detection, in: Proceedings of IEEE International Conference on Multimedia & Expo (ICME), 2006, pp. 1477 -1480. • Two-part method: – Scale-invariance makes this method suitable for determining salient features of many sizes. – Region-enhancement aids in making logical crops later on.

Scale-invariant Saliency • Image transformed to perceptually uniform color space (Lu*v*). – Difference in color values proportional to perceived difference. • Gaussian pyramid created and a contrast map is calculated for each level. – The contrast of each pixel is the sum of its distances from its neighbors (in Lu*v* space). – Pixels closer to the center are given more weight. • The contrast maps are summed to find scale invariant saliency. • Algorithm implemented in Matlab.

Region-enhanced Saliency • Regions extracted from the image using any preferred method. – Mean shift method used in paper (and in this project). – C++ code from EDISON, Matlab wrapper function from shawnlankton. com. • Scale-invariant saliency is averaged over each region.

Saliency-based Cropping • B. Suh, H. Ling, B. B. Bederson, and D. W. Jacobs, "Automatic Thumbnail Cropping and Its Effectiveness, " in 16 th Annual ACM Symposium on User Interface Software and Technology. 2003, pp. 95 -104. • What is the smallest crop that exceeds a specified saliency threshold (proportion of the total image saliency)? – Can be calculated by brute force, but inefficient

A greedy approach • User specifies a target output size and saliency threshold. • Select the most salient region. • Until we reach our threshold: – Expand selection to include next most salient region. – Adjust selection to match aspect ratio of output size. • Not guaranteed optimal crop – But much faster – Guaranteed to include whole regions

Demonstration • Bounding box in red, correct aspect ratio in green

Determining Saliency Threshold Automatically • Find the point of diminishing returns: when does increasing the crop area not become worth it? • One simple method: select the image coming after the biggest increase in saliency.

Results Original Thumbnail

Further possible improvements • Usability testing – Test users on how quickly they can locate a specified image in a set given a table of thumbnails. • Tweak region parameters • Saliency threshold selection – Try out more sophisticated metrics • Basic feature recognition – Face detection