Since 2008 Google street view has blurred faces

Since 2008 Google street view has blurred faces for privacy.

The system mostly works well, but does have both false positive and false negatives

Let us see if we could solve this problem, with what we have learned in this class. First, a small insight. Let us solve the skin detection problem, instead of the face detection problem. What features can we use? Every color pixel in an image, is represented as a combination of three colors, typically Red Green Blue, but there are other possibilities, HSV (hue, saturation, value), HSL (hue, saturation, lightness/luminance) etc. G R B

G R B

B G R R= If we project a skin pixel into 3 D space, then you could use the following, human coded rule to classify skin pixels: G= If (R > 95) AND (G > 40) AND (B > 20) AND (max{R, G, B} − min{R, G, B} > 15) AND (|R − G| > 15) AND (R > G) AND (R > B) Then Is. Skin However, this rule is not perfect (next slide) B=

The rule works pretty good here, but it has a lot of false positives, the plate on the wall, the stripy sweater, parts of the couch on the bottom left etc.

Let us try to do better, with machine learning. First we need to get training data to build our classifier. Recall that for machine learning problems, it can sometimes be useful to transform features. Instead of height and weight, use weight/ height 2 Maybe there is a transformation of the color space that makes this problem easer? We could ask an expert, or just compare all color spaces using leave-one-out classification.

In YCb. Cr, the Y is the brightness (luma), Cb is blue minus luma (B-Y) and Cr is red minus luma (R-Y). Y= Cb = What does skin look like in this space? Next slide… Cr =

In YCb. Cr, the Y is the brightness (luma), Cb is blue minus luma (B-Y) and Cr is red minus luma (R-Y). 250 200 150 Cr Y= 100 50 0 0 50 100 Cb 150 200 250 Cb = 250 200 The RGB space, (only R and G are shown) 150 R 100 Cr = 50 0 0 50 100 G 150 200 250

Cr > 124? 250 200 150 Cr Cr < 174? 100 Not Skin 50 0 0 50 100 Cb 150 200 250 Cb < 90? Cb > 140? Skin Not Skin

250 200 150 Cr 100 50 0 0 50 100 Cb 150 200 250 RGB-H-Cb. Cr Skin Colour Model for Human Face Detection. Nusirwan Anwar Abdul Rahim, Kit Chong Wei, John See. MMU International Symposium on Information & Communications Technologies (M 2 USIC 2006),

This is a hand coded piecewise linear model 250 Cr ≤ 1. 5862 × Cb + 20 Cr ≥ 0. 3448 × Cb + 76. 2069 Cr ≥ -4. 5652 × Cb + 234. 5652 Cr ≤ -1. 15 × Cb + 301. 75 Cr ≤ -2. 2857 × Cb + 432. 85 200 150 Cr 100 50 0 0 50 100 Cb 150 200 250 RGB-H-Cb. Cr Skin Colour Model for Human Face Detection. Nusirwan Anwar Abdul Rahim, Kit Chong Wei, John See. MMU International Symposium on Information & Communications Technologies (M 2 USIC 2006),

A simple nearest neighbor algorithm in the YCb. Cr space works very well. How could we make it better?

Extensions We can use basic classifiers as a building blocks for more complex questions For example, if we wanted to find examples of Fed. Ex trucks on streetview, we could build a classifier for each color in the logo (Purple, Orange and White). If a small area of the image had all three colors, we could classify truck seen. Why not just “read” the text? Consider the below…

Less we get too complacent I How well could you do on this problems? examples Google “Karen Zack” for more

Less we get too complacent II How well could you do on this problems?