Estimation of Skin Color Range Using Achromatic Features

Estimation of Skin Color Range Using Achromatic Features Wen-Hung Liao Department of Computer Science National. Chengchi. University November 27, 2008

Outline Motivation and Related Work l Color Spaces l Fixed vs. Dynamic Range Approach l Experimental Results l l Skin color segmentation l Hand & finger detection l Conclusion

Background l Previous claims: skin color is restricted to a “fixed” range in certain color coordinates: l l l Sobottka & Pitas: Hue: [0, 50º], Saturation: [0. 23, 0. 68] Chai & Ngan: Cb: [77, 127], Cr[137, 177] Kawato & Ohya: Decision boundary in normalized RGB space

Decision Boundary in Normalized RGB Space

Sobottka& Pitas: Fixed Hue + Saturation

Chai& Ngan: Fixed. Cb, Cr

Kawato& Ohya

Comparative Analysis From: Phung et al, Skin segmentation using color pixel classification: analysis and comparison, IEEE Transactions on PAMI, 2005.

Observation l l It is true that the skin color lies in a small range, yet this range tends to shift under different lighting conditions. Question: Is it possible to dynamically adjust the range of skin color to enhance the robustness of color-based segmentation?

The Proposed Solution Use achromatic information (face detection) to help determine the range. l Limitation: l l l Face must be present and detected. Suitable for vision-based human computer interface.

Five Classes of Color Space Color space Representative color space Basic color spaces RGB、normalized RGB Perceptual color spaces HSV、HIS Orthogonal color spaces YCb. Cr、YUV Perceptually uniform color spaces CIELab、CIELuv Other color spaces Mixture

Color Spaces Investigated color space domains RGB Red、Green、Blue HSV Hue、Saturation、Value CIELab L、a、b YCb. Cr Y、Cb、Cr CIELuv L、u、v * Dynamically set the threshold in Hue domain

Determining the Threshold (I) l l Step 1: detecting and locating the face Step 2: mark the cheek area X = X 0 +(W 0 /5) Y = Y 0 +(H 0 /2) width = W 0 /5 height = H 0 /5 l (X 0, Y 0) W 0 H 0 Step 3: obtain the hue distribution of the marked area.

Determining the Threshold (II) l Step 4: assume that the histogram is peaked at A: l search to the left and right of A until l Local minimum <A/10 is uncovered l A non-zero global minimum is found 0 255

Face Detection using DSE l Directional Sobel Edges

Experiment: Skin Color Segmentation l Compare the performance of 5 different methods: l l l Dynamic threshold Fixed threshold – fixed Hue Kawato & Ohya – fixed Normalized RGB Sobottka & Pitas – fixed Hue & Saturation Chai & Ngan – fixed Cb & Cr Material l l Images captured by a low-cost webcam under different lighting conditions. A total of 400 images (taken indoor) are manually segmented and labeled.

Skin Color Segmentation: Experimental Results false positive false negative true positive Dynamic Threshold 0. 0736 0. 1706 0. 9264 0. 8294 fixed Hue 0. 2125 0. 3361 0. 7875 0. 6639 fixed Normalized RGB 0. 0504 0. 5303 0. 9496 0. 4697 fixed Hue & Sat 0. 0588 0. 5747 0. 9412 0. 4253 fixed Cr &Cb 0. 0857 0. 2996 0. 9143 0. 7004

Best and Worst Case Performance best TP worst TP Dynamic Threshold 0. 9947 0. 3494 fixed Hue 0. 9977 0. 0733 fixed Normalized RGB 0. 9055 0. 0002 fixed Hue & Sat 0. 8891 0. 0005 fixed Cr &Cb 0. 9447 0. 2234

Recall and Precision Recall = TP/(TP+FP) Precision = TP/(TP+FN)

Speed-up the Process 1. Detecting Face (After K frames) 2. Record color distribution 3. Tracking face of cheek area 4. Local search 5. Update color distribution

Performance Improvement

Experiment: Hand Detection l l l Color-based hand segmentation No post-processing Does not involve statistical modeling and classifier

Plamar vs. Dorsal Side Hue histogram

Hand Detection: Experimental Results Hand detection Accuracy Dorsal side (fingers) Plamarside (fingers) 92. 65% 94. 26% 90. 78% 95. 01%

Fingertip Detection l 150 images # of fingers detected Dynamic threshold Fixed Threshold 5 108 72% 17 11% 4 21 14% 22 15% 3 10 7% 23 15% 2 5 3% 20 13% 1 1 1% 20 13% 0 5 3% 48 33%

Conclusion l l Perform comparative evaluation of several color-based segmentation methods. Propose and implement a dynamic range estimation algorithm using achromatic features. Superior performance in terms of skin-color segmentation, hand finger detection. Suitable for vision-based HCI.

Thank you Q & A

Experimental Result l Dynamic Threshold worst TP

Experimental Result l Fixed Hue worst TP

Experimental Result l Fixed Normalized RGB worst TP

Experiment Result l Fixed Hue & Saturation worst TP

Experiment Result l Fixed Cb & Cr worst TP

Recall = TP/(TP+FP) Precision = TP/(TP+FN)