Microsoft Kinect How does a machine infer body

Microsoft Kinect How does a machine infer body position?

Disclaimer A lot of the information presented comes from reverse engineering performed by a number of agencies. So they might not reflect true ground data, which is proprietary and thus not publicly available. 1

Microsoft Kinect �Series of motion sensing input devices for the Xbox �Launched in November 2010 �Removes the need of a game controller �User interacts through gestures and spoken commands. 2

Two step Process �Compute Depth Map �Apply Machine Learning 3

Depth Map? 4

So how does it manage something so cool? 5

So how does it manage something so cool? Projects a speckle pattern of Infrared Laser Light Depths Inferred from the distortions seen in the pattern 6

(Cont) The cool combined with the Classic ● Depth from Focus ○ What is far will be more blurry ○ Combine different focal lengths for X Axis and Y Axis ● Depth from Stereo ○ Combine image obtained from cameras in different positions ○ The speckle pattern projected from one point, observed from another 7

We have the depth. What do we do next? Infer the Body Position, of course 8

Body Position Inference �Kinect starts with millions of labelled depth images �Learns a randomized decision forest, mapping depth images to body parts �This RDF takes days to be trained, but the process is made faster using parallel computing 9

Body Position Inference �Kinect uses the depth image and the RDF and computes a probability of a pixel belonging to each body part. �Uses mean shift algorithm to select the body parts and sketches a skeleton. �All of these computations are very fast. 10

Advantages �Works separately per frame as all the computations are very fast �Runs at 31 fps �Significantly more efficient and robust than body trackers �Works real-time with great results �Handles multiple players �Reported Accuracy: ~72% 11

References http: //www. i-programmer. info/news/105 -artificial-intelligence/2176 -kinects-ai-breakthroughexplained. html http: //users. dickinson. edu/~jmac/selected-talks/kinect. pdf http: //research. microsoft. com/pubs/145347/Body. Part. Recognition. pdf 12

Thank You Arhan Sibal Rudresh Pande