DEV 330 Coding 4 Fun Build Fun Cool

DEV 330 Coding 4 Fun: Build Fun, Cool, Commercial Applications Using the Kinect for Windows SDK Dan Fernandez Director Microsoft Corporation Rick Barraza Senior Technical Evangelist Microsoft Corporation

IR Emitter Color Sensor IR Depth Sensor Tilt Motor Microphone Array

Vertical Horizontal

BGR 32 Format – Every Pixel (0, 0 | 0, 1 | 0, 2) has blue, green, red, empty B G R Empty 255 255 255 0 Kinect Image Sizes: 80 x 60, 320 x 240, 640 x 480 DPI: 96 Stride: # of bytes per single line: 80 x 60 320 x 240 640 x 480

DEMO

D D D D P P P 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Depth Player Index

0. 4 m/1. 3 ft Near 0. 8 m/2. 6 ft Default

Feet Meters 1. 3’ 2. 6’. 4 . 8 9. 8’ 13. 1’ 26. 2’ 3 4 8 Default Mode Near Mode Unknown Normal

Feet Meters 1. 3’ 2. 6’. 4 . 8 9. 8’ 13. 1’ 26. 2’ 3 4 8 Default Mode Near Mode Unknown Too Near Normal Too Far

DEPTH FORMULAS Depth data short[] all. Data = new short[depth. Frame. Pixel. Data. Length]; Get depth for a point (X: 160, Y: 120, Width: 320) int index = (width * y) + x; // 38, 560 Get X, Y for an index (Index: 5, 000, Width: 320) int x = index % width; // 200 int y = index / width; //15 Distance Formula short depth. Point = all. Data[index]; int depth = depth. Point >> Depth. Image. Frame. Player. Index. Bitmask. Width; // 2, 000 Player Formula int player = depth. Point & Depth. Image. Frame. Player. Index. Bitmask; // 1

DEMO

Shoulder. Center Default (20) Spine Hip. Center Max of two skeletons per Kinect 6 Player Proposals Shoulder. Right Head Shoulder. Left Elbow. Right Elbow. Left Wrist. Right Wrist. Left Hand. Right Hand. Left Hip. Right Hip. Left Knee. Right Knee. Left Ankle. Right Ankle. Left Foot. Right Foot. Left Seated (10)

Joints in meters from camera Positive Y Axis Positive Z Axis Positive X Axis Negative

Each joint has associated tracking state Tracked, Not tracked, or Inferred - Occluded, clipped, or low confidence joints Use Transform. Smooth. Parameters to smooth joint data to reduce jitter Skeleton. Clipped. Edges = Frame. Edges. Top; Kinect transform smoothing uses Holt Double Exponential Smoothing - http: //en. wikipedia. org/wiki/Holt-Winters

Fatigue Kills Gestures Use Coding 4 Fun Scale. To extension method

DEMO

KINECT AS AN OFFICE MONITOR.

KINECT AS AN OFFICE MONITOR.

DEMO

http: //download. codeplex. com/Download? Project. Name=kinectservice&Download. Id=336928 Kinect data over the network using sockets Color, Depth, Skeleton, and Audio Client libraries for WPF, Windows Phone & Win. RT

Real-time tracking of multiple faces 2 D/3 D Mesh and Points 3 D Head Pose Animation Units Z Axis

+20 +90 -90 Roll +45 -45 Yaw -20 Pitch

DEMO

SPEECH RECOGNITION GRAMMAR Speech Recognition: English/Great Britain, English/Ireland, English/Australia, English/New Zealand, English/Canada, French/France, French/Canada, Italian/Italy, Japanese/Japan, Spanish/Spain and Spanish/Mexico <!-- Confirmation_Yes. _value: string ["Yes"] --> <rule id="Confirmation_Yes" scope="public"> <example> yes </example> <example> yes please </example> <one-of> <item> yes </item> <item> yeah </item> <item> yep </item> <item> ok </item> </one-of> <item repeat="0 -1"> please </item> <tag> out. _value = "Yes"; </tag> var grammar = new Choices(); grammar. Add(“yes please"); grammar. Add(“yes"); grammar. Add(“yeah"); grammar. Add(“yep"); grammar. Add(“ok"); Ensure Automatic. Gain. Control = false Sample : C: Program FilesMicrosoft SDKsSpeechv 11. 0SamplesSample Grammars

DEMO

Rotation in camera space, or amount of rotation from parent bone Player orientation stored in Hip. Center, or Shoulder. Center when seated Matrix or Quaternion format for animations

DEMO

Boxing. Bots – In Expo Hall

Learning Connect. Share. Discuss. Microsoft Certification & Training Resources http: //europe. msteched. com www. microsoft. com/learning Tech. Net Resources for IT Professionals Resources for Developers http: //microsoft. com/technet http: //microsoft. com/msdn

Evaluations Submit your evals online http: //europe. msteched. com/sessions