Sens IT Technology Presented By Motion Capture Group

Sens. IT Technology Presented By Motion Capture Group: Azadeh Jamalian Ata Naemi Sa'ed Abu-Alhaija Sunghoon Ivan Lee

Overview § Introduction § System Overview § Hardware unit § Image Processing § Modeling and Animation § Results § Financing § Applications and Marketing § Future Missions § Question Period § Demonstration Welcome to our Presentation!

Introduction Study dance movements Robotic Our Mission: Study dance movements Sports Medicine

System Overview Image Processing Unit ü Real time ü User friendly ü Easy to customize for different applications ü Compatible with other systems ü Low cost Animation Image Processing Unit

System Overview Image Processing Unit Animation Unit

Hardware Block Diagram Microcontroller PIC 16 LF 84 A 4 Sel 4 -16 Turn on One LED Decoder ü Turn on LED’s for 10 frames/sec ü Timing issues ü Components from local vendors and engineering department

Hardware Components ØMicrocontroller (PIC 16 LF 84 A) - Assembly Language - Watch Dog Timer (WDT) - Low power (1 m. A), Sleep mode (3. 5μA) ØRegulator (LM 317) - Regulate 3. 3 V power supply off a 9 V battery - Power off operation as well as staple WDT ØInfrared LED’s (KIE-7305) - 1. 6 V, draw a current of 2 m. A - Each LED is on for 50 msec

Test Models Ø Angle Sensitivity Ø Problem solving as project progressed Ø Use high efficient IR LED’s

System Overview Image Processing Unit Animation Unit

Software Components Ø Low Level Design - Capture Control Unit (CCU) - Image Processing Unit (IPU)

Capture Control Unit (CCU) ØThree main tasks 1. Adjusting Phase Difference 2. Capturing Image 3. Convert Raw Image into RGB

Image Processing Unit (IPU) Ø Calculate position of IR light source in the frame Ø Placing film in front of the web camera - Film absorbs all visible light - Only IR lights penetrate the film

IPU- Challenges Ø Synchronization with Hardware Ø Distorted Images and Virtual Images

Synchronization Frequency of IR sensors and frame rate can be identical Tb IR Freq. t Frame Rate t Tb

Synchronization Frequency of IR sensors and frame rate can be identical Tb IR Freq. t Tphase Frame Rate t Tb

Unexpected Images ØDistorted Images & Virtual Images ØSolutions: - Intensity (changing threshold value) - Width & Height

System Overview Image Processing Unit Animation Unit

Data Flow Diagram Captured Frames Calculated Coordinates of the LED’s Animated Model X and Y values

Animation Unit (AU) ØSoftware: -C++ (Open. GL) ØMain Tasks: -Reads the coordinates from an external file -Assign the coordinates of each joint and record them in a file -Model 2 D motion in real time

AU- Challenges ØSynchronization with IPU - Timing - Start Point - Missing Points ØSimulation and Modeling - Coordinate Systems - Matrix Transformation and Rotation

Simulation Algorithm- Cont. It’s not that easy to think smart!

AU- User Interface

Results Ø Frame Rate: 10 frames/sec Ø Max Motion Frequency: 2 Hz Ø Max Distance from camera: 1 m Ø Precision: 100% (Clear angle of sight) Ø Accuracy: ± 10%

Finance Estimated Cost: Available Funding: Equipment Estimated Cost Sensors/markers + interface-electronics $400 Sources + interface-electronics $400 Computer interface-electronics $100 User Interface $80 Cables $20 Total Cost $1000. 00 Actual Cost: Equipment Estimated Cost Sensors/markers + interface-electronics $60 Sources + interface-electronics $80 Computer interface-electronics 0 User Interface $60 Cables $10 Total Cost $200 $400 from EUSS

Supplementary Application Sens. IT Mouse: Tracks the motion of finger

Marketing Ø Different applications: - Art - Robotic - Medicine - Sports - Game Industry Ø Competitors: - Electronic Arts Canada - C 4 Digital Entertainment - Phoeni. X Technologies Incorporated

Future Improvements Known Limitations: - Captures the motion only in 2 D - Line of sight - Angle of sight - Limited range of monitoring - Interference from other IR sources - Frequency limitation of object motion Improvements: - Capture the motion in 3 D - Multiple marking - Enhance the speed

What We Learned “If you’re not sure what makes the problem Guess what causes it, Make it worse, Try to solve it, If it didn’t work, try another source!”

Conclusion Ø Our system: - Research based project - Proof of concept to design a low cost motion capture system - Initial step to designing a 3 D motion capture system Ø In future: - Unifying music and dance - Future research opportunities - Future Coop positions

Acknowledgements Special Thanks to: Dr. Lucky One Mr. Steve Whitmore Dr. Pavaneh Dr. Patrick Leung Dr. Ivan Bajic Dr. Payandeh Mr. Vinay Kotamraju Mr. Amir Masoud Niroumand Mr. Brad Oldham Thanks a ton!

Questions ?

IPU- Cont. Points to make here Camera Recognizes the IR as a white light True white color pixel in 24 -bit RGB gives [R, G, B] = [255, 255] Only Use one of R, G, B matrices

IPU Algorithm

IPU Algorithm

IPU Algorithm

IPU Algorithm

Simulation Algorithm - One LED blinks twice indicating the start point - An array to hold the coordinates Useful for applications that track the motion of a specific object However, not smart enough to detect the position of various joints in real time