Senior Capstone Project Gaze Tracking System what you
- Slides: 36
Senior Capstone Project Gaze Tracking System what you see is finally what you get Team IRALAR Breanna Heidenburg -- Michael Lenisa -- Daniel Wentzel Advisor: Dr. Malinowski DISCLAIMER: This presentation was used specifically for the Bradley University Student Expo on April 4 th, 2008. Other versions may contain a more comprehensive description of the project.
Presentation Outline The Project ◦ Why is it important The Goals ◦ System breakdown Image recognition Point transformation User Interface The Results
Brief overview What is our project?
Brief Overview Track a user’s eye and use the information to control a computer cursor Wherever you look on the display is where the cursor goes
Brief Overview Enhances Human Computer Interaction ◦ Speed of use ◦ Hands-free use
System Development 3 Part System ◦ Image Processing Application ◦ Calibration and Mapping system ◦ GUI designed for gaze-based interaction Systems developed concurrently and independently Separate Applications at run-time
System Development Hardware and Image Processing Application
System Development User Interface Hardware ◦ Camera Quick. Cam Pro for Notebooks Visible Spectrum Camera ◦ Polarizer Tiffen 25 mm polarizing filter Removes glare from eye reflections ◦ Lighting Diffuse LEDs Slightly distracting to the user, but necessary to provide light for the camera
System Development User Interface Lit. Eye LE-500 ◦ ◦ High resolution (SVGA) Color Display Translucent or opaque operation Stationary relative to user’s eye
System Development Software – Image Processing Real time pupil tracking system ◦ Developed in C using Open. CV image processing libraries ◦ Traditional image processing and blob tracking Capabilities ◦ ◦ ◦ Locate and determine center of pupil in image Low light and high reflection environments All eye colors Data logging and static test modes Packaged into self contained Windows installer for easy deployment onto any computer
System Development Software – Image Processing The Process ◦ ◦ ◦ ◦ ◦ Capture Image Extract Red Channel Smooth image Apply binary threshold Locate blobs Reject false positives Determine center of pupil blob Adapt threshold Repeat
System Development Software – Image Processing Summary ◦ The Good Dynamically adapts to changing lighting conditions and eye types Maintains performance in low-light and specularly noisy conditions ◦ The Bad Still relies on Logitech camera drivers Extreme reflections still cause problems
System Development Image Processing in Action Raw Pre-Processed Completed Capture Recognition Image
System Development Image Processing Examples of performance in poor conditions Low Light Difficult False Positive
System Development Calibration and point mapping
System Development Eye Mapping System for mapping the location of the center of the pupil to a pixel on a computer screen Reasons ◦ Geometry The eye is not flat but a screen is not ◦ User Customization All eyes are different Everyone wears the HMD differently ◦ User Training Calibration system also acts as a quick tutorial
System Development Point Transformation 3 dimensional best fit plane ◦ Currently using a 4 th degree best fit Xpix = A 1 + Xeye*B 1 + Yeye*C 1 Ypix = A 2 + Xeye*B 2 + Yeye*C 2 Calibration sub-system determines these coefficients
System Development Calibration How do we solve the problem? ◦ Multiple Variable Linear Regression – Least Squares Y = B 0 + B 1 x 1 + … + B k x k ◦ Uses matrix algebra to obtain a coefficient matrix B[] = (X’X)-1 X’Y
System Development Calibration Results B[] = (X’X)-1 X’Y ◦ pixel. X = 224. 9 + 4. 8*eye. X – 2. 9*eye. Y ◦ pixel. Y = 1612. 5 + 2. 0*eye. X – 5. 6*eye. Y
System Development ‘Clicking’ How do we click? ◦ Monitor eye movements ◦ Identify pauses ◦ When eye position is within a small percentage for a certain amount of time ◦ Currently set at 5 frames (~200 m. S) ◦ Generally, it takes 230 m. S for a hand to click a mouse.
System Development Communications and Custom GUI interface
System Development Communication Why is it important? ◦ Allows multiple processes to communicate ◦ Allows relay of time sensitive information UDP vs. TCP ◦ UDP -> ‘Connectionless’ ◦ TCP -> ‘Connection oriented’ Multi-threading ◦ Necessary for running multiple pieces of code in a single process
System Development Communication Multi-threading issues present
System Development Communication Multi-threading ◦ Issues with public variable usage ◦ Solutions we are using: Critical Section Raises thread priority (does not allow for interrupts) Receive request for data (over UDP) Raise Thread priority Read Variable Lower Thread Priority Reply to request
System Development User Interface Custom GUI for Gaze Tracking Applications Why? ◦ Gaze tracking accuracy limited by inherent properties of human vision system ◦ Traditional GUI too small and intrusive for use with transparent HMD ◦ Demonstrate applications of gaze tracking
System Development User Interface Achieved via a combination of Hardware and Software: WPF & C# ◦ XAML (e. Xtensible Application Markup Language) ◦ Very similar to XML ◦ Uses ‘code-behind’ in a similar style to javascript ‘code-behind’ is in C#
System Development User Interface Multiple applications have been created within the interface Multiple screens for functionality testing ◦ even games Additional ability to minimize interface and interact with external applications is present software
System Development User Interface Calibration Screen Main Screen
Results What did our test results show?
Results Calibration
Results Calibration
Results Timing
Results Schedule
Thanks Who helped us out?
Thanks Northrop Grumman Lit. Eye Dr. Malinowski and the EE faculty Mr. Mattus & Mr. Schmidt
Questions
- The male gaze
- Capstone tracking
- Middle school capstone project ideas
- Capstone proposal outline
- Fluor atls
- Strand for culinary arts
- Bdm capstone project
- Mini capstone project
- Java capstone project
- Capstone project
- Capstone project portfolio
- Career life connections capstone project examples
- Enrollment system capstone
- Student tracking system project
- Disadvantages of eye gaze communication system
- Eye gaze communication system
- Final project presentation template
- Senior project outline
- Senior geography project
- Senior exit project examples
- Machine learning senior project ideas
- Ee senior design project ideas
- What is capstone
- Capstone home automation
- Pasco match graph
- Swot analysis of organic products
- The capstone approach
- How to use pasco capstone
- Cvat tool
- Vulnerability capstone
- Quest framework ap seminar
- Capstone logiciel
- Capstone video games
- Capstone video games
- Richard quinn ucf
- Me 4182
- Capstone meeting