Difference between revisions of "Eye Tracking for ALS Patients"

From ESE497 Wiki
Jump to navigationJump to search
 
(36 intermediate revisions by the same user not shown)
Line 1: Line 1:
<sidebar>Trilateration_in_Robotic_Sensing_using_Acoustic_Sensors_Nav</sidebar>
+
 
 +
 
 
===Background===
 
===Background===
This research was conducted by Chuck Holmes and Joseph Eisner in the Fall 2010 Semester at Washington University in Saint Louis. It was part of the Undergraduate Research Program and taken for credit as the course ESE 497 under the Electrical and Systems Engineering Department. The project was overseen by Dr. Arye Nehorai, Patricio La Rosa, and Ed Richter.
 
  
===Acknowledgments===
+
This research was conducted by Sana Naghipour and Saba Naghipour in the Fall 2011 Semester at Washington University in Saint Louis. It was part of the Undergraduate Research Program and taken for credit as the course ESE 497 under the Electrical and Systems Engineering Department. The project was overseen by Dr. Arye Nehorai,Ed Richter and Phani Chavali.
We would like to thank the following people who helped us on this project:
+
===Project overview===
 +
 
 +
The Eye tracker project is a research effort to empower people, who are suffering from Amyotrophic Lateral Sclerosis (ALS), to write using their eyes by tracking the movement of the pupil. The project will be implemented in two main phases:
 +
 
 +
First Phase: development of the software for pupil tracking
 +
 
 +
Second Phase: building the hardware necessary to capture the images of the eye and transfer the images to a processing unit.
 +
 
 +
===Acknowledgment===
 +
 
 +
We would like to show our greatest appreciation to Prof. Nehorai and Ed Richtor for providing us the research opportunity and feedbacks for the successful completion of the research.
 +
 
 +
We are highly indebted to our mentor, Phani Chaveli for his guidance and constant supervision as well as for providing necessary information regarding the project.
 +
 
 +
===Introduction===
 +
 
 +
First Phase: In this phase of the project, we focus on software development. We will use an infrared camera to capture the images of a human eye using Lab view. We employ template matching methods to locate the center of the pupil, where we will use a small patch of dark pixels as a template. We will propose adaptive search methods, in which we choose the search space in each frame based on the estimates of the pupil location from the previous frames. The proposed adaptive search will greatly reduce the computational complexity of the algorithm, which is essential for the real time tracking. We will also work on developing hybrid methods that use template matching along with feature selection to develop robust and computationally inexpensive algorithms that are insensitive to noise in the images and orientation of the camera that is used to obtain the images.
 +
Second Phase: In the second phase of the project, we will build the hardware system that can record the images using a camera mounted on a pair of sunglasses. We will port the software that we develop during the first phase on to a suitable microcontroller, which does the processing and generates the control signal which can be used to move the prosthetic limbs. We will also develop hardware to transmit the recorded images to the micro-controller wirelessly.
 +
Eye tracking project would help patients in various tasks such as communication, writing emails, drawing and making music. More advanced applications of this project are: cognitive studies, laser refractive surgery, computer usability, translation process research, infant research, sport training and commercial eye tracking.
 +
 
 +
 
 +
===System Setup===
 +
 
 +
We use Lab view to capture the video using an infrared camera. There is support for recording of videos with several frame rates, and formats.  After obtaining the video, we perform sequential frame by frame processing.
 +
 
 +
 
 +
 
 +
[[File:SETUP1.jpg|200px|Experiment Setup]]
 +
[[File:SETUP2.jpg|500px|Experiment Setup]]
 +
 
 +
===Tracking===
 +
 
 +
In this part we used different methods that helped us overcome the challenges we faced during the project. The technics we used are:
 +
 
 +
'''Discarding Color information:'''
 +
We convert the images from all the frames into to their corresponding gray scale images. To do this, we average the pixel values in the entire three color channel to obtain a gray scale image.
 +
 
 +
'''Low pass filtering:'''
 +
We use low-pass filtering to remove the sharp edges in each image. This also helps to remove the undesired background light in the image.
 +
[[File:LPF1.jpg|200px|Low Pass Filtering]]
 +
[[File:LPF2.jpg|200px|Low Pass Filtering]]
 +
[[File:lpf.jpg|200px|Low Pass Filtering]]
 +
 
 +
 
  
Dr. Nehorai for putting together this program and giving us the opportunity to work with physical systems and learn from all the failures that involves.
+
'''Scaling:'''
 +
We scale down the filtered images to obtain lower resolution images. This serves two purposes. First, since the dimension of the image decreases, scaling improves the processing time. Second, the averaging effect removes the undesired background light.
  
Patricio La Rosa for his direction, insight, and patience.
+
'''Template Matching:'''
 +
We used a template matching algorithm to segment the darkest region of the image. Since after discarding the color information and, low-pass filtering, the pupil corresponds to the darkest spot in the eye, this method was used. We used a small patch of dark pixels as a template. The matching is done using exhaustive search over the entire image. Once a match is found, the centroid of the block was determined to the pupil location.  For the experiments, we used a block size of 5 x 5 pixels.
  
Ed Richter for being able and willing to troubleshoot and solve any problem.
+
'''Determining the search space:'''
 +
Since the exhaustive search over the entire image to find a match is computationally intensive, we propose an adaptive search method.  Using this method, we choose the search space based on the pupil location from earlier frame. In this manner, using the past information, we were able to greatly reduce the complexity of the search.  We used a search space of 75 X 75 pixels around the pupil location from the last frame.
  
Phani Chavali for supporting us and loaning us his laptop (which was thankfully '''not''' stolen).
 
  
Raphael Schwartz, Zachary Knudsen, and Andrew Wiens for their web reports which helped us structure this one.
+
===Results===
  
Joshua York for his project which we reference.
+
[[File:RESULT4.jpg|200px|RESULT]]
 +
[[File:RESULT1.jpg|200px|RESULT]]
 +
[[File:RESULT3.jpg|200px|RESULT]]
 +
[[File:RESULT2.jpg|200px|RESULT]]
  
===Project Overview===
 
Abstract: The use of global positioning systems (GPS) for accurately locating targets of interest has become ubiquitous over the last decade. However, contrary to the intuition, the localization becomes more difficult when the source is located within a smaller region, for example, inside a building. This phenomenon is the result of the slow processing time of the computers relative to the time taken by the light to travel back and forth from the satellite to the source. The aim of this research is to study the localization using trilateration methods employing acoustics sources.  Since the speed of propagation of sound is less than the speed of propagation of light, we expect to obtain better resolution with this setup. We mount a microphone on a robot which can move in 2-D plane and track its position by measuring the signals that the microphone records from four speakers whose position is known. We later extend this setup to perform trilateration using other existing infrastructure like WLAN.
 
  
 +
===Conclusion & Future work===
  
[[EyeTracking_Intro | Introduction]]
+
An algorithm for estimating the position and the movement of the pupil is implemented using a template matching method. In the future, we will build the necessary hardware that uses the algorithm for prosthetic limb control.
  
[[EyeTracking_SystemSetup | System Setup]]
+
====Source Files ====
 +
[[File:Code.zip|Final Code]]
  
[[EyeTracking_ImageProcessing | Tracking]]
+
[[File:Code.zip|Final Code]]
  
[[EyeTracking_Results | Results]]
+
[[File:powerpoint.ppt|Presentation]]
  
[[EyeTracking_Conclusions | Conclusions and Future Work]]
+
[[File:poster.ppt|poster]]

Latest revision as of 21:38, 19 December 2011


Background

This research was conducted by Sana Naghipour and Saba Naghipour in the Fall 2011 Semester at Washington University in Saint Louis. It was part of the Undergraduate Research Program and taken for credit as the course ESE 497 under the Electrical and Systems Engineering Department. The project was overseen by Dr. Arye Nehorai,Ed Richter and Phani Chavali.

Project overview

The Eye tracker project is a research effort to empower people, who are suffering from Amyotrophic Lateral Sclerosis (ALS), to write using their eyes by tracking the movement of the pupil. The project will be implemented in two main phases:

First Phase: development of the software for pupil tracking

Second Phase: building the hardware necessary to capture the images of the eye and transfer the images to a processing unit.

Acknowledgment

We would like to show our greatest appreciation to Prof. Nehorai and Ed Richtor for providing us the research opportunity and feedbacks for the successful completion of the research.

We are highly indebted to our mentor, Phani Chaveli for his guidance and constant supervision as well as for providing necessary information regarding the project.

Introduction

First Phase: In this phase of the project, we focus on software development. We will use an infrared camera to capture the images of a human eye using Lab view. We employ template matching methods to locate the center of the pupil, where we will use a small patch of dark pixels as a template. We will propose adaptive search methods, in which we choose the search space in each frame based on the estimates of the pupil location from the previous frames. The proposed adaptive search will greatly reduce the computational complexity of the algorithm, which is essential for the real time tracking. We will also work on developing hybrid methods that use template matching along with feature selection to develop robust and computationally inexpensive algorithms that are insensitive to noise in the images and orientation of the camera that is used to obtain the images. Second Phase: In the second phase of the project, we will build the hardware system that can record the images using a camera mounted on a pair of sunglasses. We will port the software that we develop during the first phase on to a suitable microcontroller, which does the processing and generates the control signal which can be used to move the prosthetic limbs. We will also develop hardware to transmit the recorded images to the micro-controller wirelessly. Eye tracking project would help patients in various tasks such as communication, writing emails, drawing and making music. More advanced applications of this project are: cognitive studies, laser refractive surgery, computer usability, translation process research, infant research, sport training and commercial eye tracking.


System Setup

We use Lab view to capture the video using an infrared camera. There is support for recording of videos with several frame rates, and formats. After obtaining the video, we perform sequential frame by frame processing.


Experiment Setup Experiment Setup

Tracking

In this part we used different methods that helped us overcome the challenges we faced during the project. The technics we used are:

Discarding Color information: We convert the images from all the frames into to their corresponding gray scale images. To do this, we average the pixel values in the entire three color channel to obtain a gray scale image.

Low pass filtering: We use low-pass filtering to remove the sharp edges in each image. This also helps to remove the undesired background light in the image. Low Pass Filtering Low Pass Filtering Low Pass Filtering


Scaling: We scale down the filtered images to obtain lower resolution images. This serves two purposes. First, since the dimension of the image decreases, scaling improves the processing time. Second, the averaging effect removes the undesired background light.

Template Matching: We used a template matching algorithm to segment the darkest region of the image. Since after discarding the color information and, low-pass filtering, the pupil corresponds to the darkest spot in the eye, this method was used. We used a small patch of dark pixels as a template. The matching is done using exhaustive search over the entire image. Once a match is found, the centroid of the block was determined to the pupil location. For the experiments, we used a block size of 5 x 5 pixels.

Determining the search space: Since the exhaustive search over the entire image to find a match is computationally intensive, we propose an adaptive search method. Using this method, we choose the search space based on the pupil location from earlier frame. In this manner, using the past information, we were able to greatly reduce the complexity of the search. We used a search space of 75 X 75 pixels around the pupil location from the last frame.


Results

RESULT RESULT RESULT RESULT


Conclusion & Future work

An algorithm for estimating the position and the movement of the pupil is implemented using a template matching method. In the future, we will build the necessary hardware that uses the algorithm for prosthetic limb control.

Source Files

File:Code.zip

File:Code.zip

File:Powerpoint.ppt

File:Poster.ppt