Difference between revisions of "Eye Tracking for ALS Patients"
| (25 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | Project | + | ===Background=== |
| + | |||
| + | This research was conducted by Sana Naghipour and Saba Naghipour in the Fall 2011 Semester at Washington University in Saint Louis. It was part of the Undergraduate Research Program and taken for credit as the course ESE 497 under the Electrical and Systems Engineering Department. The project was overseen by Dr. Arye Nehorai,Ed Richter and Phani Chavali. | ||
| + | ===Project overview=== | ||
| − | + | The Eye tracker project is a research effort to empower people, who are suffering from Amyotrophic Lateral Sclerosis (ALS), to write using their eyes by tracking the movement of the pupil. The project will be implemented in two main phases: | |
| − | The | ||
| − | === | + | First Phase: development of the software for pupil tracking |
| + | |||
| + | Second Phase: building the hardware necessary to capture the images of the eye and transfer the images to a processing unit. | ||
| + | |||
| + | ===Acknowledgment=== | ||
| + | |||
| + | We would like to show our greatest appreciation to Prof. Nehorai and Ed Richtor for providing us the research opportunity and feedbacks for the successful completion of the research. | ||
| + | |||
| + | We are highly indebted to our mentor, Phani Chaveli for his guidance and constant supervision as well as for providing necessary information regarding the project. | ||
| + | |||
| + | ===Introduction=== | ||
| + | |||
| + | First Phase: In this phase of the project, we focus on software development. We will use an infrared camera to capture the images of a human eye using Lab view. We employ template matching methods to locate the center of the pupil, where we will use a small patch of dark pixels as a template. We will propose adaptive search methods, in which we choose the search space in each frame based on the estimates of the pupil location from the previous frames. The proposed adaptive search will greatly reduce the computational complexity of the algorithm, which is essential for the real time tracking. We will also work on developing hybrid methods that use template matching along with feature selection to develop robust and computationally inexpensive algorithms that are insensitive to noise in the images and orientation of the camera that is used to obtain the images. | ||
| + | Second Phase: In the second phase of the project, we will build the hardware system that can record the images using a camera mounted on a pair of sunglasses. We will port the software that we develop during the first phase on to a suitable microcontroller, which does the processing and generates the control signal which can be used to move the prosthetic limbs. We will also develop hardware to transmit the recorded images to the micro-controller wirelessly. | ||
| + | Eye tracking project would help patients in various tasks such as communication, writing emails, drawing and making music. More advanced applications of this project are: cognitive studies, laser refractive surgery, computer usability, translation process research, infant research, sport training and commercial eye tracking. | ||
| + | |||
| + | |||
| + | ===System Setup=== | ||
| + | |||
| + | We use Lab view to capture the video using an infrared camera. There is support for recording of videos with several frame rates, and formats. After obtaining the video, we perform sequential frame by frame processing. | ||
| − | |||
| − | + | [[File:SETUP1.jpg|200px|Experiment Setup]] | |
| + | [[File:SETUP2.jpg|500px|Experiment Setup]] | ||
| + | ===Tracking=== | ||
| − | + | In this part we used different methods that helped us overcome the challenges we faced during the project. The technics we used are: | |
| − | |||
| − | |||
| + | '''Discarding Color information:''' | ||
| + | We convert the images from all the frames into to their corresponding gray scale images. To do this, we average the pixel values in the entire three color channel to obtain a gray scale image. | ||
| − | + | '''Low pass filtering:''' | |
| + | We use low-pass filtering to remove the sharp edges in each image. This also helps to remove the undesired background light in the image. | ||
| + | [[File:LPF1.jpg|200px|Low Pass Filtering]] | ||
| + | [[File:LPF2.jpg|200px|Low Pass Filtering]] | ||
| + | [[File:lpf.jpg|200px|Low Pass Filtering]] | ||
| − | |||
| + | '''Scaling:''' | ||
| + | We scale down the filtered images to obtain lower resolution images. This serves two purposes. First, since the dimension of the image decreases, scaling improves the processing time. Second, the averaging effect removes the undesired background light. | ||
| − | + | '''Template Matching:''' | |
| − | + | We used a template matching algorithm to segment the darkest region of the image. Since after discarding the color information and, low-pass filtering, the pupil corresponds to the darkest spot in the eye, this method was used. We used a small patch of dark pixels as a template. The matching is done using exhaustive search over the entire image. Once a match is found, the centroid of the block was determined to the pupil location. For the experiments, we used a block size of 5 x 5 pixels. | |
| − | + | '''Determining the search space:''' | |
| − | + | Since the exhaustive search over the entire image to find a match is computationally intensive, we propose an adaptive search method. Using this method, we choose the search space based on the pupil location from earlier frame. In this manner, using the past information, we were able to greatly reduce the complexity of the search. We used a search space of 75 X 75 pixels around the pupil location from the last frame. | |
===Results=== | ===Results=== | ||
| − | + | [[File:RESULT4.jpg|200px|RESULT]] | |
[[File:RESULT1.jpg|200px|RESULT]] | [[File:RESULT1.jpg|200px|RESULT]] | ||
[[File:RESULT3.jpg|200px|RESULT]] | [[File:RESULT3.jpg|200px|RESULT]] | ||
[[File:RESULT2.jpg|200px|RESULT]] | [[File:RESULT2.jpg|200px|RESULT]] | ||
| − | |||
| + | ===Conclusion & Future work=== | ||
| + | |||
| + | An algorithm for estimating the position and the movement of the pupil is implemented using a template matching method. In the future, we will build the necessary hardware that uses the algorithm for prosthetic limb control. | ||
| + | ====Source Files ==== | ||
| + | [[File:Code.zip|Final Code]] | ||
| − | + | [[File:Code.zip|Final Code]] | |
| − | + | [[File:powerpoint.ppt|Presentation]] | |
| − | + | [[File:poster.ppt|poster]] | |
| − | |||
| − | |||
| − | |||
Latest revision as of 21:38, 19 December 2011
Contents
Background
This research was conducted by Sana Naghipour and Saba Naghipour in the Fall 2011 Semester at Washington University in Saint Louis. It was part of the Undergraduate Research Program and taken for credit as the course ESE 497 under the Electrical and Systems Engineering Department. The project was overseen by Dr. Arye Nehorai,Ed Richter and Phani Chavali.
Project overview
The Eye tracker project is a research effort to empower people, who are suffering from Amyotrophic Lateral Sclerosis (ALS), to write using their eyes by tracking the movement of the pupil. The project will be implemented in two main phases:
First Phase: development of the software for pupil tracking
Second Phase: building the hardware necessary to capture the images of the eye and transfer the images to a processing unit.
Acknowledgment
We would like to show our greatest appreciation to Prof. Nehorai and Ed Richtor for providing us the research opportunity and feedbacks for the successful completion of the research.
We are highly indebted to our mentor, Phani Chaveli for his guidance and constant supervision as well as for providing necessary information regarding the project.
Introduction
First Phase: In this phase of the project, we focus on software development. We will use an infrared camera to capture the images of a human eye using Lab view. We employ template matching methods to locate the center of the pupil, where we will use a small patch of dark pixels as a template. We will propose adaptive search methods, in which we choose the search space in each frame based on the estimates of the pupil location from the previous frames. The proposed adaptive search will greatly reduce the computational complexity of the algorithm, which is essential for the real time tracking. We will also work on developing hybrid methods that use template matching along with feature selection to develop robust and computationally inexpensive algorithms that are insensitive to noise in the images and orientation of the camera that is used to obtain the images. Second Phase: In the second phase of the project, we will build the hardware system that can record the images using a camera mounted on a pair of sunglasses. We will port the software that we develop during the first phase on to a suitable microcontroller, which does the processing and generates the control signal which can be used to move the prosthetic limbs. We will also develop hardware to transmit the recorded images to the micro-controller wirelessly. Eye tracking project would help patients in various tasks such as communication, writing emails, drawing and making music. More advanced applications of this project are: cognitive studies, laser refractive surgery, computer usability, translation process research, infant research, sport training and commercial eye tracking.
System Setup
We use Lab view to capture the video using an infrared camera. There is support for recording of videos with several frame rates, and formats. After obtaining the video, we perform sequential frame by frame processing.
Tracking
In this part we used different methods that helped us overcome the challenges we faced during the project. The technics we used are:
Discarding Color information: We convert the images from all the frames into to their corresponding gray scale images. To do this, we average the pixel values in the entire three color channel to obtain a gray scale image.
Low pass filtering:
We use low-pass filtering to remove the sharp edges in each image. This also helps to remove the undesired background light in the image.
Scaling: We scale down the filtered images to obtain lower resolution images. This serves two purposes. First, since the dimension of the image decreases, scaling improves the processing time. Second, the averaging effect removes the undesired background light.
Template Matching: We used a template matching algorithm to segment the darkest region of the image. Since after discarding the color information and, low-pass filtering, the pupil corresponds to the darkest spot in the eye, this method was used. We used a small patch of dark pixels as a template. The matching is done using exhaustive search over the entire image. Once a match is found, the centroid of the block was determined to the pupil location. For the experiments, we used a block size of 5 x 5 pixels.
Determining the search space: Since the exhaustive search over the entire image to find a match is computationally intensive, we propose an adaptive search method. Using this method, we choose the search space based on the pupil location from earlier frame. In this manner, using the past information, we were able to greatly reduce the complexity of the search. We used a search space of 75 X 75 pixels around the pupil location from the last frame.
Results
Conclusion & Future work
An algorithm for estimating the position and the movement of the pupil is implemented using a template matching method. In the future, we will build the necessary hardware that uses the algorithm for prosthetic limb control.
