Case Studies
Creating a Real-Time Embedded Vision System to Monitor Driver’s Visual Attention
POSTED 07/13/2011 | By: Aurobinda Routray, Indian Institute of Technology
The Challenge:
Developing a real-time, noninvasive, reliable method to ascertain early symptoms of driver fatigue to prevent car accidents.
The Solution:
Using NI Compact Vision System hardware and software to create a stand-alone embedded system with a real-time eye detection algorithm alongside a percentage closure of eyes and eye saccade measurement algorithm.
The significant increase in the number of traffic accidents over the last decade has become a serious concern. According to the Department of Road Transport and Highways in India, the number of road accidents increased by 34 percent from 1994 to 2004. The majority of these accidents (about 90 to 95 percent) are caused by human error. More recent data reveals that driver inattentiveness, including distraction and falling asleep at the wheel, accounts for at least 25 percent of crashes. Leading laboratories are conducting research to ascertain early signs of fatigue using various methods based on psychometric and ocular measures as well as physiological signals. However, because of the nature of the problem, invasive methods to acquire physiological signals such as heart rate and brain waves are not feasible. Therefore, the challenge is developing a noninvasive, real-time, computationally inexpensive, and reliable method to ascertain early symptoms of driver fatigue to prevent such accidents.
Recently, Dinges, et al conducted a wide-ranging ocular measurement technique evaluation that conclusively proved that the most effective fatigue-level estimator in a driver is the measure of percentage closure of eyes (PERCLOS). Another important estimator is the rate of eye saccades. Eye saccades are quick, simultaneous movements of both eyes in the same direction. As fatigue level increases, the rate of saccadic movement slowly decreases. We created an image-based, nonintrusive, real-time driver attention monitoring system to measure both PERCLOS and the rate of saccadic eye movement as an estimator of fatigue level in a driver.
Design Considerations and Offline Results
We created a stand-alone embedded system with a real-time eye detection algorithm alongside a PERCLOS and saccade measurement algorithm. We tested several existing eye tracking algorithms and selected the appearance-based principal component analysis (PCA) method. Its high accuracy and low complexity make it attractive for real-time embedded applications. We proposed a novel eye state estimation method based on offline pattern classification compared to the PCA generated weight vectors to measure the PERCLOS and saccadic eye movement rate.
We implemented the algorithm on the NI CVS-1456 device (see Figure 1). The NI Compact Vision System devices are easy-to-use, distributed, real-time imaging systems that acquire, process, and display images from IEEE 1394 cameras. With an Ethernet connection between the CVS-1450 device and a development computer, the system can display measurement results and status information and configure the CVS-1450 device settings. When configured, the CVS-1450 device can run applications without a connection to the development computer. Its front panel consists of a vector generator analyzer (VGA) connector, an RS232 serial port, a 10/100 Ethernet connector, and three IEEE 1394a ports.
NI-IMAQ for IEEE 1394 Cameras is the interface path between the application software and the CVS-1450 device. The NI-IMAQ for IEEE 1394 Cameras driver software performs all functions necessary for acquiring and saving images. The CVS-1450 consists of a 733 MHz Intel Celeron processor with a 128 MB SDRAM, but we used only 32 percent of that for final implementation. This device works with NI LabVIEW software and the algorithm runs at 6 fps with an accuracy of more than 90 percent on the separate CVS-1456 device. Figure 1 shows the system operating in real time on the NI CVS-1456. It acts as a stand-alone system for different users under various conditions.
Conclusion
We created a system to monitor the state of a driver in real time for signs of fatigue. We selected an in-vehicle, online approach to estimate the state of a driver’s attention. Eye PERCLOS and saccade measurements are the best estimators of fatigue among the passive image-based approaches. We used a PCA algorithm for eye detection because it has high accuracy and low complexity. We proposed a new pattern-based method using the PCA for eye state estimation. We implemented the algorithm on an NI CVS-1456 device working as a stand-alone unit and achieved 6 fps with a success rate of more than 90 percent.
Author Information:
Aurobinda Routray
Indian Institute of Technology, Kharagpur
RTES Lab, Indian Institute of Technology
Kharagpur
India