Abstract
Driver cognitive distraction is a hazard state, which can easily lead to traffic accidents. This study focuses on detecting the driver cognitive distraction state based on driving performance measures. Characteristic parameters could be directly extracted from Controller Area Network-(CAN-)Bus data, without depending on other sensors, which improves real-time and robustness performance. Three cognitive distraction states (no cognitive distraction, low cognitive distraction, and high cognitive distraction) were defined using different secondary tasks. NLModel, NHModel, LHModel, and NLHModel were developed using SVMs according to different states. The developed system shows promising results, which can correctly classify the driver’s states in approximately 74%. Although the sensitivity for these models is low, it is acceptable because in this situation the driver could control the car sufficiently. Thus, driving performance measures could be used alone to detect driver cognitive state.
1. Introduction
Driver distraction is a major factor in traffic accidents, and it is estimated that up to 23 percent of crashes and near-crashes are caused by driver distraction. As the use of in-vehicle information systems (IVISs) such as cell phones, navigation systems, and satellite radios, will increase these figures will likely increase [1–6]. Thus enabling drivers to benefit from IVISs without diminishing safety is an important challenge [7]. One way to solve this problem is to detect the driver state in real time, and when distraction occurs, the corresponding warning system works to mitigate the effects of distraction [8].
Obviously, measuring driver state in real time is the core function in such systems. There has been an explosion research on these topics including the definition, classification, and detection of distraction. Donmez et al. [8] proposed a general definition that is “driver distraction is a diversion of attention away from activities critical for safe driving toward a competing activity.”
Generally, visual distraction and cognitive distraction are the two major types. Visual distraction can be described as “eye-off-road” and cognitive distraction as “mind-off-road” [9]. Both of them can undermine drivers’ performance.
Visual distraction is straightforward, occurring when drivers look away from the roadway (e.g., to adjust a radio), which can be reasonably measured by the length and frequency of glances away from the road [10]. Unlike visual distraction, cognitive distraction occurs when drivers think about something not directly related to the current vehicle control task (e.g., conversing on a hands-free cell phone or route planning) [6]. Therefore, in this paper we only detect driver cognitive distraction.
2. Measures Selection
There are five types of measures for driver inattention detection [11]: (1) subjective report measures (e.g., SSS, KSS); (2) driver biological measures (e.g., EEG, ECG); (3) driver physical measures (e.g., PERCLOS, gaze direction); (4) driving performance measures (e.g., steering wheel angle, yaw angle); (5) hybrid measures.
Since cognitive distraction needs to be done in real time and nonintrusively, the subjective report measures and driver biological measures are not suitable for a real-life context.
2.1. Driver Physical Measures
The most common used driver physical data for driver cognitive distraction are eye movements [6, 12]. Azman et al. [13] found that mouth and eyes are correlated to each other when a person is thinking or cognitively distracted and they could be used to detect driver’s cognitive distraction. Victor et al. [9] found that cognitive distraction causes drivers to concentrate their gaze in the center of the driving scene, as defined by the horizontal and vertical standard deviation of gaze distribution, and diminishes drivers’ ability to detect targets across the entire driving scene. Fletcher and Zelinsky [14] utilized faceLAB to obtain information such as eye gaze direction, eye closure, and blink detection, as well as head position.
2.2. Driving Performance Measures
A change in the mental state can induce the change in driving performance. Many studies prove the fact that compared to the attentive drivers the distracted ones steer their car in a different way; the same applies for throttle use and speed [15]. Some lines of evidence show that drivers adjust their behavior according to cognitive demand of secondary tasks. Drivers tend to increase the distance to the leading vehicle in the car-following scenario when they engage in cognitively demanding secondary tasks [16–18]. This suggests that drivers may compensate for the impairments that secondary tasks have imposed. One study found that drivers drove faster than the normal when distracted by a cognitive task [19]. Liang et al. [6] found that the driving measures improved the cognitive distraction detection model performance dramatically and built SVM models used driving performance only. The driving measures consist of standard deviation of steering wheel position, mean of steering error, and standard deviation of lane position. But compared with gaze behavior, they found that gaze-related features led to much better prediction accuracy than using the driving performance measures alone. The similar conclusion was found in [4]. Wollmer et al. [1] introduced a technique for online driver distraction detection that used LSTM recurrent neural nets to continuously predict the driver’s state based on driving and head-tracking data. The measured signals include steering wheel angle, throttle position, speed, heading angle, lateral deviation, and head rotation. These links between driving performance and cognitive state show that driving performance measures are good candidates to predict cognitive distraction.
2.3. Hybrid Measures
In [6], driver physical measures and driving performance measures were combined to detect driver distraction in real time. Comparing support vector machines (SVMs) to traditional logistic regression models, the results showed that the SVMs models performed better. In [20], machine-learning techniques were used to detect driver cognitive distraction based on the standard deviations of eye gaze, head orientation, pupil diameter, and average heart rate (RRI). The eye and head parameters were obtained using faceLAB, whereas the RRI data came from ECG. Sathyanarayana et al. [21] detected distraction by combining motion signals from the leg and head with driving performance signals using a -nearest neighbor classifier, the driving performance signals adopted including vehicle speed, braking, acceleration, and steering angle.
Among all of these measures, eye movements are one of the most promising ways to assess driver distraction [4, 6, 12]. While most of the eye movements parameters were obtained by faceLAB or SmartEye, these systems are not common in vehicles today, owing to their higher price for installation into a vehicle. At the same time there are limits in the process of extracting eye movements’ parameters [6].(1)Complex calibration: before each experimental drive, the calibration of the gaze vector with the simulator screen must be verified. After that, in the process of the experiment eye tracker must be calibrated to every participant and the calibration takes 5 to 15?min. After the complex calibration, the tracking error was approximately 5% of visual angle for most participants.(2)Driver restriction: the participants cannot wear glasses or eye make-up because these conditions can negatively affect tracking accuracy.(3)Environmental restriction: eye trackers may lose tracking accuracy when vehicles are traveling on rough roads or the lighting conditions are variable.(4)Time delay: the Seeing Machines’ faceLAB eye tracking system takes approximately 2.6?s to transfer camera image to numerical data.
These requirements limit the application of cognitive distraction system using eye movements parameters obtained from faceLAB or SmartEye; therefore, up till now, this scheme is only for research offline. More robust and real-time eye tracking techniques are needed to make these detection systems become a reality. While driving performance parameters could be obtained in real time from CAN-Bus directly, driving performance measures are used in this study for cognitive distraction detection. In this method, the characteristic parameters could be directly extracted without depending on other sensors, and system real-time performance and robustness are improved.
3. Model Development
Driving performance data were collected in a simulator experiment. The driving simulation directly outputs driving performance original data, which is collected each 10?Hz. After extracting characteristic parameters from the original data, SVM model was trained for each participant. Twelve subjects participated in the experiment to detect driver’s cognitive distraction state.
3.1. Experiment
3.1.1. Participants
Twelve participants (4 women and 8 men) aged 21–40 years old took part in this study. All participants were experienced drivers with valid licenses. Participants were recruited via an advertisement in school website.
3.1.2. Driving Simulator
The driving simulator used in this experiment is shown in Figure 1. The highway scenario is a 133?km long highway of a sampled actual ChangPing highway located between ChangChun and SiPing city, with two lanes in one direction. The traffic situation selected in this experiment was only sparse oncoming traffic and no traffic driving in the same direction as the test subject.
3.1.3. Driving Task
The driving experiment was to drive the simulator at 80–120?km/h. Every participant was asked to drive 4 sessions. In the first session, the participant drove 20 minutes to be familiar with the driving condition. In the following three sessions, the participants were asked to perform secondary tasks, including no workload tasks, low workload tasks, and high workload tasks. Each session took 25–35 minutes.
In the no workload task session, subjects drove the simulator without secondary task introduced, called no cognitive distraction (NCD). During the events, researcher randomly recorded ten different 60-second periods driving performance original data as NCD data.
In the low workload tasks session, subjects were asked to talk with the researcher. During the events, researcher ensures the talk diverts part of subjects’ attention away from activities critical for safe driving. The talk added subjects’ workload and made subjects think about something not directly related to the current vehicle control task but not led to be lost in thought, called low cognitive distraction (LCD). During the talking process, another researcher randomly recorded ten different 60-second periods driving performance original data as the LCD data.
In the high workload tasks session, subjects were asked to answer the researcher’s questions (intelligent test questions). These intelligent test questions diverted subjects’ attention away from activities critical for safe driving, and led subjects to think and be lost in thought, called high cognitive distraction (HCD). During the events, when another researcher ensured subjects thought about a question, driving performance original data were recorded as the HCD data. In this session, the original data record period depends on driver’s distraction state. This session took much more time until enough original data were recorded.
3.1.4. Original Data
The driving performance original data were directly obtained from the driving simulator including (1) vehicle velocity, (2) vehicle acceleration, (3) steering wheel angle, (4) steering wheel angular velocity, (5) throttle position, (6) yaw angle, and (7) yaw angular velocity.
3.2. SVM Model
3.2.1. Support Vector Machines (SVMs)
Support vector machine (SVM) is a popular machine learning method for classification, regression, and other learning tasks, which is first proposed by Vapnik [22].
The basic idea of classification using SVMs in 2D space is shown in Figure 2. Labeled binary-class training data , where is a vector containing multiple features and is a class indicator with value either -1 or 1, are illustrated as circles and dots in Figure 2, respectively.
They are mapped onto a high-dimensional feature space via a function . When the mapped data are linearly separable in the feature space, a hyperplane maximizing the margin from it to the closest data points of each class exists. The hyperplane yields a nonlinear boundary in the input space. The maximum margin represents the minimized upper bound of generalization error. The function is written in the form of a kernel function used in the SVM calculation. When data are not linearly separable in the feature space, the positive penalty parameter allows for training error by specifying the cost of misclassified training instances.
As cognitive distraction affects driving performance complexly, the learning technique of the SVM method makes it very suitable for measuring the cognitive state of humans.
3.2.2. Characteristic Parameters
The original data were preprocessed and generated a vector of characteristic parameters as listed in Table 1 as model input.
3.2.3. Window Size
Window size denotes the period over which characteristic parameters were averaged. In order to improve the real-time performance, the window size 1?s was chosen in this paper. The characteristic parameters were summarized across the window size to form instances as model input. For every participant there were 600 training instances to each distraction state (NCD, LCD, and HCD).
3.2.4. Model Training
Cognitive distraction affects driver behavior in a subtle, inconsistent manner, which can be easily washed out by individual differences associated with driving style [23]. Thus SVM model was trained for each participant. We randomly selected 200 training instances to each distraction state (NCD, LCD, and HCD) and used the remaining instances, which accounted for at least two thirds of total instances for testing.
The “LIBSVM” Matlab toolbox [24] was used to train and test SVM models, and LIBSVM is currently one of the most widely used SVM software. Linear, polynomial, radial basis function (RBF), and sigmoid are the four basic kernels. The RBF can nonlinearly map samples into a higher dimensional space, which can handle the case when the relation between class labels and attributes is nonlinear. At the same time, the RBF can reduce numerical difficulties and tend to obtain more robust results than other kernels, such as polynomial. Furthermore, compared to the polynomial kernel, the RBF has less hyperparameters which influence the complexity of model selection [25]. Therefore the RBF was chosen as the kernel function for the SVM models: where and represent two data points and is a predefined positive parameter. There are two parameters for the RBF kernel (the penalty parameter) and , and it is not known beforehand. In order to improve the model prediction performance, grid-search is recommended to identify good using cross-validation (CV). LIBSVM provides a tool to check a grid of parameters, and CV accuracy was obtained for each parameter setting. When the highest CV accuracy returns, are selected to the model.
3.2.5. Model Performance Measures
Model performance was evaluated with three different measures: accuracy, sensitivity, and specificity [26], which were calculated according to
The definition of sensitivity and specificity is shown in Table 2. where TU represents the number of true prediction instances; TON represents the total number of instances; TP represents true positive, which is defined as when the driver is distracted and the model detect result is distracted; TN represents true negative, which is defined as when the driver is not distracted and the model detect result is no distracted; FP represents false positive, which is defined as when the driver is not distracted and the model detect result is distracted; FN represents false negative, which is defined as when the driver is distracted and the model detect result is not distracted.
4. Experimental Results
Different SVM models were developed by using the same training instances from two of the three distraction states (NCD, LCD, and HCD) to compare the accuracy of this detection system based on driving performance measures when the driver was in different cognitive distraction states. NLModel, NHModel, LHModel, and NLHModel were developed, where NLModel was developed to distinguish LCD from NCD, NHModel was developed to distinguish HCD from NCD, and LHModel was developed to distinguish LCD from HCD.
The selection process for the best of a LHModel is shown in Figure 3. We searched for and in the growing sequences ranging from to according to (4.1). At last the best , were selected when the CV accuracy was 95.75%:
(a)
(b)
After all of the models developed, the mean detecting performance is shown in Figure 4. The mean accuracy for all NLModels was 78% (), the mean sensitivity was 56% (), and the mean specificity was 100% (). The mean accuracy for all LHModels was 66.14% (), the mean sensitivity was 84.86% (), and the mean specificity was 47.43% (). The mean accuracy for all NHModels was 76.87% (), the mean sensitivity was 54.07% (), and the mean specificity was 99.75% ().
The results thus far have demonstrated that driver cognitive distraction affects driving performance obviously, and driving performance measures could be used alone to detect driver state. The accuracy for LHModel is 66.14%. It shows that when driver workload changes from LCD to HCD, the driver performance changes. However, the accuracy for NLModel (78%) and NHModel (76.87%) is higher than LHModel (66.14%), which means the change rate of driving performance between NCD and LCD (or NCD and HCD) is higher than it is between LCD and HCD.
The sensitivity for NLModel (LHModel, NHModel) means that the accuracy rate of it predicts NCD (LCD, NCD) test instances as NCD (LCD, NCD), and the specificity for NLModel (LHModel, NHModel) means that the accuracy rate of it predicts LCD (HCD, HCD) test instances as LCD (HCD, HCD). From the detecting performance, specificity for NLModel is apparently higher than sensitivity, NLModel can predict NCD test instances accurately as NCD, while it classifies some LCD test instances as NCD. It suggests that the characteristics of NCD are more significant than that of LCD.
One low NLModel prediction performance is shown in Figure 5, where NCD label is 1, and LCD label is 2. It shows that NLModel can predict NCD class accurately (100%) and the LCD class prediction accuracy is only 46%. Though NLModel classifies some LCD as NCD, it can predict NCD accurately. False alarm rate for this system is low, which increases the system acceptance.
It appears that low sensitivity of this system increases the missing alarm rate. However cognitive distraction state (LCD or HCD) is defined as the driver performs a secondary task in this paper, and the subject’s distraction state is distinguished by researcher’s subjective judgment. At the same time, driving performance changes when the driver is in distraction state, but which is not totally different from NCD at any moment during LCD or HCD period. Especially for experienced drivers, the characteristic change between NCD and LCD (or HCD) is short and smooth. Thus part of the train and test instances from distraction state (LCD or HCD) is basically similar with NCD, and the sensitivity is relatively low. Fortunately, as long as driving performance is not affected by cognitive distraction, driver can control the activities critical for safe driving effectively, a situation where the driver is safe. Therefore low sensitivity is acceptable.
NLHModel is developed to classify NCD, LCD, and HCD, and NLHModel performance shows in Figure 6. The mean accuracy was 73.86% (), NCD prediction accuracy was 97.86% (), LCD prediction accuracy was 65% (), HCD prediction accuracy was 60.14% (), sensitivity was 61.86% (), and specificity was 97.86% (). The sensitivity for NLHModel means the accuracy rate of that it predicts NCD test instances as NCD, and the specificity for NLHModel means the accuracy rate of it predicts LCD and HCD test instances as LCD and HCD. Thus, NLHModel could be used to predict different cognitive state, and it has the similar conclusion as NLModel and NHModel.
5. Discussions and Conclusion
Compared with driver physical measures, using driving performance measures to detect driver cognitive distraction is more effective, simple, and of real time, so it is used to detect driver state in this paper. NCD, LCD, and HCD were defined as three different cognitive distraction states using different secondary tasks. Twelve drivers were recruited to take part in the experiment. For every participator, 7 original data about driving performance were obtained from the driving simulator directly, and 14 characteristic parameters were extracted as SVM models input. In order to improve real-time performance of the developed models, window size used in this research was 1?s. At last, different SVM models (NLModel, NHModel, LHModel, and NLHModel) were developed by using the same training instances from two of the three distraction states (NCD, LCD, and HCD) to compare the accuracy of this detection system when the driver was in different cognitive distraction states. The mean accuracy of each SVM model is approximately 74%; thus driving performance can be used alone to detect driver cognitive state. The specificity is up to 99%, and false alarm rate for this system is low, which increases the system acceptance. The sensitivity of each SVM model is low, which is acceptable, because in this situation the driver could control the car sufficiently.
At the same time, the participator’s cognitive state is distinguished by researcher’s subjective judgment, which affects the model’s accuracy seriously. Therefore it could be helpful to use driver biological measures as the standard for future research to select correct and reasonable training instances as model’s input. Furthermore, SVM models were trained for each participant, and it might be interesting to select characteristic parameters which cannot be affected by individual differences driving style.
Acknowledgments
This research was supported partly by Doctoral Study Special Research Foundation of Higher Education (no. 20110061110036), Major Projects of Jilin Science and Technology Department (no. 20116017), New Century Excellent Talent Foundation Program under grant and National Nature Science Foundation of China (no. 50908098).