Abstract
In the driver fatigue monitoring technology, the essence is to capture and analyze the driver behavior information, such as eyes, face, heart, and EEG activity during driving. However, ECG and EEG monitoring are limited by the installation electrodes and are not commercially available. The most common fatigue detection method is the analysis of driver behavior, that is, to determine whether the driver is tired by recording and analyzing the behavior characteristics of steering wheel and brake. The driver usually adjusts his or her actions based on the observed road conditions. Obviously the road path information is directly contained in the vehicle driving state; if you want to judge the driver’s driving behavior by vehicle driving status information, the first task is to remove the road information from the vehicle driving state data. Therefore, this paper proposes an effective intrinsic mode function selection method for the approximate entropy of empirical mode decomposition considering the characteristics of the frequency distribution of road and vehicle information and the unsteady and nonlinear characteristics of the driver closed-loop driving system in vehicle driving state data. The objective is to extract the effective component of the driving behavior information and to weaken the road information component. Finally the effectiveness of the proposed method is verified by simulating driving experiments.
1. Introduction
At present, the identification methods of dangerous driving are identification method based on physiological information, identification method based on driver operation information, machine vision recognition method, and vehicle motion identification method.
The physiological information method is to study the dangerous driving state by using biometrics, in particular, the physiological parameters such as electroencephalogram (EEG), electroencephalogram (EOG), electromyography (EMG), respiratory flow, pulse, and blood pressure to identify dangerous driving conditions. Lal et al. found that fatigue driving status correlated with EEG and used changes in EEG to identify fatigue [1]. Fu et al. used EMG and ECG signals to analyze the fatigue status of driver during driving and selected the characteristic parameters of fatigue driving to identify fatigue driving status [2]. Hostensa and Ramonb found that when the driver was in fatigue, the frequency of the EMG signal showed a downward trend, and the amplitude of the EMG signal showed an upward trend, which could serve as a recognition feature for fatigue driving [3]. Japanese researchers at the University of Tokyo found that driver’s lactic acid, ammonia, and alcohol content in human sweat in fatigue and normal driving are different [4]. Jagannath and Balasubramanian report a study that evaluated driver fatigue using multimodal fatigue measures, that is, surface electromyography (sEMG), electroencephalography (EEG), seat interface pressure, blood pressure, heart rate, and oxygen saturation level [5].
Since the method of judging the driver’s dangerous driving condition based on physiological information needs to install the sensor on the driver and the application requires accurate correction or debugging of the sensor, it can not be widely used in practice.
Machine vision recognition of driver information is a common method of identifying dangerous driving behavior. Driver head movement, eye closure frequency, eye movement, and mouth movement are commonly used to identify dangerous driving status indicators. Wierwile et al. established the percent eyelid closure over the pupil over time (PERELOS) indicator to identify the driver’s dangerous condition by comparing the eye closure time between a dangerous driving and a normal driving [6]. Azim et al. also demonstrated the validity of the PRERLOS index. National Aeronautics and Space Administration (NASA) has also developed a driver fatigue detection system using PERELOS. The Drowsy Diver Detection System (DDS) detects the driver’s eye movement, blink duration, and blink frequency using a camera to identify the driver’s dangerous driving [7]. Jackson et al. proposed using different brightness of the dual light source and the parity difference method to achieve the pupil positioning in order to determine the PERELOS indicator to identify the driver’s dangerous driving state [8]. Javier et al. propose a new framework for driver assistance systems based on driver state, especially emotions: advanced driver monitoring for assistance systems [9]. Yawning is also a driver in the fatigue state of a performance. Fan et al. proposed the use of gray projection to detect the driver’s mouth position, use the Gabor texture feature to identify the driver’s yaw type, and then identify the dangerous driving state [10].
Although the dynamic identification of driver’s behavior on machine vision can be realized, the influence of environmental factors is great, and the driver’s habitual action has great influence on the recognition accuracy of driving behavior, and its application universality is poor.
The dangerous driving recognition based on the driver’s operating action (such as turning the steering wheel frequency, step on the accelerator, and brake strength) is performed by the association analysis between the driver’s operation information and the driving state to identify the dangerous driving state. For example, Mortazavi et al. found that the driver’s frequency of operating the steering wheel during fatigue was significantly reduced. This information could be used for driver fatigue identification [11]. The United States Electronic safety Product Company also uses the driver’s steering wheel operation to identify the driver’s fatigue and the development of the driver fatigue detection device steering attention monitor. The device is mounted on the steering wheel, if the steering wheel does not turn for 4 seconds, is considered to be a feature of the driver at the beginning of fatigue and can be used to make early warning of the driver’s fatigue. Liang and Lee found that the driver operating the steering wheel slowed down in distraction. And through this index established the driver distracted state recognition method [12].
It is a convenient and practical method to identify dangerous driving status based on vehicle motion information. Because the driver in dangerous driving state, the reaction speed and attention will be reduced, resulting in abnormal movement of vehicles, such as lane deviation. Therefore, many scholars have established a way to identify the driver’s dangerous driving conditions by means of vehicle speed variation, lateral deviation, acceleration, and deceleration [13–15]. The United States AssistWare Technology company developed the SafeTrace system; the system uses the front camera to detect the front lane line, according to the deviation of the vehicle to identify the driver’s dangerous driving status. United States Ellison Research Lab developed DAS2000 road warning system; its principle is the use of lane deviation to identify the driver’s dangerous driving state.
Vehicle driving information can be used to identify the driver’s dangerous driving behavior. However, not all vehicle motion characteristics help to identify the driver’s dangerous driving behavior. Some characteristics not only have not contributed but also affect the accuracy and calculation speed of the driver’s dangerous behavior identification. Steering wheel angle and lateral acceleration are an important part of vehicle driving state. The time series of steering wheel angle is the reflection of driver’s input to external environment after mental judgment and analysis. The vehicle movement dynamics study shows [16] that the driver can be regarded as an intelligent controller in the closed-loop drive system, when the mental state of the driver changes, the steering wheel angle and lateral acceleration will change.
The driver behavior indirect analysis method based on vehicle driving status data can realize the monitoring and diagnosis of driving behavior under “zero interference.” The specific principle of this method is to use signal processing technology to extract features from the vehicle operation status and driver operation behavior and then compare it with the known characteristics of normal driving status and dangerous driving situation and determine the status of the driver according to the comparison results. However, the change of steering angle caused by the driver’s mental state is mixed with the change of steering angle information caused by road information; they can not be effectively separated.
The above research shows the feasibility of extracting driver’s behavior information from vehicle driving status information, but the biggest bottleneck of this method is how to eliminate or weaken the driving path information contained in steering wheel angle or lateral acceleration signal. The purpose of this paper is to propose a new preprocessing method of dangerous driving behavior identification based on vehicle motion information. This paper studies the driver state information component extraction method based on information entropy statistics in vehicle driving state information.
2. The Method of Removing Road Component
In the dangerous driving state, the reaction speed and attention deficit lead to the abnormal state of vehicle driving status data, such as steering wheel angle, lateral acceleration, trajectory, yaw rate, vehicle speed, and longitudinal acceleration. Taking the steering wheel angle as an example, actual steering wheel angle signal can be divided into two parts: one part reflects the curvature of driving path and contains the path information to a maximum degree. Another part is the driver’s habit information and mental state information, which is modulated by the curvature information of the road. Driving habits are formed during the long period of driving; when the driver is in the initial stage of dangerous driving state, the habit behaviors will increase. This feature is reflected in a high frequency rhythm in the steering wheel operation signal. According to the low pass filtering characteristics of the closed-loop driving system, the path information is reflected in a low frequency signal after being processed by the driver.
Take a typical example to explain the following: even the road is a sharp turn, the skilled driver also wants to make the vehicle run smoothly; this is a low pass filtering process. This shows that the road information has a different frequency band with the driver state information, but the nonstationary and nonlinear characteristics of the closed-loop driving system lead to the unpredicted intersection of the road information and the driver’s state information. The empirical mode decomposition (EMD) can adaptively decompose the high and low frequency fluctuation in the signal into different internal model functions; it can effectively deal with the problem of frequency band overlapping. Therefore, it is proposed to decompose the vehicle driving state into different internal model functions by using EMD and then look for the effective IMF component which can reflect the driver’s state by using the approximate entropy as index; its synthetic signal reflects the driver’s state information in a maximum degree.
Empirical mode decomposition is an adaptive decomposition method; it is the time scale decomposition based on the local characteristic of signal. The method adaptively decomposes the signal into different internal model functions (IMF); the performance of basic function in the decomposition process is driven by its own data. Its outstanding advantage is to achieve adaptive division of high and low frequency signal and applicable to both steady-state signal and nonstationary signal [17]. This method is considered to be a breakthrough for the traditional linear spectrum analysis and has caused widespread concern of experts in related fields.
The personality information contained in the driver’s operation habit is formed during the long driving course; it shows a certain rhythm in the steering wheel angle signal. Road environment information is completely real-time information to the driver and is the information that can not be expected in advance. Although the driver may know the macroeconomic situation of next section, but it is impossible to predict the instant traffic of next time, the instant road information determines the operation of the driver and further determines the vehicle driving state information. So the road information has the characteristic of pseudorandom signal. And approximate entropy is an index to evaluate the complexity of time series; the greater the entropy, the more complex the time series; namely, the disorder is higher. Inspired by this feature, the approximate entropy of each IMF after EMD decomposing from white noise is taken as the template, so as to compare and analyze all the IMF approximate entropy of normalized steering wheel angle. Those beyond the template can consider that the IMF component is an effective component of driver’s driving information. Those in the template can be considered as invalid component that does not contain driving dynamic behavior. The specific steps are shown in Figure 1.
3. Empirical Mode Decomposition and Basic Concepts of Approximate Entropy
3.1. Basic Principle of EMD
In 1998, American scientist Norden E. Huang put forward the new signal processing method of empirical mode decomposition. The method adaptively decomposes the signal into the sum of different internal model functions IMF. During the decomposition process, the performance of basic function is driven by its data; its outstanding advantage is able to achieve the adaptive division of high and low frequency signals, applicable to both steady signal and unsteady signal [18]. This method is considered to be a breakthrough for the traditional linear spectrum analysis and has caused widespread concern of experts in related fields. EMD decomposition method can decompose any signal by following steps.
Step 1. Find all local minimum and maximum values included in the sequence to be decomposed, then apply the cubic spline interpolation method to calculate all upper and lower envelopes and formed by local minimum and maximum values, and calculate their average curve :
Subtract with and get
Judge whether the minimum and maximum value number of are equal to the number of zero crossing point and whether the average value of its envelope is zero. If does not meet the above conditions, the above process should be repeated to obtain with as the original signal:
In order to avoid the excessive decomposition which may make the decomposition result have no practical application value, EMD decomposition algorithm defines termination conditions: standard deviation (short for SD) should be less than a certain value between 0.2~0.3. The definition of standard deviation can be expressed by the following formula:
Screen until meets the foundation conditions of IMF or meets the termination conditions; namely,
Then the first IMF has been decomposed form original signal after above steps, and it is marked as .
Step 2. Subtract the first-order IMF component from original signal , and get the rest signal :
Because the rest signal after the first decomposition contains the low frequency component of original signal, the rest signal can be regarded as an original signal; repeat Step in the EMD algorithm; then the th-order IMF and the th-order rest signal can be obtained:
When the rest shows a monotonic trend or the value is rather small, which meets the screening condition, stop the continuing decomposition process. And the following formula can be finally obtained:
The signal can be decomposed into the linear combination of a set of internal model functions by above method. The decomposition process of EMD is a kind of innovation from the perspective of signal decomposition basis function theory. The basis function of EMD is a series of harmonic function with variable amplitude and variable frequency and is obtained based on the data driven adaptive of signal. Through Monte Carlo experiments, literature [19, 20] proved that the EMD method has the band pass filter characteristics of the constant quality factor which is similar to the wavelet decomposition; its cut-off frequency and bandwidth are changed with the change of signal. According to the literature [21], the IMF component obtained by EMD decomposition contains different characteristic time scales, and the frequency resolution of the th IMF can be expressed in the following formula:In this, indicates the maximum frequency of the th IMF and is number of sampling point.
It can be seen that the frequency resolution of each IMF component is different; this is different from the characteristic in wavelet analysis that the time and frequency interact. From above analysis, we can know that the EMD method preserves the nonlinear and nonstationary characteristics of original signal. As for the analysis of nonstationary signal, EMD results have more physical meaning undoubtedly than the Fourier transform, wavelet analysis that are based on harmonic analysis.
It is worth noting that the end effect must be considered in EMD decomposition. This effect can destroy the whole data sequence and make the filter result in a serious distortion. In order to solve this problem, some researchers at home and abroad have put forward a series of effective methods to reduce the end effect. Common methods are polynomial fitting continuation method [22]; extension method based on AR model prediction [23]; extension method based on neural network prediction [24]; extension method based on support vector machine prediction [25]; mirror extension method [26]. Combined with the actual, this paper applies the mirror extension method to inhibit the end effect.
3.2. Basic Concept of Approximate Entropy
Approximate entropy is an index which reflects the complexity of random time series from the statistical point of view; it was proposed by Pincus [27]. Approximate entropy index is widely used in the field of neural biological signal analysis [28, 29], machinery and equipment fault diagnosis field [30, 31], and current signal analysis in electric arc welding [32]. The specific calculation process is given as follows.
Suppose a time series data ; when giving the embedding dimension and the initial value of similar tolerance , the specific calculation steps of approximate entropy are as follows.
Step 1. Arrange the time series into dimensions vector in accordance with the order; namely,
Step 2. Calculate the distance between vectors and for each
Step 3. According to the initial given similar tolerance , the similarity between vector and the rest vector can be expressed with following form:where
Step 4. Get the natural logarithm of , then calculate its average value to all , and record as
Step 5. Repeat the process from Step to Step for and get .
Step 6. Theoretically, approximate entropy of series can be expressed as Generally, limit of formula (15) exists when tends to . But in the actual operation, can not be infinite great, and when its value is limited, the estimated value of approximate entropy calculated according to above steps is
According to the calculation process of approximate entropy, the calculated value of approximate entropy is related to the embedding dimension and similar tolerance . If similar tolerance reduces, the pattern that meets the similar conditions will reduce, and the estimation accuracy of approximate entropy will be poor. On the contrary, if similar tolerance increases, the similar pattern is excessive; detailed information of the time series will be lost. In order to optimize the embedding dimension and similar tolerance , literature [33] proves that when and ( represents the standard deviation of time series , the estimated approximate entropy can reasonably reflect the complexity of the signal after a lot of experiments. This paper applies and on studying the IMF approximate entropy property of EMD of the white noise.
3.3. The Distribution of IMF-ApEn of White Noise
EMD is quite suitable for analyzing the nonstationary and nonlinear signals; however, even with these advantages, it can not process the complex situation when the interference level is unknown. Therefore, it has become a primary task to study the IMFs characteristics of white noise for extracting useful information from the mixed noise.
Due to lack of the analysis method for IMF-ApEn of white noise, this paper applies numerical test method. In fact, many important conclusions of scientific research come from numerical experiments. The statistical template of white noise IMFs approximate entropy is the key problem to be solved to identify effective modal component.
In order to study the statistical properties of IMF-ApEn of unit white noise, the method described in the literature [34] is taken to produce the unit white noise. Reveal the relationship between the probability density function of each white noise IMF-ApEn and sampling rate, data length through numerical simulation experiment. Before discussing a more detailed statistic characteristic of each white noise IMF-ApEn, we should firstly understand the concept of IMF average cycle and two-dimensional probability density estimation.
3.3.1. IMF Average Cycle Time
The EMD decomposition of white noise or fractal Gauss noise is equivalent to a set of two filter banks; the Fourier variation spectrum of each IMF is approximate to a single shape along with the logarithm of cycle or frequency. Taking into account the scale loss effect, IMF number of white noise should be less than ; is the length of noise series. IMF is defined as an envelope function formed by local maximum and minimum value; its extreme value number is equal to zero crossing point. From the definition we know, the IMF average cycle is determined by the number of local peaks. The practical calculation method can be obtained by fast Fourier transform: where refers to the Fourier transform.
3.3.2. Two-Dimensional Probability Density Estimation
Suppose that is a multidimensional continuous random variable and is the probability density function of multidimensional joint distribution. is probability density estimation of unknown distribution based on sample points; then the kernel estimation of multivariate random variables can be defined as where is random vector sample; is number of samples; is window width of kernel estimation; is -dimensional kernel function; it needs to meet the following conditions:
Suppose ; then 2-dimensional probability density can be written as
The difference between actual probability density and estimated probability density can be expressed with the mean integrated square error
Therefore, the optimal kernel estimation window width can be obtained through derivation for MISE:
When the number of samples is fixed, the accuracy of probability density estimation is mainly affected by the window width of kernel estimation. According to formula (22), the optimal window width depends on the unknown probability density function , so the optimal window width calculation formula is not given above. If the unknown probability density function takes kernel function as the base function, the actual optimal window function can be obtained by the least squares method:and it is easy to prove
According to formula (25), when the value of parameter makes to be the smallest, also has the smallest value, so the optimal window width can be determined with formula (23) in practical application.
3.3.3. Joint Probability Density of Average Cycle and All IMF Approximate Entropy of White Noise
In order to count the joint probability density of average cycle and IMF approximate entropy of white noise after EMD decomposition, 200 groups of IMF-ApEn from white noise decomposition and the Log-M-P of its average cycle are taken as sample data. Obtain the probability density function of joint distribution with 2-dimensional kernel function estimation method. Specific steps are as follows.
Step 1 (initialization). Produce sets of white noise data .
Step 2 (normalization). Transform the white noise value to 0-1, suppose that the value in white noise is , and complete the normalization through .
Step 3 (EMD decomposition). Carry out EMD decomposition for to produce sets of IMF, which is expressed as .
Step 4. Calculate the approximate entropy of all and its average cycle and obtain a series of , for is the serial number of IMF, is the length of noise time series, and is noise sample number.
Step 5 (probability density estimation). After obtaining the IMF-ApEn of sets noise, calculate the joint probability density of approximate entropy-average cycle according to formula (20).
Log-M-P in Figure 5 is the natural logarithm of IMF average cycle after normalization. Suppose that is the sampling frequency of white noise, is point number of random signals, is the sampling time of random signal, is the mean IMF cycle of white noise, and is the point number contained in the average cycle of IMF. Then normalized average cycle is defined as follows:
Obviously, the range of average cycle after normalization is between 0 and 1. White noise has a flat power spectral density and self-similarity; namely, the local signal is similar to the whole signal. From the above features, the conclusion can be drawn: increasing the sampling time of white noise signal when the sampling rate is fixed is equivalent to increase the sampling rate of white noise when the sampling time is fixed. So studying the influence of different sampling rates on the statistical properties of white noise is equivalent to the influence of different length on the statistical characteristics of IMF-ApEn at different levels when the sampling rate is fixed.
4. Experiment
Considering that it is not allowed to carry out fatigue driving experiments on the real road conditions by China’s traffic laws and regulation, it is dangerous for drivers to carry out fatigue driving study in the experimental site, and the environmental stimulation of experimental site is not consistent with the actual, so the construction of simulation driving experiment platform is the most important problem to be solved in the dangerous driving experiment.
Currently, driving simulator has become a common means of training drivers; it can reflect the closed-loop driving characteristic of man-car-road. Moreover, most pieces of current research literature of driver behavior use driving simulator as one of the main experimental means. Driving simulator is mainly used to establish a virtual driving environment. Through the virtual input of different road, dynamic characteristics, and environment, it obtains the driver’s real driving action and response of control state. This shows that the vehicle driving state data contains the driver’s real operation and its own state information.
So, although the collected vehicle driving state data comes from the virtual simulation data of driving simulator, it can meet the needs of collaborative coupled diagnosis experiment of dangerous driving behavior.
The driving simulator used in the experiment should have a high simulation ability of vehicle dynamics and road performance, so as to meet the requirements of dangerous driving experiment; meanwhile, it should also record the running state data in the virtual running of the vehicle, so as to solve the data acquisition in dangerous driving research.
4.1. Structure of the Simulation Driving System
Structure of the simulation driving system is shown in Figure 2; the system includes driver operation information input module, vehicle dynamics, kinematics analysis model, virtual-reality driving environment generation model, and vehicle running state parameter recording module.
4.1.1. Input Module
Mainly providing the input of vehicle operation signal, through this module, the steering wheel angle, throttle, brake, shift, and other information are input to the simulation driving system.
4.1.2. Dynamics Analysis Module
This module is the basic part of the whole simulation driving system. According to the parametric model of the vehicle and driver operation signal, it calculates the vehicle driving state information data, in real-time, so as to provide the basis for driving 3D vehicle model in virtual environment. The data source of vehicle dynamic simulation module is input model; the real-time vehicle state can be calculated by solving vehicle dynamic model, so that the driver can really feel the driving effect of the vehicle in virtual environment.
4.1.3. Virtual-Reality Driving Environment Generation Model
The main function of this module is to provide a realistic virtual driving experiment environment for the driver; it is the core of the whole system. 3D interactive scene display of driving link can be produced through the virtual-reality technology, so as to obtain the input response of real driver and the driving state data of virtual vehicle.
4.1.4. Vehicle Running State Parameter Recording Module
This module collects and records the driver’s input data, such as steering wheel angle, throttle opening, brake pedal opening, gear position, and vehicle output data (including speed, acceleration, position, and attitude angle), which provides data support for the collaborative coupled diagnosis experiment of dangerous driving.
4.1.5. Interface Module
Through the interface model, the real-time vehicle simulation, virtual running environment, steering wheel, and other interface devices, drivers are organically linked into an entire. This not only includes the coordination of hardware but also includes human-computer interaction and other technical contents.
4.2. Driver’s Mental State Calibration
4.2.1. An Evaluation Method of Driver Proficiency
In order to avoid inexperienced drivers to participate in the experiment, the method of combining the orbital tracking error index and the vehicle direction error index is used to evaluate the driver’s proficiency. Sophisticated drivers are able to keep the trajectory of the vehicle to the maximum extent consistent with the intended trajectory, whereas unskilled drivers have relatively weak lane retention. In addition, the skilled driver is able to keep the direction of the vehicle body axis in the same direction as the longitudinal speed of the vehicle, and the unskilled driver is less stable to the vehicle control. To this end, the following two indicators are defined to evaluate the proficiency of the driver.
The expression for the trajectory tracking ability index is as follows:
wherein is the desired trajectory of the driver at time , is the actual trajectory of the vehicle, is the number of samples recorded.
The expression for smooth driving ability index is as follows:
wherein is the longitudinal speed of the vehicle at time , is the center-of-mass side-slip angle of the vehicle at time , is the number of samples recorded.
Figure 3 is 11 drivers track tracking ability indicators and smooth driving ability indicators statistical box diagram. The -test (level = 0.05) showed that the trajectory tracking ability index and smooth driving ability index were related to the driver’s proficiency. From this we can see that the trajectory tracking ability index , smooth driving ability index can reflect the driver’s proficiency.
(a) The trajectory tracking ability index
(b) Smooth driving ability index
4.2.2. An Evaluation Method of Driver Fatigue
The driver’s fatigue appears as a driver of short-term memory loss, reduced judgment, response time longer, and distraction. Attention and reaction time are used as the objective evaluation indexes of fatigue driving.
In the simulated driving experiment platform, a screen mask program is embedded (a region of the computer screen becomes black after the program is run, so that the driver can not perceive the environment). When the driver sees the mask, he can click the button to obtain the opening time of the shield. The value of the attentional indicator can be calculated by calculating the time difference between the closing and reopening of the mask. The formula is as follows:
wherein is the number of times the mask has been opened during the test period; is Sets the opening time of the mask after one key press; is the time interval between the closing of the mask and the reopening; is the average reaction operation time from the detection of the closing of the mask to the reopening of the shadow mask.
In the driver in front of the red, yellow, and green three colors of the image, the experiment requires the driver to immediately respond to see the corresponding color. Red press A, yellow press B, and green press C record the driver’s reaction time. The reaction time index is calculated as follows:wherein is the number of valid tests; is test sequence number; are maximum reaction time and minimum reaction time; is the validity parameter, if the key is correct and , then ; otherwise ; is the driver’s reaction time at the th test.
A total of 11 skilled drivers participated in the experiment; the reaction time of different experimental stage indicators and attention indicators were analyzed; the results are displayed as statistical box diagram. The first stage of the experiment: before the experiment, a reasonable driver fatigue assessment method was used to measure the mental state of the driver. After completing the test, volunteers were required to drive the virtual vehicle for 10 minutes to 20 minutes to become familiar with the virtual driving environment. The second stage of the experiment: volunteers were asked to simulate continuous driving for 120 minutes, and then we measured the mental state of the volunteers. The third phase of the experiment: continue for 120 minutes of simulated driving, and then measure the mental state of the volunteers. The fourth stage of the experiment: volunteers, 10 minutes after the break, continue driving for 120 minutes and then measure the mental state of volunteers.
Figure 4 shows the driver’s reaction time and the change of the attention index during different driving phases during the simulation experiment. The variance analysis showed that the response time index and the attention index of the four stages were significantly different (), and the driver’s attention index decreased significantly after completion of the experiment (stage 4) compared to the preparation stage (stage 1).
(a)
(b)
4.3. Extraction Experiment for Driving State Principal Component
After completing the virtual driving experiment platform and formulating the evaluation method of driver’s mental state, the driver’s state information principal component extraction experiment can be done. It is worth noting that the experiment is not in order to classify the mental state of the driver, but in order to remove the road information contained in the vehicle driving state data. The state of the driver in the experiment was evaluated only to ensure that the driver involved in the experiment was a qualified driver and avoided human error in the removal of the road information.
The working flow of virtual driving system is as follows: firstly, the parametric model of the vehicle and other pieces of information are imported into the virtual environment, according to user’s control information input, calculate the vehicle’s state information in real-time with vehicle dynamics analysis module (add driver’s real-time control information), and then combine the calculated data with vehicle in virtual scene, thus realizing the virtual simulation of driving process. By virtual-reality technology, users can not only operate the vehicle in virtual environment but also obtain the visual motion feedback of the virtual driving process and enhance immersion.
The vehicle dynamics model is a mathematical model that reflects the driving characteristics and control of the vehicle and is an important basis for measuring the authenticity of virtual operation in the virtual driving system. According to actual condition of virtual driving system, vehicle dynamics model applies a model of eight freedom degree. After the vehicle dynamics model is established, the exportable simulation driving system of vehicle running state has been established by 3D model generation technology and 3D model generation technique for the road and surrounding environment.
Figure 5 is the virtual driving experiment platform that has been successfully developed. The platform is mainly composed of software and hardware, of which the software part is composed of virtual-reality module, vehicle dynamics solution module, vehicle dynamics parameter record module, and collision detection module. The hardware part includes a computer used for simulation and a steering wheel used for simulation driving. Simulation steering wheel has the dual engine force feedback function, which can simulate the state of real driving vehicle, and it is a more advanced vehicle virtual simulation input device.
Vehicle driving state information in driver’s driving, such as steering wheel angle and lateral acceleration, is obviously affected by the road path. How to remove or weaken the road path information contained in the steering wheel angle and lateral acceleration signal is a key step for the research of feature extraction of dangerous state driving behavior and classification on the basis of vehicle driving state signal.
This part tries to use the IMFs approximate entropy noise template matching method to weaken the road information component in the vehicle driving state. Here we take steering wheel angle as an example.
Steering wheel angle information is divided into two parts:where is steering wheel angle signal; is principal component of road information; reflects the driver’s driving habits, mental status, and other information.
The personality information of driver’s operation habits is formed in the long-term driving process; it shows a certain rhythm in the steering wheel angle signal. Inspired by this feature, the approximate entropy of each IMF after EMD decomposing from white noise is taken as the template, carries out EMD decomposition for the normalized steering wheel angle information, and solves the approximate entropy of each IMF and then compares with the normalized white noise template. Those beyond the template can be considered that the component decomposed by EMD represents a regular rhythm signal; those in the template can be considered as invalid component that does not contain driving dynamic behavior.
The driving simulator is shown in Figure 5. Figure 6 is the experimental path of simulation driving (the length of 8-shape-like road is 1600 meters) and part of vehicle driving state data (steering wheel angle and lateral acceleration).
(a) Experimental road
(b) Overview
(c) Overlooking
(d) Driver view at A point
(e) Steering wheel angle
(f) Lateral acceleration
Figure 5 is the site distribution of each IMF component’s approximate entropy on the joint probability density function contour map of IMFs-ApEn and Log-M-P (normalized average natural logarithm) after the EMD decomposition of normalized steering wheel angle signal.
The outermost contour is determined by confidence level ; its meaning is that if the Log-M-P, IMFs-ApEn of measured signal is in this region, it has a probability of 95% to be invalid IMF component (FIMF). If it locates in the close area identified by the outermost contour, it has a probability of 95% to be valid IMF component (EIMF).
Through the comparison between the IMFs approximate entropy and the white noise IMFs approximate entropy of normalized steering wheel angle signal, it can be seen that, after the steering wheel angle signal is decomposed by EMD, the IMF2, IMF3, IMF4, and IMF5 are outside the confidence region of template; then they can be determined as the effective modal component; the superposition of IMF can be regarded as the driver’s driving habits and driving state information. IMF1, IMF6, IMF7, and IMF8 fall outside the confidence region; they can be regarded as invalid modal component of driving behavior information; it may be caused by noise interference and low frequency road information interference.
Figure 6 shows the result of road information component method in steering wheel angle signal, of which the road information component method of vehicle driving state is weakened by the IMFs approximate entropy noise template matching method. Figure 7(a) is the steering wheel angle signal of simulation driving experiment carried by the driver in specified path (shown in Figure 6). Figure 7(b) is the synthetic signal of effective IMFs component; namely, ; it mainly reflects the driver’s driving habits, mental state, and other pieces of information. Figure 8(c) is the related components of road environment information; namely, ; it mainly reflects the false component caused by noise interference and low frequency road information interference. From the figure, it can be seen that has a strong correlation with the road curvature. Thereby, the method can effectively peel off the road information contained in steering wheel angle information.
(a) IMF1
(b) IMF2
(c) IMF3
(d) IMF4
(e) IMF5
(f) IMF6
(g) IMF7
(h) IMF8
(a) The actual steering angle
(b) The driver’s behavior information component
(c) The path information component
(d) Centroid trajectory curvature
4.4. Discussion on Experimental Results
4.4.1. Positive Results of the Experiment
It can be seen from the experimental results that the method can remove the path information mixed in the steering wheel angle information. An advantage of this approach is that its removal rules are self-data driven.
Figure 8 is a typical example of using this method to remove a road information component contained in steering wheel corner information. In order to evaluate the effectiveness of this method, the center-of-mass curve (Figure 8(d)) of the vehicle is generated by the record of the vehicle track data. From the experimental results on the simulated driver (Figures 8(d) and 8(c)) the method of this paper effectively stripping the road information component can be seen. The most obvious difference between the method of this paper and traditional filtering method is that it can effectively decompose the vehicle travel status information into the IMF domain and effectively avoid the frequency confusion of traditional filtering methods.
4.4.2. A Disadvantage or an Improvement
The method does not solve the driver’s mental state identification. The preprocessed information also needs to be further processed by the feature classification algorithm.
Because the road information needs to be calculated and retrieved in advance, the feature matching template is acquired, and its computation time is long and the real-time performance is poor. These features restrict its application in real-time warning. The efficiency of the algorithm needs to be studied in order to improve its real-time. The second limitation is that the accuracy of the feature template requires the relevant parameter information of the vehicle. The real-time acquisition of this parameter information is difficult and limits the popularity of its application. The next step is to study the real-time acquisition algorithm of vehicle state parameters to improve the practicability of the method.
4.4.3. The Idea of Future Research
In this paper, the study of dangerous driving behavior is carried out in the simulator, and the actual driving situation remains to be done to further study the differences. The research of this paper is only the pretreatment of the driver state analysis based on the vehicle driving state information, and the postprocessing technology needs further study. The method opens up a new way for the dynamic monitoring of dangerous driving behavior, which is of great impetus to the early warning technology development of dangerous driving behavior. In addition, the method can also improve the level of traffic accident investigation. Through the historical record analysis of the vehicle state data, it can restore the driver’s dangerous driving level before the accident, which is of great significance to the traffic accidents or hidden dangers.
5. Conclusion
Dangerous driving behavior is one of the main causes of vehicle collision accidents. Vehicle assisted driving system is an important means to reduce vehicle collision accidents. Therefore, the identification method of dangerous driving behavior of vehicles is of great significance to the development of vehicle safety auxiliary system and the safety of driving. The main contribution of this paper is to propose a method to extract the effective components of driving behavior information from vehicle driving state data, so as to provide an effective preprocessing method for the driving behavior recognition algorithm based on vehicle driving state data. The main conclusions are as follows:(1)A virtual experiment platform of vehicle motion information is established to study the dangerous driving behavior of vehicles. In this paper, a vehicle motion information acquisition experimental platform is established by virtual simulation technology. Based on this, the method of separating the path information from vehicle state data is studied.(2)For the nonstationary and nonlinear characteristics of closed-loop driving system, considering the frequency distribution characteristics of driver behavior information component and road information component in vehicle status data, an effective IMF selection method based on EMD decomposition template is proposed. A method of effective component separation of driver’s driving behavior information in vehicle driving state data is constructed.(3)The distribution characteristics and regularity of normalized white noise IMF-ApEn in approximate entropy and average periodic plane are studied to understand the evolution rule of approximate entropy of each IMF component in EMD decomposition. The driver-state-extraction IMF-ApEn template is established.(4)Using the simulation driving experimental platform derived from the vehicle driving state data, the change rate of the curvature of the vehicle track in the virtual driving process is obtained. Based on this, the proposed method can eliminate the road curvature information in the vehicle driving state data.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work is supported by National Natural Science Foundation of China (Project 51305339) and China Postdoctoral Science Foundation (Project 2015M572578).