Abstract
Driver fatigue detection (DFD) is an effective method to prevent traffic accidents. The existing research on DFD using facial features is an effective and noninvasive fatigue detection method. However, this approach is affected by facial occlusions (glasses, sunglasses, masks, etc.) and the large facial pose deformations in the extraction of effective fatigue features. In this paper, we introduce a novel DFD method using human pose information entropy. The method first estimates human pose from video sequences and then uses them as clues to extract multiple fatigue-related features which can reduce the influence of facial occlusion and head pose deformation. Information entropy and sliding window algorithm are applied to analyse and calculate sufficient consecutive video frames to obtain more robust and accurate fatigue-related values than by using a single frame. These information entropy values are combined resorting to the support vector machine (SVM) to recognize the driver fatigue state. Experimental results show that the method can achieve much higher accuracy and robustness, and the detection speed meets the requirements of real time.
1. Introduction
The World Health Organization (WHO) released “Road safety” in 2017 and emphasized the number of deaths caused by traffic has continued to rise. About 3,200 people die in everyday traffic globally [1]. Many studies have shown that fatigued driving is an important cause of road accidents. In the process of fatigue driving, it will cause impairments to normal driving; therefore, impacting driving performance and attention which can result in traffic may happen [2].
Therefore, it is necessary to develop advanced driver assistance systems (ADAS) to reduce road accidents, and the driver fatigue detection research is significant to improve road safety. A driver fatigue detection (DFD) system with good performance can give a warning when the driver is fatigued and is helpful to prevent road accidents [3].
When the driver is fatigued, the driver’s electroencephalogram (EEG), electrocardiogram (ECG) and skin conductance, and other neurophysiological signal fluctuations will be quite different from the normal driving state, so the physiological signal-based DFD method has been widely studied [4, 5]. Physiological signals can timely and correctly reflect the fatigue state, but physiological signal acquisition requires the help of various sensors in direct contact with the driver, which will affect the driver’s normal operation [6, 7]. Therefore, in the actual driving scene, the practical application of this detection method is limited. The noninvasive fatigue detection method can effectively avoid direct contact with the driver. Among them, an example of noninvasive fatigue detection is using vehicle behaviour metrics since a measuring device is not required on the driver’s body while the car’s operation details are observed [8]. This method can effectively avoid direct contact with the driver and pay attention to the operation details of the vehicle itself [9, 10]. However, it is possible that in the early stages of drowsiness, there may be normal operation of the vehicle, and therefore, the vehicle signal is difficult to timely and accurately reflect the driver’s actual driving state, and this detection method takes too much time to detect driver fatigue, and the real-time performance is insufficient [11]. Therefore, sometimes the vehicle behaviour-based DFD method only detects the fatigue state when the driver is extremely sleepy. In addition, the computer vision-based DFD method is also a noninvasive detection method. At present, this method mainly uses facial behaviours (blinking, yawning, and nodding) to detect fatigue. This method has a high probability of reflecting whether the driver is fatigued. Literature [12, 13] noted that yawning is an indication of fatigue in drivers and delivered good results in fatigue tests. Among them, the PERCLOS (percentage of eyelid closure) metric can directly and effectively reflect the changes of the eye-opening and closing state [14]. Therefore, this method is widely used in the research of driver fatigue detection [15, 16]. Bergasa et al. [17] combined eye and head features including PERCLOS, eye closure duration, and nodding frequency, and the driver’s fatigue state was detected using a fuzzy classifier. Sigari et al. [18] extracted the information related to the eye region (PERCLOS, eyelid distance change) and face region (head rotation) and achieved good results in fatigue detection. However, there are still some challenges in the detection method based on facial features [19], including the following:(i)Facial Occlusion. The challenge of facial occlusion comes from the fact that when the driver wears glasses, masks, and other facial occlusions, it will seriously affect the extraction of key facial features (such as eye or mouth information), resulting in a decrease in the performance of the fatigue detection model.(ii)Driver’s Head Posture Deformation. The driver’s head posture will change at any time during driving. When the change exceeds a certain angle, facial images cannot be obtained, which may cause the fatigue detection model to fail. And when the driver’s head is nodding, the sleepiness is quite developed, and it is already too late to detect the driver’s fatigue state.
To solve the problems discussed above, in this paper, we introduce a novel DFD method using human pose information entropy. The overall framework of our system is shown in Figure 1.
The main contributions of this work are as follows:(1)A novel driver fatigue feature extraction method based on human pose is proposed, which integrates the action information of the shoulder, wrist, and hand using the key points.(2)The information entropy theory is introduced to describe the change of fatigue features. The amount of disorder of fatigue feature values over a specified period will vary depending on the driver’s state. Therefore, the change of information entropy can reflect the difference between fatigue and normal state driving.(3)Considering the time accumulation characteristics of the driver fatigue state, we have designed a sliding window to calculate the information entropy values of consecutive frames. The SVM classifier with information entropy values used as input realizes the driver’s fatigue state prediction.
2. Related Work
2.1. Vision-Based Fatigue Detection
At present, there have been many studies on the computer vision-based DFD method, and many researchers have established fatigue features using the driver’s facial information or head posture. Khan et al. [21] detect the curvature of the eyelid and evaluate the sleepiness based on the correlation between the curvature and the openness of the eyes. Zhao et al. [22] take the driver’s head posture as a fatigue feature, count and analyse the head posture differences between different fatigue states, and then predict the driver’s state. Chen et al. [23] combine the facial features to predict the fatigue level. Although the fatigue detection technology using the driver’s facial features [24–26] can obtain good detection results, the extraction of facial feature points will be affected by the driver’s head posture offset deformation or occlusion. We investigate these problems to find a stable method to extract key points and more robust fatigue features. At present, the research on human pose has become a new direction to study human behaviour, and the extracted human pose information belongs to the overall feature, which has great advantages in the robustness of detection and extraction. The research on human pose has been widely used in safety supervision, human-computer interaction, and other fields [27, 28]. The study by Bin et al. [29] showed that abnormal behaviour of the human body is detected based on the human pose, detects the pose of human behaviour in different scenes, and judges whether human behaviour is in the abnormal state. In the related field of safe driving, the research on driver attitude has become one of the most important research directions of ADAS [30]. Dua et al. [31] consider that some drivers covering their mouth with hands would lead to wrong judgment of yawning. Therefore, the change of the driver’s hand posture is integrated into the fatigue detection method. In the work of Yang et al. [32], a motion capture system is used to capture the driver’s human pose trajectory, and the extracted human pose motion data are used to evaluate the driver’s motion quality during driving based on kinematics and dynamics analysis.
Through the research on the application of human pose information in the field of safe driving, it is found that human pose data contain rich information that can reflect the current state of drivers. In addition, the driving fatigue detection method based on computer vision is a noninvasive method, which is easier to be accepted by drivers. Therefore, in this paper, we extract human pose data based on computer vision and take this as the basis of fatigue feature extraction. We classify and predict the fatigue state by analysing the changes of the driver’s human pose.
2.2. Fatigue Status Classification Prediction
After the fatigue feature is extracted, the fatigue state classification prediction needs to consider the interaction between multiple fatigue features. The classifiers are not universal for different fatigue features. In order to improve the performance of classification models, researchers combine a variety of methods to design the most suitable classifier. Du et al. [33] combined heart rate, eye, and mouth to detect driver fatigue and proposed a new multimode fusion recurrent neural network (MFRNN), which can combine these three features, and different time information extraction networks are designed for different information sources to improve the accuracy. Ansari et al. [34] used the capture system to monitor the driver’s head posture movement to measure the driver’s mental fatigue and drowsiness, and combined with this fatigue feature, they adopted a new and improved deep learning network reLU-BILSTM to observe and identify its related different driving head activity patterns. In a similar method of using appearance actions to establish fatigue features, Huang et al. [35] adopted a new multigranularity deep convolution model called RF-DCM, which integrates multigranularity extraction, feature recalibration, and feature fusion used for driver fatigue detection and has better improvement effects in extraction and classification prediction. Ye et al. [36] proposed an improved method based on sample entropy and kernel principal component analysis for fatigue feature extraction. The SVM classifier is used to realize the effective recognition of driver fatigue state.
In this paper, the driver’s pose information is selected as the fatigue feature, and information entropy is introduced to quantify the amount of disorder of the fatigue feature, and with that, the driver’s fatigue state is identified by combining with SVM. In addition, driver fatigue is a continuous process, and the time change of fatigue feature is very important for the identification of fatigue driving. In the study by Ouabida et al. [37], a single frame or fixed frame was used for fatigue detection, ignoring the time change of fatigue. Therefore, in the calculation of the fatigue feature classification, the sliding window is used to extract the feature change over a while, which can have a better prediction of the driver’s fatigue state.
3. Methodology
3.1. Fatigue Feature Extraction
In this paper, OpenPose [38] is applied to detect 18 key points of the human body. It takes an image as input and outputs the two-dimensional coordinates of key points. The human-skeleton structure located by OpenPose is shown in Figure 2.
When the driver is driving normally, the driver will constantly adjust vehicle status according to the road conditions, and even on straight roads, the driver always has to make some slight rotation in the steering wheel. When drivers are fatigued, the operating speed of the vehicle slows down and the range decreases. Consequently, the frequency and amplitude of changes can reflect the fatigue state of the driver [39]. When the driver is fatigued, not only drowsy facial behaviours (blinking, yawning, and nodding) will occur but also the range of body movements becomes smaller and the frequency becomes lower. To study the relationship between human pose and driver fatigue, this paper intends to calculate the projected Euclidean distance of the arms, the area between the arms, and the dispersion degree of the wrist coordinate using human body key points and takes them as the fatigue features. The rest of this section is the description of extracting fatigue features.
3.1.1. Euclidean Distance of the Arms Projection ( and )
The driver’s arm posture contains the information of operating the steering wheel. The change in Euclidean distance of the projection of the two arms on the projection surface can reflect the differences of drivers in different states (normal or fatigue). In the fatigue state, the Euclidean distance of arms changes little or remains unchanged because the driver’s operation slows down or even stops. Figure 3 shows the movement of the driver’s arm posture. The black solid line and the red dashed line are the driver’s arm posture before and after the body pose change, respectively.
Based on the driver’s arms movement information, the key points , , , and are selected to calculate the Euclidean distance between and , and the Euclidean distance between and , which are defined as (1) and (2).
The schematic diagram of the changes of and is shown in Figure 4. When the driver is driving normally, the Euclidean distance of the arms on the projection surface changes greatly in value. When the driver is fatigued, the frequency of the arms movement will be reduced, and the fluctuation of and is small or even unchanged.
As shown in Figure 4, with the change of the driver human pose, and change greatly. Fatigue leads to slow body pose change, which makes the difference between and value change between normal state and fatigue state. This difference can be reflected in the level of driver fatigue.
3.1.2. Projected Area between the Arms ()
The area S between the arms can be calculated using the four key points , , , and of the two upper arms as shown in Figure 5. The value of changes with the rotation of the steering wheel, and the shaking of the body will also affect the change of . The gray and red shadows are the changes in caused by the driver’s pose variety.
will change with the movement of the arm. As shown in Figure 5, and are, respectively, the shape of the area between the elbows before and after the change. Where is calculated by (3)
3.1.3. Wrist Coordinate Point Dispersion ()
When the driver is driving normally, the wrist moves as the driver operates the steering wheel, causing the wrist coordinate point to change across a wide range. During fatigue driving, body movements become less frequent and have a smaller amplitude, and the movement frequency and amplitude of the corresponding wrist coordinate point are lower than when driving normally. is the wrist coordinate point as indicated in Figure 2. In a period, the wrist coordinate point set is {, , …, }. The coordinate distribution is shown in Figure 6.
According to the coordinate point information, the dispersion degree of the coordinate point is quantified, and the fluctuation of the dispersion degree can reflect the movement of the wrist. First, calculate the center point of the coordinate point set generated in the time :
Then, calculate the distance between each coordinate point and the center point and define it as the wrist coordinate dispersion as follows:
3.2. Human Pose Information Entropy
After extracting the four groups of fatigue features , , , and , the generated fatigue feature data are processed based on information entropy theory. When the driver’s body pose changes, resulting in an angle change or a distance variation from the camera, it will interfere with the result of information entropy. With the introduction of the correction factor, an improved calculation method for information entropy is proposed so that the human pose information entropy reaches a better degree of discrimination during classification training. In the four groups of features , , , and , the coordinate information is different from the other three features, so corresponding methods are used in the correction calculation.
3.2.1. , , and Correction Method
(1)According to the data in the time , the fatigue features , , and are calculated, respectively, and then, the mean values , , and of the period are calculated as follows:where N is the total number of data generated in a sliding window.(2)The correction value for reducing interference with , , and is obtained by combining the correction factors , , and with the mean value of each fatigue feature.(3)Use the correction value to correct each fatigue feature in time to calculate the final , and the calculation method is given in the following equation:
3.2.2. Correction Method
(1)The correction factor is added to the center point to yield the correction point . The correction point setting formula is shown as given in the following equation:(2)Calculate the distance between each coordinate point and the correction point in time:
3.2.3. Calculation of Human Pose Information Entropy
Calculate the data in a fixed sliding window, where n is the number of fatigue feature data falling into the same interval in this period, and N is the total amount of data in this period. When driving with fatigue, the change range of human pose becomes smaller, and the value of fatigue feature (, , , and ) remains unchanged or changes very little. It leads to a large number of repeated fatigue feature data, and the information entropy obtained is relatively low. When the driver is driving normally, the body moves frequently, causing changes in the fatigue feature data, which raises the fatigue feature’s information entropy value.
According to the correction fatigue feature value calculated by (8) and (10), further, calculate the mean value and variance and the equal interval and obtain the information entropy value used to detect fatigue and normal state.
Calculate the fatigue features in a sliding window and divide them equally by (11), that is, (0, ), (, )……(, ) count the number of data in each set of sliding windows in distinct equal divisions and then insert them into equation (12), (13). is the proportion of the quantity dropping in each interval in the entire amount of data.
(13) is the information entropy calculation formula. Different information entropy values are derived from different fatigue features in this method, and the four information entropy values are designated as to further classifier training and prediction.
3.3. The Classifier
The classification algorithm’s content is to determine the class of features using the extracting features. In our model, four human pose information entropy are extracted from the human pose, and fatigue labels are corresponding to the information entropy. Given a training data set , , …, on a feature space, is the input sample, including the information entropy of , , , and , and is the label corresponding to . When , is called a normal sample, and when , is a fatigue sample. We use three classification methods for comparative experiments, namely, Naive Bayes, Multilayer Perceptron (MLP), and SVM.
The experiment establishes a classification model using a supervised classification approach and performs tests using three classification methods to verify the method’s accuracy and select the best classifier.
4. Experiments
4.1. Data Set Acquisition
Currently, the majority of computer vision-based driver fatigue detection technologies attempt to conduct research focused on the driver’s facial traits. The available public data sets are the YawDD data set [40], NTHU driver drowsiness data set [41], DROZY [42], etc. Fatigue detection based on facial features alone is limited and not comprehensive enough for actual application scenarios, so it is necessary to mine more fatigue features to improve detection performance. As we know, the driver’s pose also contains a wealth of fatigue-related information. However, there is currently no public data set based on the human pose for the DFD. Therefore, we construct a driver data set based on the human pose and carry out experiments to verify the accuracy and validity of the proposed method.
4.1.1. Hardware and Environment Setting
The computer configuration is AMD Ryzen5 4600H CPU with 16G DDR4 RAM, and the graphics card is GTX1650. The driving simulation platform is “Euro Truck 2,” and the steering wheel kit uses the Logitech steering wheel kit, which contains a steering wheel, a manual gear lever, and an adaptive angle linear pedal. The camera is positioned on the upper-left corner of the driver with a fixed height. The resolution of video capture is 1280720. Each video is approximately five minutes in length. The road environment during acquisition is shown in Figure 7.
4.1.2. Record Information
Thirteen healthy subjects (9 males and 4 females), aged 20–50, are selected for two rounds of driving simulation and video recording. It should be noted that these selected subjects usually have lunch break habits. In the process of collecting the data set, the participants in the capture drive freely, without being directed, and the road is not fixed. Each driver maintains their driving habits such as the range of steering wheel operation and the way of observing road conditions.
Before data collection, the subjects will understand the division standard of the Karolinska Sleepiness Scale (KSS) [43], and the levels of KSS are divided into two categories—normal state (KSS≤3) and fatigue state (KSS≥7). Since KSS is a subjective self-evaluated method, subjects’ understanding of it will vary from person to person. Therefore, in order to reduce the misjudgment of mental state caused by this difference during data collection, the time of data collection is determined according to the subjects’ sleep habits. The first round of data collection is carried out from 8:00 to 9:00 in the morning so that the subjects can get enough sleep the night before, drive in the self-evaluated normal state, and collect five minutes of normal-state driving data. The second round of data collection is conducted from 14:00 to 15:00. From the end of the first round to the beginning of the second round, the subjects are not allowed to take a lunch break, nor did they involve any refreshing items. When the self-evaluation is fatigued, they begin to simulate driving and collect driving data under fatigue. The recording result is shown in Figure 8:
The self-built data set contains 28 video samples (22 training and 6 tests). The training set is divided into 11 positives (fatigue state) and 11 negatives (normal state).
4.2. Driver Fatigue Feature Extraction and Information Entropy Calculation
The purpose of this section is to compare and analyse the variation range of fatigue features under different conditions. The human body key points are located based on the self-built data set before fatigue feature extraction, and the result is shown in Figure 9:
When the driver is driving in a normal state, he/she can observe the road traffic situation in time and make driving responses. When the driver is fatigued, the reaction to the road situation is relatively slow, and the corresponding driving action amplitude and frequency become smaller [44].
The fatigue features between normal and fatigue states are compared. Figures 10 and 11 show the values of fatigue features and in normal and fatigue states, respectively. Among them, the black line represents the normal state, and the red line denotes the fatigue state. Since the fluctuations of and are similar, we take as an example here. Figure 12 shows the coordinate information of the left wrist in normal and fatigue states.
As shown in Figures 10 and 11, it can be seen that there is a clear difference in the degree of disorder between the Euclidean distance and the area in the two states. When the driver is fatigued, the magnitude of the change is significantly smaller. In the left wrist coordinate comparison chart, it can also be seen that there is a clear difference between the dispersion information of the wrist coordinate in the different states. When the driver is fatigued, the overall coordinate information of the wrist is more concentrated.
This paper uses the sliding window method to calculate information entropy. The processing interval is 50 frames, and the information entropy value is generated every 1.67 seconds according to the camera speed of 30 frames per second.
The information entropy will change with the driver’s movement, but under the interference of some angles or distance from the camera, the different effect of information entropy in different states is not obvious. Hence, the correction factor is introduced to regularize the fatigue feature value, which can reduce the interference. The value of the correction factor will have a greater impact on the result of the information entropy. After the experimental test on the value of the correction factor, Table 1 lists the specific value of the determined optimal correction factor. Figure 13 shows the change in information entropy before and after the modification.
(a)
(b)
(c)
(d)
From the above results, the interference caused by the projection is reduced by using the correction factor. The discrimination of information entropy is more obvious in different states. The information entropy calculated by the final method can well indicate the driver's fatigue state.
4.3. Evaluation Index
To evaluate the performance of the proposed driver fatigue detection method on the self-built data set, the evaluation index of accuracy, precision, and recall is used as in the following equation:
4.4. Fatigue Detection Result
According to the evaluation index in Section 4.3, the comparison result among the different driver fatigue detection classifiers on the data set is listed in Table 2.
As shown in Table 2, in both training and test, the results show that SVM and MLP have stronger classification effects than Naive Bayes. MLP can be used for classification experiments, and the number of neurons can be adjusted to find the best classification model based on MLP. Increasing the number of neurons improves the classification effect; however, when it continues to increase, the classification performance will decline. When the hidden layer is three layers of 128 neurons, the classification effect of MLP reaches the best, and the accuracy reaches 98.30%. SVM can use different kernel functions to better handle classification problems such as linear kernel function (T = 0) and polynomial kernel function (T = 1), and adjusting the parameter d in the polynomial kernel function also has an impact on the classification effect. Many experiments using the SVM model show that better results will be obtained when the kernel function is selected as a linear kernel function (t = 0). The result of the test set under this classification model reaches the optimal result, the accuracy reaches 99.35%, and the accuracy and recall are in the best performance of all classification results. Figures 14 and 15 show the information entropy and prediction results in the test set.
The selected information entropy results have two states: fatigue and normal state. As shown in Figure 14, comparing the information entropy results of the two states, the information entropy in the normal state is significantly higher than that produced by the fatigue state. The classification results in Figure 15 show that the prediction results closely reflect the driver's current state, among which the red line shows the prediction result. The prediction result produces wrong results in frames 383–387, and fatigue is predicted as the normal state. The driver's body pose will shake at a moment even if tired due to physiological reactions, resulting in a sudden change in information entropy, and affecting the classification results. The result shows that the information entropy based on the human pose can predict the driver's fatigue level well.
4.5. Runtime Performance
According to the hardware environment listed in Section 4.1.1, an experiment is carried out on the test set to calculate the runtime of the proposed DFD method (Table 3)
It can be seen that a single detection requires a total of 41.874 ms, and the detection speed can reach 23.88 FPS. Consequently, the proposed detection method has a good runtime performance and can meet the requirements of real-time fatigue detection.
5. Discussion
5.1. Selection of Fatigue Feature
We propose a method using human pose information entropy to recognize driver fatigue, and we combine the information entropy of four fatigue features for classification, which differs from the result of fatigue classification based on only a single feature. The fatigue detection accuracy of different features is shown in Table 4.
The experimental result shows that using only H(Lleft) and H(Lright) for fatigue detection, the accuracy is lower, 74.45% and 69.19%, respectively. Using H(S), Hv(x, y) can get higher detection accuracy. After the fusion of the information entropy of four fatigue features, the overall accuracy is increased to 99.35%. This is because the driver's body movements during driving are complicated, and the overall pose needs to be considered. Each fatigue feature changes with the driver’s pose during driving, and different fatigue features vary in different periods. Therefore, the fusion of multiple fatigue feature information entropy can effectively improve the accuracy and robustness of fatigue detection.
5.2. Correction Optimization
In this paper, the correction factor is introduced to the optimal calculation of the fatigue feature information entropy. The calculated information entropy has a higher degree of difference between fatigue and normal state. At the same time, the information entropy before and after correction also has a great impact on the classification results. We resort to the information entropy results before and after correction to classify the fatigue state. When all classification parameters are kept the same, the results are shown in Table 5:
The arms are one of the most active components of the human body when the vehicle is operating regularly. The projection of the arm on the plane will fluctuate with the driver's driving action, and the area between the arms and the dispersion of wrist coordinates will also change. Therefore, according to the changing trend of these four characteristics, they are selected as fatigue features. However, during the calculation process, the uncorrected fatigue features will interfere with fatigue prediction results. After the correction factor is added, the interference can be reduced and the classification result between fatigue and normal can be optimized. When uncorrected data are used for classification, the accuracy is only 90.68%, but when corrected data are used for classification, the accuracy is improved by 8.67%. The results show that the corrected calculation method is more effective in driver fatigue detection.
5.3. Comparison
This paper proposes a human pose information entropy-based DFD method and evaluates it on a data set. Table 6 shows the result of the proposed method compared with other methods.
Due to a limitation of research on the DFD using the human pose and the limited content that can be utilized for comparison, the latest fatigue detection method has been chosen for comparison and analysis in this paper. The fatigue feature of the chosen approach combines multiple features of the face and heart rate, expression, and head posture.
The stability of the process of extracting fatigue features is significant for detecting driver fatigue. The facial feature is prone to fail to detect feature information when the driver's face is covered (such as wearing sunglasses or masks). According to the comparative experiment, facial occlusion affects the extraction of fatigue features and the accuracy of driver fatigue detection. Although the head posture-based DFD method is unaffected by face obstructions when the driver's head posture is deformation, certain key points information will be lost, decreasing the accuracy and robustness. This paper detects the fatigue state using the human pose information, and the facial mask or head posture movement will not affect the detection process, making it more robust than the approaches mentioned before.
Driver fatigue is a sequential process. Huang et al. [35] use a single frame or a set frame to detect the fatigue state. Although the approach achieves high accuracy, it is a transitional problem to ignore fatigue when judging the fatigue process. This paper considers the importance of the time change of fatigue feature for the identification of fatigue driving because the final classification prediction is a quantifiable process, and it has a higher performance in properly predicting the driver's state.
In addition, in the comparison of these works, this paper applies the mathematical theory of information entropy to fatigue features and uses the disorder degree of the fatigue feature as the final fatigue evaluation index.
6. Conclusion
This paper proposes a DFD method using human pose information entropy. This method extracts fatigue features from the driver's human pose, avoiding interference from the driver's face occlusion and head posture deformation. The theory of information entropy is used to quantify the fatigue feature after it has been extracted. The fatigue feature's quantified information entropy value represents the fatigue feature's disorder degree with time, which is more suitable for subsequent classification prediction. Finally, SVM is used to classify and predict the results of information entropy. The experiments have constructed a driver fatigue detection data set, and the efficiency of the proposed method is proved on the self-built data set. The results show that the human pose-based DFD method can accurately predict the driver's fatigue state and has high detection accuracy and robustness. However, when the driver drives with one hand, it will affect the accuracy of the proposed method. Future research should study this problem to make this method applicable to more drivers with different driving habits as much as possible. In addition, this work will study how to solve the problem of the effect of light change on fatigue detection. At the same time, it is considered to fuse facial information with human pose to further improve the performance of driver fatigue detection.
In the future, the method will seek cooperation or funds. In addition to improving KSS-based self-evaluation, the driver's EEG, ECG, and other signals will be collected to formulate more accurate and suitable fatigue standards. And then, the driver fatigue test experiment will be conducted in combination with the new state standard to compare with our test results so as to improve our method and make the driver fatigue test results based on human posture information entropy more accurate. In order to support deeper research, the data set collection scheme will be further optimized (e.g., sleep restriction) to induce the driver to experience a fatigued state closer to the real driving scene.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This research was supported by the Science and Technology Program of Gansu Province (Grant nos. 21JR7RA303 and 21JR1RA235) and the Lanzhou Jiaotong University Youth Science Foundation Project (Grant no. 2020002).