Ambient Intelligence in the Internet of ThingsView this Special Issue
DrivingSense: Dangerous Driving Behavior Identification Based on Smartphone Autocalibration
Since pervasive smartphones own advanced computing capability and are equipped with various sensors, they have been used for dangerous driving behaviors detection, such as drunk driving. However, sensory data gathered by smartphones are noisy, which results in inaccurate driving behaviors estimations. Some existing works try to filter noise from sensor readings, but usually only the outlier data are filtered. The noises caused by hardware of the smartphone cannot be removed from the sensor reading. In this paper, we propose DrivingSense, a reliable dangerous driving behavior identification scheme based on smartphone autocalibration. We first theoretically analyze the impact of the sensor error on the vehicle driving behavior estimation. Then, we propose a smartphone autocalibration algorithm based on sensor noise distribution determination when a vehicle is being driven. DrivingSense leverages the corrected sensor parameters to identify three kinds of dangerous behaviors: speeding, irregular driving direction change, and abnormal speed control. We evaluate the effectiveness of our scheme under realistic environments. The results show that DrivingSense, on average, is able to detect the driving direction change event and abnormal speed control event with 93.95% precision and 90.54% recall, respectively. In addition, the speed estimation error is less than 2.1 m/s, which is an acceptable range.
Owing to the rise in the popularity of automobiles over the last century, road accidents have become one of the leading causes of death in many countries around the world . For instance, in 2010, there were almost 280000 injured and 70000 killed in traffic accident in China alone . A study shows that over 90% of traffic accidents are associated with human errors . The human behaviors, such as speeding, drunk driving, and using a mobile phone while driving, are the major factors which lead to inattention of drivers. Since large scale fields studies have proved that when a driver is monitored, his/her behavior is relatively safer, thus, to reduce the road accident, various technologies have been developed to detect driver’s state while driving. For example, in , the authors proposed to monitor the loss of attention of drivers by determining the percentage of eye closure. In addition, in , the authors proposed to leverage the existing car stereo infrastructure to monitor whether a phone is used by the driver. However, since the unsafe state of a driver is presented as dangerous driving behaviors of a vehicle, it is more meaningful to monitor driving behaviors of the vehicle rather than detecting a specific unsafe driving behavior of the driver.
Currently, several companies have provided products for drivers to monitor driving behaviors of vehicles with the aim of avoiding the traffic accident. In [6–8], the products collect real-time vehicular sensor data, such as GPS trajectory, and transmit them to a data center through the Internet or cellular wireless networks. Thus, we can troubleshoot and monitor the vehicle from our smartphone or computer. However, as with the sensing technology, the data collection raises severe privacy concerns among users who may perceive the continuous monitoring by the operator as intrusive . To overcome this drawback, products for personal use have been designed [10, 11]. The product is installed on the vehicle, to monitor parameters that determine the driving behavior of the vehicle and provide feedback on a regular basis for drivers. Then, the driver can ensure where they need to improve according to the feedback. However, the problem is that these products bring in high cost. For example, the camera-based product unit is roughly $800 each . Nowadays, only a tiny percentage of cars on the road are equipped with these driver assistance devices and it will take a decade for this new technology to be commonplace in most cars across the globe.
In recent years, there has been tremendous growth in smartphones which own advanced computing capability and is embedded with numerous sensors such as accelerometers, GPS, magnetometers, and cameras. This consequently results in that a massive smartphone augmented reality applications are proposed [12–14], including combining the smartphone with cars to offer assisted service to drivers [15–17]. The advantage of the smartphone-based approach is that it overcomes the high investment cost of those commercial systems. However, we find, through our thoroughly test in practice, that the data provided by the embedded sensors of the smartphone is presumably inferior. The simple integration over these data may result in large deviation from the ground truth of the vehicle’s states, which has significant impact on real-world usability of these proposed applications.
In this paper, we propose DrivingSense, which is a reliable dangerous driving behavior identification scheme based on smartphone autocalibration. In DrivingSense, it uses three-axis accelerometer, gyroscope sensor, GPS, and microphone embedded into the smartphone, to periodically collect data of a vehicle. In order to decrease the impact of sensor noise, an autocalibration algorithm based on an improved Kalman filter algorithm is introduced. During this process, the primary challenge is how to determine the sensor noise distribution while the vehicle is being driven. According to Newton’s first law, when the velocity of the vehicle is constant, the data readings of accelerometer and gyroscope sensors of smartphones are theoretically equal to zero. Based on this intuition, we propose a pseudo-second-order differential method to determine the sensor noise distribution. After that, DrivingSense uses the corrected data to identify the dangerous driving behavior of speeding, irregular driving direction change, and abnormal speed control, which are corresponding to the three biggest causes of fatalities on the road: speeding, distracted driving, and drunk driving .
In our scheme, the accelerometer sensor and GPS readings are used to estimate the driving speed. If the driving speed exceeds the road speed limits which can be obtained from a navigation system, the speeding behavior is detected. To identify the irregular driving direction change behavior, we first use the gyroscope sensor to infer the spin movement of a vehicle. Then, the microphone is used to detect its turn signal. If the turn signal is not detected when the driving direction change happens, the irregular driving direction change is detected. For the abnormal speed control, it is related to abrupt acceleration and deceleration or erratic braking, which are reflected on the changes of longitudinal acceleration of a vehicle. If the acceleration exceeds a safe threshold, the abnormal speed control is detected. The main advantage of DrivingSense is that it can sense features of vehicles in natural driving conditions through smartphone sensors, providing a reliable vehicle state estimation. Furthermore, DrivingSense is easy to implement and lightweight so that it can run on standard smartphones. Our extensive experiments validate the accuracy and feasibility of our scheme in real driving environment.
We highlight our main contributions as follows:(i)We propose a sensor noise distribution determination algorithm for the smartphone on a vehicle. Specifically, we exploit different change trends of smartphone sensor data between uniform moving and motion change to infer which data segments are from the vehicle in uniform moving. Then we can use this data segment to estimate the sensor noise distribution.(ii)To correct the smartphone sensor data error, we propose an improved Kalman filter based autocalibration algorithm. The experimental results show that this method can effectively correct the data error.(iii)We deduce an accurate driving speed of a vehicle estimation method that only uses the corrected acceleration data and GPS.(iv)To detect the turn signal audio beep, an algorithm based on Fast Fourier Transform and cross-correlation is proposed. The Fast Fourier Transform is used to analyze the audio beep frequency, filtering out the background noises. The cross-correlation algorithm is used to detect the turn signal.(v)We conduct extensive experiments in urban city, Chengdu, China. The results show that, in the real world, DrivingSense can identify the vehicular driving behavior with high accuracy.
The remainder of this paper is structured as follows. Section 2 presents a brief overview of related works. Section 3 gives a data error analysis, which illustrates the impact of the data error on the vehicle state estimation. In Section 4, we present an overview of DrivingSense and the design details of our scheme step by step, including sensor noise distribution determination, data error correction, coordinate reorientation, and dangerous driving behavior identification. We evaluate the performance of our scheme and present the results in Section 5. Finally, we give the conclusion in Section 6.
2. Related Work
Due to the popularity of smartphones and multiple sensors they are equipped with, there is a growing interest in driving safe research based on smartphones. In , the authors used a smartphone as a sensor platform to detect aggressive driving. Specifically, it used sensor-fusion output of accelerometer, gyroscope, and magnetometer sensors of a smartphone to detect and classify vehicle movement. The drawback of this approach is that it cannot tell the driver where he/she drives improperly in detail. In , authors proposed CarSafe, which is an app than runs on the smartphone. In CarSafe, it uses the time series GPS to estimate the vehicle’s speed and uses the phone’s front camera to recognize the head position of the driver to ensure whether the driver is in a safe lane change mode. A similar scheme has been proposed in [19, 20] where GPS or subsampled GPS is used to drive the vehicle speed. Since the vehicle is highly dynamic, the low update rate of GPS is hard to keep up with the frequent change of the vehicle speed. Additionally, continuously using GPS drains the phone battery quickly. Thus, it is hard to obtain the accurate speed estimation from GPS trajectory. Besides vehicle speed estimation based on GPS, an alternative method based on OBD-II was developed . It leverages the Bluetooth communication between a smartphone and OBD-II adapter to monitor the vehicle driving speed and provide feedback for the driver. Although the speed obtained from OBD-II is quite accurate, this approach relies on an additional OBD-II adapter. In , the authors proposed SenSpeed, which estimated the vehicle speed by integration of the accelerometer’s readings over time. The problem is that the initial velocity only can be calculated at the turn reference point through the angular speed. Besides, there are accumulative errors of the speed estimation caused by the biased acceleration. In , the authors proposed to use sensory data of accelerometer and orientation sensor of smartphone to detect the drunk driving. However, all these methods are suffering from the problem of sensor noise.
To tackle this problem, up to now, serval methods have been proposed to process noisy signal for robust detection. In , a signal filter was used over the raw data. But it can only filter out the noise from the vibrations of the vehicle interior. In , authors leveraged a mechanism that when a vehicle reached its maximum speed, the vehicle changed from acceleration to deceleration during normal driving. Thus, when the reference speed from OBD-II reaches its local maximum, the acceleration should be equal to zero. Acceleration adjustment is calculated by reducing the bias. Although this mechanism can obtain more accurate speed estimation, it not only requires additional hardware, but also cannot correct other sensor errors, such as gyroscope sensor. In , the authors proposed to use the Kalman filter algorithm to correct sensor noise. It assumed that the noise was drawn from a zero mean multivariate normal distribution and the variance was measured when a smartphone was still. However, there are some problems. (1) As we have tested, the sensor data error does derive not only from the white noise but also with a bias. Thus, the Kalman filter algorithm cannot be used directly. (2) We find that the data error is different every time even in still state. This means that we have to remeasure the data error when we use it. In , authors proposed to sense the natural driving conditions to identify the reference points to measure the acceleration error and further eliminate accumulative error from the biased acceleration. However, this method cannot be used in highway scenario in which there are less reference points. In this paper, we propose DrivingSense, which can efficiently eliminate the accumulate error when vehicles are driving, providing more accurate detection of the dangerous driving behaviors.
3. Data Error Analysis
As mentioned above, in this paper we utilize the smartphone as a sensing platform to collect the driving information of vehicles and identify their dangerous driving behaviors. However, we find that the data collected by smartphone sensors are noisy. In this section, we will verify the impact of the data error of sensors on the vehicle driving behavior estimation.
We first conduct experiment to learn about how the sensor data error is. To achieve this objective, we lay a smartphone in a horizontal plane and keep it stable to collect the sensor data. The sample frequency is set to 1 Hz. Under the ideal condition, the value of each sensor data reading is equal to zero. We take the -acceleration and -gyroscope readings as an example and plot the measurement data, as shown in Figure 1. The measurement data are volatile and deviate from the ground truth. The primary error sources of smartphone sensors are uncorrected bias errors and white noise , which are consistent with our practice test. From the figure, we can see that the data error of the accelerometer sensor is larger than gyroscope sensor’s. Thus, we mainly focus on analyzing the impact of acceleration error on the vehicle driving speed estimation.
Let be a series of collected data, , is the acceleration, and is the vehicle location. Suppose the interval of sampling is . Therefore, the travelling distance during the time span can be expressed as
With the initial velocity at the beginning of data collection, the travelling distance can be computed as
Then, we have the velocity estimation function of as
Thus, the vehicle speed at the time point can be estimated as
From (4), we can see that the velocity of a vehicle is comprised of the acceleration and the travelling distance which is obtained by GPS. As we know, the GPS data is unreliable as well. Even the GPS readings corrected by WAAS have an error of 3 m (standard deviation), not to mention the ones in the area without WAAS. Fortunately, analyzing the GPS trajectories of different vehicles, we observe that the GPS error is highly correlated for a long driving distance, which is reflected by the fact that the vehicle trajectory is nearly paralleled with the real roads, as shown in Figure 2. That is to say, for a series of GPS trajectories, they have the similar data bias. It is worth to note that we are not the first ones to make such observations; similar characteristics have already been discovered and utilized by many works [27, 28]. Based on this result, we can conclude that the travelling distance computed through the relative motion distance superposition is reliable. Using (4), we can figure out the estimation speed error of the vehicle aswhere is the ground truth value of the acceleration.
From (5), we find that the estimation error is accumulated when integrating the accelerometer’s readings and the latter accelerometer’s readings have greater impact on the vehicular speed estimation. Suppose the accelerometer’s -axis is along the moving direction of the vehicle. The error mean of acceleration readings in Figure 1(a) is . For 200 samples, the speed estimation error is up to 7.48 m/s, which enough affects the vehicular driving behavior identification. Therefore, it is very necessary to correct the sensor data error before using them.
4. The Detailed Design of DrivingSense
Since DrivingSense is designed to run on the smartphone, it should be lightweight and fast so that the dangerous driving behavior can be detected in real time and a warning message can be sent to the driver as accurately as possible. In this section, we present the design of our DrivingSense and describe this scheme in detail.
4.1. The DrivingSense Overview
The vehicle driving behavior can be estimated by integrating of sensor data reading over time. However, there are two problems. Firstly, the sensor data are noisy. The accumulative error can cause a large deviation between the ground truth value and the estimation result. Secondly, since the smartphone can be in any orientation in the vehicle, its coordinate system is different from the vehicle’s. Thus, before using the sensor data, DrivingSense must perform data processing to correct the obtained data and align the smartphone’s coordinate system with the vehicle’s.
The workflow of DrivingSense is shown in Figure 3. It is mainly divided into three components: (1) data collection; (2) data processing; (3) dangerous driving behavior identification. For data collection, DrivingSense uses two kinds of sensors, accelerometer and gyroscope, GPS device, and microphone in smartphones. The accelerometer is used to monitor the vehicle acceleration and the gyroscope is used to monitor the vehicle angular speed, the GPS device is used to obtain the vehicle location which will be used to calculate the relative motion distance over a period of time, and the microphone is used to monitor the audio beep in the vehicle. For data processing, DrivingSense first determines the sensor error distribution. It can be estimated by the data segment that derives from when the vehicle moves in uniform motion. Then, it uses an improved Kalman filter algorithm to correct the collected data. After that, DrivingSense utilizes the corrected data to align the smartphone’s coordinate system with the vehicle’s to obtain meaningful data. For dangerous driving behavior identification, DrivingSense uses the corrected readings to identify the dangerous driving behavior of speeding, irregular driving direction change, and abnormal speed control. Speeding, which is one of the main causes of traffic accident, means the vehicle driving over the speed limit of the road. It is identified by comparing the estimated speed with the predefined speed obtained from a navigation system. Irregular driving direction change is when the vehicle makes a lane change or turn without turning on the turn signals. Abnormal speed control is abrupt accelerating, deceleration, or erratic braking. This is very common when drivers are under the drunk or fatigue driving conditions. In our scheme, we utilize a threshold scheme to identify this dangerous driving behavior.
4.2. Sensor Noise Distribution Determination
The smartphone is used to measure the vehicle movement parameters; the collected sensor data are derived from that when the vehicle is being driven. How to calculate the data error distribution under this state becomes the key issue of the data error correction. In the following parts, we will first present a method to determine the sensor noise distribution of an on-board vehicle smartphone.
The spatial movement of a rigid body can be described as a combination of translation and rotation in space. Suppose the -acceleration represents the vehicle’s longitudinal acceleration, and the vehicle motion of the lane change or turn is determined by the -gyroscope. When vehicle motion changes (speeding up and making a turn), the two parameter readings have an obvious change. As shown in Figure 4, is the data reading deviation when the vehicle is in uniform motion. is the data reading deviation when the vehicle motion changes. Compared with , has a much larger change. Based on this observation, we can infer which data segment derives from when the vehicle moves in uniform motion. After that, we utilize the mean and variance of the data segment to estimate the sensor noise distribution.
The key issue during this process is how to determine the change point and the algorithm should be lightweight so that it can run on the smartphone efficiently. Let be the raw data reading. We make a first-order difference on the obtained data and then extract all the nonzero values. After that, we make a first-order differential again on the absolute value of the extracted data. The absolute values of the results are calculated. We name this process as pseudo-second-order differential. Based on the result, data reading change trends can be determined. As Figure 5 shows, they are the results of the pseudo-second-order differential of -acceleration and -gyroscope in Figure 4. It can be seen that the results grow rapidly when the vehicle motion changes. Let be the pseudo-second-order differential set of the raw data. If , we can see that there is a regular change in the raw data. Thus, we only need to consider the case ; the change point of satisfieswhere TH is a threshold that is chosen empirically as two. After determination, we can find the change point of raw data using data index. The 100 consecutive samples between two change points can be used to determine the sensor error. The detailed sensor noise distribution determination method is depicted in Algorithm 1.
4.3. Data Error Correction
Once the sensor noise distribution is determined, DrivingSense next uses this information to correct the sensor data. As described above, the sensor data error is mainly caused by a constant bias and a white noise. If we subtract the constant bias from the collected data, the remaining data error is mainly a white noise. Then, we can use Kalman filter algorithm to correct the remaining data. In our scheme, the constant bias is the mean value of sensor noise distribution.
Let be the th measurement vector. Thus, is the new measurement vector with a white noise. Let be the th state vector which denotes the rough estimate before the measurement update correction. To obtain the corrected data, we introduce a discrete control process of the system; it can be given aswhere is the state transfer matrix of the system and is the measurement matrix. Since in our system and are just numeric values, and are identity matrixes. and are the process noise and the measurement noise, respectively. Usually, can be assumed as white Gaussian noise , and is white Gaussian noise with the variance derived from Algorithm 1. Their covariances are and .
According to (7), we utilize the previous corrected sample to predict the current state that is given aswhere is the corrected result of the th sample.
After that, we should calculate the current measurement data by the raw data value minus the mean value of the sensor noise. Based on the combination of the current prediction result and the measurement, the optimal correction result can be given aswhere Kg is the Kalman gain; it can be computed aswhere is the covariance of ; it is computed aswhere is the covariance of .
In order to implement Kalman filter algorithm until the end of the system, we should update the covariance of as
Initially, in our scheme we choose , . Through this process iteration, we can obtain more accurate data.
4.4. Coordinate Reorientation
In DrivingSense, we utilize -acceleration and -gyroscope of the smartphone to obtain the longitudinal acceleration and angular speed of vehicles. However, the smartphone can be fixed in the vehicle body in any orientation. That is to say, there are two coordinates in the system, one for the vehicle and the other for the smartphone , as illustrated in Figure 6. Thus, to derive the meaningful vehicle dynamics from sensor readings on the smartphone, DrivingSense must align the phone’s coordinate system with the vehicle’s.
Figure 7 depicts the relationship between the vehicle’s coordinate and the phone’s. Thus, our coordinate alignment aims to find the rotation angle, , , and , of -axis, -axis, and -axis of the smartphone. Based on the rotation angle, we can determine a rotation matrix to rotate the phone’s coordinate to match the vehicle’s. Let denote the acceleration of gravity. The angles of the coordinate on the phone to the vertical direction are , , and . When the vehicle moves with a constant speed, the acceleration readings are caused by the projection of gravity acceleration. The corrected values of the acceleration on the three directions of the smartphone are denoted as , , and . Therefore, we have the following results:
According to the rotation matrix, the smartphone will go through a self-learning process to complete reorientation. After that, DrivingSense can obtain meaningful data readings that represent the vehicle’s movement.
4.5. Dangerous Driving Behavior Identification
In our scheme, DrivingSense collects sensor data from smartphones in real time to identify three dangerous driving behaviors: (1) speeding; (2) irregular driving direction change; (3) abnormal speed control. To achieve these functions, we should carefully design the detection method so that it can reduce the potential false negative dangerous driving behavior detection.
After the smartphone’s coordinate reorientation and data error correction, DrivingSense obtains the meaningful data. Based on -acceleration and GPS readings, we can apply (4) to estimate the vehicle speed. In order to avoid the estimation error caused by the GPS bias, DrivingSense will reestimate vehicle speed every 150 m (usually, the GPS error correlation continues to more than 200 m ). When the vehicle speed exceeds the road speed limit obtained from a navigation system, DrivingSense identifies that the vehicle is in the speeding mode.
Different from the existing speed estimation algorithm [22, 30], we propose a novel speed estimation method. Not only does it not depend on the additional infrastructure, such as base station, but also there is no accumulative error during the speed estimation process. From (4), we can see that DrivingSense just utilizes the corrected the sensor data, providing drivers with an accurate speed estimation.
4.5.2. Irregular Driving Direction Change
DrivingSense utilizes the -gyroscope to detect the driving direction change of vehicles. In our system, driving direction changes under three conditions, which are lane change, sharp turn, and gentle turn. We define the irregular driving direction change as the driver does not provide any caution signal to the drivers around him when the event of driving direction change happens. This is to say, the host vehicle’s turn signal is off during this period. Therefore, the irregular driving direction change detection is divided into two stages: (1) the driving direction change detection; (2) the turn signal detection.
The Driving Direction Change Detection. The spatial movement of a vehicle can be divided into two kinds of movements: translation movement and spin movement. The spin movement is the key factor to distinguish turning style. The gyroscope of a smartphone is a sensitive device that can be used to detect angular speed in three dimensions according to the coordinate system of the phone. After aligning the phone’s coordinate with the vehicle’s, -gyroscope is used to reflect the spin movement of vehicles. The primary work of DrivingSense is to distinguish the driving direction change event from all the spin movement based on the corrected -gyroscope readings.
In order to achieve this objective, in our initial experiment we collect three sets of -gyroscope of each driving direction change event. Based on the datasets, Figure 8 plots the -gyroscope cumulative distribution function (CDF) of lane change, sharp turn, and gentle turn. According to the figure, we find that when the -gyroscope reading exceeds 0.56 rad/s, one type of driving direction change happens. To filter out outliers due to any sudden change of vehicle, a window is used. We set in our implementation.
The Turn Signal Detection. As we know, when the turn signal is on, the vehicle will send an audio beep to respond to the driver. In our system, we let the smartphone detect the audio beep which is a distinct beep in the vehicle interior. In order to detect the audio beep of the turn signal, we collect an audio clip in a vehicle at the audio sampling rate of 44.1 kHz with a smartphone. Figure 9(a) plots the raw audio signal that contains background signal and turn signal beep in the time domain. The background signal is that there is no sound except for the engine sound of a car. The turn signal starts beeping approximately from the 520000th sample and lasts to the 1010000th sample. We crop the segment of the audio signal and the background signal. Then, we convert the time domain signal to the frequency domain through Fast Fourier Transform, as shown in Figures 9(b) and 9(c). We observe that the frequency domain of the background signal is almost equal to zero. The frequency band of the turn signal beep is between 4 kHz and 6 kHz, which can rule out the background signal.
(a) The raw audio signal in the time domain, which contains background signal and turn signal beep
(b) The frequency domain of the background signal
(c) The frequency domain of the turn signal beep
With the knowledge of the frequency range of the audio beep send out by the turn signal, in our system we first utilize a band-pass filter  to filter out some of noise caused by the talking or music, improving the detection accuracy. After that, a sound cross-correlation algorithm  is used to detect the audio beep. Particularly, sound features of the turn signal are captured in advance. When filtering out a sound signal, DrivingSense implements the cross-correlation between the filtered sound signal and the previous captured signal. When there is a spike in the result, it means that there is turn signal sound. Figure 10 shows a sound wave cross-correlation result. It can be seen that there are spikes in the figure, which indicates that the audio signal contains the turn signal sound.
4.5.3. Abnormal Speed Control
Since the abnormal speed control is related to abrupt acceleration or deceleration and erratic braking, it will all be reflected on the changes of longitudinal acceleration. Usually, the abnormal speed control indicates that the driver is in drunk driving or fatigue driving state, which is one of the main causes of traffic accident . The abrupt acceleration of vehicle will lead to a great increase in longitudinal acceleration. On the contrary, the abrupt deceleration or erratic braking will cause a great decrease of longitudinal acceleration. Therefore, the vehicle acts abnormally in either acceleration or deceleration, resulting in a large absolute value of .
To detect the abnormal speed control, DrivingSense keeps checking the maximum and minimum value of longitudinal acceleration and in the raw data. If the amplitude of the value exceeds a threshold , a speed control problem is considered detected. Since the features of the acceleration and deceleration during driving are different even for the same driver, we set different thresholds for the acceleration and deceleration, denoted as and , respectively. In this paper, we set the threshold as two times the values of and .
In this section, we evaluate the performance of the DrivingSense with different types of smartphones. We first present the experimental setup. Then, we test and evaluate each component of DrivingSense, including smartphone sensor data correction, turn signal audio beep detection, speed estimation accuracy, driving direction change, and abnormal speed control detection. The following details the experimental methodology and findings.
5.1. Experimental Setup
5.1.1. Experimental Equipment
To test the practicability of DrivingSense, we conducted our experiments on two Android smartphones. One is Nubia Z5S and the other is MX3. Both of them are equipped with accelerometers, gyroscope, and support 44.1 kHz audio signal sampling from microphones. The Nubia Z5S has a 2 GB RAM and Quad-Core 2.2 GHz Adreno Snapdragon 800 processor, while the MX3 has a 2 GB RAM and Quad-Core 1.6 GHz Exynos 5410 processor.
5.1.2. Experimental Scenarios
To evaluate the generality and robustness of DrivingSense, we need to test our designs in a realistic driving environment. Since it is irresponsible to run an experiment that promotes dangerous behaviors without taking the sort of measures which car manufacturers take, it is challenging to build suitable experimental environment. To finish the experiment, we let DrivingSense sense the natural driving of a vehicle. We conduct experiments under a real-world condition, which is derived from Chengdu, a city in China. Figure 11 shows the area that the trace covered and there are two routes used for data collection. For route 1, the total length of the trace is up to 4.8 km. At the end of the trace it is an empty space. For route 2, the total length of the trace is up to 0.65 km and it is a straight road.
We implement our system using the Android platform. DrivingSense records sensed data from GPS, accelerometers, gyroscopes, and microphone during the natural driving of a vehicle. In order to verify the effectiveness of driving direction change detection, we deliberately let the vehicle make the driving direction change behaviors on route 1. Similarly, to verify the effectiveness of abnormal speed control detection, we let the vehicle make the abnormal speed control behavior at the end of route 1 which is an empty space. Table 1 summarizes the details of the two events. Since it is difficult to obtain the various accurate acceleration readings from on-board devices, to evaluate sensor data correction, we also collect data from route 2 in which the vehicle did little motion change. It also means that the true value of the accelerometer and gyroscope of a smartphone should be equal to zero.
5.2. Sensor Data Correction Performance
Our accurate vehicle driving behavior detection is built upon the inerrant data source that derives from the natural driving conditions. Thus, we first evaluate the performance of smartphone sensor data correction algorithm.
As we depict above, -acceleration has a comparatively large deviation from the true value. In this section, we use -acceleration readings as the test set. Since it is difficult to obtain the various accurate acceleration readings from the on-board device, we choose the dataset that is between two change points from route 2 as the test data. At this time, the true value of this dataset is zero. It also can be verified by a constant speed. Figure 12 presents the corrected result. From Figure 12(a), we can observe that the data errors of Z5S and MX3 are obvious differently. For the smartphone of Z5S, the data error is positive deviation. But for the MX3, the data error is negative deviation. In addition, we can also observe that the corrected data gradually converge to the true value with time. From Figure 12(b) which is the a posteriori error estimation of the corrected set, we find that the convergence rate and correction accuracy of Z5S are better than MX3’s. The reason is that the raw data of Z5S has more convergence than MX3’s. Furthermore, the error variances of the smartphone are less than 0.05 after the sample. It is an encouraging result.
(a) The correction result
(b) The a posteriori error estimation
Furthermore, we also compare our correction algorithm with Kalman filter based correction algorithm  and SenSpeed  using the data from Z5S. We leverage the parameter of error variance which denotes variance between correction results and ground true values to evaluate the performance of the correction algorithms. Figure 13 presents the error variance of the three algorithms. We can observe that the error variance of DrivingSense is much lower than the other two algorithms. For Kalman filter based method, the error variance increases initially and tends to be stable with the number of samples. But it has a larger error variance as well. The reason is that it can only correct sensor data discreteness but cannot correct a bias. For SenSpeed, it has the highest error variance. The reason is that SenSpeed can only use the acceleration error at the beginning of route 2 to correct the following data readings. As we mentioned above, the sensor data error is not fixed. So it has larger cumulative errors than DrivingSense and SenSpeed.
5.3. Turn Signal Audio Beep Detection
To evaluate the robustness of our turn signal audio beep detection algorithm, we collect audio signals in the other two scenarios: turn signal together with talking and music. Firstly, we analyze the spectrum characteristics of talking and music. As shown in Figures 14(c) and 14(d), the frequency band of talking is between 0.2 kHz and 1 kHz and the frequency band of music is between 0.5 kHz and 1.5 kHz. Thus, they can be well ruled out from the turn signal. As shown in Figures 15(a) and 15(c), they are the raw audio signal segments that derive from turn signal together with talking and music environment, respectively. It can be seen that human voices and music submerge into the turn signal. The corresponding detection results of the two scenarios are shown in Figures 15(b) and 15(d). From the two figures, we can find that there are obvious spikes for each situation, which indicates the existence of the turn signal. Thus, we can conclude that, based on the noise filtering, our audio beep detection algorithm has a higher accuracy of turn signal identification in various environments.
(a) The raw audio beep of talking in the time domain
(b) The raw audio beep of playing music in the time domain
(c) The frequency domain of talking
(d) The frequency domain of playing music
(a) The raw audio beep of talking environment
(b) The detection result of talking
(c) The raw audio beep of playing music environment
(d) The detection result of playing music
5.4. Speed Estimation Accuracy
We evaluate the speed estimation accuracy of our system using two test smartphones under two routes. To verify the effectiveness of our speed estimation method, we compare the estimated speed by our system with the SenSpeed  and the GPS. DrivingSense and SenSpeed both use the acceleration integration scheme to estimate speed. We compare the estimated speed with that of the ground truth, which is obtained from a calibrated OBD-II adapter.
Figure 16 presents the average estimation error in the two routes. For route 1, since the vehicle changed frequently, GPS cannot well keep up with the dynamic; it has the highest estimation error. Although DrivingSense and SenSpeed both use the acceleration integration scheme to estimate speed, SenSpeed cannot eliminate the accumulated error caused by sensor noise until at the reference point (the turning point). Thus, DrivingSense leveraging the sensor noise correction scheme has the lowest error compared with SenSpeed and GPS. For route 2, since the motion of the vehicle changed a little and there is no reference point, SenSpeed is worse than GPS and DrivingSense. Furthermore, we can observe that the average estimation error of GPS is lower than the DrivingSense’s. The reason is that, under this scenario, the acceleration integration scheme would incur more estimation error caused by sensor noise correction error. But we can find that the bias is very small. Thus, we can conclude that DrivingSense has more greater universality.
To further evaluate the accuracy and robustness of DrivingSense, we analyze the speed estimation error. Figure 17 shows the CDF of the speed estimation error of the smartphone MX3 and Z5S. It can be seen that we get a relatively accurate speed estimation for our scheme. For the smartphone Z5S, the estimation error is less than 1.9 m/s; by comparison, the max estimation error for the smartphone MX3 is 2.06 m/s. We analyze the datasets that are used for the speed estimation, finding that the speed estimation error is mainly caused by two reasons: (1) the existing outlier point; (2) the changing of the error deviation of the collected data. To tackle these problems, an outlier point filter algorithm and an error deviation recalculation method can be used. Anyway, the speed estimation error of our system is within an acceptable range, which indicates DrivingSense can detect the speeding driving behavior with high accuracy.
5.5. Driving Direction Change and Abnormal Speed Control Detection
The main function of our scheme is to detect the dangerous driving behavior under real-world conditions. During the route, we deliberately let the vehicle make the driving direction change behavior and at the end of the route, which is an empty space, we let the vehicle make the abnormal speed control behaviors. There are 56 driving direction change events and 20 abnormal speed control events for the two smartphones. The confusion matrix in Table 2 shows the precision and recall results of the two events. The average precision and recall for driving direction change and abnormal speed control detection are 93.95% and 90.54%, respectively. After checking the test data, we find that the false negative of the driving direction change event is mainly caused by gentle shifting of the vehicle when it makes a lane change or gentle turn. During this process, the -gyroscope does not exceed the threshold that identifies the driving direction change occurrence. The false negative of the abnormal speed control detection event is mainly caused by the slow driving. When we make the experiment for the abnormal speed control, the speed of the vehicle is relatively low. At that time, when the vehicle is in abrupt deceleration, -acceleration is not greater than the predefined threshold.
In this paper, we propose DrivingSense that makes the best of smartphones for dangerous driving behaviors detection, so that it can provide drivers with a warning to avoid traffic accidents. DrivingSense can detect three dangerous driving events: speeding, irregular driving direction change, and abnormal speed control. To achieve the high accuracy detection objective, we first propose a smartphone sensor data correction algorithm based on an improved Kalman filter algorithm. After that, we utilize the corrected data to estimate the vehicle’s behaviors in real time. To calculate the vehicle driving speed, we propose a novel speed estimation method which is based on the kinematics knowledge. To detect the turn signal, we propose a two-step based method: filtering out noise that submerges into the raw audio beep and a cross-correlation process over the filtered audio data. At last, the experimental results show that DrivingSense detects dangerous driving behaviors effectively.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
The authors sincerely would like to thank their shepherd. This work was supported in part by the National Natural Science Foundation of China (Grants nos. 61572113 and 61272526), the Fundamental Research Funds for the Central Universities (Grant no. ZYGX2012J083), and the Doctoral Fund of Tianjin Normal University (043-135202XB1615).
W. H. Organization, The Top Ten Causes of Death Who Fact Sheet, WHO, Geneva, Switzerland, 2007.
The 2010 national road traffic accident, http://www.mps.gov.cn/n2254314/n2254486/c3898566/content.html.
J. Yang, S. Sidhom, G. Chandrasekaran et al., “Detecting driver phone use leveraging car speakers,” in Proceedings of the 17th Annual International Conference on Mobile Computing and Networking, pp. 97–108, ACM, Las Vegas, Nev, USA, September 2011.View at: Google Scholar
“Vehicle monitoring,” http://www.uscellular.com/vehicle-services/vehicle-monitoring.html.View at: Google Scholar
Vehicle tracking solutions, https://vehicletracking.com/news/.
“Driver behaviour monitoring,” http://www.verilocation.com/driver-behaviour-monitoring/.View at: Google Scholar
“Driver fatigue driving behaviour monitoring cctv system,” http://recodadvr.en.made-in-china.com/product/oKzQrhbOLPWZ/China-Driver-Fatigue-Driving-Behaviour-Monitoring-CCTV-System.html.View at: Google Scholar
C.-W. You, N. D. Lane, F. Chen et al., “CarSafe App: alerting drowsy and distracted drivers using dual cameras on smartphones,” in Proceedings of the 11th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '13), pp. 13–26, ACM, June 2013.View at: Publisher Site | Google Scholar
J. Paefgen, F. Kehr, Y. Zhai, and F. Michahelles, “Driving behavior analysis with smartphones: insights from a Controlled Field Study,” in Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia (MUM '12), p. 36, ACM, December 2012.View at: Publisher Site | Google Scholar
Safety on the road, http://www.nsc.org/Pages/nsc-on-the-road.aspx.
B. Hoh, M. Gruteser, R. Herring et al., “Virtual trip lines for distributed privacy-preserving traffic monitoring,” in Proceedings of the 6th International Conference on Mobile Systems, Applications, pp. 15–28, ACM, 2008.View at: Google Scholar
“itunes preview,” https://itunes.apple.com/sg/app/you-jia/id738980410?mt=8.View at: Google Scholar
Y. Wang, J. Yang, H. Liu, Y. Chen, M. Gruteser, and R. P. Martin, “Sensing vehicle dynamics for determining driver phone use,” in Proceedings of the 11th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '13), pp. 41–54, ACM, June 2013.View at: Publisher Site | Google Scholar
J. Almazan, L. M. Bergasa, J. J. Yebes, R. Barea, and R. Arroyo, “Full auto-calibration of a smartphone on board a vehicle using IMU and GPS embedded sensors,” in Proceedings of the 2013 IEEE Intelligent Vehicles Symposium (IEEE IV '13), pp. 1374–1380, City of Gold Coast, Australia, June 2013.View at: Publisher Site | Google Scholar
O. J. Woodman, “An introduction to inertial navigation,” Tech. Rep. UCAMCL-TR-696, University of Cambridge, Computer Laboratory, 2007.View at: Google Scholar
“Kalman filter for dummies,” http://bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx.View at: Google Scholar
A. V. M. G. G. Chandrasekaran and T. Vu, “Tracking vehicular speed variations by warping mobile phone signal strengths,” in Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom '11), pp. 213–221, IEEE, 2011.View at: Google Scholar