- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
International Journal of Distributed Sensor Networks
Volume 2012 (2012), Article ID 319276, 8 pages
You Take Care of the Drive, I Take Care of the Rule: A Traffic-Rule Awareness System Using Vehicular Sensors and Mobile Phones
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan 611731, China
Received 28 September 2012; Accepted 23 October 2012
Academic Editor: Nianbo Liu
Copyright © 2012 Xing Zhang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Traffic rules are used to regulate drivers’ behaviours in modern traffic systems. In fact, all driving behaviours are presented by vehicles’ behaviours. If vehicles have awareness of their behaviours, it is possible that traffic rules are able to regulate vehicles instead of drivers. There are three advantages of vehicle regulation: (1) without worrying about violations of traffic rules and searching for traffic signs, drivers can pay more attention on emergency situations, such as jaywalking. (2) Many traffic violations are due to attention distraction; machines do not have the attention issues; therefore they can provide good traffic-rule obeying. (3) New traffic rules can be spread and applied more quickly and effectively through the Internet or Vehicular Ad hoc NETworks (VANETs). In this paper, we propose a novel traffic-rule awareness system using vehicular sensors and mobile phones. It translates traffic rules into combinations of vehicular sensors, GPS device, and Geography Information System (GIS); the system can tell whether a driver violates the traffic rules and help him to amend his driving behaviour immediately. Experiments in real driving environments show that our system can be aware of the traffic rules accurately and immediately.
During the year 2011, there were 11856 traffic accidents that happened in Sichuan province of China and 95% of the accidents are caused by traffic violations . If traffic-violation rate can be significantly decreased, a lot of lives can be saved from traffic accidents. In fact, most traffic violations are not on purpose; the reason of high traffic-violation rate is that the traffic rules are designed to regulate drivers’ behaviours. As long as drivers are human beings, they will suffer from memory issues and attention distraction. Their memory and concentration will be severely affected by mood, alcohol, drugs, and environment. Even a short conversation during driving will distract drivers and cause unnecessary traffic violations.
Google self-driving car  is a good attempt to decrease the traffic-violation rate, because machines do not have the memory and concentration issues like humans. However, google self-driving car only focuses on self-driving; it has not yet taken traffic rules into consideration except for traffic lights. Some other works like  monitor dangerous driving behaviours like aggressive turns, acceleration, and braking and help drivers to correct these unsafe behaviours. However, they did not take traffic rules into consideration. Actually, even if driving behaviours are not aggressive, as long as the driver violates the traffic rule, there is still a big possibility that traffic accidents may happen.
If vehicles have awareness of whether they violate traffic rules, they can warn the drivers about the violations and help them to amend their driving behaviours. In fact, although it is a human who drives the vehicle, the driving behaviours are presented by vehicles’ behaviours. No matter who drives a vehicle or how he drives a vehicle, the driving behaviours can be described by two factors of vehicle’s behaviours: speed and direction. Traffic rules are location based; different location has different traffic rule; therefore location is the third factor to describe driving behaviours for traffic rules. For example, if we want to describe a traffic rule of which speed limit is 40 kilometers per hour and one way from south to north at road A, we can set the three factors as Table 1 shows.
Modern mobile phones are equipped with various sensors like accelerometer, gyroscope, and orientation. They also equipped with GPS localization devices. Accelerometer can be used to monitor the speed of a vehicle; gyroscope and orientation can be used to monitor the direction changes. GPS device can be used to monitor the location. It seems that mobile phones alone are enough to build the traffic-rule awareness system. In fact, traffic rules not only regulate the driving vehicles, but also regulate the parking vehicles. We need to know whether a vehicle is parking or not. No speed does not mean the vehicle is parking; it can be waiting for traffic lights or something else. We need to know whether the engine has been flamed out for a while to judge if the vehicle is parking, therefore we have to take advantage of vehicular sensors to monitor the engine states.
In this paper, we propose a novel traffic-rule awareness system. It translates traffic rules into combinations of the three aforementioned factors. It will judge whether there is the tendency of violating the traffic rules while driving and warn drivers of this tendency. Therefore, it will free drivers from searching for traffic signs and remembering all the traffic rules; drivers can put more energy on driving. Mobile phones can provide 2G/3G/4G data connection for downloading translated traffic-rule from remote servers or construct Mobile Ad-hoc NETwork (MANET) to get the traffic rules from neighbors.
2. System Design
As mentioned before, mobile phones alone can not meet all the requirements of the system; vehicular sensors are good complements. On-Board Diagnostics (OBD) is widely equipped in modern automobiles. The OBD implementations use a standardized digital communications port to provide real-time data in addition to a standardized series of diagnostic trouble codes , or DTCs, which allow one to rapidly identify and remedy malfunctions within the vehicle . With the help of OBD scanner, the values of vehicular sensors can be easily obtained by our system.
2.1. System Overview
As Figure 1 shows, there are three modules in the system: communication module, process module, and alert module. Communication module is used to download traffic rules from remote server via internet or neighbors via MANET; also it provides download service for its neighbors. Process module contains two parts: information collecting part and information processing part. Information collecting part collects three kinds of information which are related to the vehicle: speed, direction, and location. Speed can be obtained from accelerometer of mobile phone; however, as mentioned before, we also need to know whether the vehicle is parking when the speed is zero. Engine state is a way to judge the parking state, and it can be read from OBD scanner. ELM 327 Diagnostic Scanner  is a common off-the-shelf product which provides an interface to access the OBD system. The scanner uses a female 16-pin SAE J1962 connector to connect to the OBD system, and it uses Bluetooth to connect to a control device such as mobile phone. Gyroscope and orientation which are embedded in mobile phones are used to generate the direction information. GPS device which is embedded in mobile phone is used to obtain the location information. The information process part will compare the collected information with the traffic rules stored in DB to judge whether the driver violates the traffic rules. Once there is a trend of traffic violation, the violation alert module will warn the driver.
2.2. Traffic Rules Translation
As mentioned before, vehicles’ behaviours can be described by three factors: speed, direction, and location. In this research, we use a collection of triples TR (speed, direction, and location) to represent the traffic rules. Speed is a 2-tuple Speed (lower, upper); the lower and upper elements are the lower and upper bounds of an interval of speed limit. For example, for the highway, the speed limit is not less than 80 km/h and not more than 120 km/h, so the 2-tuple is Speed . Direction is a 2-tuple Direction (road direction, turns); road direction is the direction allowed on the road and it is a set of five values: NORTH, EAST, WEST, SOUTH and TWOWAY. If the road is one way street, road direction gives the heading direction. If the road is two way street, road direction gives the TWOWAY value. Turns is a quadruples Turn (Left, Right, U, Lane Change), each element indicates whether this kind of turn is allowed. Location is a 2-tuple Location (road, area). Element road is the road name on the map; area is a 2-tuple area (from, to) which indicate the area of road applying this traffic rule. From and to are GPS coordinates. For example, as Figure 2 shows, Chengdu road is an one-way street and the speed limit is not more than 40 km/h. Between Locations 2 and 3, vehicles can choose to turn right or go straight. The traffic rules at Chengdu road can be translated to three TR (speed, direction, location) triples as Table 2 shows. We can see that the lower bound of Speed (lower, upper) is 1; the reason is that parking on the road is not allowed. If a location allows parking, the lower bound of speed will be set to zero.
2.3. Information Collection
There are three kinds of information need to be collected: speed, direction and location. The process module uses OBD parameter IDs (PIDs) codes to request data from a vehicle. SAE standard J/1979 defines many PIDs, but manufacturers also define many more PIDs specific to their vehicles . In this research, we use standard PIDs for good compatibility.
Speed can be calculated through GPS coordinates or accelerometer, but a more easier way is that sending a PID code VEHICLE_SPEED to OBD scanner, the OBD system will return the vehicle speed. However, speed is not enough for traffic-rule translation; we also need to know the engine states. To detect whether the vehicle is parking, we need to know whether the engine is off for a period of time. There is an OBD PID code ENGINE_RPM which indicate the rotation speed of engine, if the engine is off, the rotation speed is zero. Therefore, a vehicle is parking when VEHICLE_SPEED is zero and ENGINE_RPM is zero for a period of time. As (1) shows, only when the vehicle is parking, we set the speed of the vehicle to zero, otherwise the speed is larger than zero:
To detect the direction, we need three kinds of sensors: accelerometer, orientation, and gyroscope. Before discussing about how to collect direction information, we will first discuss the coordinate system of mobile phone. As Figure 3 shows, the -axis is horizontal and points to the right, the -axis is vertical and points up, and the -axis points towards the outside of the front face of the screen . Data from all the sensors embedded in mobile phone is three-dimensional and obeying this coordinate system.
To obtain the value of road direction, we need to use orientation sensor. The data from orientation sensor follows three dimensions: Azimuth (degrees of rotation around the -axis); Pitch (degrees of rotation around the -axis); Roll (degrees of rotation around the -axis) . In this research, we fix the mobile phone head forward and flatwise; therefore, Azimuth is the degree we needed. Table 3 shows the mapping relation between Azimuth and road direction.
We can use accelerometer to detect the turn event of driving behaviour, as Figure 4 shows; three kinds of turns, lane change, left turn and turn are easily distinguished from each other. There are many works discussed about the driving patterns and gestures recognition [9–11]. However, we cannot tell the driver: “You cannot turn left” when he already turned left. We have to make prediction of drivers’ turning intentions. Gyroscope is a very sensitive device to detect the rotation action; we can recognize the turning intentions at the very beginning of drivers’ turning behaviours by taking full advantage of gyroscope. Detailed discussion is in the next section.
Modern mobile phones are equipped with GPS devices; therefore, the location of vehicles can be easily obtained. With the help of Geography Information System (GIS), we can locate the vehicle on the map by inputting GPS coordinates.
The main role that the traffic-rule awareness system plays is to warn drivers about the possible future traffic violations. Therefore, different from other works of recognizing driving patterns which are already happened, the system must predict drivers’ intentions. Existing researches in prediction of driving intentions are mostly using cameras which would bring inconvenience and privacy issue to the drivers. In this research, we only take advantage of sensors from mobile phones to achieve the prediction.
2.4.1. How to Predict Drivers’ Intention?
According to velocity formula , to predict the velocity of a vehicle, we need to know current velocity and the acceleration . can be obtained through OBD system can be obtained through accelerometer; therefore, can be predicted.
In the coordinate system of mobile phone, data from accelerometer are three dimensions: (acceleration at -axis), (acceleration at -axis), and (acceleration at -axis). Acceleration can be calculated as (2) shows. To predict the velocity , we need to average , as (3) shows, to reduce measurement error. is the current time and is the time interval of prediction. Equation (4) gives the final prediction. Once the prediction does not belong to Speed (lower, upper), a warning will be alerted:
Predicting the turning event is a great challenge. Firstly, turning event is a short-time process; the prediction must be made at the very beginning to give enough response time for drivers. Secondly, normal driving behaviours contain a lot of turning actions, such as dodging and parking. To distinguish these turning actions from real turning event, we have to take advantage of turn signal. Turning on turn signal is a standard process in turning event; every time drivers want to make turns, they will firstly turn on the turn signal. Therefore, if we can detect the turn signal, we can recognize the real turning event.
2.4.2. How to Detect the Right/Left Turn Signal?
OBD system is no doubt the first choice of detecting the left/right turn signal; it can accurately tell whether the left or right turn signal is on. For example, Chevrolet Corvette OBD-II Codes contain the codes to return the information of left and right turn signals . However, the OBD codes are not the standard OBD codes; they are exclusive to a certain vehicle manufacturer. To find a more compatible way to recognize the turn signal, we use a method called Sound Cross-Correlation (SCC).
The main idea of SCC is that the sound features of turn signal are precaptured; every time a sound signal received, the system will cross-correlate the received sound signal with the sound of turn signal. When a spike occurs in the correlation, it means THAT turn signal is in the received sound signal. Figure 5 shows the sound wave feature of turn signal. Cross-correlation is a widely used method in recent research works . Suppose the known symbol pattern of sound wave of turn signal is of length , is the complex number representing the received symbol, then the cross-correlation at a shift position is
is the complex conjugate of and is the correlation coefficient. Once the is aligned with , there will be a sudden spike in the correlation as Figure 6 shows.
SCC cannot distinguish right turn and left turn; however, it will not affect the recognition of left turn and right turn in out system, because turn signal is only used for detecting drivers’ turning intention.
2.4.3. How to Distinguish Lane Change, Left Turn, and U Turn?
Only if we can hack in drivers’ brains, otherwise we cannot know whether they want to turn left or change lane or make turn in advance. Now the turn signal is detected, the driver’s turning intention is known, and the problem is there are three kinds of turning; all of them are related to the traffic rules; therefore the three kinds of turning must be distinguished. Only when the driver makes turns, we can know what kind of turns he makes. If we recognize the kind of turning after the turning behaviour is finished like the pattern recognition or gesture recognition does, it is meaningless to issue a warning to the driver after the traffic violation. The challenge is that the recognition must be finished at the very beginning of the turning behaviour.
The movement of turning can be divided into two kinds of movements: translation movement and spin movement. Spin movement is the key factor to distinguish turning style. Accelerometer measures the acceleration of vehicles; it can be used to detect the translation movement, but it can not detect the spin movement; therefore it is not suitable for turning detection. Gyroscope is a sensitive device which can detect angular speed in three dimensions according to the coordinate system of mobile phone. If the mobile phone is fixed flatwise, the rotation around -axis can be used to reflect the spin movement; therefore gyroscope can be used to distinguish the turning style.
Figure 7 shows the gyroscope readings of lane change (from right to left), left turn, and turn (from right to left); we choose the Azimuth reading to reflect the spin movement. We can see that the first positive peak of three turning style all appears at the beginning (there is a process of approaching to right side before turn, so the first positive peak appears a little late). The peak represents the start of spin movement of the three turning styles, and the three turning style have different peak intervals. The peak interval of lane change is ; the peak interval of left turn is ; the peak interval of turn is . Therefore, we can distinguish the three turning styles at the beginning of turning by which intervals of the first positive peak belong to.
3. Implementation and Evaluation
The experiment evaluates three aspects of the system: turn signal detection, turning event prediction, and traffic rules awareness. All the experiments are running in real driving environment. The automobile we use is Buick Regal which provides a standard OBD system. The mobile phone we use is Lenovo S2 which has 1 GB ram and 1.4 GHz CPU core frequency which can provide enough processing ability, also the phone has all the sensors we need including orientation sensor, acceleration sensor, and gyroscope sensor.
3.1. Turn Signal Detection
Cross-correlation can well recognize a signal in a mixed signal; however, it has difficulties in recognizing when SINR is low; here we treat the sound wave of turn signal as Signal, the others as Interference and Noise. We test the cross correlation in three different environments: quiet, talking, and playing music. Figure 8 shows the results of cross correlation in the three environments. We can see from Figure 8(a) that cross correlation can well recognize turn signal in a quiet environment; four separate spikes indicate four turn signals; in a talking environment as Figure 8(b) shows, human voices sometimes submerge the turn signals and make the recognition impossible. However, talking is a discrete process, it has nontalking gap for us to recognize the turn signal, and there is a spike at the beginning which indicates the existence of turn signal. Other spikes are caused by interference which must be excluded; the spikes are usually mixed together; therefore we choose the separated spikes as the indication of turn signal; Figure 8(c) shows the cross correlation in a music playing environment; we can see that there is no separated spike; therefore recognition of turn signal is almost impossible in a loud and continuous interfering environment.
Loud and noisy driving environment is bad for driving safety; therefore, in most cases, driving environment is quiet and suitable for turn signal recognition.
3.2. Turning Event Prediction
We test the turning event prediction in the campus. Once the system recognizes any of three turning patterns, it will give a warning and the driver stops the car. Figure 9 shows the results of experiments; we can see that the car stops at the very beginning of turning movement and gives the driver chances to amend their driving behaviours.
3.2.1. Traffic Rules Awareness
We choose an area in the campus which is sparsely populated and convenient for us to do the tests. As Figure 10 shows, we choose four road segments and label them with four colors: red, green, blue, and yellow. Red road is for testing lane change and left turn restriction; green road is for testing turn and one-way restriction; blue road is for testing speed limit; yellow road is for testing parking restriction. The traffic rules are listed as Table 4 shows.
The results show that our system has good awareness of traffic rules; however, there are a few misjudgments during the tests; when we do the same tests again at another time, misjudgments disappear. The reason probably is that the accuracy of GPS localization is sometimes not good; it will cause a vehicle at road B violates a rule at a neighbor road A.
4. Related Works
Vehicle safety is a permanent topic; the main reason VANET and Vehicular Sensor Network are proposed as to enhance the safety. Traffic rules are closely related to driving safety in real driving environment; however, there are few researches take it into consideration. Most researches focus on recognizing drivers’ driving behaviours and try to correct some dangerous behaviours.
4.1. Intentions Prediction
Most researches use vision-based method to recognize body gestures to predict driving intentions.  uses camera to capture body pose to predict turning intention;  uses camera to capture head pose to judge whether the driver focuses on driving;  generates 3D visions to assist driving.  establishes a probabilistic model to analyse and predict driving intentions to assist brake controls.
4.2. Behaviours Recognition
Driving behaviours recognition can be used to detect some dangerous behaviours, like drunk drive . Most recognition works are using accelerometer [19, 20]; it is enough for driving pattern recognition, but not enough to describe more complex driving behaviours.  takes advantage of more sensors: gyroscope, magnetometer, to detect more complex and dangerous driving behaviours.
4.3. Traffic Lights
Traffic lights detection is related to traffic rules;  uses camera from mobile phone to detect and predict the traffic light schedule. It is a complement to our research.
Traffic rules regulate drivers’ behaviours so that there could be fewer traffic accidents and fewer lost lives. However, a human is not a machine; he has memory and attention troubles, he is easily affected by many things; therefore traffic violation rates never come down too much. Modern vehicles already have powerful processing ability and various sensors for sensing the world; they can be aware of the traffic rules for human so that drivers can focus on dealing with emergency situations such as jaywalking; it will no doubt significantly decrease traffic-accident rates and save thousands of lives.
However, how to predict drivers’ intentions as soon as possible and improve localization accuracy will be the future works.
This work is supported by the National Science Foundation under Grant nos. 60903158, 61003229, 61170256, and 61103226 and the Fundamental Research Funds for the Central Universities under Grant nos. ZYGX2010J074 and ZYGX2011J102.
- Sichuan traffic accidents statistics, http://www.sc122.gov.cn/system/2012/02/01/013433326.shtml.
- How google self-driving car works, http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/how-google-self-driving-car-works.
- D. A. Johnson and M. M. Trivedi, “Driving style recognition using a smartphone as a sensor platform,” in Proceedings of the 14th International IEEE Conference on Intelligent Transportation Systems, Washington, DC, USA, 2011.
- On board diagnostics parameter ids, http://en.wikipedia.org/wiki/Table of OBD-II_Codes#Bitwise_encoded_PIDs.
- On board diagnostics, http://en.wikipedia.org/wiki/On-board-diagnostics.
- Elm327 diagnostic scanner, http://www.tmart.com/OBD-Diagnostics/.
- Android sensor coordinate system, http://developer.android.com/reference/android/hardware/SensorEvent.html.
- Orientate sensor, http://developer.android.com/guide/topics/sensors/sensors_position.html.
- D. Mitrović, “Reliable method for driving events recognition,” IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 2, pp. 198–205, 2005.
- G. A. ten Holt, M. J. Reinders, and E. A. Hendriks, “Multidimensional dynamic time warping for gesture recognition,” in Proceedings of the 13th Annual Conference of the Advanced School for Computing and Imaging, 2007.
- R. Muscillo, S. Conforto, M. Schmid, P. Caselli, and T. D'Alessio, “Classification of motor activities through derivative dynamic time warping applied on accelerometer data,” Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS '07), vol. 2007, pp. 4930–4933, 2007.
- 2002 chevrolet corvette obd-ii codes, http://www.obd-codes.com/trouble_codes/chevrolet/2002-corvette-obd-ii-codes.php.
- S. Sen, R. R. Choudhury, and S. Nelakuditi, “CSMA/CN: carrier sense multiple access with collision notification,” in Proceedings of the 16th Annual Conference on Mobile Computing and Networking (MobiCom '10), pp. 25–36, September 2010.
- S. Y. Cheng and M. M. Trivedi, “Turn-intent analysis using body pose for intelligent driver assistance,” IEEE Pervasive Computing, vol. 5, no. 4, pp. 28–37, 2006.
- E. Murphy-Chutorian, A. Doshi, and M. M. Trivedi, “Head pose estimation for driver assistance systems: a robust algorithm and experimental evaluation,” in Proceedings of the 10th International IEEE Conference on Intelligent Transportation Systems (ITSC '07), pp. 709–714, October 2007.
- C. Tran and M. M. Trivedi, “Towards a vision-based system exploring 3D driver posture dynamics for driver assistance: issues and possibilities,” in Proceedings of the IEEE Intelligent Vehicles Symposium (IV '10), pp. 179–184, June 2010.
- J. C. McCall and M. M. Trivedi, “Driver behavior and situation aware brake assistance for intelligent vehicles,” Proceedings of the IEEE, vol. 95, no. 2, pp. 374–387, 2007.
- J. Dai, J. Teng, X. Bai, Z. Shen, and D. Xuan, “Mobile phone based drunk driving detection,” in Proceedings of the 4th International Conference on Pervasive Computing Technologies for Healthcare (Pervasive Health '10), March 2010.
- G. A. ten Holt, M. J. Reinders, and E. A. Hendriks, “Multidimensional dynamic time warping for gesture recognition,” in Proceedings of the 13th annual conference of the Advanced School for Computing and Imaging, 2007.
- R. Muscillo, S. Conforto, M. Schmid, P. Caselli, and T. D'Alessio, “Classification of motor activities through derivative dynamic time warping applied on accelerometer data,” Proceedings of the 29th Annual International Conference of the IEEE, vol. 2007, pp. 4930–4933, 2007.
- E. Koukoumidis, L. S. Peh, and M. R. Martonosi, “SignalGuru: leveraging mobile phones for collaborative traffic signal schedule advisory,” in Proceedings of the 9th International Conference on Mobile Systems, Applications, and Services (MobiSys '11), pp. 127–140, ACM, New York, NY, USA, July 2011.