Abstract
The recent technologies for vehicular networks including wireless communication have allowed vehicles to provide biosensor based various applications to a driver. This paper proposes a system to provide a driver with the application which the driver wants in a vehicle using driver’s biodata. The proposed system is composed of four components including a sensing unit, an inference unit, an application providing unit, and vehicular network unit. In this paper, the experiments were performed for correlation investigation between the number of biodata and the reliability of driver’s characteristics extraction. We found that the number of the data used had a significant influence on improving the driver’s satisfaction level, playing an important role in reflecting driver’s status. Experiments using actual vehicles are conducted to examine the effectiveness of the proposed system, and the results are analyzed and discussed. The results on the investigation of detection of biodata in gasoline and diesel vehicles by the experiments are also given. We confirmed that the proposed sensing method was capable of accurate detection irrespective of vehicle mode and vehicle type.
1. Introduction
Automobiles have been evolved from traditional mechanical system to a state-of-the-art electronic system integrated with information technology (IT). In addition, they are beginning to be regarded as a mobile device or a node carrying big data on distributed vehicular networks [1, 2]. The recent technologies for vehicular networks including wireless communication have allowed vehicles to provide various applications to a driver. A variety of applications (including music, movies, and broadcasting) have been provided in a vehicle. Many drivers can choose interesting application using button operation on driving or stopping. However, as most operations are being performed on driving, these may cause an accident due to a careless mistake. As a result, more importance is being placed on research and development (R&D) in the field of intelligent systems which provide application according to the driver’s needs.
Out of the need, technologies of human sensibility ergonomics and intelligence system were applied to the vehicle [3, 4], and studies to analyze driver’s characteristics using various sensors in the vehicle had been conducted [5–8]. In the study [9], driver’s electrocardiogram (ECG) and electroencephalogram (EEG) were measured to analyze correlation between road and driving conditions and studies [10, 11] showed the results where stress levels during driving were reviewed using the detected biodata of the driver. Reference [12] analyzed the data related to driver fatigue using sensors including a video sensor to capture the driver image and a biosignal sensor to gather the driver photoplethysmograph (PPG) signal. A warning alarm is sounded if driver fatigue is believed to reach a defined threshold. Reference [13] reviewed the driver drowsiness detection system and techniques based on ECG, EEG, local binary pattern (LBP), steering wheel movement (SWM), and optical detection. Reference [14] extracted driver yawning based video processing and [15] detected driver fatigue by transcutaneous acupoints electrical stimulations based on EEG signals. Reference [16] used the ECG measuring sensor on the steering wheel for in-vehicle driver recognition and security features of the vehicle. Reference [17] recognized driver state such as stress, fatigue, and predicting driving performance using the physiological signals from the driver (ECG, EDA, and respiration), video recordings from the driver’s face, and environmental information. A number of proofs of possibility studies for emotion detection in nonlab settings had been performed, using either commercially available sensor equipment for physiology data or experimental devices [18–21]. Healey’s research was focused on recognizing stress levels of drivers by measuring and analyzing their physiological signals (skin conductance, heart activity, respiration, and muscle activity) [22]. In order to map physiological signals (galvanic skin response, heartbeat, and temperature) to driving-related emotions (frustration/anger, panic/fear, and boredom/sleepiness), a driving experiment was designed and conducted in a highly controlled virtual reality environment [23].
While most of previous studies analyzed or extracted driver’s status (mostly driving-related emotions), they were unable to provide application that a driver wants while driving: most results using the extracted driver’s status were utilized for the study on drowsiness and stress level measurement.
This paper proposes in-vehicle application (service) providing system based on a driver’s biodata. To allow providing steady and natural measurement while stopping and even driving, heart rate (HR) and skin conductance response (SCR) are used as driver’s biodata. Personal data, vehicle data, and environment data are additionally used with the biodata. Those data are used as the inputs for inferring a driver’s status, which provides the driver with the application he/she wants in a vehicle. To investigate the performance of the proposed system, the proposed system is applied to actual vehicles with the music application (the most widely used one in a vehicle), and the results are analyzed and discussed.
2. Proposed Driver’s Biodata Based Application Providing System
The proposed system mainly consists of four units as follows:(i)driver’s biodata sensing unit;(ii)driver’s status inference unit;(iii)application providing unit;(iv)vehicular network unit.
2.1. Driver’s Biodata Sensing Unit
Biodata is the information acquired from direct sensing of a driver. Driver’s biodata covers EEG, ECG, electromyogram (EMG), HR, and SCR. Existing methods for extracting biodata of a driver need specific sensors and must be worn on the specific body parts for measurement, making it difficult to be applied in actual vehicles. Therefore, this study introduced two sensing methods for HR and SCR which are not only applicable to actual vehicle but also easy to get biodata of a driver during driving. The proposed HR sensor senses driver’s HR with the auditory-type heartbeat detector embedded in the safety belts while not only facilitating natural wearing but also measuring HR with the clothes on. To make HR sensor robust to external noise, noise cancellation filter for both high frequency and low frequency was added. The SCR sensor extracts SCR by detecting the wetting condition of the driver’s skin via an electrode panel attached on the steering wheel: it is based on the principle that SCR is lowered by the perspiration on the palm skin reflecting high sensory temperature of the driver. Figure 1 shows that the sensors (to measure HR and SCR) are attached on the safety belt and steering wheel, respectively. The measuring equipment of driver’s HR and SCR is designed as a prototype to examine the applicability into an actual vehicle. Figure 2 shows the circuit diagram for measuring HR(a) and SCR(b). Low pass filter (LPF) removes noise of initial high frequency, and it outputs certain low frequency bandwidth and direct current (DC). Cut-off frequency is 30 Hz, and alternating current (AC) for HR is extracted through the differential signal detection. Impulse of HR is maximized using differentiator and amplifier, and clear HR measurement was possible through high pass filter (HPF). Similarly, SCR measurement was clear using differential signal detecting, filter, amplification, and signal tuning.
(a)
(b)
2.2. Driver’s Status Inference Unit
Besides using the biodata obtained by the sensing unit, the proposed system uses personal data, vehicle data, and environment data as additional information for driver’s status inference: personal data of a driver includes age and gender, which is the information input by the driver or available from the previously saved profile. The environment data includes time, weather, day, and location, while the vehicle data comprises general vehicle data such as speed and travelling data including travelling time and distance. Environment and vehicle data are provided after measuring with various devices inside and outside of a vehicle such as clock for time, temperature sensors for both internal temperature and external temperature, navigation system or global positioning system for location, and odometer for travelling data. The driver’s status inference unit plays a role in extracting the driver’s status based on the above-mentioned data provided (measured and given) by the sensing unit, and the driver’s status can be obtained bywhere is initial value of EP, is weight of the heart rate, is weight of the SCR, is weight of the current time, is weight of the age, and is weight of the driving time.
The energetic point (EP) plays a role in determining the status of the driver. EP is a psychological arousal state and refers to the driver’s emotional state and is divided into three parts based on Russell’s circumplex model [24] in inference unit. The value of EP ranges from 0 to 100 with the median value of 50 EP representing the normal state of the driver. A higher EP indicates that the driver is excited while any value less than 50 can be considered as having a higher chance of drowsy driving which in turn increases the chance of the driver getting into an accident. The initial value of EP, , is usually set at 50 and varying within a range of ±10 depending on the driving environment.
The weight of the heart rate is calculated asThis is established based on previous studies [25, 26] which found that higher HR is closely related to accidents due to wild driving, whereas lower HR is associated with the accidents caused by drowsy driving.
The weight of the SCR, , is defined by the following:where normal state indicates the lack of palm perspiration, while tension state means that perspiration is detected on the palm.
Weight on time, , is calculated by the following:For example, if the driving time is between 0 (0 A.M.) and 6 (6 A.M.), is set at 0. is used as the weighting to provide application tailored to different age groups. In addition, is used as the weighting to determine when to notify the driver to take a break. This weight is used to prevent the driver from falling asleep in the event of long distance driving. Driver can preset the in the application of the proposed system, and warning alarm is sounded if time out.
2.3. Application Providing Unit
This unit is capable of providing a variety of applications as shown in Figure 3 based on the driver’s status inferred from Section 2.2. The unit provides the customized suggestions extracted from the inference unit based on the sensed data. The unit additionally displays all sensed data as well as the inferred driver’s status data on the screen. The application providing unit can also provide a variety of applications such as music, movie, and news.
2.4. Vehicular Network
This network has two units: the data network and the interface network. The interface network has a role in communicating the data and command between the other three components (sensing unit, inference unit, and application providing unit). The data network has a role in transferring the data of the three units as shown in Figure 4(a). In particular, it has a role in networking between all the units of the vehicle. Figure 4 shows the overall block diagram (Figure 4(a)) and dedicated circuit board developed for the proposed system (Figure 4(b)), respectively.
(a)
(b)
3. Application to Vehicles and Results
3.1. Experiments
In this section, the system introduced in Section 2 is applied to actual vehicles, and driver satisfaction according to the tailored application is evaluated. Figure 5 shows the experimental setup using actual vehicles. The 2007 model of Hyundai Motor’s gasoline-powered SONATA, the 2008 model of Kia Motor’s gasoline-powered SPORTAGE, and the 2005 model of Kia Motor’s diesel-powered CERATO were used in the experiment.
(a)
(b)
As mentioned in Section 2.4, the audio function is one of the most frequently used among various applications available within the vehicle. Thus, in this study, we will assess the feasibility of the proposed system by conducting an audio application (music application) providing experiment customized to the driver’s status. The three genres of music (exciting music, relaxing music, and trot music) are selected and provided as the music application considering Korean culture. Each genre of music is tagged and stored in the database. To determine the three adjectives of music mood, we derived adjectives that are representative of the mood of the different music genres based on adjective check list by Hevner [27, 28]. In addition, we derived mapping relation between the driver’s preferences and music genres by associating each genre with adjective [29]. Figure 6 shows the graphic user interface (GUI) developed for music providing application experiment of the proposed system. As shown in Figure 6, the GUI for the test is simulated under the identical environment as a subject (driver) manipulates a navigation device (MTS-255 model provided by Hyundai Motors) within a vehicle. The area inside solid line is the menu screen of music application provided to a subject via navigation device, the area inside dotted line being the internal screen to monitor various data used in the experiment.
Figure 7 shows an example of probabilistic model derived for music recommendation. For example, if a driver (54 years old) having a faster HR, tension state is driving in the morning, exciting music (5%), relaxing music (45%), and trot music (50%) will be provided by probabilistic model of proposed system. The three experiments were conducted as shown in Table 1. In Experiment 1, the data except biodata (of driver’s data) were used to provide music application: personal data (of driver’s data), environment data, and vehicle data were used. In Experiment 2, HR data was added to the data used in Experiment 1: music applications were provided based on both driver’s data used in Experiment 1 and one biodataset (HR data). In Experiment 3, SCR data was added into the data used in Experiment 2: music applications were provided based on both driver’s data used in Experiment 1 and two biodatasets (HR and SCR data).
3.2. Discussions: Analysis of the Results Obtained Using the Proposed System
12 subjects whose ages are between 20s and 40s participated in the experiments and were asked to evaluate their level of satisfaction on the music provided by the proposed system with Likert-scale five scoring levels ((5) highly satisfied (4) satisfied (3) average (2) unsatisfied (1) highly unsatisfied). Each subject conducted an experiment once daily for 3 days. To assure the reliability of the experiment process, experiments were conducted at the same CT and DT for each subject. The subjects did not know the order and content of the three experiments and the experiments were conducted randomly.
Table 2 lists all the results and Figure 8 shows the comparison results on the satisfaction levels obtained from the three experiments, respectively. The results shown in Figure 8 indicated that higher number of biodata used in the experiment is associated with higher levels of satisfaction. The satisfaction level obtained from the experiment result using Method 2 including HR data was higher than that obtained from the experiment using Method 1 without biodata, and moreover the one obtained using Method 3 in which the SCR was added was confirmed to be higher than that of Experiment 2 (while the overall satisfaction ratio (%) of Experiment 3 was identical to that of Experiment 2, the rating (4) given for “satisfied” increased from 41.7% to 50%). From the aforementioned results, it was found that the number of the data (in particular, the number of the biodata) used had a significant influence on improving the driver’s satisfaction level, playing an important role in reflecting driver’s status.
These results were analyzed for significance using one-way ANOVA from the SPSS computer program [30], and we used Scheffe based post hoc tests for the three experiments’ analysis. Table 3 shows the post hoc tests results. Satisfaction by experimental methods was dismissed because it is larger than the level of significance of 0.05. Although satisfaction results gap was not big enough, satisfaction result of Method 3 was shown to be slightly higher than Method 1 and Method 2 due to added biodata (HR and SCR).
Next, we examined whether the biodata critical to providing the aforementioned application were accurately detected within the vehicle. Figures 9 and 10 showed the HR data measured by the proposed system in both driving and stationary mode.
(a)
(b)
(a)
(b)
From the results, it was confirmed that the proposed sensing method was capable of accurate detection irrespective of vehicle mode (including driving and stationary mode) and vehicle type (such as gasoline-powered vehicle and diesel-powered vehicle). (The median voltage of waveform was approximately 4 [V] and period was approximately 0.8 [s].)
As the SCR is not affected by the state of vehicle, the experiment was conducted by switching the types of vehicle only.
The following three states were examined to determine measurement accuracy.(i)Disconnected electrode state: a state where a driver is not holding the steering wheel, with no SCR detection.(ii)Connected electrode state: a state where a driver holds the steering wheel, where output value proportional to the quantity of perspiration caused by stress must be measured.(iii)Short-circuit state: it is for the verification of sensor circuit, whose value must be 0.
As shown in Figure 11, (1) when the driver’s hands were not in direct contact with steering wheel, SCR was not measured (green circle), (2) when the driver gripped the steering wheel, SCR was measured at around 2 [V] in dry condition (blue circle) and at a little lower value in a wet condition with perspiration resulting from strain (red circle), and (3) when the electrode was forced to short for circuit verification, the voltage showed a measure of 0 [V] (black circle). From the figure, it was confirmed that the proposed measurement was able to extract the SCR data accurately regardless of vehicle type (such as gasoline-powered vehicle and diesel-powered vehicle), which was good enough to be used for deducing the driver’s status in the inference unit.
(a)
(b)
We proposed driver’s biodata based application providing system, whereas most of previous systems provided sophisticated alarming service to prevent drowsiness based on a driver’s biodata. In conclusion, there is no comparable system for music providing application in vehicle.
4. Conclusions
A driver’s biodata based application providing system was presented in this paper. To examine how well the application given by the proposed system reflects on the driver’s status, the proposed system was applied to actual vehicles providing the music application, and the results were analyzed and discussed. From the experimental results, our findings indicated that the number of the data, in particular the number of the biodata used, had a significant influence on improving the driver’s satisfaction level, playing an important role in reflecting driver’s preference. It was also confirmed that the proposed measurement was able to sense both the HR and SCR data accurately enough to reflect the driver’s status in the inference unit of the proposed system irrespective of vehicle mode (both driving and stationary mode) and power type of the vehicle (gasoline-powered or diesel-powered). The study is expected to be utilized as a guideline for providing a variety of driver customized services in vehicle as biosensor is developed to be more sophisticated and compact: our study presented in this paper dealt with the feasibility for music providing (as a single service) based on biodata, and our future study will be expanded for providing multimedia services including personal news and video.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This paper was supported by the 2015 Education and Research Promotion Program of KoreaTech.