Abstract
This study explored the relationships between subjective evaluations and objective metrics of vehicle dynamic performance. First, a real vehicle test was performed to measure the acceleration performance under different conditions, and participants’ subjective evaluations of the acceleration performance were investigated. Second, correlation analysis was conducted to explore relationships between each subjective evaluation and its corresponding objective metric as well as between the overall subjective evaluation and three individual subjective evaluations. Finally, an overall subjective evaluation model related to the three objective metrics was established based on the Probabilistic Neural Network (PNN). The analysis results demonstrated that the correlation coefficients of the three groups of data were greater than 0.5 and that each subjective evaluation was significantly correlated with its corresponding objective metric. The individual subjective evaluation of the climbing acceleration performance had the largest effect on the overall subjective evaluation, with a correlation coefficient of 0.47. The established overall subjective evaluation model was relatively reliable, with a prediction accuracy of 90%. This study furthered the existing knowledge of the methods for evaluating vehicle dynamic performance. The proposed overall subjective evaluation model improves the reliability of vehicle dynamic evaluations and offers a theoretical basis for vehicle manufacturers to improve automobile performance.
1. Introduction
Before prototype vehicles are put into production, many performance indicators are tested to assess whether a vehicle achieves the design goals and to identify differences between the prototype vehicle and competing vehicles. Among the vehicle performance indicators, the dynamic performance is the most basic and important as it indicates that vehicles can reach the average travel speed under longitudinal force when driving straight on good roads. The dynamic performance is often tested using three indices: maximum speed, acceleration time, and maximum gradient [1]. The dynamic performance directly affects vehicle efficiency and is therefore one of the most important performance indicators for users as well as a key area in which vehicle manufacturers can improve their competitiveness.
To improve the validity of vehicle dynamic performance evaluations, certain domestic and foreign scholars have continued to explore evaluation methods for this performance indicator. According to the evaluation process, these methods can be divided into subjective evaluation methods and objective evaluation methods. In subjective evaluation methods, trained evaluation personnel or experienced drivers evaluate the vehicle performance based on their experiences in road tests. In objective evaluation methods, researchers analyze vehicle operation data collected via computer simulations or road tests to assess the vehicle performance. These two evaluation methods differ in application range and evaluation effectiveness. In practical applications, researchers select the appropriate method based on actual demands. For example, vehicle manufacturers in Germany primarily adopt the subjective evaluation system presented by Bernd Heißing et al. This evaluation system includes acceleration performance, and certain poor performance issues (such as acceleration jitter and acceleration pitching) are taken as the evaluation indices for acceleration performance during vehicle startup [2]. The subjective evaluation system of UK Lotus is also relatively complete and includes dynamic performance feel as an evaluation item. The related indices mainly focus on the dynamic performance feel under different acceleration conditions, including full throttle, partial throttle, starting acceleration, and overtaking acceleration [3]. A company in Japan uses an evaluation system that examines the vehicle dynamic performance more comprehensively based on evaluation indices related to driving force and drivability [4]. Vehicles from different manufacturers have different characteristics and styles; therefore, each manufacturer adopts different subjective evaluation indices. However, manufacturers in China typically use foreign evaluation systems to evaluate vehicle dynamic performance, as the subjective evaluation systems that have been developed by Chinese manufacturers and researchers are not sufficient. Based on existing research and applications, Li Bing presented a subjective evaluation system for dynamic performance and braking performance; compared to other evaluation systems [5], this proposed system is relatively complete. Based on the existing subjective evaluation index system, Wang et al. proposed a method to determine the weight of the subjective evaluation index of vehicle dynamic performance based on entropy method [6]. Combining the subjective evaluation system of vehicle dynamic performance with the weight coefficient is helpful to improve the reliability of the evaluation results. In summary, the subjective evaluation systems developed in foreign countries are more comprehensive than those developed in China.
The subjective evaluation method is widely used to evaluate vehicle performance because it is easy to perform. This method requires items to be described with demotic language and requires the use of experienced drivers, but certain problems such as strong subjectivity and poor reproducibility cannot be avoided completely. With the development of science and technology, the objective evaluation method for vehicle performance is gradually overcoming the shortcomings of the subjective evaluation method. The objective evaluation method can evaluate vehicle performance based on vehicle operation data collected from computer simulations or road tests. These objective evaluation methods for vehicle dynamic performance can be further divided into two categories. The first category includes evaluating the performance of a vehicle through a bench test, a simulation test, or a driving simulation experiment [7, 8] or presenting evaluation methods of dynamic performance based on simulation platforms [9]. The second category evaluates the dynamic performance through road tests; for instance, Wei proposed an objective evaluation system with evaluation indices related to dynamic performance and drivability as well as the test conditions of all indices [10]. Zhou put forward a new method to evaluate the dynamic performance of the vehicle; this method evaluates the vehicle dynamic performance through the driving performance feeling index [11]. The results of the PFI evaluation method and the conventional vehicle dynamic performance evaluation method showed that the PFI is a more objective and comprehensive dynamic performance evaluation method. In addition, with the development of electric vehicles, some scholars have devoted themselves to study the dynamic and economy evaluation methods of electric vehicles or evaluate their dynamic performance [12–15]. These studies laid the foundation for objective evaluations of vehicle performance.
Compared with the subjective evaluation method, the objective evaluation method has a smaller error and more intuitive results. However, when the objective evaluation method is used for certain types of performance indicators, such as comfort and drivability, the evaluation performance is poor. In contrast, such performance indicators can be effectively tested using the subjective evaluation method; thus, the two types of evaluation methods complement each other. To further explore the relationships between these two evaluation methods, certain scholars have explored the relationships between subjective evaluation items and objective metrics of vehicle handling stability. The related research can be divided into two categories: the first category explored the linear relationship between subjective evaluation items and objective metrics through regression analysis [16, 17] and the second category used neural networks to explore the nonlinear relationship between subjective evaluation items and objective metrics (King et al., 2003) [18]. These studies provided theoretical guidance for exploring the relationships between subjective and objective evaluation methods in terms of other vehicle performance metrics. Dynamic performance is an important index for measuring vehicle performance; however, certain real vehicle tests related to dynamic performance require higher costs and longer cycles, so no relevant research has explored the relationships between the two evaluation methods for vehicle dynamic performance based on real vehicle test data. As a result, vehicle manufacturers suffer from uncertainty and a lack of knowledge when selecting a dynamic performance evaluation method.
To improve the existing knowledge of the evaluation methods for vehicle dynamic performance, this paper explores the relationships between the subjective evaluation items and objective indices for dynamic performance. This work was conducted as follows. First, the evaluation indices were selected, and road tests were performed to collect the subjective evaluation data and objective measurement data for all indices. Second, Pearson correlation analysis was applied to explore the relationships between each subjective evaluation and its corresponding objective metric as well as between the overall subjective evaluation and the three individual subjective evaluations. Finally, an overall subjective evaluation model related to the objective metrics was established based on the Probabilistic Neural Networks (PNN). This model combines the subjective evaluation method with the objective evaluation method to improve the reliability of dynamic performance evaluation.
2. Materials and Methods
2.1. Real Vehicle Test
2.1.1. Test Instrumentation and Participants
The test vehicle, with a displacement of 1.6 L, an age of six months, and mileage of 7,359 km, passed a quality review performed by senior technicians. During the test, researchers used opaque cards to cover all of the logos on the test vehicle in order to prevent the vehicle brand from impacting participant evaluations. An electronic control unit with an output frequency of 30 Hz was installed on the test vehicle and was used to record the automotive operation data, including speed and engine speed. In addition, a car recorder was installed on the test vehicle to record environmental information, which helps to extract target data during data processing.
To improve the reliability of the test results, 105 experienced drivers participated in the test, including 89 male drivers and 16 female drivers. All participants provided written informed consent. The average age of the participants was 41, and their average mileage was 390,000 km. The participants were not involved in any serious traffic accidents or traffic violations in the past 3 years. The statistics indicated that these participants had excellent driving skills, extensive driving experience, and strong safety awareness. In the course of recruiting participants, the drivers’ vehicles were investigated, and some applicants who drove the same vehicle or a similar vehicle as the test car did not participate in the test to avoid the effects of driver familiarity with the test vehicle.
2.1.2. Test Route
To test vehicle acceleration performance under different conditions, a 10-km-long closed loop route with both an urban road and a highway was selected as the test route shown in Figure 1. The urban road includes several signal-controlled intersections, which were used to test the starting acceleration, and a sloped section, which was used to test the climbing acceleration. The highway was used to test the driving acceleration. The road surface of the test route is smooth, clean, and dry asphalt pavement. The test vehicle was able to maintain a free driving condition because of the low traffic volume on this route, reducing the impacts of the external environment on the test vehicle. Figure 2 is the test process diagram.
2.1.3. Test Procedure
During the vehicle test, the participants were required to drive along the selected route and to perform assigned tasks on corresponding sections. Before the formal test, each participant was informed of the driving tasks they would conduct during the test and was asked to freely drive on the road near the starting point for 5 min to become familiar with the test vehicle. In addition, the participants were asked to notice the acceleration performance during the assigned tasks so that they could complete the subsequent questionnaire.
During the formal test, participants drove through an intersection, a sloped section (gradient of approximately 2.5°), and a highway section in turn, and they were required to complete the following three tasks: starting acceleration (0-30 km/h), climbing acceleration (from the bottom to the top of the slope), and highway driving acceleration (30-100 km/h). Although no interventions were imposed on the test vehicle or participant during the tasks, certain details still required the participants’ attention. First, the participants were directed to stop in front of the stop lines on the premise of safety when arriving at an intersection with a red signal; otherwise, they were instructed to stop behind the preceding car, leaving a distance of approximately one car. This requirement was intended to keep the preceding car from impacting the test car during the starting acceleration performance test. To ensure consistent test distance, the participants were directed to stop the car when they arrived at the starting point of the climbing test; then, they were directed to start the car and to naturally accelerate to the top of the slope. The participants were instructed to start the driving acceleration test when the speed slowed to 30 km/h in the merging area of the highway and were again reminded to freely drive when the speed reached 100 km/h for the first time.
The test time lasted from November 2 to November 5, 2016. Throughout the test, the weather was clear, and the ambient relative humidity was no greater than 75%. The maximum temperature was approximately 283.15 K, and the temperature fluctuations were minimal.
2.2. Questionnaire Survey
After the vehicle test was completed, each participant filled out the subjective evaluation questionnaire based on the driving experience. This questionnaire asked questions related to the participants’ subjective feelings regarding the starting acceleration, climbing acceleration, and highway driving acceleration. The subjective evaluation questionnaire included three components. The first component covered basic information about the participant, including gender, age, driving experience, driving type, and total mileage. The second component consisted of individual subjective evaluation items intended to investigate the participants’ feelings toward each type of acceleration performance. The third component addressed the overall subjective evaluation item to investigate the participants’ feelings toward the overall acceleration performance. The second and third components contained 6 items in total, and the specific contents of each item are shown in Table 1.
2.3. Test Indices
Li Bing proposed a set of dynamic performance evaluation indices that include acceleration jitter, acceleration pitch, starting acceleration performance, and climbing performance [5]. However, it is difficult to collect objective data for certain indices, such as acceleration jitter and acceleration pitch, in real vehicle tests. In contrast, it is easy to obtain objective data for the starting acceleration performance, climbing performance, and driving acceleration performance in real vehicle tests. Therefore, these three indices were selected for evaluating the automotive dynamic performance.
The acceleration time of each test was obtained according to the collected data. As a result, this study used acceleration time for the objective evaluation data. The objective evaluation data for the starting acceleration test, climbing acceleration test, and driving acceleration test corresponded to the acceleration time from 0 to 30 km/h at the start of the intersection, the acceleration time from the bottom to the top of the slope, and the acceleration time from 30 to 100 km/h on the highway, respectively. The three groups of objective evaluation data were denoted as , , and , respectively.
The questionnaire was used to obtain the subjective evaluation data for the dynamic performance. As shown in Table 1, the starting acceleration performance and climbing acceleration performance each included two items. The mean of item 1 and item 2 was taken as the subjective evaluation for the starting acceleration performance and was denoted as . Similarly, the mean of item 4 and item 5 was taken as the subjective evaluation for the driving acceleration performance and was denoted as . Item 3 was considered the subjective evaluation for the climbing acceleration performance. In addition, the subjective evaluation of the overall acceleration performance was denoted as .
3. Results
3.1. Descriptive Statistics
Before analysis, outliers were eliminated based on Pauta criterion which is used to eliminate outliers in a large sample that follows a normal distribution [19]. After processing, 95 sets of effective data remained.
3.1.1. Individual Subjective Evaluations
The subjective evaluation results of the selected indices are shown in Figure 3, where the X-axis represents the five evaluation ratings of each item and the Y-axis represents the number of participants corresponding to each evaluation rating. Figure 3 shows that more than half of the participants thought that the driving acceleration of the test vehicle was moderately fast or fast, approximately 43% of the participants deemed that the starting acceleration of the test vehicle was moderately fast or fast, and only 21% of the participants stated that the climbing acceleration of the test vehicle was moderately fast or fast. Overall, the driving acceleration performance of the test vehicle was better than the starting acceleration performance, whereas the climbing acceleration performance was worse than both. Figure 3 illustrates that the subjective evaluation results of the three indices have certain differences, which need to be verified.
3.1.2. Acceleration Time
As shown in Table 2, in the starting acceleration test, the maximum and minimum durations for the test vehicle to accelerate from 0 to 30 km/h were 15.5 s and 6.5 s, respectively, and the average time was 10.2 s. The maximum and minimum durations for the test vehicle to climb from the bottom of the slope to the top were 29.5 s and 18.5 s, respectively, and the average time was 23.5 s. The maximum and minimum durations for the test vehicle to accelerate from 30 to 100 km/h were 70.5 s and 20.5 s, respectively, and the average time was 38.1 s.
To compare the subjective evaluation results with the objective acceleration time data, the continuous acceleration time data were divided into five groups because the subjective evaluation results had five ratings. The grouping method is described as follows. Assuming that , denote the acceleration time of index of participant ) and that , then the acceleration time data of all participants for index can be divided into five groups according to , i.e., , , , , and . These groups were labeled group 5, group 4, group 3, group 2, and group 1, respectively. The group with larger sequence numbers had shorter acceleration time. In other words, better dynamic performance is indicated by a larger sequence number. The number of participants in each group is shown in Figure 4.
As shown in Figure 4, in the starting acceleration test, the acceleration time of relatively few participants fell within the ranges of 13.7-15.5s (corresponding to Group 1) and 11.9-13.7s (corresponding to Group 2) and only 17% of the participants accelerated from 0 to 30 km/h with no less than 11.9s. In the climbing acceleration test, the acceleration time for the participants was primarily distributed in Group 3 (between 22.9s and 25.1s) and Group 4 (between 20.7s and 22.9s), and the proportion of participants in Group 3 and Group 4 was 58%. In the driving acceleration test, the acceleration time for the participants was mainly distributed in Group 4 (between 30.5s and 40.5s) and Group 5 (between 20.5s and 30.5s), and 57% of the participants accelerated from 30 km/h to 100 km/h in less than 40.5 s.
3.1.3. Comparison between Two Groups of Data
The subjective evaluation results used 5 ratings: slow, moderately slow, moderate, moderately fast, and fast. Therefore, the acceleration time data were divided into 5 corresponding groups: Groups 1, 2, 3, 4, and 5. Comparing the subjective evaluation results with the acceleration time data for the starting acceleration performance and driving acceleration performance shows that an approximately equal number of participants had fast or moderately fast evaluation results and had acceleration time that was distributed in Groups 4 and 5. As shown in Figures 3 and 4, a total of 41 participants considered the starting acceleration performance to be better than average (evaluation results of moderately fast or fast), and 43 participants had a starting acceleration time that was distributed in Groups 4 and 5. A total of 50 participants considered the driving acceleration performance to be better than average, and 54 participants had a driving acceleration time that was distributed in Groups 4 and 5. However, for the climbing acceleration performance, the number of participants showed larger differences in certain evaluation ratings and corresponding groupings. The above analysis indicated that certain relationships might exist between the subjective evaluation results and the acceleration time data; thus, further research should be conducted.
3.2. Correlation Analysis
To further explore the relationships between the subjective and objective evaluation methods, the correlation analysis method was adopted to determine the relationships between each subjective evaluation and its corresponding acceleration time as well as between the overall subjective evaluation and three individual subjective evaluations; this method has also been adopted to explore the relation between the objective and subjective workload assessment measures [20]. The correlation analysis method takes the Pearson Product Moment Correlation Coefficient (PPMCC) as statistical index and this index has values between -1 and 1 and is often denoted as r. The formula for calculating the PPMCC is as follows:
where is subjective evaluation of the th participant; is average subjective evaluation of all participants; is acceleration time of the th participant; is average acceleration time of all participants.
3.2.1. Individual Subjective Evaluations and Acceleration Time
In this study, the starting acceleration performance, climbing acceleration performance, and driving acceleration performance were selected to measure the automotive dynamic performance. Each index contains two groups of data: one group contains the acceleration time data, and the other group consists of the subjective evaluation data. Although the two groups of data used different evaluation methods, they both measured the automotive dynamic performance. To explore whether or not the two types of evaluation results have a relationship, the individual subjective evaluation and the corresponding objective metric of the three indices were used in a correlation analysis. The analysis results are shown in Table 3, where and denote the subjective evaluation data and objective metric data, respectively.
Table 3 illustrates that each individual evaluation result was significantly negatively correlated with the corresponding acceleration time data. In other words, the evaluation score was lower when more time was spent on each test. This relationship is in agreement with the actual situation; thus, the evaluation results of the subjective method and objective method are consistent with each other.
As shown in Table 3, the subjective evaluation result for the driving acceleration performance had the strongest correlation with the corresponding acceleration time data (r=-0.678), and the correlation coefficient of the starting acceleration performance was similar to that of the climbing acceleration performance (r=-0.561 and r=-0.543, respectively). The two evaluation results had the strongest correlation for the driving acceleration performance, possibly due to the test conditions. Unlike the other two indices, the driving acceleration performance was tested on the highway, which had better conditions than the other roads and few disturbances; these improved conditions might improve the validity of the test. The correlation coefficient of the two evaluation results for the climbing acceleration performance was lower than that for the other two indices, possibly due to the geometry of the sloped road section. The sloped road section had a dip angle that might hinder the participants from perceiving the acceleration performance. Therefore, a closed, flat test route should be selected for performance tests to avoid the external environment (such as road line type) affecting test participant perceptions.
3.2.2. Individual Subjective Evaluations and the Overall Evaluation
The overall subjective evaluation investigated the participants’ feelings toward the overall acceleration performance, whereas the individual subjective evaluations investigated the participants’ feelings toward each type of acceleration performance. To explore the relationships between the two groups of data, the corresponding data were subjected to correlation analysis to explore the impacts of the individual subjective evaluations on the overall subjective evaluation. The analysis results are shown in Table 4, where denotes the individual subjective evaluation data and denotes the overall subjective evaluation data.
Table 4 shows that the individual subjective evaluation items were significantly positively correlated with the overall subjective evaluation items. In other words, the scores of the overall evaluations were associated with the scores of the individual evaluation. However, the three individual subjective evaluation items have different impacts on the overall subjective evaluation. The subjective evaluation for the climbing acceleration performance had the largest effect on the overall subjective evaluation (r=0.473), followed by that of the driving acceleration performance (r=0.319) and then that of the starting acceleration performance (r=0.221).
As noted above, the individual subjective evaluations for climbing acceleration and driving acceleration had greater effects on the overall evaluation than that of the starting acceleration. This observation could be caused by the high demands that were made on the dynamic performance during climbing acceleration and driving acceleration; thus, these two types of conditions can more directly reflect the dynamic performance. As a result, the climbing acceleration and driving acceleration had larger effects on the overall evaluation of the dynamic performance. This result showed that car manufacturers should focus on the dynamic performance of climbing acceleration and high gear acceleration to improve driver satisfaction.
3.3. Model of the Overall Subjective Evaluation
The overall subjective evaluation of vehicle dynamic performance is an important index and one that vehicle manufacturers focus on. If the acceleration time in different situations could reflect the overall subjective evaluation of dynamic performance, then the overall subjective evaluation results can be predicted through these objective metrics to avoid the effects of individual differences of drivers on the overall evaluation results.
As noted above, the relationships between the individual subjective evaluations and objective metrics showed that the acceleration time can affect the corresponding subjective evaluation. In addition, the results indicated that the individual subjective evaluations can affect the overall subjective evaluations of drivers. As a result, the acceleration time in different situations may affect the overall subjective evaluation of the dynamic performance.
To explore the relationships between the three acceleration times and the overall subjective evaluation, a four-dimensional scatter plot was drawn using the test data. In Figure 5, the X-, Y-, and Z-axes represent the acceleration times of the starting test, climbing test, and driving test, respectively, and the five colors correspond to the five ratings of the overall subjective evaluation. This figure shows that the overall subjective evaluation nonmonotonically decreased with increasing climbing acceleration time and that the acceleration times of the other two tests also affected the overall subjective evaluation. The overall subjective evaluation decreased with increasing driving acceleration time when the acceleration time of the climbing test was approached, and this trend was more significant when the starting acceleration time was longer. In summary, a nonlinear relationship existed between the overall subjective evaluation and the acceleration times under the three different conditions. Thus, further research must be conducted to explore this complex relationship.
3.3.1. PNN Model Summary
As shown in Figure 5, the relationships between the overall subjective evaluation and the acceleration times under different situations were nonlinear; thus, methods used for exploring nonlinear relationships between variables should be adopted. A previous study that explored the relationships between subjective and objective evaluations of vehicle handling showed that artificial neural networks (ANNs) are a powerful tool for exploring nonlinear relationships between subjective evaluations and objective metrics [21]. Therefore, this paper adopted ANNs to explore the relationships between subjective evaluations and objective metrics of dynamic performance.
ANNs include a variety of algorithms, and each algorithm has different features and application ranges. Because the output variable is a categorical variable and the values of the input and output variables in the training data set were both obtained, the Probabilistic Neural Networks (PNN) were adopted. The structure of PNN is as shown in Figure 6.
PNN is a forward neural network that was proposed by Specht in 1989 [22]. Parzen proposed treating the Gauss function as the basis function to form the estimation method of joint probability density and Bayesian optimization rule. Based on this proposal, Specht constructed a neural network of the classification estimation of probability density and parallel processing. As a result, the PNN has not only the characteristics of general neural networks but also excellent generalization ability and fast learning ability. As shown in Figure 6, the PNN structure consists of an input layer, a hidden layer, and an output layer. The input layer is responsible for introducing characteristic vectors into the networks, and the number of neurons in this layer is equal to the dimensions of the input eigenvector. The hidden layer is connected to the input layer through connection weights; this layer calculated the matching degree between the input characteristic vectors and each pattern in the training set and delivered the distance of these two groups of data to the Gauss function to obtain the outputs of this layer. The number of neurons in the hidden layer is equal to the number of samples. The output layer is responsible for connecting hidden layer units of each category and outputting the class with the highest score; the number of neurons in this layer is equal to the number of sample categories.
3.3.2. Model Establishment
In this paper, the PNN was used to establish an overall subjective evaluation model, and the following steps were taken.
First, the input variables and output variables were determined.
In this paper, the input variables for the neural network were the acceleration time data from the three tests, and the output variable was the participants’ overall subjective evaluation, with values of 1-5 corresponding to bad, moderately bad, moderate, moderately good, and good, respectively.
Second, the PNN model was trained. The training process of PNN is as follows:
(1) Determine the center of radial basis function of neurons in hidden layer.
Supposing and denote the input matrix and the output matrix of the training set, and are as follows:
In the matrix and , the variable represents the th input variable of theth training sample; the variable represents the th output variable of theth training sample; R represents dimensions of input variables and the K represents dimensions of output variables corresponding to the number of the classification; Q represents the number of sample size of training set.
In the PNN algorithm, each neuron in the hidden layer corresponds to a training sample. In other words, the radial basis function center of Q neurons in hidden layer is as follows:
(2) Determine the threshold of the hidden layer neurons.
Assume that the threshold of neurons in hidden layer is , and , denotes the expansion speed of radial basis function.
(3) Determine the weights between the hidden layer and the output layer.
When the center and the threshold of radial basis function of the hidden layer neurons are determined, the output of the hidden layer neurons can be calculated by the following formula:
where is the th training sample vector.
The connection weight W between the hidden layer and the output layer in PNN algorithm is taken as the output matrix of the training set:
(4) Calculate the output of output layer neurons.
When the connection weights between the hidden layer and the output layer neurons are determined, the output of the output layer neurons can be calculated as follows:
The training and optimization programs were developed using MATLAB. All the effective data were randomly divided into two groups: one group included 75 samples that were used to train the neural network and the other group included 20 samples that were used to verify the prediction effect of the model.
Finally, the validity of the model was verified.
The prediction error rate was determined by comparing the true values with the values predicted by the established model.
3.3.3. Model Verification
The prediction values and actual values for 20 groups of test samples are shown in Figure 7.
As shown in Figure 7, among all the testing samples, only two prediction values (Nos. 4 and 6) were not in agreement with the actual values, and the remaining values were in absolutely agreement with the actual values, meaning that the prediction accuracy was 90%. In addition, for Nos. 4 and 6, the prediction values were only one rating higher than the actual values. In other words, little difference was noted between the actual values and the incorrect prediction values. In summary, the prediction accuracy of the overall evaluation model was relatively high. Although the correct prediction rate failed to reach 100%, the incorrect prediction values were relatively close to the actual values. Therefore, according to the acceleration time of the three indices, the participants’ overall subjective evaluations can be predicted using the established PNN model, and the subjective evaluation results can be used to improve the dynamic performance of vehicles to meet the diverse demands of users.
4. Conclusions
Evaluating the dynamic performance is an important task in vehicle production, and subjective evaluation and objective evaluation are two commonly used methods for this task. However, no relevant research has explored the relationships between the subjective evaluation and objective evaluation so that the validity of two evaluation methods cannot be assessed.
In this study, first, a real vehicle test was conducted to collect vehicle acceleration data under different conditions, and the participants’ subjective feelings of the dynamic performance were investigated using a questionnaire. Second, based on the collected data, the correlation analysis method was adopted to explore the relationships between each individual subjective evaluation item and the corresponding objective metric as well as between the overall subjective evaluation item and the three individual subjective evaluation items. The results showed that individual subjective evaluations were significantly correlated with the corresponding objective metrics, and the correlation coefficients were greater than 0.5 (the correlation coefficients between the subjective evaluations and objective metrics for the starting acceleration performance, climbing acceleration performance, and driving acceleration performance were 0.561, 0.543, and 0.678, respectively). The climbing acceleration performance had the largest effect on the overall subjective evaluation, and the correlation coefficient of the two groups of data was 0.473. Finally, an overall subjective evaluation model related to the three objective metrics was established based on the PNN, and the verification results showed that the prediction accuracy of the proposed model was relatively high.
This study explored the relationships between subjective evaluations and objective evaluations of dynamic performance based on selected indices. This research furthered the theoretical knowledge of the methods for evaluating vehicle dynamic performance and offered methodological guidance for exploring the relationships between subjective evaluations and objective indices. In addition, an overall subjective evaluation model of dynamic performance was established based on real vehicle test data; this model used the acceleration time data obtained under different situations as input variables. Furthermore, because this model predicted the overall subjective evaluation through objective test data, it can avoid the effects of subjectivity on the overall evaluation of dynamic performance. As a result, the proposed model had the advantages of high practicability and reliability and can be used by vehicle manufacturers to optimize vehicle designs. However, in the current study, the acceleration times under different conditions were identified as the objective indices. Although these indices can strongly reflect the acceleration performance of the vehicle, they cannot completely reflect the drivability of the car. Therefore, in future research, the evaluation indices, particularly the objective indices that correspond to the subjective evaluations, should be further perfected so that the relationships between the two methods could be explored in depth to improve the reliability of the evaluation results and to further improve the competitiveness of vehicles in the market.
Data Availability
The data in this research was obtained through experiment.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This paper is supported by the “National Natural Science Foundation of Anhui Province” (no. 1708085ME125) and the “Scientific Research Projects Foundation in Higher Education of Anhui Educational Department” (no. KJ2017A485).