Abstract
This paper presents an approach for weighting indices in the comprehensive evaluation. In accordance with the principle that the entire difference of various evaluation objects is to be maximally differentiated, an adjusted weighting coefficient is introduced. Based on the idea of maximizing the difference between the adjusted evaluation scores of each evaluation object and their mean, an objective programming model is established with more obvious differentiation between evaluation scores and the combined weight coefficient determined, thereby avoiding contradictory and less distinguishable evaluation results of single weighting methods. The proposed model is demonstrated using 2,044 observations. The empirical results show that the combined weighting method has the least misjudgment probability, as well as the least error probability, when compared with four single weighting methods, namely, G1, G2, variation coefficient, and deviation methods.
1. Introduction
In the process of comprehensive evaluation, a complete set of evaluation indices must be determined, with the corresponding weight of each evaluation index. The index weight reflects the relative importance of each index, that is, the status and function of each factor in the evaluation and decision-making process. Hence, the determination of index weight relates to the reliability and validity of ranking results of the project. For example, an enterprise credit risk evaluation is related to its financial factors, nonfinancial factors, and macro factors. In terms of large enterprises, financial condition can reflect the repayment ability and willingness to reimburse well; for this reason, there must be substantial weights for the financial factors in a large enterprise credit risk evaluation. When the financial system of small enterprises is not standard and sound, financial information cannot fully reflect their operating condition, and nonfinancial and macro factors have greater influence. As a result, if we cannot allocate the weights of financial, nonfinancial, and macro factors, the irrational phenomenon that small enterprises with poor credit might have a high ranking can result in losses to the bank. Therefore, weight determination methods have been a major focus in comprehensive evaluation research.
Weighting methods for comprehensive evaluation can be divided into three categories: subjective, objective, and combined. A subjective weighting method is a means whereby decision makers obtain index weights on the basis of their own experience and the emphasis that they subjectively place on each index. For this reason, the index weight obtained using subjective weighting depends on experts’ knowledge, experience, and personal preferences, with no consideration for the characteristics and regularities of actual sample data [1]. Subjective weighting models include, for example, the analytic hierarchy process (AHP) [2, 3], analytic network process (ANP) [4, 5], and Delphi method [6]. In contrast, an objective weighting method is a means whereby the index weight is determined using objective information regarding each index. Hence, the index weight obtained using this method does not involve subjective opinions of decision makers. Objective weighting models include, for example, the mean weight [7], goal programming [8, 9], entropy [10] standard deviation (SD) [11], criteria importance through intercriteria correlation (CRITIC) [12], and preference selection index (PSI) [13] methods.
Though an objective weighting method provides weights according to the actual sample data, it is easily subject to the influence of sample data. Moreover, different objective weighting methods tend to yield different results. Therefore, many scholars have proposed combined weighting methods that consider the advantages and disadvantages of both subjective and objective weighting. A combined weighting model has the advantage of integrating the benefits from both subjective and objective weighting models and mitigating limitations of single weighting models [14, 15]. Sun and Bao integrated the rough set weighting method and AHP model based on variance maximization theory to improve the traditional combination weights determination method and made the weights determined more reasonably and more helpful in supporting decision making [16]. Li and Chi proposed a combined weighting model that integrates the evaluation values from single evaluation methods by maximizing the deviation of single evaluation values [17].
Although existing research has made great progress, there are still drawbacks. First, there is usually randomness in the method selection for the determination of objective and subjective weights. Meanwhile, the method for the coefficient of objective and subjective weights is not appropriate. Second, a reasonable test is lacking to verify the rationality of a combined weighting method.
This paper proposes a combined weighting method based on difference maximization. Different evaluation scores for the same object are obtained using different weighting methods. According to the principle that the entire difference of different evaluation objects must be maximally differentiated, an adjusted weighting coefficient is introduced. Based on the idea of maximizing the difference between the adjusted evaluation scores of each evaluation object and their mean, an objective programming model is established with more obvious differentiation between evaluation scores guaranteed and the combined weight coefficient determined. At the same time, with 2,044 small private businesses from a Chinese government-owned commercial bank as empirical samples, a higher differentiation accuracy of the combined weighting method compared with four kinds of single weighting methods, namely, G1, G2, variation coefficient, and deviation methods, is ensured by using misjudgment probability (MP) and error probability (EP) to test evaluation results from the use of different weighting methods.
The rest of the paper is structured as follows. Section 2 introduces the preliminary models. Section 3 discusses the proposed model. Section 4 presents the data and the empirical results. Conclusions are given in Section 5.
2. Preliminary Models
2.1. Data Standardization
The purpose of index data standardization is to transform the index data into to eliminate the inconsistency of units and dimensions.
There are four types of indexes: positive, negative, interval, and qualitative. Positive indices are those for which greater values are better, such as “ Net income,” negative indices are those for which smaller values are better, such as “ Liquidity ratio,” and interval indices are those for which the values are reasonable only when they lie in particular intervals, such as “ Age.”
Let denote the standard score of the th sample on the th index. Let denote the original data of the th sample on the th index. is the total number of indices and is the total number of samples. The standardization equations of the positive and negative indices are shown as (1) and (2), respectively [18]. Consider the following:
Equation (1) is the ratio of the deviation between the index original data and the minimum value to the range . It indicates that the closer the index of the original data is to the maximum value , the greater the standardized value will be.
The two equations have similar meanings. Equation (2) indicates that the closer the index of the original data is to the minimum value , the greater the standardized value will be.
Let and be the left and right boundaries of the ideal interval, respectively. The standardization of the interval factors is shown as follows [18]. Equation (3) indicates that an index value will be given one point if it falls in the optimal range . Greater deviation from this interval leads to lower :
For all qualitative indices, we establish a grading standard suitable for small private businesses. Note that the data of qualitative indices fall in after standardization, indicating that we need not normalize them through (1)–(3).
2.2. Single Weighting Models
2.2.1. Subjective Weighting Model: G1
G1 reflects the importance of indices by their sequential order. After the order is determined, rational weights are assigned based on comparing the adjacent indices. Therefore, the relative importance of any two adjacent indices is certain, assuring that the importance corresponds to the weight. The steps of G1 are as follows.
Step 1. Experts specify the sequence order of indices.
Step 2. Define as the G1 weight of the th index and as the total number of index. Experts specify the rational ratio of indices next to each other, that is, specifying the relative ratio of importance between adjacent index and [19]:
Step 3. Based on the rational ratio of (4), the weight of the th index in the entire index system is calculated using (5), as follows [19]. also denotes the total number of index:
Step 4. Equation (4) can be transformed into (6). Based on weight of (5), the index’s weights are calculated using the following:
Note that ; hence the normalization of is not necessary. is proved as follows.
The symbolic meanings in (7)–(10) are the same as in (4)–(6).
Equation (6) can be iterated to obtain the following:
Equation (7) is substituted into (6) to obtain the following:
Similarly, iteration continues until can be expressed by , shown as follows:
In (9), each can be expressed by , with the result that
As a result of (5), (10) clearly equals one; hence .
2.2.2. Subjective Weighting Model: G2
The essential role of G2 is to reflect index importance by index order; a more important index corresponds to a greater weight. We first specify index order by expert experience and find the least important index labeling . Then, experts specify the relative ratio of importance between and every other index . Let be the G2 weight of the th index, the relative ratio of importance between all other indices and , and the total number of index, with calculated using the following [20]:
Equation (11) indicates that a greater weight comes with a greater relative ratio .
2.2.3. Objective Weighting Model: Variation Coefficient
Let be the variation coefficient weight of the th index, the variation coefficient of the th index, the standard deviation of the th index, the total number of index, and the mean value of all samples of the th index. Thus, the variation coefficient weight of the th index is calculated using the following [21]:
Then, , , where is the standard score of the th sample on the th index, and is the total number of samples.
Equations (12) and (13) indicate that a greater variation coefficient corresponds to a greater distribution variation in comprehensive evaluation. Furthermore, the information discrimination capacity of the index is stronger and its weight is greater.
2.2.4. Objective Weighting Model: Deviation
Let , , , , and be the deviation weight of the th index, the standard score of the th sample on the th index, the standard score of the th sample on the th index, the total number of indices, and the total number of samples, respectively. Then, the deviation weight of the th index is calculated using the following [22]: in (14) reflects the difference of any two samples. Greater indicates greater difference between any two samples; hence the information discrimination capacity of the index is stronger, and its weight is greater.
3. Proposed Model
3.1. Standardization of Multiple Evaluation Models
Different single weighting models lead to varying evaluation results. Standardization is necessary to establish the comparability of weighting models. Let , , and be the score of the th object of the th weighting model, the weight of the th index of the th weighting model, and the standard score of the th index and the th evaluation object, respectively. Let be the total number of samples and the total number of weighting models. The symbolic meanings of and in (16)–(19) are the same as in (15). Then, the score of the th model and the th evaluation object is shown as
Let be the standardized score of the th weighting model and the th evaluation object. Then, the standardized process is shown aswhere is the mean of all evaluation scores of the th weighting model and is the standard deviation of the evaluation scores of the th weighting model. For convenience, the normalized evaluation result matrix is denoted as .
3.2. Combined Weighting Model to Maximize the Evaluation Score Difference
The column vectors of the standardized matrix have positive indices. We introduce an adjusted weight coefficient to reflect the entire difference of various evaluation objects most fully. The adjusted evaluation score is shown as
Based on the principle of reflecting the entire difference of different evaluation objects most fully, the variance of the evaluation scores adjusted by (17) must be as large as possible. The adjusted variance of the evaluation scores is shown as Among these, is the adjusted variance of the evaluation scores, while is the th adjusted evaluation score, and is the mean of the adjusted evaluation scores. are standardized values. Combined with the right side of (18) , this leads to the following:
Matrix is the covariance matrix. According to the principle of reflecting the entire difference of different evaluation objects to the maximum, solving for transforms the problem into a programming problem, as shown in
The economic significance of (17)–(20) lies in the introduction of the adjusted weighting coefficient to satisfy the principle of reflecting the entire difference of different evaluation objects to be the maximum. An objective programming model is established to calculate the combined weight coefficients of the indices based on the idea of maximizing the difference between the adjusted evaluation scores of each evaluation object and their mean.
According to [23], the optimum solution of (20) is the characteristic vector corresponding to the largest characteristic root of the covariance matrix . The normalized characteristic vector can lead to weight vector . The calculation process is to calculate the characteristic roots of equation to obtain the greatest characteristic root , to establish equation , and to solve this equation to obtain the characteristic vector corresponding to the greater characteristic root of the covariance matrix . Adjusted weight vector is obtained by normalizing the characteristic vector. Note that this paper lets have the value of four as a result of the fact that there are four single weighting models in Section 2.2. Let , , , , and be the combined, G1, G2, variance coefficient, and deviation weights, respectively. Then, the combined weight can be expressed as
3.3. The Accuracy Test
The accuracy of the evaluation results for each weighting model is tested using the receiver operating characteristic (ROC) curve. ROC requires two indices, MP (misjudgment probability), the probability of rejecting a good sample as a bad sample by misjudgment, and EP (error probability), the probability of accepting a bad sample as a good sample by misjudgment. In the case of measuring a credit evaluation model, there are default samples and nondefault samples in the sample set; hence each weighting model can discriminate a sample as either default or nondefault. Greater MP and EP indicate that the model has less discrimination accuracy, while lower MP and EP indicate that the model has greater discrimination accuracy.
4. Empirical Study
4.1. Samples and Data Source
Based on available indices from a Chinese government-owned commercial bank, this paper selects 21 indices of small private businesses, including six feature layers, “ Basic information,” “ Guarantee and joint guarantee,” “ Capacity of repayment,” “ Capacity of profitability,” “ Capacity of operation,” and “ Macro environment,” as shown in Table 1.
Data are collected from a Chinese government-owned commercial bank that deals with 2,044 small private businesses from 30 provinces [24]. The 2,044 small private businesses consist of 348 default small private businesses and 1,696 nondefault small private businesses. Moreover, we can compute that the default small private businesses account for only 17.03% of the total sample, with the nondefault small private businesses accounting for 82.97% of the total sample. The samples are divided into two groups, experimental and test. Each group has 1,022 small private businesses, consisting of 174 default and 848 nondefault samples.
According to the type in Column 4 of Table 1, substitute the original data of positive indices , negative indices , and interval indices into (1), (2), and (3), respectively, to obtain the standard scores of indices . The scoring standard of qualitative indices can be obtained using rational analysis, as shown in Columns 2 through 6 of Table 2. According to the index type in Column 4 of Table 1, the standard scores of qualitative indices can be obtained based on the scoring criteria of the qualitative indices in Table 2.
4.2. Calculation of Five Types of Weight
4.2.1. Calculation of G1 Weight
Through interviews with many customer managers from a Chinese government-owned commercial bank, the order rational ratio between two adjacent indices was determined, as shown in Column 4 of Table 3. Substituting the order rational ratio from Column 4 of Table 3 into (5), the G1 weight of the last index “ Industry cycle index” is obtained, as shown in the last row of Column 5 of Table 3. Through (6), the other 20 indices’ G1 weights are obtained, as shown in Column 5 of Table 3.
4.2.2. Calculation of G2 Weight
Through interviews with many customer managers from a Chinese government-owned commercial bank, we can determine the least important index of 21 indices “ GDP growth rate” and, at the same time, the important degree ratio of the other 20 indices to the index “ GDP growth rate,” as shown in Column 6 of Table 3. Then, substituting the importance degree ratio into (11), the G2 weight is obtained, as shown in Column 7 of Table 3.
4.2.3. Calculation of Variation Coefficient Weight
The mean and the standard deviation of each index can be obtained from the standardized score of all samples of each index. The variation coefficient of each index , shown in Column 8 of Table 3, is obtained by substituting and into (13). Then, thevariation coefficient weight , shown in Column 9 of Table 3, is obtained by substituting the variation coefficient into (12).
4.2.4. Calculation of Deviation Weight
The deviation weights, shown in Table 3, Column 10, are obtained by substituting the standardized score of all samples of each index into (14).
4.2.5. Calculation of Combined Weight
The evaluation scores of the four kinds of single weighting methods are obtained by substituting the G1 weight from Column 5, G2 weight from Column 7, variation coefficient weight from Column 9, deviation weight from Column 10 of Table 3, and the standard score of all samples of each index into (15) separately. The standard matrix is obtained by putting the evaluation scores of four kinds of single weighting methods into (16). Then, we calculate covariance matrix . The maximum eigenvalue of is , and the corresponding eigenvector is (−0.3682, −0.4068, −0.6332, −0.5458). Last, we obtain the weight coefficients , , , and from eigenvector normalization.
The combined weight , shown in Table 3, Column 11, is obtained by substituting , , , , G1 weight from Column 5, G2 weight from Column 7, variation coefficient weight from Column 9, and deviation weight from Column 10 of Table 3 into (21).
Results of the five kinds of weighting method are shown in Figure 1. The results of combining weights demonstrate that the weight of “ Operating income” is greatest, exceeding 0.1. The weights of “ Strength of the guarantor” and “ Number of labor force” are 0.0769 and 0.0701, ranking number 2 and number 3, respectively. These three indices have a more important influence on the credit evaluation of small private businesses. In contrast, the three indices “ Educational background,” “ Gender of guarantor,” and “ Liquidity ratio” have weights smaller than 0.03, and they have less important influence on credit evaluation of small private businesses.
4.3. Accuracy Test of the Five Kinds of Weighting Method
To test the accuracy of the five kinds of weighting method, we used the remaining 1,022 small private businesses as the test sample. There are 174 default samples and 848 nondefault samples. The results of MP and EP can be obtained using the ROC curve. The accuracy of weighting results can be estimated by comparing the MP and EP of the different kinds of weighting method. In this way, we can evaluate the advantages and disadvantages of the different weighting methods as well. The results of MP and EP based on the different kinds of weighting method are shown in Figure 2.
Figure 2 demonstrates that of the five kinds of weighting method, the combined weighting method has the smallest misjudgment probability as well as the smallest error probability, with G2 having the highest MP and EP. The objective weighting models, variation coefficient and deviation, have smaller MP and EP than the subjective weighting models G1 and G2.
5. Conclusions
In a comprehensive evaluation, the index weight reflects the relative importance of each index. In other words, it reflects the status and function of each factor in the evaluation and decision-making process. Hence, the determination of index weight relates to the reliability and validity of ranking results of the project. It is for this reason that weight determination methods have been a major focus in comprehensive evaluation research. To this end, many mathematical models have been explored as decision support methods.
This paper proposed a combined weighting method based on difference maximization. Different evaluation scores for the same object are obtained using different weighting methods. An adjusted weighting coefficient was introduced in accordance with the principle that the entire difference of different evaluation objects is to be maximally differentiated. An objective programming model was established with more obvious differentiation between evaluation scores guaranteed and the combined weight coefficient determined. Our model is based on the idea of maximizing the difference between the adjusted evaluation scores of each evaluation object and their mean. The proposed model is demonstrated using 2,044 observations. The empirical results show that the combined weighting method has the smallest misjudgment probability, as well as the smallest error probability, when compared with four kinds of single weighting methods, G1, G2, variation coefficient, and deviation methods.
The research contribution in theory was as follows. An adjusted weighting coefficient was introduced in accordance with the principle of reflecting the entire difference of different evaluation objects to the maximum. An objective programming model was established to calculate the combined weight coefficients of the indices, based on the idea of maximizing the difference between the adjusted evaluation scores of each evaluation object and their mean. Therefore, our model avoids the contradictory and less distinguishable evaluation results that are typical in single weighting methods.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of the paper.
Acknowledgments
This research is supported by the National Natural Science Foundation of China (nos. 71171031, 7147027, and 71503199), Banking Information Technology Risk Management Project of China Banking Regulatory Commission (CBRC) (no. 2012-4-005), Science and Technology Research Project of Ministry of Education of China (no. 2011-10), the Bank of Dalian as credit rating and loan pricing systems for Small business (no. 2012-01), and Credit Risks Evaluation and Loan Pricing For Petty Loan Funded for the Head Office of Post Savings Bank of China (no. 2009-07). The authors thank the organizations mentioned above.