Abstract
Nowadays various new technologies such as artificial neural networks, genetic algorithms, and decision trees are used for modelling of credit rating. This paper presents design of credit rating model using a type-2 fuzzy neural networks (FNN). In the paper, the structure of the type-2 FNN is designed and its learning algorithm is derived. The proposed network is constructed on the base of a set of fuzzy rules that includes type-2 fuzzy sets in the antecedent part and a linear function in the consequent part of the rules. A fuzzy clustering algorithm and gradient learning algorithm are implemented for generation of the rules and identification of parameters. Effectiveness of the proposed system is evaluated with the results obtained from the simulation of type-2 FNN based systems and with the comparative simulation results of previous related models.
1. Introduction
Credit rating is a method of measuring the creditworthiness of potential customers by analyzing their historical bank data and is a very important problem in finance. Credit rating shows whether a company has a history of financial stability and responsible credit management. The basic factors affecting credit ratings are payment history, amounts owed, length of credit history, having new credits, and types of credit. Satisfactory results obtained for these factors determine the creditworthiness of the customers.
The basic aim of credit approval is to avoid huge amount of losses that may be associated with any type of inappropriate decision. The design of credit rating models allows reducing the cost of credit analysis, reducing possible risk, and enabling faster credit decision. The solving of these problems will allow us to increase the benefit of bank finance system.
Credit rating is a binary classification problem that classifies credit customers into predefined “good” and “bad” groups based on an observation. Numerous credit rating models have been developed in order to evaluate and classify loan customers to a good applicant group or either to a bad applicant group. The aim of these studies focused on increasing the accuracy rate of credit rating models since even little bit of improvement will result in significant cost savings.
Credit rating models are based on collecting huge amounts of data about credit customers in order to avoid making the wrong decision. Credit rating models are based on the analysis of their related attributes, such as age, marriage status, and income, or on their past records, and so forth. Recently various kinds of credit rating models have been developed and applied to support credit approval decisions. These are traditional models based on statistical analysis such as discriminant analysis, logistic regression, and decision tree [1, 2]. The statistical models can perform well in many applications, but when the relationships of the system are nonlinear, the accuracy of these models decreases. Other models are based on rough sets, neural networks, genetic algorithm, and support vector machine [3–6]. The artificial neural networks (ANN) have the ability of learning nonlinear relationships in a system. Because of dealing with nonlinear patterns, ANN has shown better performance in accuracy in contrast to the traditional statistical methods such as discriminant analysis and logistic regression [3, 7, 8]. In [8, 9] genetic programming (GP) has been used in classification [8, 9]. GP is viewed as a tree-based structure and is employed to build the discriminant function for the credit rating problems [8]. After initializing the tree, the operators of genetic algorithm (GA) such as crossover, mutation, and reproduction are applied for finding of optimal generation. The disadvantages of GP are related to the long training process.
Many times the attributes of the credit customers are characterised by uncertainty and fuzziness of information. For these conditions the previous models all have certain drawbacks and never achieve stability and accuracy simultaneously. The combination of fuzzy systems and support vector machine or neural networks (NNs) successfully has been used for credit rating purposes for increasing the accuracy of the designed model [10–13].
Type-1 fuzzy systems are widely applied to solve different real world problems. However some of the designed systems face high level of uncertainties that can affect the performances of the systems. Because type-1 fuzzy systems use crisp type-1 fuzzy sets in representation of fuzzy rules, they cannot handle high level of uncertainties associated with information or data in the knowledge bases. It has been proven that type-2 fuzzy systems can handle such kind of uncertainties and increase the performance of the system. The uncertainties in type-2 fuzzy sets can arise from different sources. Four types of uncertainties are specified in [14, 15]; uncertainties caused with noisy data, with measurements, with the extracted knowledge from a group of experts which is likely to carry uncertainties, and with the words that are used in fuzzy rules are uncertain. Because the membership functions of type-2 fuzzy systems are themselves fuzzy, they provide a powerful framework to represent and handle such types of uncertainties [14–16].
Type-2 fuzzy sets were introduced by Zadeh and have further been developed by Mendel and his coauthors [14–16]. Interval type-2 fuzzy systems found set of applications, such as for forecasting of time-series [16], robot control [17], prediction of hot strip mill temperature [18], identification, and control nonlinear system [19–21].
In this paper the combination of the type-2 fuzzy systems and neural networks is considered for credit rating. The integrated neural and fuzzy systems have self-learning characteristics and allow reducing the complexity of the data and modeling uncertainty and imprecision. In this paper, neural and fuzzy methodologies are combined to construct type-2 fuzzy neural networks (FNN) and try to model human knowledge on credit rating. Clustering and gradient techniques are used for determining the parameters of type-2 FNN.
The rest of the paper is organized as follows. Section 2 presents the structure of the proposed type-2 FNN. The parameter update rules of type-2 FNN are derived in Section 3. Section 4 presents the simulation studies for credit rating. Finally, Section 5 concludes the paper.
2. Structure of Type-2 Fuzzy Neural System
The fuzzy systems are basically designed using Mamdani or TSK type IF-THEN rules. In some research works, it has been shown that TSK type fuzzy neural systems can achieve a better performance than the Mamdani type fuzzy neural systems in learning accuracy [20]. The designs of Mamdani and TSK type-2 fuzzy systems are presented in [14]. In [14] the type-2 TSK fuzzy rules are classified into three types. In the first model, the membership functions of the antecedent and consequent parts are interval type-2 fuzzy sets. In the second one, membership functions of antecedent part are interval type-2 fuzzy sets; consequent part is type-1 fuzzy sets. In the third model, the membership functions of antecedent part are type-2, and consequent part is crisp numbers. In the paper third type of type-2 TSK rules is considered.
Many research papers use interval type-2 fuzzy sets to reduce complexity of type-2 fuzzy systems. Since the secondary memberships of interval type-2 fuzzy systems are equal to one, the computations associated with interval type-2 fuzzy sets are very manageable [14]. Additionally, in the paper, to decrease the complexity of the fuzzy system, type-2 fuzzy sets are used in the antecedent parts of the rules and linear functions in the consequent part of the rules. Because of such structure the type reduction process is significantly simplified as compared to the ordinary type-2 fuzzy system.
The type-2 TSK fuzzy rules used in this paper have the following form:
Here are the input variables, are the output variables which are linear functions, and are the parameters in the consequent part of rules, and is the type-2 fuzzy membership function for the th rule of the th input which is defined as a Gaussian membership function.
The designed multi-input-single output type-2 fuzzy neural network (FNN) is given in Figure 1. The type-2 FNN uses type-2 TSK fuzzy rules which are given in formula (1). The development of the type-2 FNN includes the determination of the proper values of the unknown coefficients of the antecedent and the consequent parts of each rule. In the paper the Gaussian membership functions are used in the antecedent part of type-2 FNN. If both and parameters of the Gaussian function are considered to be uncertain (within certain intervals), the parameter space of the system can become very large. In this paper, only the means of the membership functions are assumed to be uncertain and the standard deviations (STDs) are fixed as depicted in Figure 2.
In the first layer of Figure 1, the external input signals are distributed. In the second layer each node corresponds to one linguistic term. This layer represents antecedent part of the rule (1). The interval type-2 fuzzy sets are used to represent linguistic terms. Each membership function of the antecedent part is represented by an upper and a lower membership function. They are denoted by and or and . In the nodes of the second layer, for each external input signal , the membership degrees and to which the input value belongs to a fuzzy set are determined. Consider
The inference engine of type-2 FNN uses “min” or “prod” -norms implication operators. In the third layer the -norm prod operator is applied to calculate the firing strengths of each rule using (3): where is t-norm prod operator. The fourth layer represents the consequent part of the rules and computes the outputs of the linear functions. In the fifth layer, the output signals of the third layer are multiplied by the output signals of the linear functions. The sixth and the seventh layers perform the type reduction and the defuzzification operations, respectively. The inference engine of type-2 TSK FNN is given in [14, 22]. The output of the type-2 FNN is computed as where is number of active rules, and are determined using (3), are outputs of linear functions, and is output signal of network which is determined using (4). and are the design parameters that weight the sharing of the lower and the upper firing levels of each fired rule. These parameters can be tuned during the design of the system [20–22]. Here instead of the time-consuming Karnik-Mendel iterative procedure, the design factors and are trained to adaptively adjust the upper and lower values on the left and right end points of outputs, using the parameter update rule. The use of and design factors instead of the use of Karnik-Mendel iterative procedure allows reducing complexity of the type-2 FNN.
3. Parameter Update Rules
After determining the output signal of the type-2 FNN, the training of the network starts. The training includes the adjustment of the parameters of the membership functions , and in the second layer (antecedent parts) and the parameters of the linear functions , in the fourth layer (consequent parts). In the antecedent parts, the input space is divided into a set of fuzzy regions, and in the consequent parts the system behaviour in those regions is described. Different techniques are used for parameter update. Clustering [13, 23–25], least-squares method (LSM) [14, 24], gradient algorithms [14, 20, 21], and genetic algorithms [8, 9] have been used for designing fuzzy IF-THEN rules. In this paper, the fuzzy clustering [21] and gradient technique are applied for the updating of parameters. The aim of clustering methods is to identify a certain group of data from a large data set, such that a concise representation of the behaviour of the system is produced. Each cluster centre can be translated into a fuzzy rule for identifying the class. The fuzzy clustering is applied to design the antecedent parts of the rules, that is, to select the cluster centres of the membership functions in the antecedent part of fuzzy rules using the input data set of the plant.
After the clustering, the gradient descent algorithm is applied to update the parameters of type-2 FNN, basically to design the consequent parts of the fuzzy rules (1). At first, the output error is calculated. Consider Here is number of output signals of the network (in the given case ) and and are the desired and the current output values of the network, respectively. The parameters , , and are adjusted using the following formulas: where are the learning rates, and is the number of input neurons of the network, and is the number of hidden neurons (rules). The values of derivatives in (7) are determined as The derivatives in (10) are determined by the following formulas: Here Upper and lower membership function can be written as follows (see Figure 2): Here is determined as . Then
The parameters of the type-2 FNN can thus be updated using (7)–(9) together with (10)–(14).
As mentioned above, the parameters and in (5) enable adjustment of the lower or the upper portions in the final output. The values of and are optimized from an initial value of 0.5 using
In the following section type-2 FNN structure and its parameter update rules are applied for modelling credit rating. The comparisons of the type-2 FNN model with the models based on support vector machine (SVM), NNs, and type-1 FNN will be made.
4. Simulation Studies
The type-2 FNN structure and its learning algorithms are applied for credit rating and evaluation. For credit evaluation two real world data sets are used. The Japan and Australian credit rating data sets are selected from UCI machine learning data repository. The Japan credit data set is a real world data set that includes 690 data. From this data set, the parameter values of 37 data sets are unknown. From the remaining 653 data, 296 items are classified as creditworthy, and 357 items are classified as credit unworthy items. Data set includes customer credit scoring data with 15 input features, such as age, gender, marital status, credit history records, job, account, loan purpose, other personal information, and one output variable. Nine input variables have symbolic values; six have continuous values. The attributes’ names have been changed to meaningless symbolic data for confidential reasons. Fragment of Japan data set is shown in Table 1. The second Australian credit data set includes 307 creditworthy customers and 383 credit unworthy customers. It contains 14 attributes, where six are continuous attributes and eight are categorical attributes. Both data sets are taken from the UCI Repository of Machine Learning Databases.
To simplify data processing process, the values of symbolic variables are transformed into numeric form. As shown in Table 1 the input values are not normalised. Some input variables having continuous values are changed in very big interval, from zero to 51100. During modelling, the input variables are normalized and scaled in the interval of 0 and 1. The scaling helps the training process of type-2 FNN. Also the input variables having symbolic values are discretised into numeric one. These symbolic values are converted to the numeric values in the interval of 0 and 1. After conversion and normalization, these input data are used as input signal for the type-2 FNN. The data set is divided into two group subsets: training and testing. Training data set includes parameter values of the first 400 items; next 253 data sets are used for testing.
The training input/output data for the prediction system will be a structure whose first component is the fifteen-dimension input vector and second component is the output clusters. The type-2 FNN structure is generated with fifteen input and one output neurons.
The gradient descent and fuzzy -means clustering algorithms are applied for the adjusting of the parameters of type-2 FNN. At first stage the type-2 fuzzy classification is applied to the input space in order to determine the cluster centres. These cluster centres are then used in order to organize the antecedent parts of the fuzzy rules, which are the second layer of type-2 FNN. After designing antecedent parts, the gradient algorithm is applied for the learning of the parameters in the consequent part—fourth layer of type-2 FNN. During learning, the initial values of consequent parameters and are selected randomly in the interval . Using the parameter update rules derived in Section 3, they are updated for the given input signals. The fuzzy rules are constructed using clusters obtained from the classification of input signals. Each cluster will represent the centre of Gaussian membership function used in fuzzy rules. After clustering input space, a gradient descent algorithm is used for the learning of consequent parts of the fuzzy rules, that is, parameters of linear functions. For each obtained cluster the results of simulation are given in the tables.
For comparative analysis, the simulations results of type-2 FNN are compared with simulations results of the credit rating models based on SVM, NNs, and type-1 FNN. The root mean square error (RMSE) is used as performance criteria. Consider
As mentioned above for the designing of the system, the 400 data sets are used for training and the next 253 data sets are used for diagnostic testing. All input data are scaled in the interval . The training is carried out for 1000 epochs. As a result of training, the values of the parameters of the type-2 FNN system were determined. The simulations were performed using six, sixteen, and thirty-two hidden neurons. Once the FNN has been successfully trained, it is then used for the rating of credit. During learning, the value of RMSE for training data was 0.209510. After learning, the RMSE values for test data were 0.229653.
In Figure 3, the plot of RMSE values obtained during training has been shown. The simulation of the type-2 FNN prediction model is performed using clustering and gradient techniques. Table 2 demonstrates simulation results using different number of rule bases: 6, 16, and 32. Simulation results are averaged for ten simulations. To estimate the performance of the type-2 FNN clustering systems, the recognition rates and RMSE values of errors between clusters and current output signal are taken. RMSE is computed using formula of (16). Recognition rate is computed as Recognition_Rate = (number of exact classification results/total number of clusters)*100%. As shown in Table 2, the increase of the number of rules increases training accuracy but decreases testing accuracy. The use of clustering and gradient techniques for learning allows obtaining low RMSE value quickly and allows improving the performance of type-2 FNN for training and testing stages. In the second case for comparative analysis, the classification of credit rating has been performed. The result of the simulation of the type-2 FNN prediction model is also compared with results of simulations of the support vector machine (SVM), neural networks (NNs), and FNN (type-1) based classification models. To estimate the performance of the NN, SVM, FNN, and type-2 FNN clustering systems, the recognition rates and RMSE values of errors between clusters and current output signal are compared. In Table 3, the comparative results of simulations of different models are given. As shown in the table the performance of type-2 FNN classification system is better than the performance of the other models.
In the next stage we design type-2 FNN model using Australian credit data set. The Australian credit data set has 14 input and one output parameters. After scaling input variables, for given input-output pair, the training of type-2 FWNN has been performed. Here 400 data sets are used for training and 300 data sets are used for testing. The number of epochs used for training is 2000. Table 4 demonstrates the simulation results of type-2 FWNN using Australian data set. The simulation results satisfy the efficiency of the application of type-2 FNN in constructing a credit rating model.
5. Conclusion
Designing an effective credit rating model is an important task for saving amount cost and efficient decision making. In the paper a credit rating model is developed by integrating type-2 fuzzy logic and neural networks. The functionality of the fuzzy system is realized by the neural network structure. The design of type-2 FNN model is performed using fuzzy clustering and gradient algorithm. The designed type-2 FNN uses and factors that replace the Karnik-Mendel iterative procedure in inference engine module. The used and factors significantly reduce computational cost of type-2 fuzzy system. The parameters of type-2 FNN are trained using a gradient descent algorithm. Type-2 FNN is applied to the Japan and Australian data sets. Also for comparative study, other technologies, SVM, NN, and FNN, are employed to build the model for the credit rating problem. Simulation results have demonstrated that type-2 FNN is more flexible and performs with better accuracy than other models in the credit rating.
Conflict of Interests
The author declares that there is no conflict of interests regarding the publication of this paper.