Abstract

CNC machining centers, as the key device in modern manufacturing industry, are complicated electrohydraulic products. The reliability is the most important index of CNC machining centers. However, simple life distributions hardly reflect the true law of complex system reliability with many kinds of failure mechanisms. Due to Weibull model’s versatility and relative simplicity and artificial neural networks’ (ANNs) high capability of approximating, they are widely used in reliability engineering and elsewhere. Considering the advantages of these two models, this paper defined a novel model: Weibull neural network (WNN). WNN inherits the hierarchical structure from ANNs which include three layers, namely, input layer, hidden layer, and output layer. Based on more than 3000 h field test data of CNC machining centers, WNN has been successfully applied in comprehensive operation data analysis. The results show that WNN has good approximation ability and generalization performance in reliability assessment of CNC machining centers.

1. Introduction

Common life distributions, like normal distribution, lognormal distribution, and Weibull distribution, usually are simple for system reliability modeling [1, 2]. However, CNC machining centers are complex repairable systems in which reliability distribution could not be responded precisely by these simple life distribution models. Mixture distribution has been used popularly during the development process of modern statistics. The application of mixture distribution could trace back to the late 19th century, while Weibull mixture distribution started in 1950s [35]. At present, the most common Weibull mixture distribution is twofold Weibull distribution [6, 7]. Multifold Weibull mixture distribution has been seldom used so far. There are two reasons for this: it is hard to estimate large number of parameters, and its bad generalization performance makes it difficult to avoid overfitting.

With the rapid development of computer technology, artificial neural networks (ANNs), as machine learning model with powerful nonlinear approximation ability, have been developed and get wide applications [810]. It is often used to deal with the nonlinear relationship between input and output of complex system [11]. However, ANNs easily bring overfitting phenomenon which is a hot topic and attracts many researchers [12, 13]. Improving the generalization performance of artificial neural networks is a key point to solve overfitting problem.

In this paper, Weibull neural network (WNN) is defined based on some advantages of Weibull mixture distribution and artificial neural networks. In this network, hierarchical structure of radical-basis function network (RBF), which has simple structure and powerful nonlinear approximation performance [14, 15], is adopted. RBF was proposed by Moody and Darken [16, 17] with three layers, namely, input layer, hidden layer, and output layer. The input layer is a series of source nodes that connects the networks to reliability data of CNC machining centers. The hidden layer applies a finite Weibull mixture distribution model connecting the input layer and the output layer. The output layer is the probability density of the data. Finite Weibull mixture distribution [7] is applied as hidden layer nodes function (HLNF). Wide application and multiple distribution curve shape are the main characteristics of finite Weibull mixture distribution which suits not only the life distributions of electronic products, but also the life distributions of mechanical parts.

This paper will be focused on WNN’s two key issues: to develop an efficient learning method and to improve the generalization performance of WNN. And the rest of the paper is organized as follows: a definition of Weibull neural network (WNN) is given in Section 2 with the introduction of basic characteristics of artificial neural networks and Weibull mixture distribution. Section 3 presents the learning process of Weibull neural network (WNN). Section 4 offers field test data of CNC machining centers and applies them into comprehensive simulations and reliability assessment by WNN. For comparison, authors also analyze the data by two-parameter Weibull distribution (TPWD). Finally, conclusions are given in Section 5.

2. Weibull Neural Network

2.1. Artificial Neural Networks

Artificial neural networks (ANNs) [18] are the abstraction and simulation of certain basic characteristics of biological neural networks. As complex nonlinear approximation mathematical models, ANNs rely on the complexity of network structure by adjusting the internal connections between nodes and then achieve the purpose of training and learn any complex nonlinear relationship with strong robustness and fault tolerance [19].

Hierarchical structure is the most common structure of ANNs as BP neural network [20], RBF neural network [14, 15], and so on. Hierarchical structure of ANNs can be divided into several layers by function, such as the input layer, an intermediate layer (also called hidden layer), and the output layer. Each layer is connected in order, as shown in Figure 1. The input layer is responsible for receiving input information from the outside and transfers the information to the neurons of hidden layer. A neutron is an information processing unit which is the fundamental of neural networks. Neuron model constituted different transformation functions with various information processing abilities. The hidden layer is internal information processing layer of neural network, responsible for information conversion. According to required information processing capacity, the hidden layer may be designed as one or more layers. The final one, output layer, supplies the response of neural network to the activation pattern (signal) which is applied to the input layer.

Under the external stimulus of input samples, neural network continuously changes connection weights of network as well as topology structure, so that the output of network is close to desired output. The above process is called learning progress of neural network. In this progress, adjustments and changes of connection weights need to follow certain rules called learning rule.

2.2. Weibull Mixture Distribution

Weibull distribution, including two-parameter type and three-parameter type, is the most common life distribution, which is originally proposed by Swedish physicist Waloddi Weibull, for studying the life of components [21]. In practical application, Weibull distribution is often used as the basic model of more complex distributions, such as Weibull mixture distribution [7], Weibull competing risk distribution [22], Weibull parallel distribution, and Weibull segmentation distribution [23]. Among them, Weibull mixture distribution is the most widely used.

In many cases, a sample population may be composed of two or more subsamples. Because of the difference of design methods, raw materials, manufacturing processes, and other aspects of reasons, products may follow different life distributions in different conditions. If the sample population is composed of subsamples, corresponding subsample cumulative distribution function is expressed as , respectively, probability density is , and corresponding mixture weight of subsample is indicated as , respectively; then the cumulative mixture distribution function of sample population is shown below:

Corresponding probability density function for the mixture distribution is shown below:

The general form of Weibull mixture distribution is (2), which is called -fold Weibull mixture distribution. -fold refers to any distinct subsamples. Mixture weights should satisfy the equation: and .

If is two-parameter Weibull cumulative distribution with shape parameter and scale parameter or three-parameter Weibull cumulative distribution with shape parameter , scale parameter , and positional parameter , two-parameter Weibull cumulative distribution is shown below:

Three-parameter Weibull cumulative distribution is shown below:

The most important feature of mixture Weibull distributions is the diversity in shape. Taking twofold mixture Weibull distribution with two parameters, for example, there are four basic types of density functions shape, as shown in Figure 2. Therefore, it is significant to research on the nonlinear approximation performance of mixture Weibull distribution.

2.3. Weibull Neural Network Model

In practical application, the mixture Weibull distributions have two significant limitations: difficult to precisely estimate parameters and hard to choose a suitable folds number. However, the folds affect generalization performance of mixture Weibull distributions seriously. Low folds lead to underfitting; however, high folds result in overfitting. To improve the generalization performance of mixture Weibull distributions, Weibull neural network is proposed in this paper. Weibull neural network is a kind of mixture distribution but is different from traditional mixture Weibull distributions in structure and learning process.

The hierarchical structure of Weibull neural network is identical to radical-basis function network (RBF), including the three layers shown in Figure 3. The input layer is made up of source nodes that connect the network to reliability data. The hidden layer connects the input layer and the hidden layer and finite Weibull mixture distribution is used as hidden layer function in the network. The output layer is the probability density of the data. The connection between input layer and hidden layer is the probability, namely, that each input data and the hidden layer node are connected in a certain probability. There is a linear weighted connection between the hidden layer and output layer.

In the hierarchical structure of Weibull neural network (WNN), the input of network is expressed as , where is the number of input nodes. In this paper, the hidden layer nodes function (HLNF) is finite Weibull mixture distribution expressed as . The output of Weibull neutral network (WNN) is expressed as , where is the number of output data. Connection probability between input layer and hidden layer is expressed as which means the probability of the th data sampled by the th node. The process is random sampling with replacement, so the value of connection probability is . The connection weight between hidden layer and output layer is expressed as which means the weight of the th node to the th output . According to the characteristic of mixture distribution, needs to satisfywhere is the number of the hidden layer nodes. As every node has the same weight, .

As shown in Figure 3, the input layer achieves nonlinear mapping from input data to hidden layer nodes function (HLNF) , while the output layer achieves linear mapping from hidden layer nodes function (HLNF) to output data . The mathematical model is shown below:

According to the finite Weibull mixture distribution shown in (7), the hidden layer nodes function (HLNF) is calculated by (8). Considerwhere is the folds number of mixture Weibull distributions, .

3. Parameter Estimation of Weibull Neural Network (WNN)

As shown above, in order to define the hidden layer, we need to find the hidden layer nodes function (HLNF) and make sure of the number of hidden layer nodes. Then Weibull neural network (WNN) can be finally proposed. This process is called parameter estimation of WNN which could be divided into three steps. They are parameter estimation of HLNF, selection of HLNF, and determining the number of hidden layer nodes. The following is the detailed process.

3.1. Parameter Estimation of HLNF

Expectation maximization (EM) algorithm is widely used in mixture distribution to estimate parameters [24]. Because of the independence of learning process of neuron function, EM algorithm can be used to estimate parameters of neuron function separately [25]. EM algorithm is an iterative algorithm based on maximum likelihood estimation, each iteration is divided into two steps, namely, expectation step (E-step) and maximization step (M-step). E-step is to calculate expectations of likelihood function and estimate parameters; M-step is to maximize the expectations. According to input data, the core idea of EM algorithm is to estimate parameters by the iterations of expectations. The whole EM algorithm steps are as follows.

Step 1. Initialize parameters: means , variances , and mixture weights .

Step 2. E-step: calculate responsivity according to

Step 3. M-step: calculate weight , mean , and variance according to

Step 4. Repeat Steps 2 and 3 until the convergence of maximum likelihood function value is realized.

In Step 3, , , , and meet (11) in EM algorithm. However the following is a transcendental equation which is hard to calculate analytical solutions:

In each iteration of EM algorithm, let . When , then the following exists:

According to (12), there is a monotonic relationship between and . Therefore, RBF interpolation can be used to establish the mapping relationship between and . Then can be calculated based on . Then, according to the following, can be easily gotten:

3.2. Selection of HLNF

Selection of HLNF has a great impact on the generalization performance of neutral network and the efficiency of learning process. Single Weibull distribution as neuron function may lead to underfitting problem. What is more, multifolds Weibull mixture distribution as neuron function may lead to overfitting phenomenon. It is necessary to design an algorithm to select the most appropriate HLNF. Based on the idea of random sampling, this research chooses the value of maximum likelihood function as the index to select the most appropriate finite Weibull mixture distribution as hidden layer nodes function (HLNF) by multiple sampling. This selection algorithm is as follows.

Step 1. Based on the original sample data , use the bootstrap methodology [26] to generate groups of training samples and groups of testing samples .

Step 2. For the groups of training samples , use the EM algorithm to estimate parameters of finite Weibull mixture distributions which are from 1 to folds, separately getting mixture weight , shape parameter , and scale parameter , ,  ,  .

Step 3. According to the parameters of training samples in Step 2, maximum likelihood function values of groups testing samples can be calculated by (14), where is obtained by formulas (6) (7), and (8). Maximum likelihood function values form an evaluation matrix whose size is :

Step 4. According to (15), mean evaluation value of groups testing samples corresponding to -folds Weibull mixture distribution can be gotten, and select the maximum evaluation value corresponding to -folds Weibull mixture distribution. Then -folds Weibull mixture distribution would be the most appropriate finite Weibull mixture distribution which is the hidden layer nodes function (HLNF) for the original sample data :

3.3. Determining the Number of Hidden Layer Nodes

The number of nodes in the hidden layer is associated with not only the function between input and output, but also sample size, random noise, and so forth. Generally small amount of nodes causes poor recognition performance and fitting performance. However large numbers of nodes easily result in random noise and poor recognition performance. Therefore, choosing an appropriate nodes in hidden layer is critical for improving generalization performance of network.

Sampling study is helpful to weaken the influence of random noise. Therefore, as the number of nodes in hidden layer dynamically increases, the shape of density function of sample data tends to be stable in the learning process of WNN. For the learning process, similarity coefficient is defined to determine the stopping condition.

When the number of nodes in hidden layer is and , define the density function as and , respectively. Then the similarity coefficient (SC) between and is defined as

As is shown in Figure 4, the ratio between intersection and union of coverage area by and is the similarity coefficient (SC). The interval of theoretical value of similarity coefficient is . The larger SC means the higher similarity between and . When SC = 1, it means that two density functions are identical.

By combining HLNF, bootstrap algorithm, and EM algorithm, the specific algorithm to determine the hidden layer nodes is given as follows.

Step 1. Initialize which is the stopping condition of the algorithm.

Step 2. Using Bootstrap algorithm to generate a group of initial training sample data , estimate parameters of HLNF corresponding to sample data and ; set .

Step 3. Using Bootstrap algorithm to generate th group of training sample data , estimate parameters of HLNF corresponding to sample data , and . Then the WNN model can be gotten: where is the probability density function of th hidden layer node.

Step 4. According to (16), similarity coefficient between and layers WNN can be gotten to judge whether it meets the stopping condition: . If does not satisfy the condition, go to Step 3; if satisfies the condition, end the training.

Finally the WNN is defined. And the reliability assessment process is shown in Figure 5. In the process, at the 3rd step, when parameters of WNN are gotten, the probability density on WNN is also gotten.

4. Data Collection and Analysis

In order to validate the WNN model, we collected 23 CNC machining centers’ field test data, almost more than 3000 h running time for each. After data preprocessing, the time between failures obtained within the operation time is listed in Table 1.

For the failure data in Table 1, according to Section 3.2, twofold Weibull mixture distribution is selected as neuron function. Similarity coefficient is set to be the stopping condition in learning process of Weibull neural network. According to , the number of hidden layer nodes is calculated as 51. Follow the steps in Section 3.1, estimate the parameters for hidden layer nodes function (HLNF) , and get mixture weights , shape parameter , and scale parameter of 51 nodes that are shown in Table 2. Substitute those three parameters into (6), (7), and (8) to calculate probability density which is the blue line in Figure 6.

Distribution law of time between failures of 23 CNC machining centers is modeled by WNN. For comparison, general two-parameter Weibull distribution (TPWD) is used to analysis the same data. And the probability density curves on two methods are shown in Figure 6. The blue curve is the probability density on WNN and the red one is on TPWD. The probability density function curves of the data are continuous and derivable in this case, presenting good generalization performance which inherits Weibull method.

Different from the only one peak of the probability density curve on TPWD, the blue one has another peak around 1500 h which means the CNC machining center has more probability to be a failure after running about 2500 h. The probability density curve on WNN reveals more accurate information of the distribution law of time between failures than that on TPWD. Two-peak curve by WNN approximates actual condition. In other words, WNN has better approximation ability than TPWD in distribution modeling of life data.

Mean time between failures (MTBF) describes the expected time between two failures for a repairable system [27]. MTBF is the major index of the reliability of CNC machining centers; MTBF based on point estimation and MTBF based on interval estimation are the most famous methods for MTBF. To deeply estimate WNN and TPWD, these two MTBF are calculated based on the above two methods. The two MTBF of WNN are calculated by the method in [2830]. The comparison results are shown in Table 3.

The results show that MTBF based on point estimation under WNN is larger than that under TPWD by 14.2864 h. And MTBF based on interval estimation, under the same confidence level, are much different: the lower limit of WNN is lower than that of TPBD, and the upper limit of WNN is higher than that of TPBD by 16.192 h. What is more, the confidence interval of MTBF based on interval estimation of WNN is wider than the result of TPBD by over 23.5 h equivalent to 10.58% of TPBD. The big differences of MTBF based on interval estimation and point estimation further show that WNN and TPBD reveal the different distribution laws of time between failures of CNC machining centers. And there is a big error for TPBD in reliability assessment of CNC machining centers. Because WNN has better approximation ability than TPWD in distribution modeling of life data of CNC machining centers, combined with the above comparison, WNN is more close to the actual distribution law of time between failures.

5. Conclusions

The basic idea discussed in this paper is the study on applications of Weibull neural network for complex system reliability assessment. General reliability model easily results in overfitting or underfitting in reliability modeling process. The poor generalization performance of general reliability model could not reflect the actual life distribution law of reliability data. For the above problem, through analyzing the characteristics of artificial neural networks and Weibull mixture distribution, the authors propose Weibull neural network (WNN) for system reliability modeling. A common structure of neural network, hierarchical structure, is adopted in Weibull neural network. And three-step learning process of Weibull neural network is proposed in this paper. In the learning process, using interpolation method to solve transcendental equation significantly improves the computational efficiency of EM algorithm. Finally, a practical application case is presented. WNN is used for analyzing distribution law of time between failures of certain type of CNC machining centers. The probability density function curve of the data is continuous and derivable in the case, presenting good generalization performance. For further comparison, authors introduce two-parameter Weibull distribution (TPWD) to calculate MTBF based on point estimation and MTBF based on interval estimation. The result of the case indicates that Weibull neural network (WNN) has better approximation ability than TPWD in distribution modeling of life data. And Weibull neural network (WNN) could be popularized and applied on reliability assessment for complex systems.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publishing of this paper.

Acknowledgments

Research in this paper was supported by National Science and Technology Major Project of China: reliability promotion of thousands Chinese CNC machining centers (Grant no. 2013ZX04011-012); and Jilin province science and technology development plan: reliability system design for key functional components of CNC machine tools (Grant no. 20130302009GX).