Abstract

This paper proposes a new type fuzzy neural systems, denoted IT2RFNS-A (interval type-2 recurrent fuzzy neural system with asymmetric membership function), for nonlinear systems identification and control. To enhance the performance and approximation ability, the triangular asymmetric fuzzy membership function (AFMF) and TSK-type consequent part are adopted for IT2RFNS-A. The gradient information of the IT2RFNS-A is not easy to obtain due to the asymmetric membership functions and interval valued sets. The corresponding stable learning is derived by simultaneous perturbation stochastic approximation (SPSA) algorithm which guarantees the convergence and stability of the closed-loop systems. Simulation and comparison results for the chaotic system identification and the control of Chua's chaotic circuit are shown to illustrate the feasibility and effectiveness of the proposed method.

1. Introduction

In the past few decades, the fuzzy neural network (FNN) which provides the advantages of both neural network and fuzzy system is successfully applied to nonlinear system identification and control [14]. In the FNN, the symmetric and fixed membership functions (MFs) are commonly adopted to simplify the design procedure. However, a large number of fuzzy rules should be used to achieve the specified performance [5, 6]. Thus, an asymmetric fuzzy membership function (AFMF) has been proposed to solve this problem. The AFMF is discussed and analyzed that it can effectively improve the accuracy and reduce the fuzzy rules [7, 8].

Recently, the type-2 fuzzy sets (T2 FSs) are gaining more and more in popularity [9, 10]. The T2 FSs are described by MFs that are characterized by more parameters than the type-1 fuzzy sets (T1 FSs). Hence, the T2 FSs provide us with more design degree of freedom. Nevertheless, due to the computational complexity of using the T2 FS, most people only the interval type-2 fuzzy sets (IT2 FSs) are proposed [10]. The computations associated with IT2 FSs are very manageable which makes it quite practical [11]. In our previous research [1215], we proposed a type-2 fuzzy neural network with asymmetric membership function (T2FNN-A) which combines interval type-2 fuzzy logic system with neural network. Then, in order to improve the efficiency of the T2FNN-A, we proposed an interval type-2 fuzzy neural system with AFMFs (IT2FNS-A) which utilizes several enhanced techniques such as Takagi-Sugeno-Kang fuzzy logic system (TSK FLS) and embedded type-reduction network layer [16]. These modifications not only improve the approximation accuracy of the T2FNN-A, but also achieve the specific performance with fewer fuzzy rules. However, a major drawback of the IT2FNS-A is that its application domain is limited to static problem due to its feedforward network structure. Thus, using IT2FNS-A to process dynamic problems is inefficient. Many results have been shown that recurrent system can learn and memorize information to provide better performance [2, 1720]. In this paper, we propose an interval type-2 recurrent fuzzy neural system with AFMFs (IT2RFNS-A) which provides the memory elements and extends the basic ability of the IT2FNS-A to include the dynamic problems. In addition, since the feedback layer captures the dynamic response of the system, the approximation accuracy of the network is improved.

In training neural networks, the back-propagation (BP) algorithm is widely used. However, the differential information of system is difficult to obtain due to the piece-wise continuous property of triangular AFMFs and since there are many adjustable parameters in the IT2RFNS-A. Herein, we adopt the simultaneous perturbation stochastic approximation (SPSA) algorithm to derive the update laws of the proposed IT2RFNS-A. The SPSA algorithm is based on the approximation to gradient information from the measurements of the objective function [2123]. Hence, a great deal of computational effort is saved. However, due to the stochastic characteristic of SPSA algorithm, we cannot guarantee that every searching step length is appropriate which may lead to the invalid search. To overcome this situation, we employ the Lyapunov stability analysis to derive the optimal learning step length for guaranteeing the stability of the closed-loop system. In addition, the efficient training is also ensured.

The remainder of this paper is organized as follows. In Section 2, the construction of triangular AFMFs and the SPSA algorithm are introduced. Section 3 illustrates the proposed IT2RFNS-A and the stable SPSA learning algorithm. The simulation results on chaotic system identification and control of the Chua’s chaotic circuit are shown in Section 4. Finally, the conclusion is given.

2. Preliminaries

In this section, we briefly introduce some prerequisite material including the interval type-2 asymmetric fuzzy membership function (IT2 AFMF) and simultaneous perturbation stochastic approximation (SPSA) algorithm.

2.1. Interval Type-2 Asymmetric Fuzzy Membership Function (IT2 AFMF)

The interval type-2 membership function (IT2 MF) is considered to be a special case of a general type-2 membership function (T2 MF) which simplifies the computational effort significantly [10, 11]. In general, the symmetric MF is used for simplification. However, the symmetric MF only holds either uncertain mean or uncertain variance. In addition, tuning the parameter of MFs symmetrically may result in low precision. The AFMF can treat these problems.

In this paper, the triangular fuzzy MFs are used to construct the interval type-2 asymmetric fuzzy membership functions (IT2 AFMFs) due to their lower computational effort. Each IT2 AFMF consists of upper and lower MFs, as shown in Figure 1. The upper MF is defined as where , , and denote the positions of three corners satisfying . Similarly, the lower MF is defined as where , , and denote the positions of three corners satisfying and denotes the magnitude of lower MF which should be limited between 0.5 and 1 to avoid the invalid result (a small firing strength). As above description, the following restrictions should be constrained to avoid unreasonable IT2 AFMFs: , , , , .

2.2. Simultaneous Perturbation Stochastic Approximation (SPSA) Algorithm

This section introduces the SPSA algorithm briefly. The detailed description can be found in literature [23]. Consider the optimization problem with an objective function , the SPSA algorithm updates by where is the estimated gradient result of objective function with respect to W, that is, . denotes the learning step length which is decreased over iterations with , where , , and are positive configuration coefficients [23]. The SPSA approach estimates the gradient, , using the following method. Assume that the dimension of parameter is . Let be a -dimensional vector whose element is mutually independent zero-mean random variable. Then, the estimation of the gradient at kth iteration can be computed by where is gain sequence that is also decreased with , where and are nonnegative configuration coefficients [23]. Obviously, all elements of are perturbed simultaneously and only two measurements of the objective function are needed to estimate the gradient. In addition, is usually obtained using Bernoulli distribution with equal probability for each value.

In general, the gradient information of neural fuzzy system is not easy to obtain due to the piecewise continuous property of AFMFs and large number of adjustable parameters. Herein, we adopt the SPSA algorithm to derive the stable learning for guaranteeing the convergence and stability of the closed-loop systems.

3. Interval Type-2 Recurrent Fuzzy Neural System with Asymmetric Membership Function (IT2RFNS-A)

3.1. Fuzzy Reasoning of IT2RFNS-A

The proposed IT2RFNS-A realizes the fuzzy inference into the network structure. Assume that an IT2RFNS-A system has rules and inputs, the th rule can be expressed as where is input linguistic variable of the th rule, are interval type-2 antecedent fuzzy sets, are consequent interval fuzzy set, is the network input, and is the output of the th rule. Note that the input linguistic variable has the system input term and the past information which is introduced in Section 3.2. As the above description of T2 FLSs, the membership grade is an interval valued set which consists of the lower and upper membership grades, that is, and the consequent part is where denotes the center of and denotes the spread of . Therefore, using the product t-norm, the firing strength associated with the th rule is where and . Thus, the consequent part of the th rule is By applying the Extension principle [24], the output of the FLS is To compute , we need to compute its two end-points and by type-reduction operation. Karnik and Mendel developed an iterative algorithm which is known as KM algorithm to compute these two end-points [24, 25]. A type reducer combines all fired-rule output sets in some way, just like a type-2 defuzzifier combines the type-1 rule output sets, which leads to a T1 FS that is called a type-reduced set. Herein, we need to compute the left-end point and right-end point by utilizing the KM algorithm [24, 25]. In the KM algorithm, the left-end and right-end points are represented as Then, we have to find the proper switch point value and by iterative procedure where more detail can be referred to [24, 25]. However, this iterative procedure for finding switch point values is time wasting. Hence, in the proposed IT2RFNS-A, a simple weight-average method is used to approximate the and so that the iterative procedure is unnecessary. That is, we can calculate the left-most and right-most point of firing strength by where , , , and are adjustable weights. Then, we can obtain the following left-end point and right-end point of the output of interval type-2 fuzzy inference system Note that the above simplified type-reduction technique is adopted in layer 4 (left-most and right-most layer) and feedback layer of IT2RFNS-A system. This simplifies the computational effort in type-reduction procedure. Finally, we defuzzify the type-reduced set to get a crisp output, that is,

3.2. Network Structure of IT2RFNS-A

The proposed IT2RFNS-A consists of six feed-forward layers and a feedback one. Layer 1 accepts input variables. Layer 2 is used to calculate the IT2 AFMF grade. The feedback layer embedded in Layer 2 is used to store the past information. Layer 3 forms the fuzzy rule base. Layer 4 introduces the simplified type-reduction scheme, called left-most and right-most layer. The TSK-type consequent part is implemented into Layer 5. Layer 6 is the output layer.

Next, we indicate the signal propagation and the operation functions of the node in each layer. For convenience, the multi-input-single-output case is considered here. The schematic diagram of the proposed IT2RFNS-A is shown in Figure 2. In the following description, denotes the ith output node in layer l.

Layer 1: Input Layer
For the ith node of layer 1, the net input and output are represented as where represents the th input to the th node. The nodes in this layer only transmit input to the next layer directly.

Layer 2: Membership Layer
In this layer, each node performs a triangular IT2 AFMF, that is, where the subscript indicates the th term of the th input and is an IT2 AFMF as shown in Figure 1 and (3.2). Note that the output of layer 2 and the feedback weight are interval values. According to the above description and the results of literature [26], the type reduction is embedded in the network. Herein, the output of feedback layer is expressed as where and denote the link weight of the feedback layer. Clearly, the input of this layer contains the memory terms and which store the past information of the network.

Layer 3: Rule Layer
This layer is used for computing firing strength of fuzzy rule. From (3.4), we obtain where and are the lower and upper membership grades. Therefore, the operation function in this layer is

Layer 4: Left-Most & Right-Most layer
Similar to the feedback layer, the type reduction is integrated into the network structure by calculating their left-most and right-most values. That is, the complicated type-reduction method such as KM algorithm can be reduced as where the link weights are , , , and . These link weights are adjusted by the proposed stable SPSA learning algorithm. Note that and .

Layer 5: TSK Layer
From (3.5), the TSK-type consequent part is Then, the output of this layer is

Layer 6: Output Layer
Layer 6 is the output layer which is used to implement the defuzzification operation. Therefore, the crisp output is Obviously, the total design parameters of IT2RFNS-A are , , , , , , λ, , , , , , and . These parameters are adjusted by the proposed stable SPSA learning algorithm which guarantees the convergence of IT2RFNS-A systems.

3.3. Training of IT2RFNS-A by the Stable SPSA Algorithm

Consider the nonlinear control problem, our goal is to generate a proper control sequence such that the system output follows the desired trajectory , where is the discrete-time index. The IT2RFNS-A with the stable SPSA algorithm plays the role of controller for nonlinear plant, the adaptive control scheme is shown in Figure 3. For convenience, we consider the single-output case and the tracking error is defined as Then, we define the objective function (error cost function) as The control objective is to generate the control signal such that the tracking error approaches to zero, that is, to minimize the objective function . From the well-known gradient descent method, the parameter update law can be written as where is the tuning parameters of IT2RFNS-A. Then, Herein, we adopt the SPSA algorithm to reduce the computational complexity. The parameters update laws can be represented as where where denotes the tracking error between the desired output and system output resulted by the IT2RFNS-A with perturbed tuning parameters. Note that (3.25) does not calculate the system sensitivity or gradient functions; this simplifies the computational effort.

Note that, in training neural networks, it may not be possible to update all estimated parameters with a single gradient approximation function (3.25). Thus, we should partition the estimated parameters into several parts, that is, each kind of parameter has its corresponding estimated parameter (e.g., as the estimated link weight of feedback layer). Then, the estimated parameters are updated separately by the SPSA algorithm.

Remark 3.1. As the previous results of [1, 2, 4], the adaptive control laws can be obtained by multiplying system sensitivity , that is, where is the control input produced by the IT2RFNS-A, that is, . Hence, the gradient of the IT2RFNS-A should be calculated. However, an additional identifier (using a FNN or an IT2RFNS-A) should be developed to find the unknown system’s sensitivity (details can be found in [1, 2]). In this way, the approximation accuracy should be guaranteed for training the IT2RFNS-A controller and the computational effort is complex and huge. Therefore, an approximation term of and the optimal learning step length are derived to enhance the efficiency and guarantee the stability, where denotes and for continuous and discrete time case, respectively.

Remark 3.2. Our proposed approach is also valid for the nonlinear system identification. The series-parallel architecture shown in Figure 4 is adopted. Hence, the inputs of IT2RFNS-A are and and the IT2RFNS-A output is the estimated output . The parameters of IT2RFNS-A are tuned by the proposed stable SPSA algorithm. Thus, the parameter update law is where denotes the output of the IT2RFNS-A with perturbed tuning parameters.

3.4. Stability Analysis

In this section, the convergence theorem for selecting appropriate learning step length is introduced. The choice of learning step length is very important for convergence. If a small value is given for the learning step length, then the convergence of the IT2RFNS-A is guaranteed. However, the convergent speed may be slow. On the other hand, if a large value is selected, then the system may be unstable. Hence, we employ the Lyapunov stability approach to have the condition for convergence and find the optimal learning step length for IT2RFNS-A.

Theorem 3.3. Let be the learning step length of tuning parameters for the IT2RFNS-A controller. Consider the nonlinear control problem using IT2RFNS-A (shown in Figure 3), the asymptotic convergence of the closed-loop system is guaranteed if the learning step length is chosen satisfying where is the gradient estimation using SPSA approach. In addition, the faster convergence can be obtained by the following optimal time-varying learning step length

Proof. First, we define the discrete-time Lyapunov function as follows: where represents the tracking error. Then, the change of the Lyapunov function is According to the Lyapunov stability theorem, if the change of the positive definite Lyapunov function, denoted , satisfies the condition , for all , then the asymptotical stability is guaranteed [1, 2, 27]. Hence, our objective is to select the proper learning step length such that , for all . This implies that will converge to zero when approaches to infinity. By [1, 2, 28], the error difference can be represented as where denotes the change of . From (3.22) and (3.24), we obtain Thus, Note that is positive for all , and let . Thus, Recall that the asymptotical stability of the IT2RFNS-A is guaranteed if , for all . Thus, should be positive such that , for all . Therefore, we obtain the stability condition for : The asymptotic stability is guaranteed if is chosen to satisfy (3.36). In addition, we would like to find a condition for that guarantees the fast convergence. From (3.34) and (3.35), we have The minimum of is achieved when . Hence, the time-varying optimal learning step length is

For the system identification problem, a similar convergence theorem can be obtained.

Theorem 3.4. Let be the learning step length of tuning parameters for the IT2RFNS-A. Consider the nonlinear identification problem by series-parallel architecture using IT2RFNS-A (shown in Figure 4), and the parameters update laws are shown in (3.27). The asymptotic convergence of the IT2RFNS-A system is guaranteed if the chosen learning step length is satisfied: where is the gradient estimation of IT2RFNS-A by SPSA approach. In addition, the faster convergence can be obtained by using the following optimal time-varying learning step length:

Proof. Herein, we omitted it due to that the proof of Theorem 3.4 is similar to the proof of Theorem 3.3. Only the express of the estimated gradient function is replaced by .

Remark 3.5. As above description, the SPSA algorithm has the stochastic property in the estimation of gradient functions and due to the random values and . Therefore, the stability conditions (3.28) and (3.39) are important for training. In addition, the time-varying optimal learning step lengths and , shown in (3.29) and (3.40), have the ability of guaranteeing the high-speed convergence. This reduces the effect of random values. Simulation results in Section 4 demonstrate the performance of our approach.

4. Simulation Results

Simulation results and comparisons, including the nonlinear system identification and the control of nonlinear system, are introduced to verify the performance of our proposed approach.

4.1. Example 1: Nonlinear System Identification of Chaotic System

Consider a nonlinear system described as where denotes the input, is the output, and are positive integers, and is the unknown function . Our purpose is to identify the nonlinear system (4.1) using IT2RFNS-A with the proposed stable SPSA algorithm. From literature [2], the chaotic system is described as where and and produces a chaotic strange attractor as shown in Figure 5(a). We first choose 1000 pairs training data randomly from the system over the interval [−1.5 1.5] and the initial point is set to. Then, the IT2RFNS-A is used to approximate the chaotic system using the stable SPSA algorithm with the following coefficients listed in Table 1. After training for 100 epochs, the phase plane of the chaotic system is shown in Figure 5(b). In order to show the superiority of using optimal learning step length, the same simulation is done without using optimal learning step length for comparison. In addition, we also compare it with other neural networks such as fuzzy neural network (FNN) [1], recurrent fuzzy neural network (RFNN) [2], and interval type-2 fuzzy logic system with asymmetric membership function (IT2FNS-A) [16]. For statistical analysis, the learning process is repeated for 10 independent runs. The descending trend of the mean square error (MSE) is shown in Figure 6, solid-black line: IT2RFNS-A with the optimal learning step length ; solid-red line: IT2RFNS-A without the optimal learning step length; dashed line: IT2FNS-A; dotted line: RFNN; dash-dotted line: FNN. The worst, average, and best MSE that each neural network achieved are shown in Table 2. Obviously, the IT2RFNS-A with optimal learning step length achieves the best performance and if we focus on using optimal learning step length or not, then we can find that there is a great improvement. From Figure 6, we can observe that using optimal learning step length results in better performance in convergence. Hence, we accumulate the number of epochs that MSE decreased in 100 epochs and listed in Table 3. The result, evidently, shows that the optimal learning step length can guarantee the efficient training of IT2RFNS-A.

4.2. Nonlinear System Control of Chua’s Chaotic Circuit

Consider an nth-order nonlinear dynamic system in the companion form or controllability canonical from where and are the control input and output of the nonlinear system, and are unknown nonlinear and continuous functions, and is the external disturbance or system uncertainty. Our purpose is to develop an IT2RFNS-A controller to generate the proper control signal such that the system output can follow a given reference trajectory yr.

In this paper, we consider the typical Chua’s chaotic circuit which consists of one inductor, two capacitors, one linear resistor, and one piecewise-linear resistor [29, 30]. It has been shown that this circuit holds very rich nonlinear dynamics such as chaos and bifurcations. The dynamic equation of Chua’s circuit are described as where voltages , and current are state variables, is a constant, and denotes the nonlinear resistor which is a function of voltage across two terminals of . is defined as a cubic function as According to [30], (4.4) can be transferred into the canonical form as where , , and is the external disturbance and is assumed to be a square-wave with amplitude and period . Herein, the objective is to track the reference trajectory . We choose the initial state value as and sampling time as 0.01 second.

For comparison, the same simulations are done by using IT2FNS-A and the case without optimal learning step length. The used coefficients are listed in Table 4. The simulation results of trajectories are shown in Figure 7. Figure 7(a) shows the phase plant trajectory after being controlled; Figures 7(b)7(d) shows the state trajectories , , and , respectively (solid line: actual system output; dashed line: reference trajectory). Figure 8 depicts the control effort of the IT2RFNS-A controller. We can observe that the proposed method is valid to control a nonlinear dynamic system. The comparison results are shown in Table 5 which is obtained by 10 independent runs. From Table 5, we can easily find that the IT2RFNS-A performs better than others. This result shows that the IT2RFNS-A has ability to cope with dynamic problems. Observe the performance of the case with optimal learning step length from Table 5, we again demonstrate that adopting optimal learning step length guarantees the efficient training of IT2RFNS-A.

5. Conclusion

In this paper, we have proposed an interval type-2 recurrent fuzzy neural system with triangular asymmetric membership functions (IT2RFNS-A) and its training scheme using the proposed stable SPSA algorithm. We adopt the Lyapunov theorem to derive the appropriate range of learning step length for SPSA to guarantee the stability of the closed-loop system for nonlinear control and guarantee the convergence of IT2RFNS-A for system identification. In addition, we also obtain the optimal learning step length that ensures the efficient training for IT2RFNS-A. The feasibility and the effectiveness of the proposed method have been demonstrated by two illustration examples. The simulations on the chaotic system identification and the control of the Chua’s chaotic circuit are done, and both of them show the following advantages of proposed approach, (a) the IT2RFNS-A having few rules can achieve the specific performance or even better; (b) the IT2RFNS-A can catch the dynamic response of the system; that is, it is capable of coping with dynamic problems; (c) by using the proposed stable SPSA algorithm, the gradient information is unnecessary. In other words, a great deal of computational effort is saved; (d) for control problems, the proposed method avoids approximating the system sensitivity which leads to inaccuracy; (e) adopting optimal learning step length improves the performance of training IT2RFNS-A.

Acknowledgments

The authors would like to thank the Associate Editor and anonymous reviewers for their insightful comments and valuable suggestions. This work was partially supported by the National Science Council, Taiwan, under contract no: NSC-97-2221-E-155-033-MY3.