Abstract
This paper proposes an enhanced ant colony optimization with dynamic mutation and ad hoc initialization, ACODM-I, for improving the accuracy of Takagi-Sugeno-Kang- (TSK-) type fuzzy systems design. Instead of the generic initialization usually used in most population-based algorithms, ACODM-I proposes an ad hoc application-specific initialization for generating the initial ant solutions to improve the accuracy of fuzzy system design. The generated initial ant solutions are iteratively improved by a new approach incorporating the dynamic mutation into the existing continuous ACO (). The introduced dynamic mutation balances the exploration ability and convergence rate by providing more diverse search directions in the early stage of optimization process. Application examples of two zero-order TSK-type fuzzy systems for dynamic plant tracking control and one first-order TSK-type fuzzy system for the prediction of the chaotic time series have been simulated to validate the proposed algorithm. Performance comparisons with and different advanced algorithms or neural-fuzzy models verify the superiority of the proposed algorithm. The effects on the design accuracy and convergence rate yielded by the proposed initialization and introduced dynamic mutation have also been discussed and verified in the simulations.
1. Introduction
In contrast to the Mamdani-type fuzzy systems having good interpretability, Takagi-Sugeno-Kang- (TSK-) type fuzzy systems usually are employed for the accuracy-oriented applications. Since each rule of a TSK-type fuzzy system has a crisp output and the aggregated system output is computed via weighted average, thus avoiding time-consuming and mathematically intractable defuzzification operation, the TSK-type fuzzy system is a popular candidate for sample-based fuzzy modeling [1]. However, because fuzzy modeling usually requires high precision, the TSK-type fuzzy systems usually are designed or refined by using the optimization techniques to satisfy the requirement.
The optimization of the TSK-type fuzzy system design is to find the antecedent parameters characterizing the fuzzy sets for the inputs and the consequent parameters for the output by minimizing or maximizing the objective function. One of the major categories of solving such optimization problem is the gradient-based search methods, in which the search direction is derived based on the gradient of the objective function with respect to the parameters. To calculate the gradient, the gradient-based method needs the training input-output data pair of the fuzzy system, which, however, usually is a difficulty for fuzzy control problems because the target fuzzy outputs responding to the inputs are not available in advance. Another traditional difficulty for the gradient-based method is that it is easily trapped in a local optimum when it is applied to the optimization problems having multiple peaks in the design space such as the design of fuzzy system.
To avoid the issues encountered in the gradient-based approaches, many population-based computational techniques such as genetic algorithm [2] and swarm intelligence algorithms [3] have been proposed for solving optimization problems. These computational techniques are derivative free, so they can easily be applied to any optimization problems including the fuzzy control problems and the problems with nondifferentiable objective function. Since the search direction in these techniques is stochastic, they are more likely to traverse across the highly nonlinear design space, thus avoiding to be trapped into a local optimum. Furthermore, these approaches evaluate many candidate solutions concurrently, so they have better chance to find the better solution.
Genetic algorithm was inspired by the process of natural selection and developed based on the principle of survival of fittest. Genetic algorithms use bioinspired crossover operation between two selected parent solutions to produce child solutions and the mutation operation for further exploiting the generated individual child solutions for improving the performance. Genetic fuzzy systems [4–8] are the evolutionary fuzzy systems designed by genetic algorithms. For example, GA was used to design neural-fuzzy system for temperature control in [5] and fuzzy controller for mobile robots in [7].
Particle swarm optimization (PSO) is one of swarm intelligence models. PSO was inspired by the social behavior of fish schooling and bird flocking [9]. In the PSO, each particle represents a candidate solution to a problem and flies in the hyperspace according to its flying velocity vector, which is stochastically determined by its previously personal best and swarm best experiences. In the end of PSO operation, the experienced swarm best solution is the finally obtained solution. To address the encountered issues and improve over the parent PSO [9], many advanced PSO variants [10–15] have been proposed, some of which were applied to design the TSK-type fuzzy systems [14, 15]. These algorithms include the PSO with time-varying acceleration coefficients (PSO-TVAC) [10], a self-organizing hierarchical particle swam optimizer with TVAC (HPSO-TVAC) [10], PSO with controllable random-exploration velocity (PSO-CREV) [11], enhanced PSO by incorporating a weighted particle [12], the hybrid of GA and PSO (HGAPSO) [13], two-phase swarm intelligence algorithm (TPSIA) [14], and ant and particle swarm cooperative optimization (APSCO) [15].
Another type of swarm intelligence model is ant colony optimization (ACO) [16–19]. The ACO technique was inspired by foraging behavior of the real ant colony and was proposed initially for solving discrete combinational optimization problems such as travelling salesman problem (TSP). The ACO algorithms have been successfully applied to optimize the fuzzy systems for mobile robot control [20, 21]. In those studies, in order to apply the discrete ACO for optimization, the parameters charactering the fuzzy controller were first discretized, thus sacrificing the precision.
To overcome the precision issue, some ant-related algorithms for the optimization problems with real-value parameters have been proposed [22–24]. Among these algorithms, this paper focuses the interest on [24] because it is most related to the discrete ACO and has achieved good performances on continuous optimization of the benchmark functions as demonstrated in [24]. thus is advantageous on the application problems requiring high precision such as the design of fuzzy controllers for dynamic systems because their inputs and outputs are usually continuous cases. Since the proposal of , some variants have been proposed and applied to design fuzzy systems for accuracy-oriented problems [25–28]. In [25], a modified continuous ACO algorithm (RCACO) was proposed for the design of fuzzy-rule-based systems in order to achieve considerable learning accuracy. The paper [26] proposed a cooperative continuous ACO (CCACO) with multiple colonies of populations, each colony of which is only responsible for optimizing a single fuzzy rule. Although the simulation results demonstrated that the performance of multicolony based CCACO is better than that of the single-colony RCACO, the computation of CCACO is much more complex than that of RCACO. Inspired by the cognitive psychology concepts, the study in [27] proposed an assimilation-accommodation mixed continuous ant colony optimization (ACACO) for the designs of feed-forward fuzzy systems. In addition, the elite-guided continuous ACO (ECACO) was proposed to design recurrent fuzzy systems [28], which was claimed to be the first application of continuous ACO on recurrent fuzzy system design. Although those variants indeed helped accomplish the accurate design of fuzzy systems, there is still possible room for further improvement, especially on the balance between the exploration and convergence rate of solutions. This paper introduces the dynamic mutation [29] into to enhance such balance.
For population-based algorithms such as PSO and ACO, another major issue is population initialization [30–34]. Starting with a population of initial solutions, the population-based algorithm improves the solutions iteratively in order to find the better solution. Therefore, the population initialization affects the performance. In general, however, the initial population solutions are generated randomly because of no a priori information. The work in [31] suggested use of Sobol sequence generator for generating initial PSO particles because the generated initial solutions are uniformly distributed into the search space. The study in [32] reported that the initialization using the nonlinear simplex method (NSM) helped improve the convergence rate and success rate of PSO. By using the generators of centroidal Voronoi tessellations (CVT) as the starting point, as suggested in [33], the initial solutions can be more evenly distributed throughout the high-dimensional problem space and improve PSO performance. The study in [34] proposed a center-based sampling for the initialization of population-based algorithms. By simulations, that paper showed that the points in center region have higher chances to be closer to an unknown solution and thus suggested the center region is a promising region for the initialization. However, those initialization approaches [31–34] did not take into account application-specific information. Based on [34] and the observations of some accuracy-oriented fuzzy controller designs, this paper proposes an ad hoc population initialization method for initial ant solutions to improve the design accuracy.
This paper mainly contributes to propose an enhanced continuous ACO algorithm incorporating dynamic mutations and ad hoc initialization, ACODM-I, for the design of TSK-type fuzzy systems. ACODM-I can be regarded as a population-based evolutionary algorithm. Instead of random generation, the proposed population initialization in ACODM-I takes the observations of well-performed fuzzy systems into account. The introduced dynamic mutation in ACODM-I initially provides more diverse search directions for exploring solutions to avoid being trapped into a local optimum in the early stage. The performance superiority of ACODM-I to the parent and different advanced algorithms or neural-fuzzy models is verified by the simulation results of TSK-type fuzzy systems for the problems of tracking control and chaotic time series predication. Furthermore, the effects on the convergence rate and design accuracy yielded by the proposed initialization and introduced dynamic mutation are verified by the simulation results.
This paper is organized as follows. The next section describes the zero-order and first-order TSK-type fuzzy systems. Section 3 introduces basic concepts of discrete ACO and and proposes ACODM-I for TSK-type fuzzy system design. Section 4 presents the simulation results of TSK-type fuzzy systems designed by ACODM-I for the tracking control of dynamic plant and the prediction of chaotic time series. Section 4 also compares the ACODM-I performance with the ones of with different values of parameter and advanced algorithms or models. Furthermore, the effects of the proposed initialization and dynamic mutation are also discussed and verified in Section 4. Ultimately, conclusion remarks are given in Section 5.
2. TSK-Type Fuzzy System
In contrast to Madami-type model having good interpretability, TSK-type fuzzy systems focus on the model accuracy. The main difference is on the consequent part of fuzzy rules. In a TSK-type fuzzy system, the consequence of fuzzy rule is defined as the linear combination of the fuzzy inputs. The th fuzzy rule of a TSK-type fuzzy system is described as follows:where is the time step, are the input variables, and is the output variable of the fuzzy system. In the antecedent part of fuzzy rule , is a fuzzy set for the input and is characterized by a membership function. If the function in the consequent part is a constant the system is denoted as a zero-order TSK-type fuzzy system. If the function is defined byit is a first-order TSK-type fuzzy system. In this study, as widely used in fuzzy system designs [15–17, 27–30], Gaussian function is selected to characterize the fuzzy set and is defined bywhere and are the center and the width of fuzzy set , respectively. For a Gaussian function, the parameters and are independent variables and its function output corresponding to any input value is never zero, which will make the design task easier.
For a TSK-type fuzzy system consisting of rules, the output of the fuzzy system through the inference engine is calculated by where is the firing strength of rule excited by a given input dataset . Thus, to construct an -rule TSK-type fuzzy system with input variables, all decision variables represented byfor a zero-order TSK-type fuzzy system, or for a first-order TSK-type fuzzy system, are to be determined. However, such design task of a TSK-type fuzzy system can be treated as an optimization problem that finds the free parameters represented in (6) or (7) such that the task-dependent objective function is optimized. Following this transformation, the optimization algorithms can be used to solve such optimization problem for accomplishing the design of TSK-type fuzzy system.
3. Proposed ACODM-I for Fuzzy System Design
This section presents the proposed enhanced continuous ACO with dynamic mutation and ad hoc initialization (ACODM-I) for the design of TSK-type fuzzy systems. Since the proposed ACODM-I is inspired and related to the ACO framework, this section first reviewed the basic concept of the discrete ACO and . The simulation results optimized by the will also be presented as the benchmark for comparison in Section 4.
3.1. Basic Concept of Discrete Ant Colony Optimization
The discrete ACO algorithms [16–19] were inspired and developed by the behavior of real ant colonies and now are largely employed to find the solution of discrete combinational optimization problems (COP). When ants foraged for food, they deposited the pheromone on the trail to guide other ants. Since the pheromone will evaporate with time, the path with a higher pheromone level is the most possibly a shorter one to the food source. Ant System (AS) [16] is the first discrete ACO algorithm and proposed to solve the travelling salesman problem (TSP). In Ant System for TSP, represents the probability that ant k situated at city i at the time t chooses to visit city j and is defined by where is the pheromone trail on the link between cities and , is the corresponding a priori heuristic information of the link, and is the set of allowed neighborhood cities ant can move from city . The relative significance of the pheromone trail and heuristic information is determined by the values of α and β. Each ant is assumed to visit each city only once, and it will construct a feasible solution to TSP after it visited all cities. After all ants accomplished their visits, all feasible solutions are gathered to update the pheromone levels for the next ant cycle. The discrete ACO algorithm repeats such procedure to find the shorter path. Since that proposal, some variants of discrete ACO algorithms were proposed [17, 18]. In addition, by discretizing the parameters in fuzzy systems, the applications of the discrete ACO to optimize the fuzzy controllers for mobile robots have been demonstrated [20, 21].
3.2. Basic Concept of ACOR
Among many proposed continuous ACO algorithms, the in [24] is one of the most promising algorithms and is most related to the original discrete ACO. In [24], has demonstrated good performances for continuous optimization of the benchmark functions. The basic concept of is to extend the discrete probability distributions (8) utilized in discrete ACO to continuous Gaussian probability density functions (PDFs). In , these Gaussian PDFs are derived from a maintained solution archive as shown in Figure 1, and the sampled values from the chosen PDFs are gathered to construct new solutions.
In Figure 1, a feasible solution (ant path) to the optimization problem is represented by a row vector in the archive, and its quality is measured by the value of the predefined objective function. All feasible solutions are sorted, ranked from the best to the worst, and maintained in the fixed-size archive table. By doing this, the solution thus has rank . Then the rank of each ant solution in the archive determines its probability of being chosen to follow in the next ant cycle. The operation of operation is detailed as follows.
usually initializes all N solutions in the archive, each of which is a D-dimensional row vector , by generating random numbers within the range of search space. All initialized solutions then are evaluated, sorted according to the evaluation values, and ranked in the solution archive. Each solution of the rank l in the sorted archive is assigned with a weight :where is a parameter of the . To generate a new candidate solution, the firstly chooses one leading solution among solutions in the archive according to the probability distributionIt indicates clearly from (10) that the better-ranked solution has higher probability being chosen as the leading solution and the value of q in (9) controls the tendency for exploring the archive solutions. Once a leading solution is chosen, a new candidate solution is constructed by sampling the derived Gaussian PDF in a sequence of with the mean and the standard deviation where the pheromone evaporation rate is a positive parameter. A higher value of provides more exploration in search thus converging slower while a lower value of provides more exploitation in search, thus converging faster. By repeating such process times, new candidate solutions are generated. Then these new candidate solutions are evaluated. Together with the original solutions in the previous ant cycle, the total solutions are sorted again. The only reserved the -top-performed solutions for next ant cycle. The repeats such ant cycle until the termination condition is satisfied.
3.3. ACOR Using Dynamic Mutation and Ad Hoc Initialization (ACODM-I)
3.3.1. ACOR with Dynamic Mutation (ACODM)
In , the value of in (9) controls the exploration ability thus affecting convergence rate of the algorithm. When the value of is small, strongly prefers the best-ranked solutions as the leading solution, which focuses on exploiting the best-ranked solutions locally. This will increase the convergence rate, but the chance of convergence result being trapped into the local optimum is also increased. When value is large, the probability for each solution being chosen as the leading solution becomes nearly uniform. This can enhance the exploration ability for possibly obtaining globally better solution, but the convergence rate will be decreased.
In order to avoid or lessen this issue, this paper introduces mutation technique into the original to balance the exploration ability and the convergence rate. In addition to the Gaussian sampling technique, the introduced mutation provides another option for changing (generating) a newly constructed solution component by “jumping” to the neighboring of the other archive solutions. Since the probability for the mutation is not fixed, which will be clearly seen, this modified algorithm is named as with dynamic mutation, ACODM, and its operation is detailed as follows.
In ACODM, without loss of generation, the range of each decision variable to be identified is assumed in the interval . If a leading solution is chosen, the value of the dth component of a new candidate solution is generated.
If ,
else In (12), the denotes the sampled value from a Gaussian PDF with the mean and the standard deviation . The ranges of the uniform random numbers and are in and , respectively. The index in (13) is a uniform random integer number in . Finally, the mutation probability for the th component is set to be linearly proportional to the biased standard deviation among all ants and is calculated bywhere is an adjustable parameter determining the mutation probability and is the average value among all the dth components in the ant population.
In ACODM, in addition to the exploitation using Gaussian sampling in (12), the dynamic mutation in (13) can explore more diverse search solutions to avoid being trapped into a local optima in the early stage of the optimization process. Moreover, the mutation probability for each solution component in ACODM depends on the convergence status of the ant population, so its value is not fixed but dynamic and is not the same for each solution component. In (14), a higher value of h suggests a possibly larger probability for dynamic mutation by directly jumping to the neighboring of the other archive solutions. Therefore, the higher the value of h is, the lower the convergence rate is. Finally, the change of value for the mutation of ACODM, as presented in (13), is dynamic in contrary to the fixed change used in the mutation operation of general genetic algorithms. Equation (13) indicates the points around the center region have larger possible deviation than those points close to the boundary when they are chosen as the neighbors for learning. The initial motivation for this thought is to hope the central points can wander and explore in more diverse search directions in the early stage of optimization process.
The ACODM algorithm repeats the selection of leading solution using (10) followed by the Gaussian sampling in (12) or dynamic mutation (13) to generate L new candidate solutions. These L new candidate solutions are evaluated and sorted together with the N solutions in the previous ant cycle. Similarly, only the N-top-best solutions are reserved for next ant cycle. ACODM repeats such ant cycle to find the better solution until the termination condition is met.
3.3.2. Ad Hoc Population Initialization for Fuzzy System Design
The initialization of the ant solutions in is another issue, which is also a major problem for all other population-based algorithms [30–35]. Starting with a population of initial ant solutions, improves the solutions iteratively to find the better solution. Therefore, the initialization of ant solutions in affects the performance. In general, good initialization can help achieve better optimum while bad initialization usually ends on a poor local optimum. However, because no a priori information is available, in general the initial ant solutions are generated randomly. As mentioned in Section 1, some advanced random initialization approaches over the multiple-dimensional search space were proposed [31–34]. However, those initialization approaches did not take into account application-specific information, and thus they are regarded as generic initialization.
The study in [34] proposed a center-based sampling for the initialization of population-based algorithms. By simulations, that paper showed that the points in in the search space have higher chances to be closer to an unknown solution, and thus the center region is a promising region for the initialization. In addition, in our observation of fuzzy systems, especially accuracy-oriented fuzzy controllers, one of the membership functions for each input in its antecedent part is usually designed at the location around the input value arising more frequently or most concerned, which is usually at the center of the search range. Therefore, this paper proposes an ad hoc central initialization range for initializing the parameters and in the antecedent part of fuzzy rule for the fuzzy system designs. The rest of free parameters in the consequent part of a TSK-type fuzzy system are initially generated uniformly in the search space because of no a priori information.
If ACODM generates the initial ant solutions using the proposed initialization method, the resultant algorithm is denoted by ACODM-I. For clarity, the pseudocode of the ACODM-I algorithm is shown in Algorithm 1.
| ||||||||||||||||||||||||||||||
4. Simulations
Three application examples of the designs of TSK-type fuzzy systems are demonstrated in this section to validate the proposed algorithm. Two zero-order TSK-type fuzzy systems are designed for nonlinear dynamic plant control, and one first-order TSK-type fuzzy system is optimized for the prediction of the chaotic time series. In the simulations, the population size N is 20 and the number of newly generated temporary solutions L is 20. The value of is 0.85 as used in [24, 25]. The value of is set to have the mutation probability of 1.0 when the decision variable in each dimension is uniformly distributed among all ants. The performances of fuzzy controllers and fuzzy predictor optimized by ACODM-I are validated and compared with those by with different values of q. In the simulations for , each initial parameter value in the fuzzy systems was generated randomly and uniformly within its search space. In addition, the resulting ACODM-I performance is also compared with the reported results of some advanced population-based algorithms or neural-fuzzy models on the same problem. The advantage on the optimization accuracy yielded by the proposed initialization and dynamic mutation are also discussed through simulation results. The personal computer for conducting all simulations possesses an Intel Core i5 2.8 GHz dual-core-processor and runs on Windows 7.
Example 1. As the first example, a zero-order TSK fuzzy system is designed to control the nonlinear plant as taken in [14, 25] and described byThe initial state of the system is assumed to zero and is the control input of the plant. The objective for the fuzzy system (controller) to be optimized is to control the plant output to track the reference trajectory as given byIn this example, the fuzzy controller is fed with two signals: the current plant output and the target output . As the response to these two inputs, the produced output of the fuzzy system is used to control the nonlinear plant (15). For a designed fuzzy controller, its performance is evaluated by the root mean square error (RMSE) between the plant output and the reference trajectory over the 250 time steps and is calculated by
The fuzzy controller in this example consists of five fuzzy rules in order to compare with other algorithms later. Then the values of the free parameters , , and for constructing a fuzzy system are searched using ACODM-I such that the error in (17) is minimized. For each single run of optimization process, 10000 evaluations were performed to conclude a solution. Over 50 runs of simulations, the average best-so-far RMSE at each performance evaluation of ACODM-I is shown in Figure 2. The learning results of the parent with different values of are also shown in Figure 2 for comparison. The learned statistical numerical results of RMSE errors are shown in Table 1. The results show that the average RMSE, minimum RMSE, and maximum RMSE of ACODM-I are smaller than the ones of algorithms with different values of .
The previous study [25] reported the performance results of some advance population-based evolutionary algorithms when they were applied to the same fuzzy control problem. These algorithms include a hierarchical PSO-TVAC (HPSO-TVAC) [10], a PSO with controllable random-exploration velocity PSO (PSO-CREV) [11], a hybrid of GA and PSO (HGAPSO) [13], a two-phase swarm intelligence algorithm (TPSIA) using discrete ACO in the first phase and PSO in the second phase [14], and fuzzy-rule-based continuous ant colony optimization (RCACO) [25]. The operations of these algorithms and their settings of parameters used on this control problem were detailed in [14, 25]. The fuzzy system optimized by each of these algorithms was also composed of five rules to ensure the same number of free parameters as that by ACODM-I. Moreover, for each single run, each of these algorithms also performed the same number of evaluations as that by ACODM-I. Table 2 presents the reported performance results of these algorithms. The results show that ACODM-I achieved the smaller average error than all other algorithms in comparison.
The performance results of the fuzzy system optimized by , ACODM, and -I (representing the using the proposed initialization) are also provided in order to discuss or validate the effectiveness of the proposed initialization and dynamic mutation. Table 3 presents these learned statistical numerical results when the value of is 0.1. Simulation results show that ACODM achieved the smaller average error than . The average error of ACODM-I is also smaller than that of the -I. This indicates that the dynamic mutation can improve the optimization accuracy thus validating the effectiveness of the introduced dynamic mutations. The main reason to this is the ability of the dynamic mutation providing more diverse search directions for the ant colony when the value of is not large. Similarly, the effectiveness of the proposed initialization on improving the accuracy of the fuzzy system design is also validated by the observation that the average errors of -I and ACODM-I are smaller than the ones of and ACODM, respectively. Moreover, the learning curves of , ACODM, -I, and ACODM-I are shown in Figure 3. It is also observed that the algorithms incorporating dynamic mutation, ACODM, and ACODM-I converged slower than the ones without incorporating dynamic mutation, , and -I, respectively. The possibly best explanation is that the exploration on more search directions by the dynamic mutation brings the accuracy improvement at the expense of the convergence rate. Finally, ACODM-I achieved the smallest error. The tracking results controlled by ACODM-I optimized fuzzy system are presented in Figure 4. The results show that the controlled plant output is very close to the reference trajectory.
Example 2. The nonlinear plant for control using a zero-order TSK-type fuzzy system as taken in [15, 25] is described byThe initial states and are assumed to , and is the control input of the plant. The objective of the zero-order TSK-type fuzzy controller is to control the plant output to track the reference trajectory , where is governed by the reference modelwhere the initial states of the reference model are assumed. Two previous system outputs and the target output are the three input variables of the fuzzy controller. The produced output of the fuzzy system is to control the nonlinear plant (18). The performance evaluation of a designed controller is defined as the sum of absolute error (SAE) between the plant output and reference trajectory over 250 time steps and is computed by
The fuzzy system to be optimized consists of four fuzzy rules. The values of the free parameters , , and are searched to find a better-performed fuzzy controller yielding smaller SAE. The 60000 evaluations were performed to conclude a solution in each run of optimization process. Over 50 runs of simulations, the average best-so-far SAE at each performance evaluation of each algorithm for fuzzy system design is shown in Figure 5. The learned numerical results of SAE errors are shown in Table 4. The results show that ACODM-I achieved the smaller error than algorithms with different values of q.
ACODM-I performance is also compared to the reported results of HPSO-TVAC, HGAPSO, PSO-CREV, RCACO, and ant and particle swarm cooperative optimization (APSCO) using the parallel combination of ACO and PSO [15] when they were applied to the same fuzzy control problem. The comparison is shown in Table 5. The results show that ACODM-I achieved smaller average error than all other algorithms in comparison.
In this example, when the value of q is 0.1, simulation results of the zero-order TSK-type fuzzy controller optimized by , ACODM, -I, and ACODM-I are shown in Table 6 and Figure 6. In a similar way to Example 1, the effects of the proposed initialization and dynamic mutation on the design accuracy and convergence rate are easily identified in the simulation results. The tracking results of the fuzzy system optimized by ACODM-I are shown in Figure 7. The simulation results show the controlled plant output is very close to the desired reference trajectory.
Example 3. In this example, a first-order TSK-type fuzzy system is used to predict the chaotic time series. The time series to be predicted is generated by the well-known Mackey-Glass chaotic system, which is defined by the following time-delay differential equationwhere is set as 17, the initial point is assumed to be 1.2, and for as in the previous studies [26, 27]. Four past values of , , , and are the inputs of the first-order TSK-type fuzzy system. The produced output of the fuzzy system excited by these four past inputs is the prediction of . The fuzzy predictor is optimized by ACODM-I and evaluated in the following experiment. First, the solution of (21) was solved using Runge-Kutta numerical method. Secondly, the time series data were generated by sampling the solution every second. The sampled data patterns from to 623 were used as the training set for optimizing the fuzzy system while the remaining patterns from to 1123 were used as test set for validating the designed fuzzy model.
The first-order TSK-type fuzzy system consists of four rules. The values of the free parameters , , and are searched to find a better-performed fuzzy predictor. The objective function is defined as the RMSE between the sampled output of the chaotic system governed by (21) and the predicted output by the fuzzy system. The 300000 evaluations were performed in a single run of training. The learning results of the average RMSE at each evaluation over 50 runs are shown in Figure 8. Table 7 shows the learned numerical results of training and testing RMSE errors. The results show that ACODM-I achieved the smaller prediction error than the algorithms with different values of .
Similar to the two previous examples, simulation results of the first-order TSK-type fuzzy systems optimized by , ACODM, -I, and ACODM-I are provided in Table 8 and Figure 9 to verify the effects of the proposed initialization and dynamic mutation. The results indicate that the proposed initialization and dynamic mutation in ACODM-I as applied to the design of first-order TSK-type fuzzy system have the same effects as those used in the zero-order TSK-type fuzzy systems for control problems. It is also observed that the proposed initialization helps improve much more on design accuracy than dynamic mutation. The predication results of the first-order TSK-type fuzzy system optimized by ACODM-I are shown in Figure 10. The results show the prediction values by the fuzzy system are very close to the desired ones.
The performance of fuzzy system designed by ACODM-I is also compared to the reported results of different algorithm and neural-fuzzy systems that were applied to the same prediction problem [28]. These neural-fuzzy systems include a Hybrid Neural-Fuzzy Inference System (HyFIS) [35], a Dynamic Fuzzy Neural Network (D-FNN) [36], a Subsethood-Product Fuzzy Neural Inference System (SuPFuNIS) [37], and a Generalized Fuzzy Neural Network (G-FNN) [38]. The algorithm in comparison is a multiple-colony topology based Cooperative Continuous ACO- (CCACO-) designed neural-fuzzy system [26]. The comparison results are shown in Table 9. The results show the fuzzy predictor optimized by ACODM-I achieved the smaller test error than the most of reported neural-fuzzy systems. Though the ACODM-I performance is very close to the G-FNN and CCACO, ACODM-I optimized fuzzy system uses smaller number of rules than G-FNN. As reported in [26], the average training RMSE of the CCACO is 0.0094 which is larger than 0.0078 of ACODM-I. Moreover, the CCACO algorithm is a multiple-colony algorithm, whose computation is more complex than the simple single-colony algorithm such as ACODM-I.
5. Conclusions
This paper proposes an enhanced ant colony optimization with dynamic mutation and ad hoc initialization, ACODM-I, for improving the accuracy of TSK-type fuzzy systems design. ACODM-I is developed based on and is regarded as a new population-based evolutionary optimization algorithm. Based on the observations of some accuracy-oriented fuzzy controllers, this paper proposes an ad hoc population initialization for initial ant solutions to improve the design accuracy. This is an application-specific initialization rather than generic initialization currently used in most population-based algorithms. ACODM-I incorporates the dynamic mutation technique into to balance exploration ability and convergence rate. In addition to exploiting local search around the chosen leading solution using Gaussian sampling, the dynamic mutation probabilistically provides more search directions by “jumping” to the neighboring of other archive solutions in order to avoid being trapped into a local optimum.
To validate the proposed algorithm, three application examples of the TSK-type fuzzy system designs have been simulated: two zero-order TSK-type fuzzy systems for nonlinear dynamic plant control and one first-order TSK-type fuzzy system for the prediction of the chaotic time series. The simulation results have demonstrated that ACODM-I achieves smaller error than the ones by the algorithms with different values of q. The effects on the design accuracy and convergence rate yielded by the proposed initialization and introduced dynamic mutation have also been discussed and verified in the simulations. In addition, the comparison with some advanced population-based algorithms also shows that the error achieved by ACODM-I is smaller than those by those algorithms used for comparison including HPSO-TVAC, PSO-CREV, HGAPSO, TPSIA, APSCO, and RCACO for the design of zero-order TSK-type fuzzy controller and CCACO for the design of first-order TSK-type fuzzy predictor.
In the future, the performances of ACODM-I for the designs of various types of feed-forward or recurrent fuzzy systems will be studied. The adaptive mechanism or algorithm of adjusting the value of in (14) determining the mutation probability will be investigated in the future study. The hybridization of ACODM-I with other computational techniques may further improve optimization accuracy; thus it is possibly worthy of being studied in the future.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by Ministry of Science and Technology, Taiwan, under Grant MOST 105-2221-E-415-018.