Mathematical Problems in Engineering

Volume 2017 (2017), Article ID 2427309, 11 pages

https://doi.org/10.1155/2017/2427309

## Robust Kernel Clustering Algorithm for Nonlinear System Identification

^{1}National Higher Engineering School of Tunis (ENSIT), University of Tunis, 5 Av. Taha Husein, BP 56, 1008 Tunis, Tunisia^{2}Laboratoire d’Ingenierie des Systemes Industriels et des Energies Renouvelables (LISIER), University of Tunis, ENSIT, Tunis, Tunisia

Correspondence should be addressed to Mohamed Bouzbida; rf.liamtoh@demahom_adibzuob

Received 12 December 2016; Revised 16 March 2017; Accepted 30 March 2017; Published 14 May 2017

Academic Editor: Francisco Gordillo

Copyright © 2017 Mohamed Bouzbida et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

In engineering field, it is necessary to know the model of the real nonlinear systems to ensure its control and supervision; in this context, fuzzy modeling and especially the Takagi-Sugeno fuzzy model has drawn the attention of several researchers in recent decades owing to their potential to approximate nonlinear behavior. To identify the parameters of Takagi-Sugeno fuzzy model several clustering algorithms are developed such as the Fuzzy -Means (FCM) algorithm, Possibilistic -Means (PCM) algorithm, and Possibilistic Fuzzy -Means (PFCM) algorithm. This paper presents a new clustering algorithm for Takagi-Sugeno fuzzy model identification. Our proposed algorithm called Robust Kernel Possibilistic Fuzzy -Means (RKPFCM) algorithm is an extension of the PFCM algorithm based on kernel method, where the Euclidean distance used the robust hyper tangent kernel function. The proposed algorithm can solve the nonlinear separable problems found by FCM, PCM, and PFCM algorithms. Then an optimization method using the Particle Swarm Optimization (PSO) method combined with the RKPFCM algorithm is presented to overcome the convergence to a local minimum of the objective function. Finally, validation results of examples are given to demonstrate the effectiveness, practicality, and robustness of our proposed algorithm in stochastic environment.

#### 1. Introduction

Modeling and identification are significant steps in the design of the control system. Typical applications of these models are the simulation, the prediction, or the control system design. Generally, the modeling process consists of obtaining a parametric model with the same dynamic behavior of the real process. However, when the process is nonlinear and complex, it is very difficult to define the different mathematical or physical laws which describe its behavior [1, 2]. In this context, the modeling of nonlinear systems by the conventional methods is very difficult and occasionally ineffective [3, 4]. So, other nonconventional techniques based on fuzzy logic are used more often in modeling this kind of process due to excellent ability of describing its behavior [4–6].

Among the best fuzzy modeling approaches developed in literature we mention the Takagi-Sugeno fuzzy model. In effect, this model is described by if-then rules. Each rule includes a fuzzy set antecedent and mathematical functions as consequent representing the process behavior in each region [2, 3, 7]. The identification problem consists of estimating the model parameters. In this context, to identify the parameters of Takagi-Sugeno fuzzy model, many techniques were developed such as the Adaptive schemes, heuristic approaches, nearest neighbor clustering, and support vector learning mechanisms. Besides fuzzy clustering algorithms are widely used in fuzzy modeling. Fuzzy -Means (FCM), Gustafson-Kessel (G-K), Gath-Geva (G-G), Possibilistic -Means (PCM), Fuzzy -Regression Model (FCRM), Enhanced Fuzzy -Regression Model (EFCRM), and Possibilistic Fuzzy -Means (PFCM) are popular clustering algorithms used in structure identification part and LS, WRLS, and orthogonal least square (OLS) technique were applied in consequent parameter estimation. Among the clustering algorithms, Fuzzy -Means (FCM) developed by Bezdek is well-known but this algorithm is sensitive to noise or outliers and susceptible to local minima [8, 9]. However, noise in the data sets can make the situation worse by creating many inauthentic minima. These are able to distort the global minimum solution found by FCM algorithm. This flaw has stimulated the researchers to overcome this inconvenience. To fight against the effects of outlying data, various approaches are considered such as the possibilistic clustering (PCM) proposed by Krishnapuram and Keller [10] and fuzzy noise clustering approach of Dave [11]. The possibilistic approach executed a possibilistic partition, in which a membership relation calculates the absolute degree of typicality of a point in a cluster. Although the PCM algorithm is robust against the noise points and allows identifying these outliers, it is very responsive to initializations and occasionally generates coincident clusters. To solve this deficiency of identical clusters, a Possibilistic Fuzzy -Means (PFCM) algorithm was suggested by Wu and Zhou in 2006. Nonetheless, these algorithms are not efficient for unequal dimension clusters and cannot separate clusters that are nonlinearly separable in input space and their limits between two clusters are linear. In our work to overcome this shortcoming, kernel methods [12] are regarded as the way of dealing with this problem. We propose a new clustering algorithm called Robust Kernel Possibilistic Fuzzy -Means (RKPFCM), which adopts a kernel induced metric in the data space to replace the original Euclidean norm metric. By changing the inner product with an appropriate hyper tangent kernel function, one can implicitly affect a nonlinear mapping to a high dimensional feature space where the data is more clearly separable. However, our proposed algorithm RKPFCM has an iterative nature that makes it sensitive to initialization and sensitive to converge to a local minimum. To overcome these problems, several solutions have been proposed in the literature. Among them is combining the clustering algorithm with a heuristic optimization technique. In this context, many researches have proposed the evolutionary computation technique based on Particle Swarm Optimization (PSO). They have been successfully applied to solve various optimization problems [13]. Thus, we introduce PSO into the RKPFCM algorithm to achieve global optimization. The efficacy of our algorithm compared to many algorithms is tested on noisy nonlinear systems defined by recurrent equations and an application to Box Jenkins system. This paper will be presented as follows. The second part of work is reserved for introducing the Takagi-Sugeno fuzzy model. The third part will be devoted to identifying the premise parameters of this model where we used the proposed RKPFCM algorithm and PSO algorithm is introduced. In the fourth part, we will focus on identification of consequent parameters. The simulations results and the model validity of RKPFCM and RKPFCM-PSO are presented in part five. Finally, we conclude this paper.

#### 2. Takagi-Sugeno Fuzzy Model

Takagi-Sugeno fuzzy model (T-S) is one of the best techniques used for modeling a nonlinear system represented by the recurrent equation . T-S model is constructed by a rule-based type “if … then” in which the consequent uses numeric variables rather than linguistic variables (case of Mamdani). The consequent can be expressed by a constant, a polynomial, or differential equation depending on the antecedent variables. The T-S fuzzy model allows approximating the nonlinear system into several locally linear subsystems [4, 17].

In general, a Takagi-Sugeno fuzzy model is based on if then rules of the formThe “if” rule function defines the premise part and “then” rule function constitutes the consequent part of the T-S fuzzy model. : represents the th rule; : is the parameters vector, such as ; : is a scalar; : is observations vector; : represents the fuzzy subsets,where .

Here, the fuzzy sets are represented by the following membership function [7] where are centers and is the width of the membership function.

The estimated output model is defined by the following equation [4]:Asso,

#### 3. Identification Algorithm for Premise Parameters

To identify the premise parameters of a Takagi-Sugeno fuzzy model described by equation (1), we used the Possibilistic Fuzzy -Means (PFCM) algorithm and our proposed algorithms (RKPFCM and RKPFCM-PSO).

##### 3.1. Possibilistic Fuzzy -Means (PFCM) Algorithm

The Possibilistic Fuzzy -Means (PFCM) algorithm, which uses Euclidean distance, finds the partition of the collection of measures, specified by -dimensional vectors , into fuzzy subsets by minimizing the following objective function [18]:where : the number of clusters; : the membership of in cluster satisfying : the typicality of in classes ; : the set of cluster centers (); : the suitable positive numbers described by Typically, is chosen to be 1. are the terminal membership values of FCM. is a weighting degree; this parameter has a significant impact on the form of clusters in data space*.*To minimize equation (6), we take its partial derivative of variables, , and , equal to zero and obtain the following equations:

*PFCM Algorithm Steps.* Given a set of observations .

*Initialization* () Set the number of clusters . Set the level of weighting : . Set the parameters , . Set the stopping criterion *ε*: . Execute a FCM clustering algorithm to find initial fuzzy partition matrix and cluster centers . Initialize the typicality matrix randomly. Compute by (9).

*Repeat for *.

*Step 1. * Compute the cluster centers by (12).

*Step 2. * Compute the membership matrix by (10).

*Step 3. * Compute the typicality matrix by (11).

*Until *; then stop. Otherwise, set and return to Step 1.

##### 3.2. Proposed Robust Kernel Possibilistic Fuzzy -Means (RKPFCM) Algorithm

The PFCM can deal with noisy data better than FCM and PCM; nevertheless, these conventional clustering algorithms become more effective when applied on linearly separable data or with a reasonable quantity of errors. In reality, the linearly separable data are rare. Therefore, FCM, PCM, and PFCM share the same negative point in that they are unable to get good separation of data that are nonlinearly separable in input space. To correct the imperfections found in PFCM particularly the nonlinear separable problem, kernel [7] methods are regarded as the way of dealing with this problem. In this context, we proposed a new extension of Possibilistic Fuzzy -Means algorithm based on kernel method (RKPFCM). The present work proposes a way of increasing the accuracy of the PFCM algorithm by exploiting hyper tangent kernel function to calculate the distance used in its objective function.

The kernel function is defined as a generalization of the distance metric that measures the distance between two data points mapped into a future space in which the data are more clearly separable [12, 19–21].

Define a nonlinear map as , where is the transformed feature space with higher or even infinite dimension. denotes the data space mapped into [20–22].

The RKPFCM algorithm minimizes the following objective function:Then is mapped into space [22]:where is an inner product kernel function.

If we adopt the hyper tangent kernel function, that is, then . Thus (15) can be written asConsidering (17), the objective function (13) is transformed as follows: The derivation of the objective function (18) according to , , and , defines the relationship update of cluster centers and membership coefficients.

*(i) Derivative of ** ) with respect to *.So,Equating (20) to zero leads toThen,

*(ii) Derivative of ** ) with respect to *. In this part we used the Lagrange multiplierFrom expression (25), we can write in this form:Substituting expression (26) in expression (24): It is alsoThe two expressions (26) and (28) give the following expression:Therefore the updating relationship is

*(iii) Derivative of ** ) with respect to *.Therefore, the updating typicality matrix isSimilarly (9) is rewritten by

*RKPFCM Algorithm Steps*. Given a set of observations .

*Initialization* ( = 0) Set the number of clusters . Set the level of weighting : . Set the parameters , , and . Set the stopping criterion *ε*: . Execute a FCM clustering algorithm to find initial fuzzy partition matrix and cluster centers . Initialize the typicality matrix randomly. Compute by (34).

*Repeat for *.

*Step 1. * Compute the cluster centers by (22).

*Step 2. * Compute the membership matrix by (30).

*Step 3. * Compute the typicality matrix by (33).

*Until *; then stop. Otherwise, set and return to Step 1.

##### 3.3. Robust Kernel Possibilistic Fuzzy -Means Algorithm Based on PSO (RKPFCM-PSO)

###### 3.3.1. PSO

The Particle Swarm Optimization is a heuristic search method proposed by Kennedy and Eberhart (1995). This technique uses random population solution particles to find an optimal solution to problems. Each particle moves in the search space with a dynamically adjusted position and velocity for the best solution. The particle is characterized by data structure that contains the coordinates of the current position in the search space, the best solution point visited so far, and the subset of other agents that are seen as neighbors. These adjustments are based on the historical behaviors of itself and other agents in the swarm. The change of speed (acceleration) and the position of each particle in the optimization landscape (search space) are iteratively [6, 12, 23]where : size of particles; : size of the landscape (search space); : the speed of particle; : the position of particle; : the best previous position of particle; : index represents the best particle among all particles in the group; : the constriction factor described by the following relationship: where and are two positive constants satisfying the following relationship: and : random variables defined as follows: where and are two random variables between 0 and 1; : the weight of inertia according to this equation: where and are the initial and final weight, is the maximum iterations, and iter is the current iteration number.

###### 3.3.2. Fitness Function

The fitness function defines our optimization problem described by the following expression:where is a positive constant. represents the objective function of the RKPFCM algorithm.

*RKPFCM-PSO Algorithm. *Given a set of observations , the RKPFCM-PSO algorithm is described by the following steps.

*Initialization* () Select the number of clusters , fuzzy degree , the parameters* a *and* b*, the population size NP, the constants and , the random variables and , the weight of inertia and , the size of the search space , the constant , and the stopping criterion . Set the 1st particle generation clusters centers. Initialize the fitness function and speed of each particle. Compute by (34).

*Repeat * = + 1

*Step 1. * Compute the fuzzy partition matrix by (30).

*Step 2. * Compute the typicality matrix by (33).

*Step 3. * Calculate the new value of fitness for each particle using (40).

*Step 4. * Compare the fitness of each particle with *best*, if the value is better than *best* and then set the *best* value.

*Step 5. * Compare the fitness value of *best* with the following: if the value is better than *best*, *best* then is set equal to this value.

*Step 6. * Update position and speed of each particle by (35).

So this algorithm is converged when ; that is to say, stop iteration and find the best solution in the last generation. If not, go back to Step1.

#### 4. Identification for Consequent Parameters

The defuzzification method, used in the Takagi-Sugeno fuzzy model, is linear with the consequent parameters which can be obtained as a solution of a weighted least squares problem according to the following equation:where represents an extension of regression matrix; ; is the output vector; is a diagonal matrix of dimension ( × ) containing the coefficients of fuzzy memberships.The RKPFCM and RKPFCM-PSO clustering algorithm are used to find width of the membership functions by the following equation [24]:

#### 5. Simulation Results and Validation Model

##### 5.1. Identical Data with Noise

In this example we have used 12 data which are composed of 10 models and two noises; this data set (12) is presented in [25]. The FCM, PCM, PFCM, KPFCM, and our algorithms RKPFCM and RKPFCM-PSO were used in clustering the data set in tow groups ().

In this example the parameters settings are , , , , , , = 0.9, = 0.4, , , = 20, , and = 10^{−9}.

Figure 1 shows the clustering results for our proposed method RKPFCM-PSO. The Ideal (true) centroids areTable 1 shows the results of center clusters using the six algorithms. The error between the results prototypes and ideal center clusters is calculated by the next expression:where is the FCM, PCM, PFCM, KPFCM, RKPFCM, and RKPFCM-PSO.