Abstract
This paper presents a reference function based 3D FLC design methodology using support vector regression (SVR) learning. The concept of reference function is introduced to 3D FLC for the generation of 3D membership functions (MF), which enhance the capability of the 3D FLC to cope with more kinds of MFs. The nonlinear mathematical expression of the reference function based 3D FLC is derived, and spatial fuzzy basis functions are defined. Via relating spatial fuzzy basis functions of a 3D FLC to kernel functions of an SVR, an equivalence relationship between a 3D FLC and an SVR is established. Therefore, a 3D FLC can be constructed using the learned results of an SVR. Furthermore, the universal approximation capability of the proposed 3D fuzzy system is proven in terms of the finite covering theorem. Finally, the proposed method is applied to a catalytic packed-bed reactor and simulation results have verified its effectiveness.
1. Introduction
Many industrial processes and systems are “distributed’’ in space [1] and are usually called spatially distributed systems. Recently, a novel three-dimensional fuzzy logic controller (3D FLC) [2] has been developed for the control of such systems. The 3D FLC employs a three-dimensional (3D) fuzzy set [2], which is composed of the traditional fuzzy set plus a third dimension for the spatial information, and carries out a 3D rule inference engine; thus, it has the inherent capability to process spatiotemporal dynamic systems. The control strategy of the 3D FLC is similar to how human operators or experts control the temperature in a space domain. Actually, it is a kind of spatiotemporal fuzzy control system with the traditional model-free advantage.
Currently, most 3D FLC designs are based on expert knowledge [2–5], which requires that the human knowledge to the control solution must exist and be structured [6]. However, in many real-world applications, experts may have problems structuring the knowledge. Sometimes, although experts have the structured knowledge, they may sway between extreme cases: offering too much knowledge in the field of expertise or tending to hide their knowledge [6]. On the contrast, data sets hidden with effective control rules are usually available. The motivation of this study is to design a 3D FLC using spatiotemporal data information.
To date, few literatures are found to be focused on data-based 3D FLC design methods. In [7], a table look-up scheme was employed to design 3D FLC in terms of input-output pairs. In [8], a fuzzy c-means algorithm (FCM) and gradient-descent approach were used to design a data-based 3D FLC, where FCM was used to learn the initial 3D fuzzy rule base and then the gradient-descent approach was used to optimize the parameters of MFs. In [9, 10], a clustering and linear support vector regression based 3D FLC design method was proposed, where the nearest neighborhood clustering was used to construct the antecedent part of 3D fuzzy rules and a linear support vector regression (SVR) was used to learn the consequent parameters. These methods either yield lots of fuzzy control rules (e.g. in [7]) or require additional algorithm to reduce redundant 3D fuzzy sets or 3D fuzzy rules [8–10]. As a complementary, Zhang et al. proposed a data-based 3D FLC design method using SVR learning [11], where the learned support vectors and associated learning parameters are directly used to design antecedent part and consequent part of 3D fuzzy rules. The best advantage of this method is that reasonable 3D fuzzy control rules are directly extracted and constructed by SVR learning. The limitation of the design is that Gaussian shape membership function (MF) is the only choice for MF design.
In this study, we focus on a reference function based 3D FLC design using SVR learning, which integrates the merits of SVR learning and flexible MF choice. Utilizing the concept of reference function, the 3D FLC can cope with more kinds of MFs, for example, Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc. A nonlinear mathematical description of a reference function based 3D FLC can be derived, and spatial fuzzy basis functions are defined. Via relating spatial fuzzy basis functions of a 3D FLC to kernel functions (KFs) of an SVR, an equivalence relationship between a 3D FLC and an SVR is established. Therefore, a 3D FLC can be constructed using the learned results of an SVR. In addition, the universal approximation capability of the proposed 3D fuzzy system is proven in terms of the finite covering theorem.
The paper is organized as follows. Preliminaries about the reference function, 3D MF generated by reference function, and the nonlinear mapping of reference function based 3D FLC are addressed in Section 2. In Section 3, the methodology and design scheme of the reference function based 3D FLC design using SVR learning are presented. Then, the finite covering theorem is used to prove that the 3D FLC is a universal approximator in Section 4. In Section 5, a catalytic packed-bed reactor is presented as an example to illustrate the proposed 3D FLC and validate its effectiveness. In Section 6, conclusions are given.
2. Preliminaries
2.1. Reference Function
Definition of Reference Function (see [12, 13]). A function is a reference function if and only if the following two conditions hold:(1), (2). Many functions may be reference functions. For instance, Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc as listed in Table 1 are reference functions. The reference functions can be used to generate 3D MFs, which provide a way for 3D FLC to access more kinds of 3D MFs.
2.2. Reference Function Based 3D FLC
2.2.1. 3D MF Generated by Reference Function
A 3D MF is an extension of a traditional MF by adding a third coordinate for the spatial information. In detail, the 3D MF has three coordinates: one is for the universe of discourse of the variable, another one is for the spatial information, and the third one is for the membership degree. If finite sensors are used, the 3D MF can be considered as the assembly of the traditional 2D MFs at each sensing location. In this way, we can generate a 3D MF by location transformation of a reference function at each sensing location.
For example, we have a spatial input variable defined in a discrete spatial domain . A 3D MF of can be an assembly of the traditional 2D MFs at each sensing location . The MF distribution of at sensing location can be shown in Figure 1, where is a Gaussian type reference function; , , and are generated by location transformation of .
2.2.2. Reference Function Based 3D FLC as a Nonlinear Mapping
The basic structure of a 3D FLC is composed of 3D fuzzifier, 3D rule inference, and defuzzifier. Due to its unique 3D nature, some detailed operations of a 3D FLC are different from a traditional one for spatial information expression, processing, and compression. For their detailed operations, one can refer to [2]. Once each component of a reference function based 3D FLC is set, the nonlinear mathematical description of the 3D FLC can be derived (see Appendix A for a brief derivation). Assuming that we employ 3D singleton fuzzifier, 3D fuzzy rules as shown in of Appendix A, “product’’ t-norm and “weighted aggregation’’ dimension reduction [3] in the 3D rule inference, singleton fuzzy sets for the output variable, and “linear’’ defuzzifier [14], the reference function based 3D FLC can be mathematically expressed as where denotes the input of the th spatial input variable from the sensing location ; denotes MF generated by the reference function ; denotes a location parameter, that is, the location transformation of the MF of with respect to ; denotes the spatial weight from theth sensing location; and are constants; denotes sensor number.
In (1), let then (1) can be rewritten as We define as a Spatial Fuzzy Basis Function (SFBF) [11]. Each SFBF corresponds to a 3D fuzzy rule, and all the SFBFs correspond to a 3D rule base. Mathematically, a 3D FLC is a linear combination of all the SFBFs. Furthermore, we rewrite (2) into (4) where .
From (4), we can find that, at each sensing location, there exists a traditional FBF [15] ; in the whole space domain, multiple traditional FBFs are assembled by the spatial weights into a SFBF . All the spatial information expression and processing as well as the fuzzy linguistic expression and rule inference are integrated into SFBFs.
Equation (1) (or (3)) shows that the reference function based 3D FLC is a nonlinear mapping from the input space to the output space . In particular, using (3) a reference function based 3D FLC can be represented by a three-layer network structure as show in Figure 2.
3. Reference Function Based 3D FLC Design Using SVR Learning
3.1. Design Methodology
The design methodology can be depicted by Figure 3. The SFBFs from a 3D FLC are input to an SVR as the KFs, and the learned spatial support vectors as leading spatiotemporal data points from the SVR are imported for the design of a 3D fuzzy rule base. The design theory will involve two crucial issues. The first one is whether SFBFs from a 3D FLC can be used as KFs in an SVR. The second one is what the relationship between a 3D FLC and an SVR would be like on the basis of the first issue.
3.1.1. Spatial Fuzzy Basis Function as Mercer KF
When relating the SFBFs with the KFs in an SVR, for instance, SFBFs are regarded as KFs, the SVR and the 3D FLC will have the same network structures and then have the same mathematical expressions, which will be discussed in Section 3.1.2. Generally speaking, a function satisfying Mercer theorem can be used as a KF for an SVR [16]. In this study, we will prove that an SFBF is a Mercer KF.
In (3), we rewrite into , which can be further expressed as follows: where is a spatial input, is the location transformation parameter of 3D MF in the th rule, and and are translation invariant KFs [13].
In terms of [13], if the reference functions are positive definite functions, then we do get a Mercer kernel. The reference functions as listed in Table 1 are positive definite functions. Using these reference functions to generate MF, from [13], we can conclude that is a Mercer kernel. Since the linear combination of KFs is still a KF [16], we can derive that is still a Mercer KF. Therefore, SFBFs are Mercer KFs, which can be used as KFs for SVR learning.
3.1.2. Mathematical Equivalence of a Spatial SVR and a 3D FLC
Once the SFBFs from the 3D FLC are employed as the KFs for an SVR, an inherent equivalence relationship will be built between the SVR and the 3D FLC (comparing Figure 2 and Figure 10). By combining (1) and , we have the following mathematical expressions: where , , and .
From (6), we can find that each spatial support vector and its associated learning parameter correspond to one 3D fuzzy rule, where is applied to set the center of the 3D MF of the 3D fuzzy set in the th rule, that is, the location transformation of the 3D fuzzy set with respect to reference function , and is used to set (the constant for the consequent set of the th rule in 3D FLC).
3.2. Design Scheme
The design of a reference function based 3D FLC consists of five parts: data collection, KF generation, SVR learning, 3D fuzzy rule construction, and 3D fuzzy controller integration, as shown in Figure 4.
(1) Data Collection. A set of spatiotemporal data will be collected. The data should contain effective control laws. Essentially, the reference function based 3D FLC design is a fuzzy modeling [17] that extracts fuzzy control rules from the spatiotemporal data.
(2) KF Generation. Before SVR learning, KFs should be properly designed. In this step, via properly selecting reference function, SFBFs from a 3D FLC will be formulated (as in (4)) to set KFs for SVR learning.
(3) SVR Learning. With proper KFs, the SVM learning algorithm directly executes the spatiotemporal data set and yields spatial support vectors and their associated learning parameters .
(4) 3D Fuzzy Rule Construction. The spatial support vectors and their associated learning parameters, as leading control laws, are used to construct 3D fuzzy control rules. In detail, the spatial support vector is employed to construct the antecedent part of the th rule; is employed to construct the consequent part of the th rule. The form of each 3D fuzzy rule is shown as below It is shown that the result of the SVM learning can be easily interpreted using structured linguistic knowledge. Finally, a 3D rule base with rules is established.
(5) 3D Fuzzy Controller Integration. Once the 3D rule base is established, a 3D FLC can be achieved by integrating other components including 3D fuzzifier, 3D rule inference, and defuzzifier. The detailed settings are given in Section 2.2.2. Finally, we obtain a complete 3D FLC, which can be used as a controller for a spatially distributed dynamic system.
4. Universal Approximation of Reference Function Based 3D FLC
The reference function based 3D FLC design method is used to construct a 3D FLC from spatiotemporal data hidden with effective control laws. In other words, the 3D FLC aims at approximating an unknown nonlinear control function. In this study, we use the finite covering theorem to prove that the 3D FLC is a universal approximator; that is, it can approximate continuous control functions to arbitrary accuracy.
The universal approximation capability of the reference function based 3D FLC can be described by Theorem 1.
Theorem 1. Let be a continuous function defined on a compact . For each , there exists a reference function based 3D FLC such that
From , it is shown that is an universal rule, namely, for any spatial input , will be fired. In , the fired rule will produce the constant . Let can be regarded as a 3D FLC generated by rule base . Then, Theorem 1 can be restated as Theorem 2 as follows.
Theorem 2. Under the condition of Theorem 1, let be a continuous function defined on a compact . For any constant , one has . For each , there exists a reference function based 3D FLC such that
Before the proof of Theorem 2, we first present some preparation work.
When , the firing level of the fired rule is The inference result of is given by where is the output variable of the 3D FLC, which is corresponding to the “” of the consequent part of the fired rule. The composition result of all the fired rules is given as follows:
Based on the above preparation, Lemma 3 is presented as follows.
Lemma 3. Under the condition of Theorem 1 there exists a reference function based 3D FLC such that
Proof. Let . As is continuous at , for each there exists a such that For each , set Then, is open on and . As is compact, there exists a finite subfamily such that We can construct a 3D FLC , defined by When , we have the following.(1)If , the lemma is trivial.(2)If , then ; hence, there exists a such that . Therefore, we further have that the following.(a)From , it follows that ; that is, . In terms of the continuity of , we have .(b)From , it follows that , and then . We have .Hence, we have
In terms of Lemma 3, the proof of Theorem 2 can be given as follows.
Proof of Theorem 2. Consider
5. Application
5.1. A Catalytic Packed-Bed Reactor
This designed 3D FLC is applied to a catalytic packed-bed reactor [1, 4, 18] shown in Figure 5, where a reaction of the form takes place on the catalyst. The reaction is endothermic and a jacket is used to heat the reactor. A dimensionless model that describes this nonlinear tubular chemical reactor is provided as follows: subject to the boundary conditions where , , and denote the dimensionless temperature, the concentration of reactant , and jacket temperature, respectively; and denote the dimensionless time and space; and are the heat and mass Peclet numbers, is a dimensionless heat of reaction, is a dimensionless preexponential factor, is a dimensionless activation energy, and is a dimensionless heat transfer coefficient. The values of the process parameters are given as follows: The control problem is to maintain a desired reaction rate via tuning the jacket temperature to control catalyst temperature. In this application, the reactor began to work at one steady state; because of the requirement of operation conditions, the reference value of temperature is increased by 8%. Thus, the control objective is to make the temperature of reactor well track the new reference value along the space domain.
5.2. Design of Reference Function Based 3D FLC
(1) Data Collection. The spatiotemporal data is collected from the catalytic packed-bed reactor, which is controlled by an expert based 3D FLC [4]. Five point sensors are located along the length of the reactor with for collecting the spatial distribution of the temperature . Two spatial inputs are error and error in change; that is, and . The detailed design of the expert based 3D FLC, including 3D fuzzifier, 3D rule inference, and defuzzifier, can refer to [4]. The scaling factors for the spatial error, the spatial error in change, and the incremental output are set as 0.5, 0.1, and 0.3, respectively. The sample period is 0.1 s, and the sampling duration is 6 s. Thus, we have 60 input-output data pairs (shown in Figure 6), each of which is represented by , .
(a)
(b)
(c)
(2) KF Generation. The reference functions (Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc) as listed in Table 1 are employed, respectively, to generate 3D MFs and then formulate SFBFs. SFBFs are used for KFs in an SVR learning.
(3) SVR Learning. With the spatiotemporal data set as above, the SVM learning algorithm is used for the support vector learning. It should be noted that the SFBFs in (3) are taken as the KFs. As a result, spatial support vectors are extracted and their associated learning parameters are obtained. For instance, when the Gaussian type reference function is used for KF generation, eight support vectors were learned from 60 spatiotemporal data pairs when , , and , as shown in Figure 7, where the spatiotemporal input data is decomposed into multiple two-dimensional graphical representations over the space domain.
(4) 3D Fuzzy Rule Construction. In terms of the learned results of the SVR in the previous step, we establish 3D fuzzy rules. For instance, with the Gaussian type reference function, eight 3D fuzzy rules are constructed. The first four 3D fuzzy rules are presented as follows. : if is close to and is close to , then is close to . : if is close to and is close to , then is close to . : if is close to and is close to , then is close to 249.2288. : if is close to and is close to , then is close to 148.1335.The first four 3D fuzzy rules and their associated 3D MF distributions can be depicted in Figure 8, which show the inherent spatial nature of the 3D fuzzy control system.
(5) 3D Fuzzy Controller Integration. Based on the 3D fuzzy rules established in step , we obtain a complete 3D FLC by combining 3D fuzzifier, 3D rule inference, and defuzzifier. The resultant 3D FLC will be used as a controller for the catalytic packed-bed reactor.
5.3. Control Performance Validation
The designed reference function based 3D FLC using SVR learning is validated on the nonlinear catalytic packed-bed reactor. We employed six kinds of reference functions, that is, Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc, and finally produced six 3D FLCs. With the reference function based 3D FLC as the controller, the catalyst temperature varying with time and space, manipulated input, and the catalyst temperature at steady state are presented in Figures 9(a)–9(f). The control performance is given in Table 2, where steady-state error (SSE), integral of the absolute error (IAE), and integral of time multiplied by absolute error (ITAE) [2] are used as the performance criteria. In terms of Figures 9(a)–9(f) and Table 2, we can find that different reference functions will yield different control performance. In this application, Gaussian, Cauchy, Hyperbolic Secant, and Squared Sinc reference functions result in good control performance, while Symmetric triangle and Laplace reference functions lead to poor control performance. The results illustrate that KF will influence the control performance; thus, in the actual application, we should choose proper KF to design a 3D FLC.
(a) Controlled by a reference function based 3D FLC with Symmetric triangle reference function
(b) Controlled by a reference function based 3D FLC with Gaussian reference function
(c) Controlled by a reference function based 3D FLC with Cauchy reference function
(d) Controlled by a reference function based 3D FLC with Laplace reference function
(e) Controlled by a reference function based 3D FLC with Hyperbolic Secant reference function
(f) Controlled by a reference function based 3D FLC with Squared Sinc reference function
(g) Controlled by an expert based 3D FLC
As a comparison, we do another control experiment; that is, the expert knowledge-based 3D FLC from [4] is taken as the controller. As for its detailed design including 3D MF, 3D rule base, 3D inference, fuzzification, and defuzzification, one can refer to [4]. The scaling factors for the spatial error, the spatial error in change, and the incremental output are set as 0.5, 0.1, and 0.3, respectively. The controlled profiles and control performance are given in Figure 9(g) and Table 2, respectively.
From Figure 9 and Table 2, we can find that with a proper reference function, the reference function based 3D FLC has nearly the same control performance as the expert knowledge-based 3D FLC. It means that the proposed spatial SVR learning method can well extract the control laws hidden in a spatiotemporal input–output dataset and formulate them in the form of 3D fuzzy rules.
6. Conclusions
A reference function based 3D FLC design methodology using SVR learning is proposed for spatially distributed dynamic systems. Utilizing the concept of reference function, the 3D FLC can access more kinds of 3D MFs, such as Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc. Based on the mathematical expressions of reference function based 3D FLC, we define spatial fuzzy basis functions and then find an equivalence relationship between a 3D FLC and an SVR by connecting spatial fuzzy basis functions in the 3D FLC to KFs in the SVR. On the basis of the equivalence relationship, a 3D FLC can be designed using the SVR learning; that is, the learned spatial support vectors as the optimal leading data points can be directly used for 3D fuzzy control rule generation. The proposed reference function based 3D FLC design can be carried out in five steps: data collection, KF generation, SVR learning, 3D fuzzy rule construction, and 3D fuzzy controller integration. Besides, the universal approximation capability of the proposed 3D fuzzy system is discussed. Finally, effectiveness of the proposed 3D FLC design methodology is validated on a catalytic packed-bed reactor.
Appendices
A. Nonlinear Mapping Derivation of a Reference Function Based 3D FLC
Let be a spatial input vector. Then, the 3D MF of the th spatial input is given as and the Gaussian type 2D MF of the th spatial input at the sensing location is given as Via a 3D fuzzifier, the spatial input vector in the universe of discourse can be transformed into a spatial fuzzy input as below: where denotes the t-norm operation.
Assume that 3D fuzzy rules are designed with the following form: where is a universal 3D fuzzy set, whose MF at sensing location is ; is a 3D fuzzy set, whose MF at sensing location is , ; and are constants.
Then, for each fired rule, a fuzzy relation is obtained as below: A 3D rule inference integrates the spatial information processing and the traditional inference and contains three main operations: spatial information fusion, dimension reduction, and traditional inference. Firstly, using the spatial information fusion operation, we have a spatially distributed set over the space domain with the grade of the MF derived as where “product’’ is used for t-norm () and singleton fuzzifier is used.
Then, utilizing a weighted aggregation [3] dimension reduction operation, a 2D set is obtained. Consider the following:
Finally, traditional inference operation (Mamdani implication operation) and linear defuzzifier [14] are carried out successively. We have the nonlinear mathematical expression as follows:
B. Mathematical Preliminaries of -Support Vector Regression
In this study, we focus on -support vector regression (-SVR). Suppose that we have a training set consisting of pairs ,,,, where the inputs are -dimensional vectors and the labels are continuous values. In -SVR, the goal is to find a function so that for all training patterns has a maximum deviation from the target values and has a maximum margin. The -insensitive loss function is defined as follows:
To make the SVR nonlinear, we may map the input vector into the vector of a high-dimensional feature space, , where represents a mapping , and formulate a linear regression problem in this feature space, and then an optimization problem will be solved. The optimization problem can also be solved in a dual space. By introducing the Lagrange multipliers and , the primal optimization problem can be formulated in its dual form as follows: subject to where the constant is a design parameter chosen by the user, which determines the tradeoff between the complexity of and the approximate error.
Solving the dual quadratic programming problem, we can find an optimal weight vector and an optimal bias of the regression hypersurface given in (B.4): Then, the best regression hypersurface is given by
The training pattern with nonzero is called support vector (SV).
To avoid a direct mapping , the kernel trick is used. AKF , which satisfies the Mercer’s theorem, is introduced as below:
Using , the SVR can be constructed which operates in an infinite dimensional space. Then, the solution of the SVR has the form
Let represent support vectors. The solution of the SVR can be described by a three-layer network structure as shown in Figure 10.
Acknowledgment
This work was supported by the project from the National Science Foundation of China under Grant no. 61273182, no. 51375293, and no. 61171145.