Universal Approximation of a Class of Interval Type-2 Fuzzy Neural Networks in Nonlinear Identification

Abstract

Neural networks (NNs), type-1 fuzzy logic systems (T1FLSs), and interval type-2 fuzzy logic systems (IT2FLSs) have been shown to be universal approximators, which means that they can approximate any nonlinear continuous function. Recent research shows that embedding an IT2FLS on an NN can be very effective for a wide number of nonlinear complex systems, especially when handling imperfect or incomplete information. In this paper we show, based on the Stone-Weierstrass theorem, that an interval type-2 fuzzy neural network (IT2FNN) is a universal approximator, which uses a set of rules and interval type-2 membership functions (IT2MFs) for this purpose. Simulation results of nonlinear function identification using the IT2FNN for one and three variables and for the Mackey-Glass chaotic time series prediction are presented to illustrate the concept of universal approximation.

1. Introduction

Several authors have contributed to universal approximation results. An overview can be found in [1–8]; further references to prime contributors in function approximations by neural networks are in [4, 9–12] and type-2 fuzzy logic modeling in [13–23]. It has been shown that a three-layer NN can approximate any real continuous function [24]. The same has been shown for a T1FLS [1, 25] using the Stone-Weierstrass theorem [3]. A similar analysis was made by Kosko [2, 9] using the concept of fuzzy regions. In [3, 26] Buckley shows that, with a Sugeno model [27], a T1FLS can be built with the ability to approximate any nonlinear continuous function. Also, combining the neural and fuzzy logic paradigms [28, 29], an effective tool can be created for approximating any nonlinear function [4]. In this sense, an expert can use a type-1 fuzzy neural network (T1FNN) [10–12, 30] or IT2FNN systems and find interpretable solutions [15–17, 31–34]. In general, Takagi-Sugeno-Kang (TSK) T1FLSs are able to approximate by the use of polynomial consequent rules [7, 27]. This paper uses the Levenberg-Marquardt backpropagation learning algorithm for adapting antecedent and consequent parameters for an adaptive IT2FNN, since its efficiency and soundness characteristics make them fit for these optimizing problems. An Adaptive IT2FNN is used as a universal approximator of any nonlinear functions. A set Ψ of IT2FNNs is universal if and only if (iff), given any process Ω, there is a system Φ∈Ψ such that the difference between the output from IT2FNN and output from Ω is less than a given 𝜀.

In this paper the main contribution is the proposed IT2FNNs architectures, which are shown to be universal approximators and are illustrated with several benchmark problems to verify their applicability for real world problems.

2. Interval Type-2 Fuzzy Neural Networks

An IT2FNN [15, 31, 35] combines a TSK interval type-2 fuzzy inference system (TSKIT2FIS) [13, 14, 33, 34] with an adaptive NN in order to take advantage of both models best characteristics. In general, when representing IT2FNN graphically, rectangles are used to represent adaptive nodes and circles to represent nonadaptive nodes. Output values of pair nodes (green color) and odd nodes (blue color) represent uncertainty intervals (Figures 1–4). In this kind of interval type-2 neurofuzzy adaptive networks, nodes represent processing units called neurons, which can be classified into crisp and fuzzy neurons.

Figure 1 

IT2FNN-1 architecture.

Figure 2 

IT2FNN-3 architecture.

Figure 3 

IT2FNN-0 architecture.

Figure 4 

IT2FNN-2 architecture.

The IT2FNN-1 architecture has 5 layers (Figure 1) [35] and consists of adaptive nodes with equivalent function to lower-upper membership in fuzzification layer (layer 1). Nonadaptive nodes in rules layer (layer 2) interconnect with the fuzzification layer (layer 1) in order to generate TSK IT2FIS rules antecedents. Adaptive nodes in the consequent layer (layer 3) are connected to the input layer (layer 0) to generate rules consequents. Nonadaptive nodes in type-reduction layer (layer 4) evaluate left-right values with the Karnik and Mendel (KM) [13, 14] algorithm. The nonadaptive nodes in the defuzzification layer (layer 5) average left-right values.

The IT2FNN-3 architecture has 8 layers (Figure 2) [35] and uses IT2FN for fuzzifying the inputs (layers 1-2). Nonadaptive nodes in rules layer (layer 3) interconnect with lower-upper linguistic values layer (layer 2) to generate TSK IT2FIS rules antecedents. Adaptive nodes in layer 4 adapt left-right firing strength, biasing rules lower-upper trigger forces with synaptic weights between layers 3 and 4. Layer 5’s nonadaptive nodes normalize rules lower-upper firing strength. Nonadaptive nodes I consequent layer (layer 6) interconnect with input layer (layer 0) to generate rules consequents. Nonadaptive nodes in type-reduction layer (layer 7) evaluate left-right values adding lower-upper product of lower-upper triggering forces normalized by rules consequent left-right values. Node in defuzzification layer is adaptive and its output 𝑦 is defined as biased average of left-right values and parameter 𝛾. Parameter 𝛾 (0.5 by default) adjusts uncertainty interval defined by left-right values [𝑦_𝑙,𝑦_𝑟].

Architectures IT2FNN-0 and IT2FNN-2, which will be shown in Sections 3.2 and 3.3, respectively, as universal approximators, are described with more details in Section 2.1.

2.1. IT2FNN-0 Architecture

An IT2FNN-0 is a seven-layer IT2FNN, which integrates a first order TSKIT2FIS (interval type-2 fuzzy antecedents and real consequents) with an adaptive NN. The IT2FNN-0 (Figure 3) layers are described as follows.

Layer 0. Inputs 𝑜⁰_𝑖=𝑥_𝑖,𝑖=1,…,𝑛. (1)

Layer 1. Adaptive type-1 fuzzy neuron (T1FN) net¹_𝑘=𝑤^1,0_𝑘,𝑖𝑥_𝑖+𝑏¹_𝑘∀𝑖=1,…,𝑛,𝑘=1,…,𝜗, (2)𝑜¹_𝑘=𝜇(net¹_𝑘), where the transfer function 𝜇 is a membership function, net¹_𝑘 is the weighted sum of inputs (𝑥_𝑖) and the synaptic weights (𝑤^1,0_𝑘,𝑖), and 𝑏¹_𝑘 is the threshold for each neuron.

Layer 2. Nonadaptive T1FN. This layer contains T-norm and S-norm fuzzy nodes 𝑜²_2𝑘−1=𝑜¹_2𝑘−1⋅𝑜¹_2𝑘∀𝑘=1,…,𝜗,T-normfuzzynode, (3) where 𝜗 is the number of nodes in layers 1 and 2 𝑜²_2𝑘=𝑜¹_2𝑘−1+𝑜¹_2𝑘−𝑜²_2𝑘−1,S-normfuzzynode, (4)𝜏=ℓ_𝑘,𝑖 for all 𝑘=1,…,𝑀, and 𝑖=1,…,𝑛, where ℓ_𝑘,𝑖 is the table of indices of the antecedents of the rules 𝜋=∑^𝑖−1_𝑗=1𝑣_𝑗+|𝜏|, where 𝜋 is a vector of indices for each node of layer 2 if 𝜏>0𝜇^𝑘_𝑖=𝑜²_{𝑖𝑙(𝜋)},𝜇^𝑘_𝑖=𝑜²_{𝑖𝑢(𝜋)} (5) else 𝜇_𝑘,𝑖=null,𝜇_𝑘,𝑖=null (6) end,

where 𝜇^𝑘_𝑖,𝜇^𝑘_𝑖 are lower and upper membership function values, respectively. 𝑖_𝑙(𝜋) and 𝑖_𝑢(𝜋) are vectors with even and odd indices of the nodes of layer 2.

Layer 3. Lower-upper firing strength (𝑤^𝑘,𝑤^𝑘). Having nonadaptive nodes for generating lower-upper firing strength of TSK IT2FIS rules (7), 𝑜³_2𝑘−1=𝑤^𝑘,𝑜³_2𝑘=𝑤^𝑘,𝑤^𝑘=^𝑛_𝑖=1𝜇𝐹^𝑘_𝑖(𝑥),𝑤^𝑘=^𝑛_𝑖=1𝜇𝐹^𝑘_𝑖(𝑥), (7) where 𝜇𝐹^𝑘_𝑖(𝑥)∈[𝜇𝐹^𝑘_𝑖(𝑥),𝜇𝐹^𝑘_𝑖(𝑥)] is the Gaussian interval type-2 membership function, igaussmtype2 (𝑥,[𝜎^𝑘_𝑖,¹𝑚^𝑘_𝑖,²𝑚^𝑘_𝑖]), defined by ¹𝜇𝐹^𝑘_𝑖𝑥_𝑖,𝜎^𝑘_𝑖,¹𝑚^𝑘_𝑖−1=exp2𝑥_𝑖−¹𝑚^𝑘_𝑖𝜎^𝑘_𝑖²₂,𝜇𝐹^𝑘_𝑖𝑥_𝑖,𝜎^𝑘_𝑖,²𝑚^𝑘_𝑖−1=exp2𝑥_𝑖−²𝑚^𝑘_𝑖𝜎^𝑘_𝑖²𝜇_𝐹𝑘𝑖,(𝑥)=²𝜇𝐹^𝑘_𝑖𝑥_𝑖,𝜎^𝑘_𝑖,²𝑚^𝑘_𝑖,𝑥_𝑖≤¹𝑚^𝑘_𝑖,₁𝜇𝐹^𝑘_𝑖𝑥_𝑖,𝜎^𝑘_𝑖,¹𝑚^𝑘_𝑖,𝑥_𝑖>²𝑚^𝑘_𝑖,𝜇_𝐹𝑘𝑖(𝑥)=¹𝜇𝐹^𝑘_𝑖𝑥_𝑖,𝜎^𝑘_𝑖,¹𝑚^𝑘_𝑖,𝑥_𝑖<¹𝑚^𝑘_𝑖,1,¹𝑚^𝑘_𝑖≤𝑥_𝑖≤²𝑚^𝑘_𝑖,₂𝜇𝐹^𝑘_𝑖𝑥_𝑖,𝜎^𝑘_𝑖,²𝑚^𝑘_𝑖,𝑥_𝑖>²𝑚^𝑘_𝑖. (8)(9)(10)(11)

Layer 4. Lower-upper firing strength rule normalization (𝜙^𝑘,𝜙^𝑘). Nodes in this layer are nonadaptive and the output is defined as the ratio between the 𝑘th lower-upper firing strength rule (𝑤^𝑘,𝑤^𝑘) and the sum of lower-upper firing strength of all rules (13) and (14): 𝑜⁴_2𝑘−1=𝜙^𝑘,𝑜⁴_2𝑘=𝜙^𝑘. (12) If we view 𝜙^𝑘,𝜙^𝑘 as fuzzy basis functions (FBF) (32) and (33) and 𝑦^𝑘(𝑥) as linear function (16), then 𝑦(𝑥) can be viewed as a linear combination of the basis functions (20) and (21): 𝜙^𝑘=𝑤^𝑘𝐷_𝑙,𝑘=1,…,𝑀, (13) where 𝐷_𝑙=∑^𝑀_𝑘=1𝑤^𝑘, 𝜙^𝑘=𝑤^𝑘𝐷_𝑟,𝑘=1,…,𝑀, (14) where 𝐷_𝑟=∑^𝑀_𝑘=1𝑤^𝑘.

Layer 5. Rule consequents. Each node is adaptive and its parameters are {𝑐^𝑘_𝑖,𝑐^𝑘₀}. The node’s output corresponds to partial output of 𝑘th rule 𝑦^𝑘 (16): 𝑜⁵_2𝑘=1=𝑦^𝑘,𝑜⁵_2𝑘=𝑦^𝑘,𝑦^𝑘=^𝑛_𝑖=1𝑐^𝑘_𝑖𝑥_𝑖+𝑐^𝑘₀;𝑘=1,…,𝑀. (15)(16)

Layer 6. Estimating left-right interval values [𝑦_𝑙,𝑦_𝑟] (18), nodes are nonadaptive with outputs 𝑦_𝑙,𝑦_𝑟. Layer 6 output is defined by 𝑜⁶₁=𝑦_𝑙(𝑥),𝑜⁶₂=𝑦_𝑟(𝑥), (17) where 𝑦_𝑙(𝑥)=^𝑀_𝑘=1𝜙^𝑘𝑦^𝑘,𝑦_𝑟(𝑥)=^𝑀_𝑘=1𝜙^𝑘𝑦^𝑘. (18)

Layer 7. Defuzzification. This layer’s node is adaptive, where the output 𝑦, (20) and (21), is defined as weighted average of left-right values and parameter 𝛾. Parameter 𝛾 (default value 0.5) adjusts the uncertainty interval defined by left-right values [𝑦_𝑙,𝑦_𝑟]: 𝑜⁷₁=𝑦(𝑥), (19) where 𝑦(𝑥)=𝛾𝑦_𝑙(𝑥)+1−𝛾𝑦_𝑟(𝑥),𝑦(𝑥)=^𝑀_𝑘=1𝛾𝜙^𝑘(𝑥)+1−𝛾𝜙^𝑘(𝑥)𝑦^𝑘(𝑥). (20)(21)

2.2. IT2FNN-2 Architecture

An IT2FNN-2 [31] is a six-layer IT2FNN, which integrates a first order TSKIT2FIS (interval type-2 fuzzy antecedents and interval type-1 fuzzy consequents), with an adaptive NN. The IT2FNN-2 (Figure 4) layers are described in a similar way to the previous architectures.

3. IT2FNN as a Universal Approximator

Based on the description of the interval type-2 fuzzy neural networks, it is possible to prove that under certain conditions, the resulting IT2FIS has unlimited approximation power to match any nonlinear functions on a compact set [36, 37] using the Stone-Weierstrass theorem [5, 6, 10, 30].

3.1. Stone-Weierstrass Theorem

Theorem 1 (Stone-Weierstrass theorem). Let 𝑍 be a set of real continuous functions on a compact set 𝑈. If (1)  𝑍 is an algebra, that is, the set 𝑍 is closed under addition, multiplication, and scalar multiplication, (2)  𝑍 separates points on 𝑈, that is, for every 𝐱,𝐲∈𝑈, 𝐱≠𝐲, there exists 𝑓∈𝑍 such that 𝑓(𝐱)≠𝑓(𝐲), and (3)  𝑍 vanishes at no point of 𝑈, that is, for each 𝑥∈𝑈 there exists 𝑓∈𝑍 such that 𝑓(𝐱)≠0, then the uniform closure of 𝑍 consists of all real continuous functions on 𝑈; that is, (𝑍,𝑑_∞) is dense in (𝐶[𝑈],𝑑_∞) [36–38].

Theorem 2 (universal approximation theorem). For any given real continuous function 𝑔(𝑢) on the compact set 𝑈⊂𝑅^𝑛 and arbitrary 𝜀>0, there exists 𝑓∈𝑌 such that sup_𝑥∈𝑈(|𝑔(𝑥)−𝑓(𝑥)|)<𝜀.

3.2. Applying Stone-Weierstrass Theorem to the IT2FNN-0 Architecture

In the IT2FNN-0, the domain on which we operate is almost always compact. It is a standard result in real analysis that every closed and bounded set in ℜ^𝑛 is compact. Now we shall apply the Stone-Weierstrass theorem to show the representational power of IT2FNN with simplified fuzzy if-then rules. We now consider a subset of the IT2FNN-0 on Figure 5. The set of IT2FNN-0 with singleton fuzzifier, product inference, center of sets type reduction, and Gaussian interval type-2 membership function consists of all FBF expansion functions of the form (38), (40). 𝑓∶𝑈⊂𝑅^𝑛→𝑅, 𝑥=(𝑥₁,𝑥₂,…,𝑥_𝑛)∈𝑈; 𝜇𝐹^𝑘_𝑖(𝑥)∈[𝜇𝐹^𝑘_𝑖(𝑥),𝜇𝐹^𝑘_𝑖(𝑥)] is the Gaussian interval type-2 membership function, igaussmtype2 (𝑥,[𝜎^𝑘_𝑖,¹𝑚^𝑘_𝑖,²𝑚^𝑘_𝑖]), defined by (27) and (31). If we view 𝜙^𝑘(𝑥),𝜙^𝑘(𝑥) as fuzzy basis functions (32) and (33) and 𝑦^𝑘(𝑥) are linear functions (34), then 𝑦(𝑥) of (38) and (40) can be viewed as a linear combination of the fuzzy basis functions, and then the IT2FNN-0 system is equivalent to an FBF expansion. Let 𝑌 be the set of all the FBF expansions (38) and (40) with 𝜙^𝑘(𝑥),𝜙^𝑘(𝑥) given by (13) and (38) and let 𝑑_∞(𝑓₁,𝑓₂)=sup_𝑥∈𝑈(|𝑓₁(𝑥)−𝑓₂(𝑥)|) be the supmetric; then, (𝑌,𝑑_∞) is a metric space [38]. We use the following Stone-Weierstrass theorem to prove our result.

(a) Antecedent IT2MFs for fuzzy rules

(b) Overall I/O curve for crisp rules

(c) An example of the interval type-2 fuzzy basis functions

(d) Overall I/O curve for IT2FNN-0

Figure 5 

An example of the IT2FNN-0 architecture.

Suppose we have two IT2FNN-0s 𝑓₁,𝑓₂∈𝑌; the output of each system can be expressed as 𝑓₁(𝑥)=𝛼𝑦¹_𝑙(𝑥)+(1−𝛼)𝑦¹_𝑟(𝑥), (22) where 𝑦¹_𝑙(𝑥)=^𝑀_𝑘=1𝜙^𝑘₁(𝑥)𝑧^𝑘₁(𝑥)=∑^𝑀1_𝑘=1𝑤^𝑘₁(𝑥)𝑧^𝑘₁(𝑥)𝐷¹_𝑙,𝑦¹_𝑟(𝑥)=^𝑀1_𝑘=1𝜙^𝑘₁(𝑥)𝑧^𝑘₁(𝑥)=∑^𝑀1_𝑘=1𝑤^𝑘₁(𝑥)𝑧^𝑘₁(𝑥)𝐷¹_𝑟, (23) where 𝐷¹_𝑙=^𝑀1𝑛_𝑘=1𝑖=1𝜇₁𝐹^𝑘_𝑖(𝑥),𝐷¹_𝑟=^𝑀1_𝑘1^𝑛_=1𝑖=1𝜇₁𝐹^𝑘_𝑖(𝑥),𝑤^𝑘₁=^𝑛_𝑖=1𝜇₁𝐹^𝑘_𝑖(𝑥),𝑤^𝑘₁=^𝑛_𝑖=1𝜇₁𝐹^𝑘_𝑖(𝑥),𝜙^𝑘₁(𝑥)=𝑤^𝑘₁(𝑥)𝐷¹_𝑙,𝜙^𝑘₁(𝑥)=𝑤^𝑘₁𝐷¹_𝑟,𝑧^𝑘₁(𝑥)=^𝑛1_𝑖=1𝑐^𝑘_𝑖𝑥_𝑖+¹𝑐^𝑘₀𝑓,𝑘=1,…,𝑀,₂(𝑥)=𝛾𝑦²_𝑙(𝑥)+1−𝛾𝑦²_𝑟(𝑥), (24)(25) where 𝑦²_𝑙(𝑥)=^𝑀_𝑘=1𝜙^𝑘₂(𝑥)𝑧^𝑘₂(𝑥)=∑^𝑀2_𝑘=1𝑤^𝑘₂(𝑥)𝑧^𝑘₂(𝑥)𝐷²_𝑙,𝑦²_𝑟(𝑥)=^𝑀2_𝑘=1𝜙^𝑘₂(𝑥)𝑧^𝑘₂(𝑥)=∑^𝑀1_𝑘=1𝑤^𝑘₂(𝑥)𝑧^𝑘₂(𝑥)𝐷²_𝑟, (26)(27) where 𝑤^𝑘₂=^𝑛_𝑖=1𝜇₂𝐹^𝑘_𝑖(𝑥),𝑤^𝑘₂=^𝑛_𝑖=1𝜇₂𝐹^𝑘_𝑖(𝑥),𝐷²_𝑙=^𝑀2𝑛_𝑘=1𝑖=1𝜇₂𝐹^𝑘_𝑖(𝑥),𝐷²_𝑟=^𝑀2𝑛_𝑘=1𝑖=1𝜇₂𝐹^𝑘_𝑖(𝑥),𝜙^𝑘₂(𝑥)=𝑤^𝑘₂(𝑥)𝐷²_𝑙,𝜙^𝑘₂(𝑥)=𝑤^𝑘₂(𝑥)𝐷²_𝑟,𝑧^𝑘₂(𝑥)=^𝑛2_𝑖=1𝑐^𝑘_𝑖𝑥_𝑖+²𝑐^𝑘₀,𝑘=1,…,𝑀. (28)

Lemma 3. 𝑌 is closed under addition.

Proof. The proof of this lemma requires our IT2FNN-0 to be able to approximate sums of functions. Suppose we have two IT2FNN-0s, 𝑓₁(𝑥) and 𝑓₂(𝑥) with 𝑀₁ and 𝑀₂ rules, respectively. The output of each system can be expressed as 𝑓₁(𝑥)+𝑓₂(𝑥)=∑^𝑀1_𝑘1=1∑^𝑀2_𝑘2=1𝑤^𝑘1₁𝑤^𝑘2₂𝛼𝑧^𝑘1₁+𝛾𝑧^𝑘2₂𝐷¹_𝑙𝐷²_𝑙+∑^𝑀1_𝑘1=1∑^𝑀2_𝑘2=1𝑤^𝑘1₁𝑤^𝑘2₂(1−𝛼)𝑧^𝑘1₁+1−𝛾𝑧^𝑘2₂𝐷¹_𝑟𝐷²_𝑟 (29) and that Φ_𝑘1,𝑘2=(𝑤^𝑘1₁𝑤^𝑘2₂)/(𝐷¹_𝑟𝐷²_𝑟) and Φ_𝑘1,𝑘2=(𝑤^𝑘1₁𝑤^𝑘2₂)/(𝐷¹_𝑟𝐷²_𝑟), where the FBFs are known to be nonlinear. Therefore, an equivalent to IT2FNN-0 can be constructed under the addition of 𝑓₁(𝑥) and 𝑓₂(𝑥), where the consequents form an addition of 𝛼𝑧^𝑘1₁+𝛾𝑧^𝑘2₂ and (1−𝛼)𝑧^𝑘1₁+(1−𝛾)𝑧^𝑘2₂ multiplied by a respective FBFs expansion (Theorem 1), and there exists 𝑓∈𝑌 such that sup_𝑥∈𝑈(|𝑔(𝑥)−𝑓(𝑥)|)<𝜀 (Theorem 2). Since 𝑓(𝑥) satisfies Lemma 3 and 𝑌∈𝑓(𝑥)=𝑓₁(𝑥)+𝑓₂(𝑥) then we can conclude that 𝑌 is closed under addition. Note that 𝑧^𝑘1₁ and 𝑧^𝑘2₂ can be linear since the FBFs are a nonlinear basis interval and therefore the resultant function, 𝑓(𝑥), is nonlinear interval (see Figure 5).

Lemma 4. 𝑌 is closed under multiplication.

Proof. Similar to Lemma 3, we model the product of 𝑓₁(𝑥)𝑓₂(𝑥) of two IT2FNN-0s. The product 𝑓₁(𝑥)𝑓₂(𝑥) can be expressed as 𝑓₁(𝑥)𝑓₂(𝑥)=1𝐷¹_𝑙𝐷²_𝑙^𝑀1_𝑘1^𝑀₌₁²_𝑘2=1𝑤^𝑘1₁𝑤^𝑘2₂𝛼𝛾𝑧^𝑘1₁𝑧^𝑘2₂+1𝐷¹_𝑙𝐷²_𝑟^𝑀1_𝑘1^𝑀₌₁²_𝑘2=1𝑤^𝑘1₁𝑤^𝑘2₂𝛼1−𝛾𝑧^𝑘1₁𝑧^𝑘2₂+1𝐷¹_𝑟𝐷²_𝑙^𝑀1_𝑘1^𝑀₌₁²_𝑘2=1𝑤^𝑘1₁𝑤^𝑘2₂(1−𝛼)𝛾𝑧^𝑘1₁𝑧^𝑘2₂+1𝐷¹_𝑟𝐷²_𝑟^𝑀1_𝑘1^𝑀₌₁²_𝑘2=1𝑤^𝑘1₁𝑤^𝑘2₂(1−𝛼)1−𝛾𝑧^𝑘1₁𝑧^𝑘2₂. (30) Therefore, an equivalent to IT2FNN-0 can be constructed under the multiplication of 𝑓₁(𝑥) and 𝑓₂(𝑥), where the consequents form an addition of 𝛼𝛾𝑧^𝑘1₁𝑧^𝑘2₂, 𝛼(1−𝛾)𝑧^𝑘1₁𝑧^𝑘2₂, (1−𝛼)𝛾𝑧^𝑘1₁𝑧^𝑘2₂, and (1−𝛼)(1−𝛾)𝑧^𝑘1₁𝑧^𝑘2₂ multiplied by a respective FBFs (Φ^𝑙,𝑙_𝑘1,𝑘2=(𝑤^𝑘1₁𝑤^𝑘2₂)/(𝐷¹_𝑙𝐷²_𝑙), Φ^𝑙,𝑟_𝑘1,𝑘2=(𝑤^𝑘1₁𝑤^𝑘2₂)/(𝐷¹_𝑙𝐷²_𝑟), Φ^𝑟,𝑙_𝑘1,𝑘2=(𝑤^𝑘1₁𝑤^𝑘2₂)/(𝐷¹_𝑟𝐷²_𝑙), and Φ^𝑟,𝑟_𝑘1,𝑘2=(𝑤^𝑘1₁𝑤^𝑘2₂)/(𝐷¹_𝑟𝐷²_𝑟)) expansion (Theorem 1), and there exists 𝑓∈𝑌 such that sup_𝑥∈𝑈(|𝑔(𝑥)−𝑓(𝑥)|)<𝜀 (Theorem 2). Since 𝑓(𝑥) satisfies Lemma 3 and 𝑌∈𝑓(𝑥)=𝑓₁(𝑥)𝑓₂(𝑥) then we can conclude that 𝑌 is closed under multiplication. Note that 𝑧^𝑘1₁ and 𝑧^𝑘2₂ can be linear since the FBFs are a nonlinear basis interval and therefore the resultant function, 𝑓(𝑥), is nonlinear interval. Also, even if 𝑧^𝑘1₁ and 𝑧^𝑘2₂ were linear, their product 𝑧^𝑘1₁𝑧^𝑘2₂ is evidently polynomial (see Figure 5).

Lemma 5. 𝑌 is closed under scalar multiplication.

Proof. Let an arbitrary IT2FNN-0 be 𝑓(𝑥) (20); the scalar multiplication of 𝑐𝑓(𝑥) can be expressed as 𝑐𝑓(𝑥)=𝛼𝑐𝑦_𝑙(𝑥)+(1−𝛼)𝑐𝑦_𝑟(𝑥)=∑^𝑀_𝑘=1𝛼𝐷_𝑟𝑤^𝑘(𝑥)+(1−𝛼)𝐷_𝑙𝑤^𝑘(𝑥)𝑐𝑧^𝑘(𝑥)𝐷_𝑙𝐷_𝑟. (31) Therefore we can construct an IT2FNN-0 that computes 𝑐𝑧^𝑘(𝑥) in the form of the proposed IT2FNN-0; 𝑌 is closed under scalar multiplication.

Lemma 6. For every 𝐱⁰,𝐲⁰∈𝑈 and 𝐱⁰≠𝐲⁰, there exists 𝑓∈𝑌 such that 𝑓(𝐱⁰)≠𝑓(𝐲⁰); that is, 𝑌 separates points on 𝑈.

Proof. We prove that (𝑌,𝑑_∞) separates points on 𝑈. We prove this by constructing a required 𝑓(𝑥) (20); that is, we specify 𝑓∈𝑌 such that 𝑓(𝐱⁰)≠𝑓(𝐲⁰) for arbitrarily given (𝐱⁰,𝐲⁰)∈𝑈 with 𝐱⁰≠𝐲⁰. We choose two fuzzy rules in the form of (8) for the fuzzy rule base (i.e., 𝑀=2). Let 𝑥⁰=(𝑥⁰₁,𝑥⁰₂,…,𝑥⁰_𝑛) and 𝑦⁰=(𝑦⁰₁,𝑦⁰₂,…,𝑦⁰_𝑛). If 𝑥⁰_𝑖=(𝑥⁰_𝑙𝑖+𝑥⁰_𝑟𝑖)/2 and 𝑦⁰_𝑖=(𝑦⁰_𝑙𝑖+𝑦⁰_𝑟𝑖)/2 with 𝑥⁰_𝑖≠𝑦⁰_𝑖, we define two interval type-2 fuzzy sets (𝐹¹_𝑖,[𝜇𝐹¹_𝑖,𝜇𝐹¹_𝑖]) and (𝐹²_𝑖,[𝜇𝐹²_𝑖,𝜇𝐹²_𝑖]) with 𝜇𝐹¹_𝑖𝑥_𝑖=−1exp2𝑥_𝑖−𝑥⁰_𝑙𝑖²,𝑥_𝑖<𝑥⁰_𝑙𝑖,1,𝑥⁰_𝑙𝑖≤𝑥_𝑖≤𝑥⁰_𝑟𝑖,−1exp2𝑥_𝑖−𝑥⁰_𝑟𝑖²,𝑥_𝑖>𝑥⁰_𝑙𝑖,𝜇𝐹¹_𝑖𝑥_𝑖=−1exp2𝑥_𝑖−𝑥⁰_𝑟𝑖²,𝑥_𝑖≤𝑥⁰_𝑙𝑖,−1exp2𝑥_𝑖−𝑥⁰_𝑙𝑖²,𝑥_𝑖>𝑥⁰_𝑟𝑖,𝜇𝐹²_𝑖𝑥_𝑖=−1exp2𝑥_𝑖−𝑦⁰_𝑙𝑖²,𝑥_𝑖<𝑦⁰_𝑙𝑖,1,𝑦⁰_𝑙𝑖≤𝑥_𝑖≤𝑦⁰_𝑟𝑖,−1exp2𝑥_𝑖−𝑦⁰_𝑟𝑖²,𝑥_𝑖>𝑦⁰_𝑙𝑖,𝜇𝐹^𝑘_𝑖𝑥_𝑖=−1exp2𝑥_𝑖−𝑦⁰_𝑟𝑖²,𝑥_𝑖≤𝑦⁰_𝑙𝑖,−1exp2𝑥_𝑖−𝑦⁰_𝑙𝑖²,𝑥_𝑖>𝑦⁰_𝑟𝑖. (32)(33)(34)(35) If 𝑥⁰_𝑖=𝑦⁰_𝑖, then 𝐹¹_𝑖=𝐹²_𝑖 and 𝜇𝐹¹_𝑖(𝑥⁰_𝑖)=𝜇𝐹²_𝑖(𝑦⁰_𝑖), 𝜇𝐹¹_𝑖(𝑥⁰_𝑖)=𝜇𝐹²_𝑖(𝑦⁰_𝑖); that is, only one interval type-2 fuzzy set is defined. We define two real value sets 𝑧¹ and 𝑧² with 𝑧^𝑘(𝑥)=∑^𝑛_𝑖=1𝑐^𝑘_𝑖𝑥_𝑖+𝑐^𝑘₀, where 𝑘=1,2. Now we have specified all the design parameters except 𝑧^𝑘; that is, we have already obtained a function 𝑓 which is in the form of (10) with 𝑀=2 and (𝐹¹_𝑖,[𝜇𝐹¹_𝑖,𝜇𝐹¹_𝑖]) given by (18), (20), and (21). With this 𝑓, we have 𝑓𝑥⁰=𝛼𝜙¹𝑥⁰𝑧¹𝑥⁰+𝜙²𝑥⁰𝑧²𝑥⁰+()×1−𝛼𝜙¹𝑥⁰𝑧¹𝑥⁰+1−𝜙¹𝑥⁰𝑧²𝑥⁰, (36) where 𝜙¹𝑥⁰=∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑥⁰_𝑖∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑥⁰_𝑖+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑥⁰_𝑖,𝜙²𝑥⁰=∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑥⁰_𝑖∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑥⁰_𝑖+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑥⁰_𝑖,𝜙¹𝑥⁰=11+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑥⁰_𝑖,𝑓𝑦⁰=𝛼𝜙¹𝑦⁰𝑧¹𝑦⁰+𝜙²𝑦⁰𝑧²𝑦⁰+()×1−𝛼1−𝜙²𝑦⁰𝑧¹𝑦⁰+𝜙²𝑦⁰𝑧²𝑦⁰, (37)(38) where 𝜙¹𝑦⁰=∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑦⁰_𝑖∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑦⁰_𝑖+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑦⁰_𝑖,𝜙²𝑦⁰=∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑦⁰_𝑖∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑦⁰_𝑖+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑦⁰_𝑖,𝜙²𝑦⁰=1∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑦⁰_𝑖.+1 (39) Since 𝐱⁰≠𝐲⁰, there must be some 𝑖 such that 𝑥⁰_𝑖=𝑦⁰_𝑖; hence, we have ∏^𝑛_𝑖=1𝜇_𝐹1𝑖(𝑥⁰_𝑖)≠1 and ∏^𝑛_𝑖=1𝜇_𝐹2𝑖(𝑥⁰_𝑖)≠1. If we choose 𝑧¹=0 and 𝑧²=1, then 𝑓(𝑥⁰)=𝛼𝜙²(𝑥⁰)+(1−𝛼)[1−𝜙¹(𝑥⁰)]≠𝛼𝜙²(𝑦⁰)+(1−𝛼)𝜙²(𝑦⁰)=𝑓(𝑦⁰). Separability is satisfied whenever an IT2FNN-0 can compute strictly monotonic functions of each input variable. This can easily be achieved by adjusting the membership functions of the premise part. Therefore, (𝑌,𝑑_∞) separates points on 𝑈.

Lemma 7. For each 𝑥∈𝑈, there exists 𝑓∈𝑌 such that 𝑓(𝑥)≠0; that is, 𝑌 vanishes at no point of  𝑈.

Finally, we prove that (𝑌,𝑑_∞) vanishes at no point of  𝑈. By observing (8)–(11), (20), and (21), we simply choose all 𝑦^𝑘(𝑥)>0 (𝑘=1,2,…,𝑀); that is, any 𝑓∈𝑌 with 𝑦^𝑘(𝑥)>0 serves as the required 𝑓.

Proof of Theorem 2. From (20) and (21), it is evident that 𝑌 is a set of real continuous functions on 𝑈, which are established by using complete interval type-2 fuzzy sets in the IF parts of fuzzy rules. Using Lemmas 3, 4, and 5, 𝑌 is proved to be an algebra. By using the Stone-Weierstrass theorem together with Lemmas 6 and 7, we establish that the proposed IT2FNN-0 possesses the universal approximation capability.

3.3. Applying the Stone-Weierstrass Theorem to the IT2FNN-2 Architecture

We now consider a subset of the IT2FNN-2 on Figure 2. The set of IT2FNN-2 with singleton fuzzifier, product inference, type-reduction defuzzifier (KM) [13, 14], and Gaussian interval type-2 membership function consists of all FBF expansion functions. 𝑓∶𝑈⊂𝑅^𝑛→𝑅, 𝑥=(𝑥₁,𝑥₂,…,𝑥_𝑛)∈𝑈; 𝜇𝐹^𝑘_𝑖(𝑥)∈[𝜇𝐹^𝑘_𝑖(𝑥),𝜇𝐹^𝑘_𝑖(𝑥)] is the Gaussian interval type-2 membership function, igaussmtype2 (𝑥,[𝜎^𝑘_𝑖,¹𝑚^𝑘_𝑖,¹𝑚^𝑘_𝑖]), defined by (8)–(11). If we view 𝜙^𝑘_𝑙(𝑥), 𝜙^𝑘_𝑙(𝑥), 𝜙^𝑘_𝑟(𝑥), 𝜙^𝑘_𝑟(𝑥) as basis functions (44), (46), (49), (50) and 𝑦^𝑘_𝑙,𝑦^𝑘_𝑟 are linear functions (41), then 𝑦(𝑥) can be viewed as a linear combination of the basis functions. Let 𝑌 be the set of all the FBF expansions with 𝜙^𝑘_𝑙(𝑥), 𝜙^𝑘_𝑙(𝑥), 𝜙^𝑘_𝑟(𝑥), 𝜙^𝑘_𝑟(𝑥) and let 𝑑_∞(𝑓₁,𝑓₂)=sup_𝑥∈𝑈(|𝑓₁(𝑥)−𝑓₂(𝑥)|) be the supmetric; then, (𝑌,𝑑_∞) is a metric space [38]. The following theorem shows that (𝑌,𝑑_∞) is dense in (𝐶[𝑈],𝑑_∞), where 𝐶[𝑈] is the set of all real continuous functions defined on 𝑈. We use the following Stone-Weierstrass theorem to prove the theorem.

Suppose we have two IT2FNN-2s 𝑓₁,𝑓₂∈𝑌; the output of each system can be expressed as 𝑓₁(𝑥)=𝛾𝑦¹_𝑙(𝑥)+1−𝛾𝑦¹_𝑟(𝑥), (40) where 𝑦¹_𝑙(𝑥)=^𝐿1_𝑘1¹₌₁𝜙^𝑘1_𝑙(𝑥)¹𝑧^𝑘1_𝑙(𝑥)+^𝑀1_𝑘1=𝐿1¹₊₁𝜙^𝑘1_𝑙(𝑥)¹𝑧^𝑘1_𝑙(𝑥)=∑^𝐿1_𝑘1=1𝑤^𝑘1₁(𝑥)¹𝑧^𝑘1_𝑙(𝑥)+∑^𝑀1_𝑘=𝐿1+1𝑤^𝑘1₁(𝑥)¹𝑧^𝑘1_𝑙(𝑥)𝐷¹_𝑙,𝑦¹_𝑟(𝑥)=^𝑅1_𝑘1¹₌₁𝜙^𝑘1_𝑟(𝑥)¹𝑧^𝑘1_𝑟(𝑥)+^𝑀1_𝑘1=𝑅1¹₊₁𝜙^𝑘1_𝑟(𝑥)¹𝑧^𝑘1_𝑟(𝑥)=∑^𝑅1_𝑘1=1𝑤^𝑘1₁(𝑥)¹𝑧^𝑘1_𝑟(𝑥)+∑^𝑀1_{𝑘1=𝑅1+1}𝑤^𝑘1₁(𝑥)¹𝑧^𝑘1_𝑟(𝑥)𝐷¹_𝑟, (41) where 𝑤^𝑘1₁=^𝑛_𝑖=1𝜇₁𝐹^𝑘1_𝑖(𝑥),𝑤^𝑘1₁=^𝑛_𝑖=1𝜇₁𝐹^𝑘1_𝑖(𝑥),𝐷¹_𝑙=^𝐿1_𝑘1=1𝑤^𝑘1₁+^𝑀1_{𝑘1=𝐿1+1}𝑤^𝑘1₁,𝐷¹_𝑟=^𝑅1_𝑘1=1𝑤^𝑘1₁+^𝑀1_{𝑘1=𝑅1+1}𝑤^𝑘1₁,₁𝜙^𝑘1_𝑙(𝑥)=𝑤^𝑘1₁𝐷¹_𝑙∀𝑘₁=1,…,𝐿₁(𝑥),₁𝜙^𝑘1_𝑙(𝑥)=𝑤^𝑘1₁𝐷¹_𝑙∀𝑘₁=𝐿₁(𝑥)+1,…,𝑀₁,₁𝜙^𝑘1_𝑟(𝑥)=𝑤^𝑘1₁𝐷¹_𝑟∀𝑘₁=1,…,𝑅₁(𝑥),₁𝜙^𝑘1_𝑟(𝑥)=𝑤^𝑘1₁𝐷¹_𝑟∀𝑘₁=𝑅₁(𝑥)+1,…,𝑀₁,₁𝑧^𝑘1_𝑙=^𝑛1_𝑖=1𝑐^𝑘1_𝑖𝑥_𝑖+¹𝑐^𝑘₀−^𝑛1_𝑖=1𝑠^𝑘1_𝑖𝑥_𝑖−¹𝑠^𝑘1₀,𝑘₁=1,…,𝑀₁,₁𝑧^𝑘1_𝑟=^𝑛1_𝑖=1𝑐^𝑘1_𝑖𝑥_𝑖+¹𝑐^𝑘1₀+^𝑛1_𝑖=1𝑠^𝑘1_𝑖𝑥_𝑖+¹𝑠^𝑘1₀,𝑘₁=1,…,𝑀₁,𝑓₂(𝑥)=𝛾𝑦²_𝑙(𝑥)+1−𝛾𝑦²_𝑟(𝑥), (42) where 𝑦²_𝑙(𝑥)=^𝐿2_𝑘2²₌₁𝜙^𝑘2_𝑙(𝑥)²𝑧^𝑘2_𝑙(𝑥)+^𝑀2_𝑘2=𝐿2²₊₁𝜙^𝑘2_𝑙(𝑥)²𝑧^𝑘2_𝑙(𝑥)=∑^𝐿2_𝑘2=1𝑤^𝑘2₂(𝑥)²𝑧^𝑘2_𝑙(𝑥)+∑^𝑀2_{𝑘2=𝐿2+1}𝑤^𝑘2₂(𝑥)²𝑧^𝑘2_𝑙(𝑥)𝐷²_𝑙,𝑦²_𝑟(𝑥)=^𝑅2_𝑘2²₌₁𝜙^𝑘2_𝑟(𝑥)²𝑧^𝑘2_𝑟(𝑥)+^𝑀2_𝑘2=𝑅2²₊₁𝜙^𝑘2_𝑟(𝑥)²𝑧^𝑘2_𝑟(𝑥)=∑^𝑅21_𝑘2=1𝑤^𝑘2₂(𝑥)²𝑧^𝑘2_𝑟(𝑥)+∑^𝑀2_{𝑘2=𝑅2+1}𝑤^𝑘2₂(𝑥)²𝑧^𝑘2_𝑟(𝑥)𝐷¹_𝑟, (43)(44) where 𝑤^𝑘2₂=^𝑛_𝑖=1𝜇₂𝐹^𝑘2_𝑖(𝑥),𝑤^𝑘2₂=^𝑛_𝑖=1𝜇₂𝐹^𝑘2_𝑖(𝑥),𝐷²_𝑙=^𝐿2_𝑘2=1𝑤^𝑘2₂+^𝑀2_{𝑘2=𝐿2+1}𝑤^𝑘2₂,𝐷²_𝑟=^𝑅2_𝑘2=1𝑤^𝑘2₂+^𝑀2_{𝑘2=𝑅2+1}𝑤^𝑘2₂,₂𝜙^𝑘2_𝑙(𝑥)=𝑤^𝑘2₂𝐷²_𝑙∀𝑘₂=1,…,𝐿₂(𝑥),₂𝜙^𝑘2_𝑙(𝑥)=𝑤^𝑘2₂𝐷²_𝑙∀𝑘₂=𝐿₂(𝑥)+1,…,𝑀₂,₂𝜙^𝑘2_𝑟(𝑥)=𝑤^𝑘2₂𝐷²_𝑟∀𝑘₂=1,…,𝑅₂(𝑥),₂𝜙^𝑘2_𝑟(𝑥)=𝑤^𝑘2₂𝐷²_𝑟∀𝑘₂=𝑅₂(𝑥)+1,…,𝑀₂,₂𝑧^𝑘2_𝑙=^𝑛2_𝑖=1𝑐^𝑘2_𝑖𝑥_𝑖+²𝑐^𝑘2₀−^𝑛2_𝑖=1𝑠^𝑘2_𝑖𝑥_𝑖−²𝑠^𝑘2₀,𝑘₂=1,…,𝑀₂,₂𝑧^𝑘2_𝑟=^𝑛2_𝑖=1𝑐^𝑘2_𝑖𝑥_𝑖+²𝑐^𝑘2₀+^𝑛2_𝑖=1𝑠^𝑘2_𝑖𝑥_𝑖+²𝑠^𝑘2₀,𝑘₂=1,…,𝑀₂. (45)

Lemma 8. 𝑌 is closed under addition.

Proof. The proof of this lemma requires our IT2FNN-2 to be able to approximate sums of functions. Suppose we have two IT2FNN-2s 𝑓₁(𝑥) and 𝑓₂(𝑥) with rules 𝑀₁ and 𝑀₂, respectively. The output of each system can be expressed as 𝑓₁(𝑥)+𝑓₂(𝑥)=^𝐿1_𝑘1^𝐿₌₁²_𝑘2=1𝛼𝐷²_𝑙𝑤^𝑘1₁¹𝑧^𝑘1_𝑙(𝑥)+𝛾𝐷¹_𝑙𝑤^𝑘2₂²𝑧^𝑘2_𝑙(𝑥×𝐷_1𝑙)𝐷²_𝑙⁻¹+^𝑀1_𝑘1=𝐿1^𝑀₊₁²_{𝑘2=𝐿2+1}𝛼𝐷²_𝑙𝑤^𝑘1₁¹𝑧^𝑘1_𝑙(𝑥)+𝛾𝐷¹_𝑙𝑤^𝑘2₂²𝑧^𝑘2_𝑙(𝑥×𝐷_1𝑙)𝐷²_𝑙⁻¹+^𝑅1_𝑘1^𝑅₌₁²_𝑘2=1(1−𝛼)𝐷²_𝑟𝑤^𝑘1₁¹𝑧^𝑘1_𝑟(𝑥)+1−𝛾𝐷¹_𝑟𝑤^𝑘2₂²𝑧^𝑘2_𝑟(𝑥)×𝐷¹_𝑟𝐷²_𝑟⁻¹+^𝑀1_𝑘1=𝑅1^𝑀₊₁²_{𝑘2=𝑅2+1}(1−𝛼)𝐷²_𝑟𝑤^𝑘1₁¹𝑧^𝑘1_𝑟(𝑥)+1−𝛾𝐷¹_𝑟𝑤^𝑘2₂²𝑧^𝑘2_𝑟(𝑥×𝐷_1𝑟)𝐷²_𝑟⁻¹. (46) Therefore, an equivalent to IT2FNN-2 can be constructed under the addition of 𝑓₁(𝑥) and 𝑓₂(𝑥), where the consequents form an addition of 𝛼¹𝑧^𝑘1_𝑙+𝛾²𝑧^𝑘2_𝑙 and (1−𝛼)¹𝑧^𝑘1_𝑟+(1−𝛾)²𝑧^𝑘2_𝑟 multiplied by a respective FBFs expansion (Theorem 1), and there exists 𝑓∈𝑌 such that sup_𝑥∈𝑈(|𝑔(𝑥)−𝑓(𝑥)|)<𝜀 (Theorem 2). Since 𝑓(𝑥) satisfies Lemma 3 and 𝑌∈𝑓(𝑥)=𝑓₁(𝑥)+𝑓₂(𝑥) then we can conclude that 𝑌 is closed under addition. Note that 𝑧^𝑘1₁ and 𝑧^𝑘2₂ can be linear interval since the FBFs are a nonlinear basis and therefore the resultant function, 𝑓(𝑥), is nonlinear interval (see Figure 6).

(a) Antecedent IT2MFs for fuzzy rules

(b) Overall I/O curve for interval rules

(c) An example of the interval type-2 fuzzy basis functions

(d) Overall I/O curve for IT2FNN-2

Figure 6 

An example of the IT2FNN-2 architecture.

Lemma 9. 𝑌 is closed under multiplication.

Proof. In a similar way to Lemma 8, we model the product of 𝑓₁(𝑥)𝑓₂(𝑥) of two IT2FNN-2s which is the last point we need to demonstrate before we can conclude that the Stone-Weierstrass theorem can be applied to the proposed reasoning mechanism. The product 𝑓₁(𝑥)𝑓₂(𝑥) can be expressed as 𝑓₁(𝑥)𝑓₂(𝑥)=𝛼𝛾𝐷¹_𝑙𝐷²_𝑙×^𝐿1_𝑘1^𝐿₌₁²_𝑘2=1𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑙(𝑥)²𝑧^𝑘2_𝑙(𝑥)+^𝐿1_𝑘1^𝑀₌₁²_{𝑘2=𝐿2+1}𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑙(𝑥)²𝑧^𝑘2_𝑙(𝑥)+^𝑀1_𝑘1=𝐿1^𝐿₊₁²_𝑘2=1𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑙(𝑥)²𝑧^𝑘2_𝑙(𝑥)+^𝑀1_𝑘1=𝐿1^𝑀₊₁²_{𝑘2=𝐿2+1}𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑙(𝑥)²𝑧^𝑘2_𝑙(𝑥)+𝛼1−𝛾𝐷¹_𝑙𝐷²_𝑟×^𝐿1_𝑘1^𝑅₌₁²_𝑘2=1𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑙(𝑥)²𝑧^𝑘2_𝑟(𝑥)+^𝐿1_𝑘1^𝑀₌₁²_{𝑘2=𝑅2+1}𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑙(𝑥)²𝑧^𝑘2_𝑟(𝑥)+^𝑀1_𝑘1=𝐿1^𝑅₊₁²_𝑘2=1𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑙(𝑥)²𝑧^𝑘2_𝑟(𝑥)+^𝑀1_𝑘1=𝐿1^𝑀₊₁²_{𝑘2=𝑅2+1}𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑙(𝑥)²𝑧^𝑘2_𝑟(𝑥)+(1−𝛼)𝛾𝐷¹_𝑟𝐷²_𝑙×^𝑅1_𝑘1^𝐿₌₁²_𝑘2=1𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑟(𝑥)²𝑧^𝑘2_𝑙(𝑥)+^𝑅1_𝑘1^𝑀₌₁²_{𝑘2=𝐿2+1}𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑟(𝑥)²𝑧^𝑘2_𝑙(𝑥)+^𝑀1_𝑘1=𝑅1^𝐿₊₁²_𝑘2=1𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑟(𝑥)²𝑧^𝑘2_𝑙(𝑥)+^𝑀1_𝑘1=𝑅1^𝑀₊₁²_{𝑘2=𝑅2+1}𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑟(𝑥)²𝑧^𝑘2_𝑙(𝑥)+(1−𝛼)1−𝛾𝐷¹_𝑟𝐷²_𝑟×^𝑅1_𝑘1^𝑅₌₁²_𝑘2=1𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑟(𝑥)²𝑧^𝑘2_𝑟(𝑥)+^𝑅1_𝑘1^𝑀₌₁²_{𝑘2=𝑅2+1}𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑟(𝑥)²𝑧^𝑘2_𝑟(𝑥)+^𝑀1_𝑘1=𝑅1^𝑅₊₁²_𝑘2=1𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑟(𝑥)²𝑧^𝑘2_𝑟(𝑥)+^𝑀1_𝑘1=𝑅1^𝑀₊₁²_{𝑘2=𝑅2+1}𝑤^𝑘1₁(𝑥)𝑤^𝑘2₂(𝑥)¹𝑧^𝑘1_𝑟(𝑥)²𝑧^𝑘2_𝑟(𝑥). (47) Therefore, an equivalent to IT2FNN-2 can be constructed under the multiplication of 𝑓₁(𝑥) and 𝑓₂(𝑥), where the consequents form an addition of 𝛼𝛾¹𝑧^𝑘1_𝑙²𝑧^𝑘2_𝑙, 𝛼(1−𝛾)¹𝑧^𝑘1_𝑙²𝑧^𝑘2_𝑟, (1−𝛼)𝛾¹𝑧^𝑘1_𝑟²𝑧^𝑘2_𝑙, and (1−𝛼)(1−𝛾)¹𝑧^𝑘1_𝑟²𝑧^𝑘2_𝑟 multiplied by a respective FBFs expansion (Theorem 1), and there exists 𝑓∈𝑌 such that sup_𝑥∈𝑈(|𝑔(𝑥)−𝑓(𝑥)|)<𝜀 (Theorem 2). Since 𝑓(𝑥) satisfies Lemma 3 and 𝑌∈𝑓(𝑥)=𝑓₁(𝑥)𝑓₂(𝑥) then we can conclude that 𝑌 is closed under multiplication. Note that 𝑧^𝑘1₁ and 𝑧^𝑘2₂ can be linear intervals since the FBFs are a nonlinear basis interval and therefore the resultant function, 𝑓(𝑥), is nonlinear interval. Also, even if 𝑧^𝑘1₁ and 𝑧^𝑘2₂ were linear, their product 𝑧^𝑘1₁𝑧^𝑘2₂ is evidently polynomial interval (see Figure 10).

Lemma 10. 𝑌 is closed under scalar multiplication.

Proof. Let an arbitrary IT2FNN-2 be 𝑓(𝑥) (51); the scalar multiplication of 𝑐𝑓(𝑥) can be expressed as 𝑐𝑓₁(𝑥)=𝛼𝑐𝑦¹_𝑙(𝑥)+(1−𝛼)𝑐𝑦¹_𝑟(𝑥)=∑^𝐿1_𝑘1=1𝑤^𝑘1₁(𝑥)𝛼𝑐¹𝑧^𝑘1_𝑙(𝑥)+∑^𝑀1_𝑘=𝐿1+1𝑤^𝑘1₁(𝑥)𝛼𝑐¹𝑧^𝑘1_𝑙(𝑥)𝐷¹_𝑙+∑^𝑅1_𝑘1=1𝑤^𝑘1₁(𝑥)(1−𝛼)𝑐¹𝑧^𝑘1_𝑟(𝑥)𝐷¹_𝑟+∑^𝑀1_{𝑘1=𝑅1+1}𝑤^𝑘1₁(𝑥)(1−𝛼)𝑐¹𝑧^𝑘1_𝑟(𝑥)𝐷¹_𝑟. (48) Therefore we can construct an IT2FNN-2 that computes all FBF expansion combinations with 𝛼𝑐¹𝑧^𝑘1_𝑙(𝑥) and (1−𝛼)𝑐¹𝑧^𝑘1_𝑟(𝑥) in the form of the proposed IT2FNN-2, and 𝑌 is closed under scalar multiplication.

Lemma 11. For every (𝐱⁰,𝐲⁰)∈𝑈 and 𝐱⁰≠𝐲⁰, there exists 𝑓∈𝑌 such that 𝑓(𝐱⁰)≠𝑓(𝐲⁰); that is, 𝑌 separates points on 𝑈.

We prove this by constructing a required 𝑓; that is, we specify 𝑓∈𝑌 such that 𝑓(𝐱⁰)≠𝑓(𝐲⁰) for arbitrarily given 𝐱⁰,𝐲⁰∈𝑈 with 𝐱⁰≠𝐲⁰. We choose two fuzzy rules in the form of (8) for the fuzzy rule base (i.e., 𝑀=2). Let 𝑥⁰=(𝑥⁰₁,𝑥⁰₂,…,𝑥⁰_𝑛) and 𝑦⁰=(𝑦⁰₁,𝑦⁰₂,…,𝑦⁰_𝑛). If 𝑥⁰_𝑖=(𝑥⁰_𝑙𝑖+𝑥⁰_𝑟𝑖)/2 and 𝑦⁰_𝑖=(𝑦⁰_𝑙𝑖+𝑦⁰_𝑟𝑖)/2 with 𝑥⁰_𝑖≠𝑦⁰_𝑖, we define two interval type-2 fuzzy sets (𝐹¹_𝑖,[𝜇𝐹¹_𝑖,𝜇𝐹¹_𝑖]) and (𝐹²_𝑖,[𝜇𝐹²_𝑖,𝜇𝐹²_𝑖]) with 𝜇𝐹¹_𝑖𝑥_𝑖=−1exp2𝑥_𝑖−𝑥⁰_𝑙𝑖²,𝑥_𝑖<𝑥⁰_𝑙𝑖,1,𝑥⁰_𝑙𝑖≤𝑥_𝑖≤𝑥⁰_𝑟𝑖,−1exp2𝑥_𝑖−𝑥⁰_𝑟𝑖²,𝑥_𝑖>𝑥⁰_𝑙𝑖,𝜇𝐹¹_𝑖𝑥_𝑖=−1exp2𝑥_𝑖−𝑥⁰_𝑟𝑖²,𝑥_𝑖≤𝑥⁰_𝑙𝑖,−1exp2𝑥_𝑖−𝑥⁰_𝑙𝑖²,𝑥_𝑖>𝑥⁰_𝑟𝑖,𝜇𝐹²_𝑖𝑥_𝑖=−1exp2𝑥_𝑖−𝑦⁰_𝑙𝑖²,𝑥_𝑖<𝑦⁰_𝑙𝑖,1,𝑦⁰_𝑙𝑖≤𝑥_𝑖≤𝑦⁰_𝑟𝑖,−1exp2𝑥_𝑖−𝑦⁰_𝑟𝑖²,𝑥_𝑖>𝑦⁰_𝑙𝑖,𝜇𝐹^𝑘_𝑖𝑥_𝑖=−1exp2𝑥_𝑖−𝑦⁰_𝑟𝑖²,𝑥_𝑖≤𝑦⁰_𝑙𝑖,−1exp2𝑥_𝑖−𝑦⁰_𝑙𝑖²,𝑥_𝑖>𝑦⁰_𝑟𝑖. (49)(50)(51)(52) If 𝑥⁰_𝑖=𝑦⁰_𝑖, then 𝐹¹_𝑖=𝐹²_𝑖 and 𝜇𝐹¹_𝑖(𝑥⁰_𝑖)=𝜇𝐹²_𝑖(𝑦⁰_𝑖), 𝜇𝐹¹_𝑖(𝑥⁰_𝑖)=𝜇𝐹²_𝑖(𝑦⁰_𝑖); that is, only one interval type-2 fuzzy set is defined. We define two interval value real sets 𝑧¹∈[𝑧¹_𝑙,𝑧¹_𝑟] and 𝑧²∈[𝑧²_𝑙,𝑧²_𝑟]. Now we have specified all the design parameters except [𝑧^𝑘_𝑙,𝑧^𝑘_𝑟]; that is, we have already obtained a function 𝑓 which is in the form of (20), (21) with 𝑀=2 and (𝐹^𝑘_𝑖,[𝜇𝐹^𝑘_𝑖,𝜇𝐹^𝑘_𝑖]) given by (8)–(11). With this 𝑓, we have 𝑓𝑥⁰=𝛼𝜙¹_𝑙𝑥⁰𝑧¹_𝑙𝑥⁰+1−𝜙¹_𝑙𝑥⁰𝑧²_𝑙𝑥⁰+(1−𝛼)𝜙¹_𝑟𝑥⁰𝑧¹_𝑟𝑥⁰+𝜙²_𝑟𝑥⁰𝑧²_𝑟𝑥⁰, (53) where 𝜙¹_𝑙𝑥⁰=11+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑥⁰_𝑖,𝜙¹_𝑟𝑥⁰=∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑥⁰_𝑖∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑥⁰_𝑖+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑥⁰_𝑖,𝜙²_𝑟𝑥⁰=∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑥⁰_𝑖∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑥⁰_𝑖+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑥⁰_𝑖,𝑓𝑦⁰=𝛼𝜙¹_𝑙𝑦⁰𝑧¹_𝑙𝑦⁰+𝜙²_𝑙𝑦⁰𝑧²_𝑙𝑦⁰+()×1−𝛼1−𝜙²_𝑟𝑦⁰𝑧¹_𝑟𝑦⁰+𝜙²_𝑟𝑦⁰𝑧²_𝑟𝑦⁰, (54) where 𝜙²_𝑟𝑦⁰=1∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑦⁰_𝑖,𝜙+1_1𝑙𝑦⁰=∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑦⁰_𝑖∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑦⁰_𝑖+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑦⁰_𝑖,𝜙²_𝑙𝑦⁰=∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑦⁰_𝑖∏^𝑛_𝑖=1𝜇_𝐹1𝑖𝑦⁰_𝑖+∏^𝑛_𝑖=1𝜇_𝐹2𝑖𝑦⁰_𝑖. (55) Since 𝐱⁰≠𝐲⁰, there must be some 𝐢 such that 𝑥⁰_𝑖=𝑦⁰_𝑖; hence, we have ∏^𝑛_𝑖=1𝜇_𝐹1𝑖(𝑥⁰_𝑖)≠1 and ∏^𝑛_𝑖=1𝜇_𝐹2𝑖(𝑥⁰_𝑖)≠1. If we choose 𝑧¹_𝑙𝑟=0 and 𝑧²_𝑙𝑟=1, then 𝑓(𝑥⁰)=𝛼(1−𝜙¹_𝑙(𝑥⁰))+(1−𝛼)𝜙²_𝑟(𝑥⁰)≠𝛼𝜙²_𝑙(𝑦⁰)+(1−𝛼)𝜙²_𝑟(𝑦⁰)=𝑓(𝑦⁰). Therefore, (𝑌,𝑑_∞) separates point on 𝑈.

Lemma 12. For each 𝑥∈𝑈, there exists 𝑓∈𝑌 such that 𝑓(𝑥)≠0; that is, 𝑌 vanishes at no point of  𝑈.

Finally, we prove that (𝑌,𝑑_∞) vanishes at no point of  𝑈. By observing (8)–(11), (20), and (21), we just choose all 𝑧^𝑘>0 (𝑘=1,2,…,𝑀); that is, any 𝑓∈𝑌 with 𝑧^𝑘>0 serves as required 𝑓.

Proof of Theorem 2. From (20) and (21), it is evident that 𝑌 is a set of real continuous functions on 𝑈, which are established by using complete interval type-2 fuzzy sets in the IF parts of fuzzy rules. Using Lemmas 8, 9, and 10, 𝑌 is proved to be an algebra. By using the Stone-Weierstrass theorem together with Lemmas 11 and 12, we establish that the proposed IT2FNN-2 possesses the universal approximation capability.
Therefore by choosing appropriate class of interval type-2 membership functions, we can conclude that the IT2FNN-0 and IT2FNN-2 with simplified fuzzy if-then rules satisfy the five criteria of the Stone-Weierstrass theorem.

4. Application Examples

In this section the results from simulations using ANFIS, IT2FNN-0, IT2FNN-1 [35], IT2FNN-2, and IT2FNN-3 [35] are presented for nonlinear system identification and forecasting the Mackey-Glass chaotic time series [39] with 𝜏=60 with different signal noise ratio values, SNR(dB) = 0, 10, 20, 30, free as uncertainty source. These examples are used as benchmark problems to test the proposed ideas in the paper. We have to mention that the IT2FNN-1 and IT2FNN-3 architectures are very similar to I2FNN-0 and IT2FNN-2, respectively [35], and their results are presented for comparison purposes. The proposed IT2FNN architectures are validated using 10-fold cross-validation [40, 41] considering sum of square errors (SSE) or root mean square error (RMSE) in the training or test phase. We use cross-validation to measure the variability of the RMSE in the training and testing phases to compare network architectures IT2FNN. Cross-validation procedure evaluation is done using Matlab’s crossvalind function. Noise is added by Matlab’s awgn function.

In 𝐾-fold cross-validation [40], the original sample is randomly partitioned into 𝐾 subsamples. Of the 𝐾 subsamples, a single subsample is retained as the validation data for testing the model, and the remaining 𝐾−1 subsamples are used as training data. The cross-validation process is then repeated 𝐾 times (the folds), with each of the 𝐾 subsamples used exactly once as the validation data. The 𝐾 results from the folds then can be averaged (or otherwise combined) to produce a single estimation. The advantage of this method over repeated random subsampling is that all observations are used for both training and validation, and each observation is used for validation exactly once. 10-fold cross-validation is commonly used. Three application examples are used to illustrate proofs of universality, as follows.

Experiment 1 (identification of a one variable nonlinear function). In this experiment we approximate a nonlinear function 𝑓∶ℜ→ℜ: 𝑓(𝑢)=0.6sin(𝜋𝑢)+0.3sin(3𝜋𝑢)+0.1sin(5𝜋𝑢)+𝜂, (56) (where 𝜂 is a uniform noise component) using a-one input one-output IT2FNN, 50 training data sets with 10-fold cross-validation with uniform noise levels, six IT2MFs type igaussmtype2, 6 rules, and 50 epochs. Once the ANFIS and IT2FNN models are identified a comparison was made, taking into account RMSE statistic values with 10-fold cross-validation. Table 1 and Figure 7 show the resulting RMSE (CHK) values for ANFIS and IT2FNN; it can be seen that IT2FNN architectures [31] perform better than ANFIS.

Table 1 

RMSE (CHK) values of ANFIS and IT2FNN with 10-fold cross-validation for identifying non-linearity of Experiment 1.

Figure 7 

RMSE (CHK) values of ANFIS and IT2FNN using 10-fold cross-validation for identifying nonlinearity in Experiment 1.

Experiment 2 (identification of a three variable nonlinear function). A three-input one-output IT2FNN is used to approximate nonlinear Sugeno [27] function 𝑓∶ℜ³→ℜ: 𝑓𝑥₁,𝑥₂,𝑥₃=1+√𝑥₁+1𝑥₂+1𝑥³₃²+𝜂. (57) 216 training data sets are generated with 10-fold cross-validation and 125 for tests; 2 igaussmtype2 IT2MFs for each input, 8 rules, and 50 epochs. Once the ANFIS and IT2FNN models are identified, a comparison is made with RMSE statistic values and 10-fold cross-validation. Table 2 and Figure 8 show the resultant RMSE (CHK) values for ANFIS and IT2FNN. It can be seen that IT2FNN architectures [31] perform better than ANFIS.

Table 2 

Resulting RMSE (CHK) values in ANFIS and IT2FNN for non-linearity identification in Experiment 2 with 10-fold cross-validation.

Figure 8 

Resulting RMSE (CHK) values obtained by ANFIS and IT2FNN for nonlinearity identification in Experiment 2 with 10-fold cross-validation.

Experiment 3. Predicting the Mackey-Glass chaotic time series.
Mackey-Glass chaotic time series is a well-known benchmark [39] for systems modeling and is described as follows: 𝑥̇(𝑡)=0.1𝑥(𝑡−𝜏)1+𝑥¹⁰()(𝑡).𝑡−𝜏−0.1𝑥 (58) 1200 data sets are generated based on initial conditions 𝑥(0)=1.2 and 𝜏=60, using fourth order Runge-Kutta method adding different levels of uniform noise. For comparing with other methods, an input-output vector is chosen for IT2FNN model with the following format: [𝑥(𝑡−18),𝑥(𝑡−12),𝑥(𝑡−6),𝑥(𝑡);𝑥(𝑡+6)]. (59) Four-input and one-output IT2FNN model is used for Mackey-Glass chaotic time series prediction, choosing 500 data sets for training and 500 test data data sets with 10-fold cross-validation test, 2 IT2MFs for each input with membership function igaussmtype2, 16 rules, and 50 epochs. ANFIS and IT2FNN models are identified, comparing RMSE statistical values with 10-fold cross-validation. Table 3 and Figures 9 and 10 show the number of 𝜍 points out of uncertainty interval 𝑌(𝑥)∈[𝑦_𝑙(𝑥),𝑦_𝑟(𝑥)] evaluated by IT2FNN model, RMSE training values (TRN) and test (CHK) obtained for ANFIS and IT2FNN models. It can be seen that IT2FNN model architectures predict better Mackey-Glass chaotic time series.

Table 3 

RMSE (TRN/CHK) and 𝜍 values determined by ANFIS and IT2FNN models with 10 fold cross-validation for Mackey-Glass chaotic time series prediction with 𝜏=60.

Figure 9 

RMSE (TRN) values determined by ANFIS and IT2FNN models with 10-fold cross-validation for Mackey-Glass chaotic time series prediction with 𝜏=60.

Figure 10 

RMSE (CHK) values determined by ANFIS and IT2FNN models with 10-fold cross-validation for Mackey-Glass chaotic time series prediction with 𝜏=60.

5. Conclusions

In this paper we have shown that an interval type-2 fuzzy neural network (IT2FNN) is a universal approximator. Simulation results of nonlinear function identification using the IT2FNN for one and three variables and for the Mackey-Glass chaotic time series prediction have been presented to illustrate the theoretical result. In these experiments, the estimated RMSE values for nonlinear function identification with 10-fold cross-validation for the hybrid architectures (IT2FNN-2:A2C0 and IT2FNN-2:A2C1) illustrate the proof based on Stone-Weierstrass theorem, that they are universal approximators for efficient identification of nonlinear functions, complying with |𝑔(𝑥)−𝑓(𝑥)|<𝜀. Also, it can be seen that while increasing the Signal Noise Ratio (SNR), IT2FNN architectures handle uncertainty more efficiently. We have also illustrated the ideas presented in the paper with the benchmark problem of Mackey-Glass chaotic time series prediction.

Acknowledgments

The authors would like to thank CONACYT and DGEST for the financial support given for this research project. The student J. R. Castro was supported by a scholarship from MYCDI, UABC-CONACYT.