Estimation of Fuzzy Measures Using Covariance Matrices in Gaussian Mixtures

Abstract

This paper presents a novel computational approach for estimating fuzzy measures directly from Gaussian mixtures model (GMM). The mixture components of GMM provide the membership functions for the input-output fuzzy sets. By treating consequent part as a function of fuzzy measures, we derived its coefficients from the covariance matrices found directly from GMM and the defuzzified output constructed from both the premise and consequent parts of the nonadditive fuzzy rules that takes the form of Choquet integral. The computational burden involved with the solution of λ-measure is minimized using Q-measure. The fuzzy model whose fuzzy measures were computed using covariance matrices found in GMM has been successfully applied on two benchmark problems and one real-time electric load data of Indian utility. The performance of the resulting model for many experimental studies including the above-mentioned application is found to be better and comparable to recent available fuzzy models. The main contribution of this paper is the estimation of fuzzy measures efficiently and directly from covariance matrices found in GMM, avoiding the computational burden greatly while learning them iteratively and solving polynomial equations of order of the number of input-output variables.

1. Introduction

Generalized fuzzy model (GFM) [1–3] is the backbone of this work that employs two norms for computing the strength of a rule: the multiplicative T-norm operator for determining the strength of a rule [4–6] and the additive S-norm operator for combining the outputs of all the rules. The effect of input fuzzy sets is taken into the defuzzified output in the form of rule strengths or weights. Gan et al. [7] have simplified the formulation of GFM by setting both the inputs and the output to be jointly Gaussian and proved that the input-output relation is as an expectation using the Bayesian framework. On simplification, this turns out to be the coveted Gaussian mixture model (GMM), which is linked by an additive function between the inputs and the output. The GMM, also known as cluster-weighted modeling (CWM) [8–12], is advocated by many researchers as means of statistical modeling of input-output systems [10, 13–15]. After the establishment of equivalence between GMM and GFM in [7], the difficult-to-compute index of fuzziness of GFM has become easy-to-compute prior probability [16–23] in GMM. Further simplification of GFM will be explored when it is converted into nonadditive case.

The use of GMM in GFM has provided a generalized framework for additive fuzzy systems. A few existing additive fuzzy systems urging our attention are due to fuzzy models given in Kosko [24] and Wang and Mendel [25]. Using the framework of GMM, Abonyi et al. [26] have made a semblance of generalizing TS model [4, 27, 28] by attaching weight to each of its rules and subsequently treating it as the strength of the rule. However, nowhere they touched upon the concept of GFM that seeks to unify both Mamdani and TS models [4, 5].

Incorporation of the nonadditive property into the fuzzy sets is done so that the corresponding output of a non-additive fuzzy system is explored here. Let us throw some light on the additive and non-additive fuzzy systems [29–31]. Conventional evaluation with multiple attributes which are independent is based on the concept of additive fuzzy systems where importance of each attribute is given a weight.

Some important applications of fuzzy measures are now mentioned. A random generation of fuzzy measures have been introduced and some subfamilies of fuzzy measures tackled in [32]. A set of isometric fuzzy measures for any isometric transformation in each family of fuzzy measures (e.g., convex) is also studied. In [33] fuzzy measure is shown to be a unifying structure for modeling knowledge about an uncertain variable. Kim Le in [34] presents an expression to evaluate a fuzzy measure from set of aggregated evidences. Hierarchical autoregression model is presented in [34] using nonmonotonic fuzzy measures and the Choquet integral. Narukawa et al. in [35] discuss a space of fuzzy measures bearing the topology introduced by Choquet integral and the space of nonnegative fuzzy measure is shown to be locally convex. Fuzzy measures and integrals (e.g., Sugeno and Choquet) are used in decision making and modeling auctions [36, 37]. Fuzzy measures also find an application in [38] to track a moving object from a dynamic image sequence.

Fascinated by the ever growing importance of the fuzzy measure theory [39–45], the additive function of the GFM model in the consequent part is replaced by a nonadditive function satisfying certain axioms of the fuzzy measures. The defuzzified output of the resultant non-additive GFM is shown to be in the form of Choquet integral [41]. This formulation is intended for real-life applications in which the information from different sources needs not to be additive; hence our efforts in this work will go a long way in evolving different types of non-additive fuzzy systems, like dynamic, adaptive, and so forth, but here our attempt is only on a simple non-additive fuzzy system. To take account of the overlapping information in the fuzzy sets, Sugeno [40] has introduced the concept of 𝜆-measure such that an appropriate value of 𝜆 can account for the interaction. In the proposed formulation, 𝑄-measure takes the role of this 𝜆-measure for reducing the computational complexity of the former.

Fuzzy integrals have come into vogue for the information fusion as they aggregate information from several sources. Out of all fuzzy integrals [40–44], the Choquet fuzzy integral [39, 41, 45] and Sugeno integral [46] are widely used. A handful of applications of fuzzy integrals include: handwriting recognition [47], landmine detection [48], pattern recognition [49], and decision making [37]. In the present study, the Choquet fuzzy integral is favored for it employs the non-additive property of fuzzy measures. The Choquet integral has not lost its luster even now; instead it is gaining prominence day by day and making inroads into new applications.

The main theme of the present work is the estimation of fuzzy densities, and hence fuzzy measures for the non-additive fuzzy model are derived from GFM; thus the fuzzy densities can be calculated straightaway from the covariance matrices used in GMM [50]. As the large number of input variables is increased for the purpose of fusion, the computational complexity grows exponentially [31, 51] with the 𝜆-measure. To overcome this problem a new fuzzy measure known as 𝑄-Measure is introduced in [52]. This new measure is just normalization of 𝜆-measure, and with this modification, the fuzzy densities are decorrelated. In this 𝜆 is taken as a variable to be learned using the training data set. The proposed modeling technique considerably reduces the computational complexity in two steps: firstly finding fuzzy densities without learning them through iterative process and secondly using 𝑄-measure that eliminates the burden of solving the exponentially increasing polynomials.

This paper is organized as follows. Section 2 briefly describes the fuzzy measures. In Section 3, the formulation for the non-additive GFM is presented. An algorithm for non-additive fuzzy modeling based on 𝑄-measure is given in Section 4. Two benchmark applications and one real-time electric load demand prediction application are provided in Section 5 to illustrate the proposed methodology. Finally conclusions are drawn in Section 6.

2. Fuzzy Measures

The fuzzy measure or fuzzy capacity is a subjective evaluation introduced by Choquet in 1953 and defined by Sugeno in 1974 for fuzzy integrals. Fuzzy measure includes a number of special class of measures like Sugeno’s 𝜆-measure and 𝑄-measure. In this work, we make use of Fuzzy measures in quantifying the influence of the interactive inputs on the output function using the fuzzy measures. The resulting model is non-additive in the sense that the information from the interactive inputs is computed non-additively. For converting GFM into a non-additive fuzzy system, the relevant properties of the fuzzy measures [46].

Definition 1. Let us denote 𝑋={𝑥₁,𝑥₂,…,𝑥_𝐷} as a finite set with its elements sorted from 𝐱 and let 𝜒 be the power set of 𝑋, that is, set of all subsets of 𝑋. Then a fuzzy measure over a set 𝑋 is a function given by 𝜁∶𝜒⟶[0,1]. (1) Satisfying the two properties(1)𝜁(𝜙)=0,𝜁(𝑋)=1.(2)If 𝑋_𝑖, 𝑋_𝑗⊂𝜒 and 𝑋_𝑖⊂𝑋_𝑗, then 𝜁(𝑋_𝑖)≤𝜁(𝑋_𝑗), which is the monotonicity relation in fuzzy measure. Next we invoke the third property, namely, the so-called 𝜆-measure due to Sugeno [40] stated as follows:(3)for all 𝑋_𝑖, 𝑋_𝑗⊂𝜒 with 𝑋_𝑖∩𝑋_𝑗=𝜙𝜁𝑋_𝑖∪𝑋_𝑗=𝜁𝑋_𝑖+𝜁𝑋_𝑗+𝜆𝜁𝑋_𝑖𝜁𝑋_𝑗. (2)

Example 2. Let 𝑋_𝑑={𝑥_𝑑,𝑥_𝑑+1,…,𝑥_𝐷} be the input sets satisfying the properties 1 and 2; then considering 𝑔 as a fuzzy density satisfying the property 3, the values of fuzzy measure, 𝜁(𝑋_𝑑) can be computed recursively as 𝜁𝑋_𝑑=𝜁𝑥_𝑑∪𝑋_𝑑+1=𝑔_𝑑+𝜁𝑋_𝑑+1+𝜆𝑔_𝑑𝜁𝑋_𝑑+1,for1≤𝑑<𝐷, (3) starting with the one-input set to find the first measure as 𝜁𝑋_𝐷=𝜁𝑥_𝐷=𝑔_𝐷. (4) It is then combined with the next input set {𝑥_𝐷−1} to compute their 𝜆-measure since these two input sets are disjoint, that is, {𝑥_𝐷}∩{𝑥_𝐷−1}=𝜙, but their fuzzy sets 𝐴_𝐷 and 𝐴_𝐷−1 are dependent, that is, 𝐴_𝐷∩𝐴_𝐷−1≠𝜙. Equation (3) indicates that we must consider every time the fuzzy measure of two disjoint input sets, {𝑥_𝑑} and 𝑋_𝑑+1, which consists of inputs 𝑥_𝑑+1,…,𝑥_𝐷. It may be noted that a single element fuzzy measure is the fuzzy density itself. The following polynomial which arises from the condition 𝜁(𝑋)=1 (see the appendix) needs to be solved for 𝜆: 1+𝜆=^𝐷_𝑑=11+𝜆𝑔_𝑑. (5) The solution of (5) is computationally intensive. We will now discuss the 𝑄-measure to address this problem.

Q-Measure
The 𝜆-measure is burdened with the problem of finding the roots of the polynomial equation in 𝜆. As the number of dimensions, 𝐷 increases, the complexity of the equation increases enormously. In order to surmount this problem, 𝑄-measure is proposed in [52]. It derives its name from the word “Quotient.” The 𝑄-measure follows the same properties of 𝜆-measure. For the finite 𝑋 the 𝑄-measure is defined as the ratio of two fuzzy measures: 𝑄𝑋_𝑑=𝜁𝑋_𝑑𝜁(𝑋)=𝜁𝑋_𝑑𝜁𝑋₁, (6) where 𝜁(𝑋)=𝜁(𝑋₁) is a function of fuzzy densities and is a polynomial in 𝜆. The 𝑄-measure [52] over a set 𝑋 is similar to 𝜁(𝑋) in (1): 𝑄∶𝜒⟶[0,1], (7) with the two properties as follows:(1)   𝑄𝜙=0,𝑄(𝑋)=1. (8)(2)If 𝑋_𝑖, 𝑋_𝑗⊂𝜒 and 𝑋_𝑖⊂𝑋_𝑗, then 𝑄(𝑋_𝑖)≤𝑄(𝑋_𝑗), which is the monotonicity relation.The third property is as follows:(3) for all 𝑋_𝑖, 𝑋_𝑗⊂𝜒 with 𝑋_𝑖∩𝑋_𝑗=𝜙, 𝑄𝑋_𝑖∪𝑋_𝑗=𝑄𝑋_𝑖+𝑄𝑋_𝑗+𝜆𝑄𝑋_𝑖𝑄𝑋_𝑗for𝜆>−1. (9) Following (9), 𝑄(𝑋_𝑑) is evaluated from 𝑄𝑋_𝑑=𝑄𝑥_𝑑∪𝑋_𝑑+1=𝑔_𝑑+𝑄𝑋_𝑑+1+𝜆⋅𝑔_𝑑⋅𝑄𝑋_𝑑+1for1≤𝑑<𝐷. (10) The first measure is computed from𝑄𝑋_𝐷=𝑄𝑥_𝐷=𝑔_𝐷. (11)

Convergence Behavior of Q-Measure
Let {𝑥₁}, {𝑥₂,𝑥₃,…,𝑥_𝐷}⊂𝑋 Such that, {𝑥₁}∪{𝑥₂,𝑥₃,…,𝑥_𝐷}=𝑋, and {𝑥₁}∩{𝑥₂,𝑥₃,…,𝑥_𝐷}=𝜙, that is, null. Supposing 𝐺₁=𝜁({𝑥₁}) and 𝐺₂=𝜁𝑥₂,𝑥₃,…,𝑥_𝐷=𝜁𝑋₂∀𝜆≥−1,𝐺₁≥0,𝐺₂≥0. (12) Then the 𝜆-measure of 𝐺₁ and 𝐺₂ is 𝜁𝑋₁=𝐺₁+𝐺₂+𝜆⋅𝐺₁⋅𝐺₂. (13) Using (10) the 𝑄-measure is written as 𝑄𝑋_𝐷−1=𝑔_𝐷−1+𝑄𝑋_𝐷+𝜆⋅𝑔_𝐷−1⋅𝑄𝑋_𝐷, (14) or 𝑄𝑋_𝐷−1=𝑔_𝐷−1+𝑔_𝐷+𝜆⋅𝑔_𝐷−1⋅𝑔_𝐷. (15) For 𝑔_𝐷−1, 𝑔_𝐷∈[0,1] and 𝜆≥−1, it is always assured that 𝑔_𝐷−1+𝑔_𝐷>𝜆⋅𝑔_𝐷−1⋅𝑔_𝐷. (16) Hence, 𝑄(𝑋_𝐷−1)>𝑔_𝐷−1 and also 𝑄(𝑋_𝐷−1)>𝑔_𝐷. Next, we have𝑄𝑋_𝐷−2=𝑔_𝐷−2+𝑄𝑋_𝐷−1+𝜆⋅𝑔_𝐷−2⋅𝑄𝑋_𝐷−1=𝑄𝑋_𝐷−1+𝑔_𝐷−21+𝜆⋅𝑄𝑋_𝐷−1, (17) where 𝜆𝑄(𝑋_𝐷−1)<1. Therefore 𝑔_𝐷−2(1+𝜆⋅𝑄(𝑋_𝐷−1))>0 and the values lie in between 0 and 1. Hence we have 𝑄𝑋_𝐷−2>𝑄𝑋_𝐷−1. (18) This will continue until 𝐷=3 when we will have 𝑄𝑋₁>𝑄𝑋₂. (19) But 𝑄(𝑋₁)=1; this means that the 𝑄-measure converges to unity.

3. From GMM to Nonadditive GFM with Q-Measure

The GFM [7] includes both the Mamdani and TS models. The Mamdani model (CRI-model) inhibits the property of fuzziness around the fixed centroid of the consequent part while the TS model gives a varying singleton for the consequent part in each fuzzy rule. The output of GFM is a fuzzy set, which is the output of Mamdani model with an index of fuzziness and with a varying centroid as the output of the TS model. To combine both of these properties, Azeem et al. [1, 53] have introduced a GFM fuzzy rule of the form𝑅^𝑘∶IF𝐱^𝑘is𝐀^𝑘THEN𝑦_𝑘is𝐁^𝑘𝑓_𝑘(𝐱),𝑣_𝑘. (20)

The fuzzy set 𝐁^𝑘(𝑓_𝑘(𝐱),𝑣_𝑘) may also be in the linguistic form with the output function 𝑓^𝑘(𝐱) which may be linear or nonlinear regression of inputs and the index of fuzziness 𝑣_𝑘. The fuzzy output of the 𝑘th rule is 𝑦_𝑘. The defuzzified output obtained by applying additive S-norm to the consequent parts of GFM rules is𝑦=^𝐾_𝑘=1𝜇^𝑘(𝐱).𝑣_𝑘∑^𝐾_𝑙=1𝜇^𝑙(𝐱)⋅𝑣_𝑙𝑓_𝑘(𝐱), (21) with 𝑓_𝑘(𝐱)=𝑏^𝑘₀+𝑏^𝑘₁𝑥₁+⋯+𝑏^𝑘_𝐷𝑥_𝐷. As noted in Section 1 on introduction, GFM represents the additive fuzzy system. Using the framework of GMM, we intend to generalize to the case when the input fuzzy sets have overlapping information, that is, 𝐴^𝑚_𝑖∩𝐴^𝑚_𝑖+1≠𝜙. So, we modify the above fuzzy rule to the nonadditive GFM as 𝑅^𝑚∶if𝐱^𝑚is𝐀^𝑚suchthat𝐴^𝑚_𝑖∩𝐴^𝑚_𝑖+1≠𝜙.Then𝑦_𝑚is𝐵^𝑚𝑓^′_𝑚(𝑋),𝜐_𝑚with𝑋⊆𝐱. (22) To take account of the above constraint, we form the input set 𝑋 from which the fuzzy measures of subsets 𝑋_𝑑=({𝑥_𝑑}∪𝑋_𝑑) are found recursively using (10) and (11). Using the fuzzy measures as the coefficients the output function is expressed as [31]𝑓^′_𝑚(𝑋)=𝛽^𝑚₀+𝛽^𝑚₁𝑥₁+⋯+𝛽^𝑚_𝐷𝑥_𝐷. (23) Equation (23) is an additive function of fuzzy measures as explained in the sequel. As the input fuzzy sets in (22) are overlapping, we take disjoint input sets from the power set 𝜒 and compute their fuzzy measure. The function 𝑓^′_𝑚(𝑋) combines the fuzzy measures from subsequent fuzzy sets, and the resulting system arising from the fuzzy rules is called nonadditive fuzzy system unlike the output function in TS model where each coefficient corresponds to its own fuzzy set and there is no concept of fuzzy measure in this model. We now impose three constraints for the model to be called non-additive:(i)the input fuzzy sets 𝐴_𝑖 associated with the power set 𝜒 are overlapping,(ii)the input sets 𝑋_𝑖 are disjoint,(iii)the output function is additive in fuzzy measures.

We will now show that the above conditions are necessary to represent the defuzzified output of the nonadditive GFM in the Choquet integral form as follows:𝑦_𝑗=^𝑀_𝑚=1𝜇^𝑚𝐱_𝑗⋅𝑝𝑐_𝑚∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙⋅𝑓^′_𝑚(𝑋)=^𝑀_𝑚=1𝜇^𝑚𝐱_𝑗⋅𝑝𝑐_𝑚∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙⋅^𝐷_𝑑=0𝛽^𝑚_𝑑⋅𝑥_𝑗𝑑, (24) where for 𝑗th set of the data we have𝜇^𝑚𝐱_𝑗=^𝐷_𝑑=1𝜇^𝑚𝐱_𝑗𝑑=^𝐷_𝑑=1𝑁¹𝐱^𝑚_𝑗𝑑;𝐱^𝑚_𝑑,_𝑚,𝑑,𝑁¹𝐱^𝑚_𝑗𝑑;𝐱^𝑚_𝑑,_𝑚,𝑑−𝐱=exp^𝑚𝑗𝑑−𝐱^𝑚_𝑑²2∑_𝑚,𝑑, (25)

with ∑_𝑚,𝑑 being the diagonal element. The firing strength of each rule is obtained by taking the multiplicative T-norm of the membership functions (by suppressing the subscripts “𝑗”) of the premise parts of the rules as 𝜇^𝑚(𝐱)=𝜇^𝑚(𝑥₁)Λ𝜇^𝑚(𝑥₂)Λ⋯Λ𝜇^𝑚(𝑥_𝐷). We intend to compute the coefficients of the output function by reformulating (21) in the Choquet fuzzy integral form. The role of GMM is to facilitate the computation of 𝑀, 𝜇^𝑚(𝐱) and 𝑝(𝑐_𝑚).

3.1. Choquet Fuzzy Integral and Q-Measure

When a fuzzy measure is available on a finite set 𝑋, we can use a fuzzy integral as a computational scheme to integrate all values from 𝜒, that is, the subsets 𝑋_𝑖 of 𝑋 nonlinearly. In other words, a fuzzy integral relies on the concept of a fuzzy measure. A general definition is that given a class of functions 𝐹⊆{ℎ∶𝜒→ℜ⁺} and a class of 𝑄-measures 𝑄(⋅), we have a functional 𝑓∶𝐹×𝑄=ℎ×𝑄→ℜ, which is a fuzzy integral [42]. The Choquet integral is a particular fuzzy integral [41] that serves as an alternative computational scheme for aggregating information.

Definition 3. Let 𝑄-measure over a set 𝑋 be a function whose elements (discrete) are 𝑥₁,𝑥₂,…,𝑥_𝐷, then the Choquet integral of a positive function ℎ with respect to 𝑄 is defined by [39] 𝑓_𝑔(ℎ)=^𝐷_𝑑=1ℎ𝑥_𝑑−ℎ𝑥_𝑑−1𝑄𝑋_𝑑, (26) or 𝑓_𝑔(ℎ)=^𝐷_𝑑=1ℎ𝑥_𝑑𝑄𝑋_𝑑−𝑄𝑋_𝑑+1, (27) where ℎ(𝑥₁)≤ℎ(𝑥₂)≤⋯≤ℎ(𝑥_𝐷). We assume here that 𝑄(𝑋_𝐷+1)=0 and ℎ(𝑥₀)=0. To convert (24) into the Choquet fuzzy integral form, we shall represent coefficients 𝛽^𝑚_𝑑 in terms of 𝑄-measures, 𝑄(𝑋_𝑑) as 𝛽^𝑚_𝑑=𝑄^𝑚𝑋_𝑑−𝑄^𝑚𝑋_𝑑+1=𝑔^𝑚_𝑑+𝜆^𝑚𝑔^𝑚_𝑑𝑄^𝑚𝑋_𝑑+1,for𝑑=1,2,3…,𝐷and𝑄^𝑚𝑋_𝐷+1=0. (28) Interchanging the summations in (24) leads to 𝑦_𝑗=^𝐷_𝑑=0^𝑀_𝑚=1𝜇^𝑚𝐱_𝑗⋅𝑝𝑐_𝑚⋅𝑥_𝑗𝑑∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙⋅𝑄^𝑚𝑋_𝑑−𝑄^𝑚𝑋_𝑑+1. (29) Next separating out the 𝑑=0 term yields, 𝑦_𝑗=𝑎₀+^𝐷_𝑑=1^𝑀_𝑚=1𝜇^𝑚𝐱_𝑗⋅𝑝𝑐_𝑚⋅𝑥_𝑗𝑑∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙⋅𝑄^𝑚𝑋_𝑑−𝑄^𝑚𝑋_𝑑+1, (30) where 𝑎₀ is a constant. Letting ℎ^𝑚𝑥_𝑗𝑑=𝜇^𝑚𝐱_𝑗⋅𝑝𝑐_𝑚⋅𝑥_𝑗𝑑∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙. (31)Equation (30) is simplified as 𝑦_𝑗=𝑎₀+^𝐷_𝑑=1^𝑀_𝑚=1ℎ^𝑚𝑥_𝑗𝑑⋅𝑄^𝑚𝑋_𝑑−𝑄^𝑚𝑋_𝑑+1. (32) Denoting [ℎ(𝑥_𝑗𝑑)]=[ℎ¹(𝑥_𝑗𝑑)ℎ²(𝑥_𝑗𝑑)⋯ℎ^𝑀(𝑥_𝑗𝑑)] and 𝑄𝑋_𝑑−𝑄𝑋_𝑑+1=𝑄¹𝑋_𝑑−𝑄¹𝑋_𝑑+1𝑄²𝑋_𝑑−𝑄²𝑋_𝑑+1⋮𝑄^𝑀𝑋_𝑑−𝑄^𝑀𝑋_𝑑+1 (33) allows us to write (32) compactly for 𝐷 input variables as 𝑦_𝑗=𝑎₀+^𝐷_𝑑=1ℎ𝑥_𝑗𝑑⋅𝑄𝑋_𝑑−𝑄𝑋_𝑑+1, (34) or 𝑦_𝑗=𝑎₀+^𝐷_𝑑=1ℎ𝑥_𝑗𝑑−ℎ𝑥_{𝑗(𝑑−1)}⋅𝑄𝑋_𝑑, (35) where ℎ^𝑚(𝑥_𝑗1)≤ℎ^𝑚(𝑥_𝑗2)≤⋯≤ℎ^𝑚(𝑥_𝑗𝐷) and ℎ^𝑚(𝑥_𝑗0)=0 for 𝑚=1,2,…,𝑀, is the monotonicity condition. Note that ℎ^𝑚(𝑥_𝑗1),ℎ^𝑚(𝑥_𝑗2),…,ℎ^𝑚(𝑥_𝑗𝐷) are the evidences provided by the input sources 𝑥₁,𝑥₂,…,𝑥_𝐷, respectively, and 𝑄^𝑚(𝑋₁),𝑄^𝑚(𝑋₂),…,𝑄^𝑚(𝑋_𝐷) are the 𝑄-measures for mth cluster. Here, we take 𝑄^𝑚(𝑋_𝐷+1) to be zero. Equations (34) and (35) similar to (26) and (27) are the two forms of Choquet fuzzy integral [42]. We have now proved that the defuzzified output of the non-additive GFM fuzzy rules is in the form of Choquet fuzzy integral. The underlying model is nonadditive GFM because both ℎ and 𝑄 are nonlinear functions of 𝑥. Thus the Choquet integral provides the functionality of non-additive GFM. The functionality of the Choquet integral for GMM will be discussed later. We will now estimate the unknown 𝑄-measures from fuzzy densities determined using (9). We then evaluate the performance of the fuzzy model with a measure, which is a function of mean square error [54], chosen as ∑𝐸=^𝑛𝑗=1𝑒²_𝑗𝑛⋅𝑦_𝑟, (36) where 𝑛 is the total number of test data vectors: 𝑒_𝑗=𝑦_𝑗−𝑦_𝑗,𝑦_𝑟=[max(𝑌)−min(𝑌)]², (37) where 𝑦_𝑗 is the estimated value, 𝑦_𝑗 is the actual output, and 𝑌={𝑦_𝑗} is the output vector.

3.2. Determination of Fuzzy Densities

The cluster-wise breakup of the estimated output of the additive fuzzy system [7] is given as𝑦_𝑗=𝜇¹𝐱_𝑗⋅𝑝𝑐₁⋅𝑓𝐱_𝑗,𝛼₁∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙+𝜇²𝐱_𝑗⋅𝑝𝑐₂⋅𝑓𝐱_𝑗,𝛼₂∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙+𝜇+⋯_𝑀𝐱_𝑗⋅𝑝𝑐_𝑀⋅𝑓𝐱_𝑗,𝛼_𝑀∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙, (38) where using GMM from [7] we estimate 𝑓(𝐱_𝑗,𝛼_𝑚) as𝑓𝐱_𝑗,𝛼_𝑚=𝑦^𝑚_𝑗−^𝐷_𝑑=1𝑥_𝑗𝑑−𝑥^𝑚_𝑑⋅𝜎_𝑚^𝑑𝑦𝜎_𝑚^𝑦𝑦. (39)

Equation (38) can also be written as𝑦_𝑗=𝑤₁⋅𝑓𝐱_𝑗,𝛼₁+𝑤₂⋅𝑓𝐱_𝑗,𝛼₂+⋯+𝑤_𝑀⋅𝑓𝐱_𝑗,𝛼_𝑀, (40) where 𝑤_𝑚=((𝜇^𝑚(𝐱_𝑗)⋅𝑝(𝑐_𝑚))/(∑^𝑀_𝑙=1𝜇^𝑙(𝐱_𝑗)⋅𝑝(𝑐_𝑙))). Substituting 𝑓(𝐱_𝑗,𝛼_𝑚)for𝑚=1to𝑀 from (39) in (40), we have𝑤₁⋅𝑦¹−^𝐷_𝑑=1𝑥_𝑗𝑑−𝑥¹_𝑑⋅𝜎₁^𝑑𝑦𝜎₁^𝑦𝑦+𝑤₂⋅𝑦²−^𝐷_𝑑=1𝑥_𝑗𝑑−𝑥²_𝑑⋅𝜎₂^𝑑𝑦𝜎₂^𝑦𝑦⋯⋯+𝑤_𝑀⋅𝑦^𝑀−^𝐷_𝑑=1𝑥_𝑗𝑑−𝑥^𝑀_𝑑⋅𝜎_𝑀^𝑑𝑦𝜎_𝑀^𝑦𝑦=𝑎¹₀+𝑤₁𝑥_𝑗1⋅𝑔¹₁+𝑥_𝑗2⋅𝑔¹₂+⋯+𝑥_𝑗𝐷⋅𝑔¹_𝐷+𝑎²₀+𝑤₂𝑥_𝑗1⋅𝑔²₁+𝑥_𝑗2⋅𝑔²₂+⋯+𝑥_𝑗𝐷⋅𝑔²_𝐷⋯⋯+𝑎^𝑀₀+𝑤_𝑀𝑥_𝑗1⋅𝑔^𝑀₁+𝑥_𝑗2⋅𝑔^𝑀₂+⋯+𝑥_𝑗𝐷⋅𝑔^𝑀_𝐷. (41) Using (32), for 𝑀 rules, we have the cluster-wise estimates of the output for the non-additive fuzzy model written as𝑦_𝑗=𝑎¹₀+^𝐷_𝑑=1ℎ¹𝑥_𝑗𝑑⋅𝑄¹𝑋_𝑑−𝑄¹𝑋_𝑑+1+𝑎²₀+^𝐷_𝑑=1ℎ²𝑥_𝑗𝑑⋅𝑄²𝑋_𝑑−𝑄²𝑋_𝑑+1⋯⋯+𝑎^𝑀₀+^𝐷_𝑑=1ℎ^𝑀𝑥_𝑗𝑑⋅𝑄^𝑀𝑋_𝑑−𝑄^𝑀𝑋_𝑑+1. (42) From (42) it is implied that 𝑎₀=𝑎¹₀+𝑎²₀+⋯+𝑎^𝑀₀ in view of (34) and (35).

Equation (42) can also be written as𝑦=𝑎¹₀+𝑤₁𝑥_𝑗1⋅𝑔¹₁1+𝜆¹𝑄¹𝑋₂+𝑥_𝑗2⋅𝑔¹₂1+𝜆¹𝑄¹𝑋₃+⋯+𝑥_𝑗𝐷⋅𝑔¹_𝐷+𝑎²₀+𝑤₂𝑥_𝑗1⋅𝑔²₁1+𝜆²𝑄²𝑋₂+𝑥_𝑗2⋅𝑔²₂1+𝜆²𝑄²𝑋₃+⋯+𝑥_𝑗𝐷⋅𝑔²_𝐷⋯⋯+𝑎^𝑀₀+𝑤_𝑀𝑥_𝑗1⋅𝑔^𝑀₁1+𝜆^𝑀𝑄^𝑀𝑋₂+𝑥_𝑗2⋅𝑔^𝑀₂1+𝜆^𝑀𝑄^𝑀𝑋₃+⋯+𝑥_𝑗𝐷⋅𝑔^𝑀_𝐷 (43) or 𝑦=𝑎₀+𝑤₁⋅𝑓^′𝐱_𝑗,𝛽₁+𝑤₂⋅𝑓^′𝐱_𝑗,𝛽₂+⋯+𝑤_𝑀⋅𝑓^′𝐱_𝑗,𝛽_𝑀. (44) Since (40) and (44) represent the estimated outputs for additive and non-additive fuzzy systems, respectively, and hence these must be equivalent to each other for 𝜆=0. Equating the right hand sides of (40) and (44), we have𝑤₁⋅𝑓𝐱_𝑗,𝛼₁+𝑤₂⋅𝑓𝐱_𝑗,𝛼₂+⋯+𝑤_𝑀⋅𝑓𝐱_𝑗,𝛼_𝑀=𝑎₀+𝑤₁⋅𝑓^′𝐱_𝑗,𝛽₁+𝑤₂⋅𝑓^′𝐱_𝑗,𝛽₂+⋯+𝑤_𝑀⋅𝑓^′𝐱_𝑗,𝛽_𝑀. (45) Thus when the fuzzy measure 𝜆^𝑚=0,𝛼_𝑖=𝛽_𝑖. As 𝛼_𝑖=𝑔^𝑚_𝑖 we can express the fuzzy densities in terms of the elements of covariance matrices of GMM as [7]𝑔^𝑚₁=𝜎_𝑚^1𝑦𝜎_𝑚^𝑦𝑦;𝑔^𝑚₂=𝜎_𝑚^2𝑦𝜎_𝑚^𝑦𝑦,⋯𝑔^𝑚_𝐷=𝜎_𝑚^𝐷𝑦𝜎_𝑚^𝑦𝑦for𝑚=1,2,…,𝑀. (46) We are in a position to estimate 𝜆 and 𝑄-measures and hence the 𝛽 coefficients using the fuzzy densities. We will then use all these in the Choquet integral to estimate the output. For convenience of notation we start with fuzzy measure of one element set 𝑋_𝑑={𝑥_𝐷} and move in backward direction to find the 𝑄-measure of 𝑋₁. Since we are computing all 𝑄-measures before the start of the Choquet integral, these become handy in computing the output. This is because in (32) the index 𝑑 is in the forward direction from 1 to 𝐷.

3.3. Functionality of GMM

As we have already proved that the Choquet integral is the functionality of non-additive GFM, It is now easy to extend to this functionality the GMM case from the fact that the output is Gaussian in the nonadditive case too as per the expression𝑝𝐲∣𝐱,𝑐_𝑚=12𝜋∑_{𝑚,(𝐷+1)}⋅exp−∑_𝑗𝑦^𝑚_𝑗−𝑓𝐱^𝑚_𝑗,𝛽_𝑚²2∑_{𝑚,(𝐷+1)}. (47) From the above it may be noted that 𝛽_𝑚 is obtained from 𝛼_𝑚 using GMM by the choice of 𝜆^𝑚 in the 𝑄-measure. Hence 𝑓(𝐱^𝑚_𝑗,𝛽_𝑚) is the mean of 𝑦^𝑚_𝑗 in the non-additive case just as 𝑓(𝐱^𝑚_𝑗,𝛼_𝑚) is the mean of 𝑦^𝑚_𝑗 in the additive case. We can conclude therefore that when 𝛽_𝑚 is evaluated from 𝛼_𝑚, the Choquet integral also assures the functionality of GMM.

4. Algorithm for Estimating the Model Parameters of the Nonadditive Fuzzy Systems

The algorithm has the following steps.

Step 1. Normalize the input-output data, so that the data values lie in between 0 and 1: 𝐱^normalized=(𝐱−min(𝐳))(max(𝐳)−min(𝐳)), (48) where 𝐳=[𝐱,𝑦] is the input-output data vector.

Step 2. Find the premise model parameters of GMM using the EM Algorithm.

Step 3. Determine fuzzy densities 𝑔^𝑚₁,𝑔^𝑚₂,…,𝑔^𝑚_𝐷 for each input variable in 𝑚th cluster/rule using (47).

Step 4. Choose initial values of 𝜆^𝑚 with the restriction 𝜆^𝑚≥−1 for 𝑚th cluster.

Step 5. Compute 𝑄^𝑚(𝑋_𝑑) by the recursive computation involving the following equations from (10) and (11):𝑄^𝑚𝑋_𝐷=𝑄^𝑚𝑥_𝐷=𝑔^𝑚_𝐷,𝑄^𝑚𝑋_𝑑=𝑔^𝑚_𝑑+𝑄^𝑚𝑋_𝑑+1,+𝜆^𝑚𝑔^𝑚_𝑑𝑄^𝑚𝑋_𝑑+1,for1≤𝑑<𝐷. (49)

Step 6. Using (31) find ℎ^𝑚(𝑥_𝑑) for each input variable 𝑑 in the cluster/rule to obtain the matrix [ℎ(𝑥_𝑑)].

Step 7. Compute the estimated output using (34).

Step 8. Learn the model. This requires the following two subtasks: (a) set up an objective function [54] as the case required. For batch:𝐽_batch=1𝑛^𝑛_𝑗=1𝑦_𝑗−𝑦_𝑗²=1𝑛^𝑛_𝑗=1𝑒_𝑗², (50) for incremental, 𝐽_incr=12𝑦_𝑗−𝑦_𝑗²=12𝑒_𝑗². (51)(b) Update the values of 𝜆^𝑚 by the Gradient-descent learning law:𝜆^𝑚_new=𝜆^𝑚_old−𝜂⋅𝜕𝐽𝜕𝜆^𝑚,𝑎_0,new=𝑎₀−𝜂⋅𝜕𝐽𝜕𝑎₀, (52) where 𝐽 may refer to either 𝐽_batch or 𝐽_incr and 𝜂 is a learning factor fixed at the value 0.9 for better performance. The parameter learning can be accomplished in either of the above two ways: batch or incremental. Derivatives required in the learning process are relegated to the Appendix.

Step 9. Repeat Step 1 to Step 8, till the model parameters are within the tolerance.

Step 10. Calculate 𝐸 from (36).

5. Experimental Results

5.1. Gas Furnace

Here, we take up the gas furnace data [55] to demonstrate the effectiveness of the proposed algorithm. In this gas furnace, the air at a feed rate of 𝑢(𝑡) is combined with methane to produce a mixture of gases with carbon dioxide of concentration 𝑦(𝑡). These are used to fit the Gaussian mixture density function with four components. The input vector after sorting is 𝐱=[𝑥₁𝑥₂𝑥₃𝑥₄]=[𝑦(𝑡−1)−𝑦(𝑡−2)𝑦(𝑡−1)𝑢(𝑡)−𝑢(𝑡−1)𝑢(𝑡)] and the output is 𝑦=𝑦(𝑡). The composition of the input vector is arrived at using the fuzzy curve approach [56]. On applying the expectation maximization (EM) [57] clustering, the number of clusters is found to be 4; hence the model will have four components. The five-variable GMM using the EM algorithm yields the parameters of GMM as shown in Table 1. These parameters are also the premise model parameters of the non-additive GFM fuzzy rules.

Table 1 

Estimated parameters of A 4-component GMM using the EM algorithm.

Fuzzy densities and estimated fuzzy measures of the model are given in Tables 2 and 4, respectively. The estimated values of 𝜆^𝑚 are given in Table 3. The estimated 𝛽-coefficient and 𝑄-measures are given in Tables 5 and 6 respectively. Note that 𝜁(𝑋₁) contains all input elements, hence its value is one. The 𝛽-coefficients constitute the consequent parameters of the GMM fuzzy rules. The performance measure is found to be 0.00208. The fuzzy measures of the constituent singleton sets satisfy the monotonic relations for 𝐷=4 shown as a lattice in Figure 2.

Table 2 

Fuzzy densities using GMM.

Table 3 

Estimated lambda (𝜆^𝑚).

Table 4 

Estimated fuzzy measures.

Table 5 

Estimated 𝛽 coefficients.

Table 6 

Estimated𝑄-measures.

For the training dataset containing the first 250 data vectors, 𝐸=6.12505×10⁻⁴. For the remaining data vectors forming the validation data set, 𝐸=0.0021. This is inferior to that of the training data set, as expected. The plots of actual output and model output for both training and prediction along with the prediction error of the model output are shown in Figure 1. The model output is very close to the actual output during training, but it displays small deviations from the actual output during the prediction period. Using the premise and consequent parameters of Table 1, we have constructed the membership functions for the four inputs and one output in Figure 3 where we depict the membership function of rules-1, 2, 3, and 4 using dashed lines, dotted lines, dots with dashes lines and solid lines, respectively.

(a)

(b)

(c)

Figure 1 

Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for Box and Jenkins’s gas furnace data.

Figure 2 

Lattice of fuzzy measure values of the constituent singleton sets [39] of rule 1 for Box and Jenkins’s data.

(a)

(b)

(c)

(d)

(e)

Figure 3 

(a) Membership function for input one. (b) Membership functions for input two. (c) Membership function for input three. (d) Membership function for input four. (e) Membership function for output.

The first rule 𝑅¹ consists of,(i) the weight of the rule; 0.2991,(ii) the membership functions of the premise parts𝑁¹𝑥₁;104066,0.0246,𝑁¹𝑥₂,𝑁;0.2897,0.0150₁𝑥₃;0.4341,0.0161,𝑁¹𝑥₄;0.5321,0.0427. (53)(iii) The consequent part of a rule can be found using Table 6.

Similarly other rules can also be formed.

5.2. Industrial Dryer

In this, we consider the industrial dryer data obtained from the DaISy database. Fuel is combusted to produce the hot gas, which is blown by an exhaust fan on the raw material fed into the dryer. It is desired to determine the relationship between the functional elements of the dryer with the moisture content of the raw material at the dryer output. The measurements are designated by(i)fuel flow rate, 𝑥₁(𝑡),(ii)hot gas exhaust-fan speed, 𝑥₂(𝑡),(iii)rate of flow of raw material, 𝑥₃(𝑡),(iv)moisture content of the raw material after drying which is 𝑦(𝑡).

Sampling period of 10 seconds is considered to generate a total of 867 samples. Input and output data are normalized using (40) to maintain compatibility in the range 0 to 1. The input variables 𝑥₁(𝑡), 𝑥₂(𝑡), 𝑥₃(𝑡), and 𝑦(𝑡−1) affect the current output 𝑦(𝑡). The input-output vectors are represented by 𝐳(𝑡)=[𝑥₁(𝑡)𝑥₂(𝑡)𝑥₃(𝑡)𝑦(𝑡−1)𝑦(𝑡)]. Out of these vectors 70% is taken as the training set. The remaining 30% is kept for prediction and evaluation of our proposed model.

The number of clusters is found to be 4 using EM clustering, which yields the parameters of GMM in Table 7, and these are also the premise model parameters of the non-additive GFM.

Table 7 

Estimated parameters of A 4-component GMM using the EM algorithm.

The estimated fuzzy densities, fuzzy measures, 𝛽-coefficients and 𝑄-measure of the model are given in Tables 8, 10, 11, and 12, respectively. The monotonicity relations for 𝐷=4 are also verified here for fuzzy measure values of the constituent singleton sets, shown as a lattice in Figure 5. Table 9 shows the estimated values of 𝜆^𝑚.

Table 8 

Fuzzy densities using GMM.

Table 9 

Estimated lambda (𝜆^𝑚).

Table 10 

Estimated 𝜆-measures.

Table 11 

Estimated 𝛽 coefficients.

Table 12 

Estimated 𝑄-measures.

The training data set yields 𝐸=9.0005×10⁻⁴ for training the model. The remaining 30% data vectors constitute the validation data set. The performance measure 𝐸 over the prediction period is obtained as 0.0019. The results of the actual output and the model output during the training and the prediction periods are shown in Figure 4 which also includes the prediction error. The results demonstrate the effectiveness of the proposed non-additive fuzzy model using 𝑄-measure. The membership functions associated with all the four rules can be obtained in the similar fashion as in Figure 3.

(a)

(b)

(c)

Figure 4 

Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for industrial dryer data.

Figure 5 

Lattice of fuzzy measure values of the constituent singleton sets of rule no.1 of Table 10 for industrial dryer’s data.

5.3. Electric Load Demand

In this, we consider the real time hourly electric load data, that is, hourly peak electric load consumption with uncertain weather information like hourly average temperature, humidity, and wind speed for the month of April 2003. The variables are designated as(i)hourly average Temperature, 𝑥₁(𝑡),(ii)hourly average Humidity, 𝑥₂(𝑡),(iii)hourly average Wind speed, 𝑥₃(𝑡),(iv)hourly peak electric load consumption which is 𝑦(𝑡).

A number of 720 samples of Input and output data are normalized using (40) to maintain the compatibility in the range 0 to 1. We consider 75% of the total input-output vectors of the form 𝐳(𝑡)=[10⋅(𝑦(𝑡)−𝑦(𝑡−2))10⋅(𝑥₃(𝑡)−𝑥₂(𝑡−2))10⋅(𝑥₂(𝑡)−𝑥₂(𝑡−2))50⋅(𝑥₃(𝑡−2)−𝑥₃(𝑡))100⋅𝑦(𝑡)] as the training set. The remaining 25% is reserved for the prediction. The weights before the components in 𝐳(𝑡) have been fixed for achieving better prediction capability. It may be noted that the criteria only facilitate the selection of proper inputs by trying out different combinations. But the final curtain falls on the selection with the verification through the performance measure.

The number of clusters in this example also is found to be 4 using the EM clustering after discarding a few datasets as outliers. Application of the EM algorithm yields the parameters of GMM in Table 13. These constitute the premise parameters of the GFM. The estimated fuzzy densities, lambda values, fuzzy measure, 𝑄-measure and 𝛽-coefficients of the model are given in Tables 14, 15, 16, 17, and 18, respectively. The monotonicity relations can also be verified as in the previous applications

Table 13 

Estimated parameters of A 4-component GMM using the EM algorithm (load).

Table 14 

Fuzzy densities using GMM.

Table 15 

Estimated lambda (𝜆^𝑚).

Table 16 

Estimated 𝜆-measures.

Table 17 

Estimated 𝑄-measures.

Table 18 

Estimated 𝛽 coefficients.

The training data set is found to give 𝐸=0.0034. The remaining 25% data vectors constitute the validation data set. The performance measure 𝐸 over this set is obtained as 0.00256. The results of the actual output and the model output during the training and the prediction periods are shown in Figure 6, which also includes the prediction error. The results demonstrate the effectiveness of the proposed non-additive fuzzy model using 𝑄-measure for this application too (Figure 7).

(a)

(b)

(c)

Figure 6 

Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for hourly electric load data.

Figure 7 

Lattice of 𝑄-measure values of the constituent singleton sets with 𝐷=4.

The performance measures of the applications A and B are given in Table 19 for comparison with some of the existing well-known methods of fuzzy modeling. This table shows that E of the proposed nonadditive fuzzy model is better than those of the additive fuzzy systems of Gan et al. [7] and WM method [25].

Table 19 

Comparison of 𝐸.

6. Conclusions

This paper deals with an important issue of computing fuzzy measures directly from covariance matrices found in GMM. Use of EM algorithm in GMM provides us with the input clusters and their Gaussian memberships. The consequent parts of the fuzzy rules, that is, the output function, of GFM are altered for incorporating the non-additive property. It has to be noted that we have been still using the multiplicative T-norm in the premise part but the additive property of S-norm in the consequent part. For this the output function is defined as a linear function of fuzzy measures serving as coefficients. Computation complexity is reduced by replacing Sugeno’s 𝜆-measure with the 𝑄-measure in the non-additive fuzzy model.

The defuzzified output of the non-additive GFM rules is then shown to be in the form of Choquet integral. By showing the Choquet fuzzy integral as the functionality of non-additive GFM, we have enhanced its capability to tackle a variety of real-life problems. We have also proved the corresponding non-additive functionality for GMM. This has been accomplished by showing that 𝑓(𝐱,𝛽_𝑚) is the mean value of 𝐲. The computation of fuzzy measures required in the Choquet integral entails a lot of computational burden, as we need to solve a polynomial in 𝜆. Using 𝑄-measure this burden has been eliminated.

The non-additive fuzzy model is applied on the three benchmark applications, namely, Box and Jenkins’s, Dryers’ data and on the real time electric load demand data. The results demonstrate the superiority of this model over the additive fuzzy model of Gan et al. [7] and Wang and Mendel method [25]. Thus this work makes an important contribution to the field of fuzzy modeling by providing a transition from additive fuzzy systems to non-additive fuzzy systems for dealing with the real-life applications. This is done without learning the fuzzy densities and eliminating the computational complexity associated with solving the polynomial function in 𝜆. In the non-additive fuzzy systems, it makes a novel contribution and will pave the way for several other applications and for further research.

Appendix

Formulation of the Learning Process

In this section we will find the derivatives required for the learning laws. To keep the thread going, we recall the relevant equations from Section 5.

For incremental learning, using (51), we obtain the derivatives in (52) as𝜕𝐽_incr𝜕𝑎₀=𝑦_𝑗−𝑦_𝑗=𝑒_𝑗,𝜕𝐽_incr𝜕𝜆^𝑚=𝜕(1/2)𝑦_𝑗−𝑦_𝑗²𝜕𝜆^𝑚=𝜕(1/2)𝑒_𝑗²𝜕𝜆^𝑚=𝑒_𝑗⋅𝜕𝑦_𝑗𝜕𝜆^𝑚. (A.1a)(A.1b)For batch learning, using (50), (52), we obtain the derivative as𝜕𝐽_batch𝜕𝑎₀=2𝑛⋅^𝑛_𝑗=1𝑦_𝑗−𝑦_𝑗,𝜕𝐽_batch𝜕𝜆^𝑚=𝜕(1/𝑛)∑^𝑛_𝑗=1𝑦_𝑗−𝑦_𝑗²𝜕𝜆^𝑚=𝜕(1/𝑛)∑^𝑛_𝑗=1𝑒_𝑗²𝜕𝜆^𝑚=2𝑛⋅^𝑛_𝑗=1𝑒_𝑗⋅𝜕𝑦_𝑗𝜕𝜆^𝑚. (A.2a)(A.2b)In the two applications that we dealt with, we have used the incremental learning. Now the formulation of the learning process required in the above two types of learning is presented in the following. For 𝑗th set of 𝐷-inputs 𝐱_𝑗=[𝑥_𝑗1𝑥_𝑗2𝑥_𝑗3⋯𝑥_𝑗𝐷], the estimated output using (34) is 𝑦_𝑗=𝑎₀+^𝐷_𝑑=1^𝑀_𝑚=1ℎ^𝑚𝑥_𝑗𝑑⋅𝑄^𝑚𝑋_𝑑−𝑄^𝑚𝑋_𝑑+1. (A.3) The partial derivatives of (A.3) with respect to 𝜆^𝑚 and fuzzy densities 𝑔^𝑚_𝑑 for 𝑑=1,2,3,…,𝐷 required in (A.1b) and (A.2b) are obtained after rearranging as𝜕𝑦_𝑗𝜕𝜆^𝑚𝜇=0+^𝑚𝐱_𝑗⋅𝑝𝑐_𝑚⋅𝑥_𝑗1∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙⋅𝜕𝑄^𝑚𝑋₁𝜕𝜆^𝑚+𝜇^𝑚𝐱_𝑗⋅𝑝𝑐_𝑚⋅𝑥_𝑗2−𝑥_𝑗1∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙⋅𝜕𝑄^𝑚𝑋₂𝜕𝜆^𝑚+𝜇+⋯_𝑚𝐱_𝑗⋅𝑝𝑐_𝑚⋅𝑥_𝑗𝐷−𝑥_{𝑗(𝐷−1)}∑^𝑀_𝑙=1𝜇^𝑙𝐱_𝑗⋅𝑝𝑐_𝑙×𝜕𝑄^𝑚𝑋_𝐷𝜕𝜆^𝑚. (A.4) Using (6), we obtain the derivatives as follows:𝜕𝑄^𝑚𝑋_𝑑𝜕𝜆^𝑚=𝜁^𝑚𝑋₁⋅𝜕𝜁^𝑚𝑋_𝑑/𝜕𝜆^𝑚𝜁^𝑚(𝑋₁)²−𝜁^𝑚𝑋_𝑑⋅𝜕𝜁^𝑚𝑋₁/𝜕𝜆^𝑚𝜁^𝑚(𝑋₁)². (A.5) The polynomial for the general case of 𝐷-inputs using (3) and (4) is𝜁^𝑚𝑋₁=𝜆^𝑚𝐷−1⋅^𝐷_𝑑=1𝑔^𝑚_𝑑+⋯+𝜆^𝑚⋅^𝐷−1𝐷_{𝑑=1𝑘=𝑑+1}𝑔^𝑚_𝑑⋅𝑔^𝑚_𝑘+^𝐷_𝑑=1𝑔^𝑚_𝑑. (A.6) The polynomial in 𝜆^𝑚 for 𝐷=4 is found as𝜁^𝑚𝑋₁=𝜑₃⋅𝜆^𝑚3+𝜑₂⋅𝜆^𝑚2+𝜑₁⋅𝜆^𝑚+𝜑₀.𝜁^𝑚𝑋₂=𝜆^𝑚2𝑔^𝑚₂⋅𝑔^𝑚₃⋅𝑔^𝑚₄+𝜆^𝑚𝑔^𝑚₂⋅𝑔^𝑚₃+𝑔^𝑚₃⋅𝑔^𝑚₄+𝑔^𝑚₂⋅𝑔^𝑚₄+𝑔^𝑚₂+𝑔^𝑚₃+𝑔^𝑚₄,𝜁^𝑚𝑋₃=𝜆^𝑚𝑔^𝑚₃⋅𝑔^𝑚₄+𝑔^𝑚₃+𝑔^𝑚₄,𝜁^𝑚𝑋₄=𝑔^𝑚₄. (A.7) The derivatives of the above with respect to 𝜆^𝑚 are𝜕𝜁^𝑚𝑋₁𝜕𝜆^𝑚=3𝜑₃⋅𝜆^𝑚2+2𝜑₂⋅𝜆^𝑚+𝜑₁,𝜕𝜁^𝑚𝑋₂𝜕𝜆^𝑚=2𝜆^𝑚𝑔^𝑚₂⋅𝑔^𝑚₃⋅𝑔^𝑚₄+𝑔^𝑚₂⋅𝑔^𝑚₃+𝑔^𝑚₃⋅𝑔^𝑚₄+𝑔^𝑚₄⋅𝑔^𝑚₂,𝜕𝜁^𝑚𝑋₃𝜕𝜆^𝑚=𝑔^𝑚₃⋅𝑔^𝑚₄,𝜕𝜁^𝑚𝑋₄𝜕𝜆^𝑚=0, (A.8) where 𝜑₃=𝑔^𝑚₁⋅𝑔^𝑚₂⋅𝑔^𝑚₃⋅𝑔^𝑚₄,𝜙₂=𝑔^𝑚₁⋅𝑔^𝑚₂⋅𝑔^𝑚₃+𝑔^𝑚₂⋅𝑔^𝑚₃⋅𝑔^𝑚₄+𝑔^𝑚₃⋅𝑔^𝑚₄⋅𝑔^𝑚₁+𝑔^𝑚₄⋅𝑔^𝑚₁⋅𝑔^𝑚₂,𝜑₁=𝑔^𝑚₁⋅𝑔^𝑚₂+𝑔^𝑚₂⋅𝑔^𝑚₃+𝑔^𝑚₃⋅𝑔^𝑚₄+𝑔^𝑚₄⋅𝑔^𝑚₁+𝑔^𝑚₁⋅𝑔^𝑚₃+𝑔^𝑚₂⋅𝑔^𝑚₄,𝜑₀=𝑔^𝑚₁+𝑔^𝑚₂+𝑔^𝑚₃+𝑔^𝑚₄. (A.9)

The above polynomial and their derivatives in (A.8) are used in (A.4) and (A.5) to obtain a solution. Thus, we can see a considerable simplification with 𝑄-measure as the solution of 𝜆^𝑚-polynomials is not needed. On the other hand without 𝑄-measure, the derivatives of 𝜆^𝑚 with respect to 𝑔^𝑚_𝑑 are required to learn 𝜆^𝑚.