Research Article | Open Access
Nishchal K. Verma, "Estimation of Fuzzy Measures Using Covariance Matrices in Gaussian Mixtures", Applied Computational Intelligence and Soft Computing, vol. 2012, Article ID 402420, 16 pages, 2012. https://doi.org/10.1155/2012/402420
Estimation of Fuzzy Measures Using Covariance Matrices in Gaussian Mixtures
This paper presents a novel computational approach for estimating fuzzy measures directly from Gaussian mixtures model (GMM). The mixture components of GMM provide the membership functions for the input-output fuzzy sets. By treating consequent part as a function of fuzzy measures, we derived its coefficients from the covariance matrices found directly from GMM and the defuzzified output constructed from both the premise and consequent parts of the nonadditive fuzzy rules that takes the form of Choquet integral. The computational burden involved with the solution of λ-measure is minimized using Q-measure. The fuzzy model whose fuzzy measures were computed using covariance matrices found in GMM has been successfully applied on two benchmark problems and one real-time electric load data of Indian utility. The performance of the resulting model for many experimental studies including the above-mentioned application is found to be better and comparable to recent available fuzzy models. The main contribution of this paper is the estimation of fuzzy measures efficiently and directly from covariance matrices found in GMM, avoiding the computational burden greatly while learning them iteratively and solving polynomial equations of order of the number of input-output variables.
Generalized fuzzy model (GFM) [1–3] is the backbone of this work that employs two norms for computing the strength of a rule: the multiplicative T-norm operator for determining the strength of a rule [4–6] and the additive S-norm operator for combining the outputs of all the rules. The effect of input fuzzy sets is taken into the defuzzified output in the form of rule strengths or weights. Gan et al.  have simplified the formulation of GFM by setting both the inputs and the output to be jointly Gaussian and proved that the input-output relation is as an expectation using the Bayesian framework. On simplification, this turns out to be the coveted Gaussian mixture model (GMM), which is linked by an additive function between the inputs and the output. The GMM, also known as cluster-weighted modeling (CWM) [8–12], is advocated by many researchers as means of statistical modeling of input-output systems [10, 13–15]. After the establishment of equivalence between GMM and GFM in , the difficult-to-compute index of fuzziness of GFM has become easy-to-compute prior probability [16–23] in GMM. Further simplification of GFM will be explored when it is converted into nonadditive case.
The use of GMM in GFM has provided a generalized framework for additive fuzzy systems. A few existing additive fuzzy systems urging our attention are due to fuzzy models given in Kosko  and Wang and Mendel . Using the framework of GMM, Abonyi et al.  have made a semblance of generalizing TS model [4, 27, 28] by attaching weight to each of its rules and subsequently treating it as the strength of the rule. However, nowhere they touched upon the concept of GFM that seeks to unify both Mamdani and TS models [4, 5].
Incorporation of the nonadditive property into the fuzzy sets is done so that the corresponding output of a non-additive fuzzy system is explored here. Let us throw some light on the additive and non-additive fuzzy systems [29–31]. Conventional evaluation with multiple attributes which are independent is based on the concept of additive fuzzy systems where importance of each attribute is given a weight.
Some important applications of fuzzy measures are now mentioned. A random generation of fuzzy measures have been introduced and some subfamilies of fuzzy measures tackled in . A set of isometric fuzzy measures for any isometric transformation in each family of fuzzy measures (e.g., convex) is also studied. In  fuzzy measure is shown to be a unifying structure for modeling knowledge about an uncertain variable. Kim Le in  presents an expression to evaluate a fuzzy measure from set of aggregated evidences. Hierarchical autoregression model is presented in  using nonmonotonic fuzzy measures and the Choquet integral. Narukawa et al. in  discuss a space of fuzzy measures bearing the topology introduced by Choquet integral and the space of nonnegative fuzzy measure is shown to be locally convex. Fuzzy measures and integrals (e.g., Sugeno and Choquet) are used in decision making and modeling auctions [36, 37]. Fuzzy measures also find an application in  to track a moving object from a dynamic image sequence.
Fascinated by the ever growing importance of the fuzzy measure theory [39–45], the additive function of the GFM model in the consequent part is replaced by a nonadditive function satisfying certain axioms of the fuzzy measures. The defuzzified output of the resultant non-additive GFM is shown to be in the form of Choquet integral . This formulation is intended for real-life applications in which the information from different sources needs not to be additive; hence our efforts in this work will go a long way in evolving different types of non-additive fuzzy systems, like dynamic, adaptive, and so forth, but here our attempt is only on a simple non-additive fuzzy system. To take account of the overlapping information in the fuzzy sets, Sugeno  has introduced the concept of -measure such that an appropriate value of can account for the interaction. In the proposed formulation, -measure takes the role of this -measure for reducing the computational complexity of the former.
Fuzzy integrals have come into vogue for the information fusion as they aggregate information from several sources. Out of all fuzzy integrals [40–44], the Choquet fuzzy integral [39, 41, 45] and Sugeno integral  are widely used. A handful of applications of fuzzy integrals include: handwriting recognition , landmine detection , pattern recognition , and decision making . In the present study, the Choquet fuzzy integral is favored for it employs the non-additive property of fuzzy measures. The Choquet integral has not lost its luster even now; instead it is gaining prominence day by day and making inroads into new applications.
The main theme of the present work is the estimation of fuzzy densities, and hence fuzzy measures for the non-additive fuzzy model are derived from GFM; thus the fuzzy densities can be calculated straightaway from the covariance matrices used in GMM . As the large number of input variables is increased for the purpose of fusion, the computational complexity grows exponentially [31, 51] with the -measure. To overcome this problem a new fuzzy measure known as -Measure is introduced in . This new measure is just normalization of -measure, and with this modification, the fuzzy densities are decorrelated. In this is taken as a variable to be learned using the training data set. The proposed modeling technique considerably reduces the computational complexity in two steps: firstly finding fuzzy densities without learning them through iterative process and secondly using -measure that eliminates the burden of solving the exponentially increasing polynomials.
This paper is organized as follows. Section 2 briefly describes the fuzzy measures. In Section 3, the formulation for the non-additive GFM is presented. An algorithm for non-additive fuzzy modeling based on -measure is given in Section 4. Two benchmark applications and one real-time electric load demand prediction application are provided in Section 5 to illustrate the proposed methodology. Finally conclusions are drawn in Section 6.
2. Fuzzy Measures
The fuzzy measure or fuzzy capacity is a subjective evaluation introduced by Choquet in 1953 and defined by Sugeno in 1974 for fuzzy integrals. Fuzzy measure includes a number of special class of measures like Sugeno’s -measure and -measure. In this work, we make use of Fuzzy measures in quantifying the influence of the interactive inputs on the output function using the fuzzy measures. The resulting model is non-additive in the sense that the information from the interactive inputs is computed non-additively. For converting GFM into a non-additive fuzzy system, the relevant properties of the fuzzy measures .
Definition 1. Let us denote as a finite set with its elements sorted from and let be the power set of , that is, set of all subsets of . Then a fuzzy measure over a set is a function given by Satisfying the two properties(1).(2)If , and , then , which is the monotonicity relation in fuzzy measure. Next we invoke the third property, namely, the so-called -measure due to Sugeno  stated as follows:(3)for all , with
Example 2. Let be the input sets satisfying the properties 1 and 2; then considering as a fuzzy density satisfying the property 3, the values of fuzzy measure, can be computed recursively as starting with the one-input set to find the first measure as It is then combined with the next input set to compute their -measure since these two input sets are disjoint, that is, , but their fuzzy sets and are dependent, that is, . Equation (3) indicates that we must consider every time the fuzzy measure of two disjoint input sets, and , which consists of inputs . It may be noted that a single element fuzzy measure is the fuzzy density itself. The following polynomial which arises from the condition (see the appendix) needs to be solved for : The solution of (5) is computationally intensive. We will now discuss the -measure to address this problem.
The -measure is burdened with the problem of finding the roots of the polynomial equation in . As the number of dimensions, increases, the complexity of the equation increases enormously. In order to surmount this problem, -measure is proposed in . It derives its name from the word “Quotient.” The -measure follows the same properties of -measure. For the finite the -measure is defined as the ratio of two fuzzy measures: where is a function of fuzzy densities and is a polynomial in . The -measure  over a set is similar to in (1): with the two properties as follows:(1) (2)If , and , then , which is the monotonicity relation.The third property is as follows:(3) for all , with , Following (9), is evaluated from The first measure is computed from
Convergence Behavior of Q-Measure
Let , Such that, , and , that is, null. Supposing and Then the -measure of and is Using (10) the -measure is written as or For , and , it is always assured that Hence, and also . Next, we have where . Therefore and the values lie in between 0 and 1. Hence we have This will continue until when we will have But ; this means that the -measure converges to unity.
3. From GMM to Nonadditive GFM with Q-Measure
The GFM  includes both the Mamdani and TS models. The Mamdani model (CRI-model) inhibits the property of fuzziness around the fixed centroid of the consequent part while the TS model gives a varying singleton for the consequent part in each fuzzy rule. The output of GFM is a fuzzy set, which is the output of Mamdani model with an index of fuzziness and with a varying centroid as the output of the TS model. To combine both of these properties, Azeem et al. [1, 53] have introduced a GFM fuzzy rule of the form
The fuzzy set may also be in the linguistic form with the output function which may be linear or nonlinear regression of inputs and the index of fuzziness . The fuzzy output of the th rule is . The defuzzified output obtained by applying additive S-norm to the consequent parts of GFM rules is with . As noted in Section 1 on introduction, GFM represents the additive fuzzy system. Using the framework of GMM, we intend to generalize to the case when the input fuzzy sets have overlapping information, that is, . So, we modify the above fuzzy rule to the nonadditive GFM as To take account of the above constraint, we form the input set from which the fuzzy measures of subsets are found recursively using (10) and (11). Using the fuzzy measures as the coefficients the output function is expressed as  Equation (23) is an additive function of fuzzy measures as explained in the sequel. As the input fuzzy sets in (22) are overlapping, we take disjoint input sets from the power set and compute their fuzzy measure. The function combines the fuzzy measures from subsequent fuzzy sets, and the resulting system arising from the fuzzy rules is called nonadditive fuzzy system unlike the output function in TS model where each coefficient corresponds to its own fuzzy set and there is no concept of fuzzy measure in this model. We now impose three constraints for the model to be called non-additive:(i)the input fuzzy sets associated with the power set are overlapping,(ii)the input sets are disjoint,(iii)the output function is additive in fuzzy measures.
We will now show that the above conditions are necessary to represent the defuzzified output of the nonadditive GFM in the Choquet integral form as follows: where for th set of the data we have
with being the diagonal element. The firing strength of each rule is obtained by taking the multiplicative T-norm of the membership functions (by suppressing the subscripts “”) of the premise parts of the rules as . We intend to compute the coefficients of the output function by reformulating (21) in the Choquet fuzzy integral form. The role of GMM is to facilitate the computation of , and .
3.1. Choquet Fuzzy Integral and Q-Measure
When a fuzzy measure is available on a finite set , we can use a fuzzy integral as a computational scheme to integrate all values from , that is, the subsets of nonlinearly. In other words, a fuzzy integral relies on the concept of a fuzzy measure. A general definition is that given a class of functions and a class of -measures , we have a functional , which is a fuzzy integral . The Choquet integral is a particular fuzzy integral  that serves as an alternative computational scheme for aggregating information.
Definition 3. Let -measure over a set be a function whose elements (discrete) are , then the Choquet integral of a positive function with respect to is defined by  or where . We assume here that and . To convert (24) into the Choquet fuzzy integral form, we shall represent coefficients in terms of -measures, as Interchanging the summations in (24) leads to Next separating out the term yields, where is a constant. Letting Equation (30) is simplified as Denoting and allows us to write (32) compactly for input variables as or where and for , is the monotonicity condition. Note that are the evidences provided by the input sources , respectively, and are the -measures for mth cluster. Here, we take to be zero. Equations (34) and (35) similar to (26) and (27) are the two forms of Choquet fuzzy integral . We have now proved that the defuzzified output of the non-additive GFM fuzzy rules is in the form of Choquet fuzzy integral. The underlying model is nonadditive GFM because both and are nonlinear functions of . Thus the Choquet integral provides the functionality of non-additive GFM. The functionality of the Choquet integral for GMM will be discussed later. We will now estimate the unknown -measures from fuzzy densities determined using (9). We then evaluate the performance of the fuzzy model with a measure, which is a function of mean square error , chosen as where is the total number of test data vectors: where is the estimated value, is the actual output, and is the output vector.
3.2. Determination of Fuzzy Densities
Equation (38) can also be written as where . Substituting from (39) in (40), we have Using (32), for rules, we have the cluster-wise estimates of the output for the non-additive fuzzy model written as From (42) it is implied that in view of (34) and (35).
Equation (42) can also be written as or Since (40) and (44) represent the estimated outputs for additive and non-additive fuzzy systems, respectively, and hence these must be equivalent to each other for . Equating the right hand sides of (40) and (44), we have Thus when the fuzzy measure . As we can express the fuzzy densities in terms of the elements of covariance matrices of GMM as  We are in a position to estimate and -measures and hence the coefficients using the fuzzy densities. We will then use all these in the Choquet integral to estimate the output. For convenience of notation we start with fuzzy measure of one element set and move in backward direction to find the -measure of . Since we are computing all -measures before the start of the Choquet integral, these become handy in computing the output. This is because in (32) the index is in the forward direction from 1 to .
3.3. Functionality of GMM
As we have already proved that the Choquet integral is the functionality of non-additive GFM, It is now easy to extend to this functionality the GMM case from the fact that the output is Gaussian in the nonadditive case too as per the expression From the above it may be noted that is obtained from using GMM by the choice of in the -measure. Hence is the mean of in the non-additive case just as is the mean of in the additive case. We can conclude therefore that when is evaluated from , the Choquet integral also assures the functionality of GMM.
4. Algorithm for Estimating the Model Parameters of the Nonadditive Fuzzy Systems
The algorithm has the following steps.
Step 1. Normalize the input-output data, so that the data values lie in between 0 and 1: where is the input-output data vector.
Step 2. Find the premise model parameters of GMM using the EM Algorithm.
Step 3. Determine fuzzy densities for each input variable in th cluster/rule using (47).
Step 4. Choose initial values of with the restriction for th cluster.
Step 6. Using (31) find for each input variable in the cluster/rule to obtain the matrix .
Step 7. Compute the estimated output using (34).
Step 8. Learn the model. This requires the following two subtasks: (a) set up an objective function  as the case required. For batch: for incremental, (b) Update the values of by the Gradient-descent learning law: where may refer to either or and is a learning factor fixed at the value 0.9 for better performance. The parameter learning can be accomplished in either of the above two ways: batch or incremental. Derivatives required in the learning process are relegated to the Appendix.
Step 10. Calculate from (36).
5. Experimental Results
5.1. Gas Furnace
Here, we take up the gas furnace data  to demonstrate the effectiveness of the proposed algorithm. In this gas furnace, the air at a feed rate of is combined with methane to produce a mixture of gases with carbon dioxide of concentration . These are used to fit the Gaussian mixture density function with four components. The input vector after sorting is and the output is . The composition of the input vector is arrived at using the fuzzy curve approach . On applying the expectation maximization (EM)  clustering, the number of clusters is found to be 4; hence the model will have four components. The five-variable GMM using the EM algorithm yields the parameters of GMM as shown in Table 1. These parameters are also the premise model parameters of the non-additive GFM fuzzy rules.
Fuzzy densities and estimated fuzzy measures of the model are given in Tables 2 and 4, respectively. The estimated values of are given in Table 3. The estimated -coefficient and -measures are given in Tables 5 and 6 respectively. Note that contains all input elements, hence its value is one. The -coefficients constitute the consequent parameters of the GMM fuzzy rules. The performance measure is found to be 0.00208. The fuzzy measures of the constituent singleton sets satisfy the monotonic relations for shown as a lattice in Figure 2.
For the training dataset containing the first 250 data vectors, . For the remaining data vectors forming the validation data set, . This is inferior to that of the training data set, as expected. The plots of actual output and model output for both training and prediction along with the prediction error of the model output are shown in Figure 1. The model output is very close to the actual output during training, but it displays small deviations from the actual output during the prediction period. Using the premise and consequent parameters of Table 1, we have constructed the membership functions for the four inputs and one output in Figure 3 where we depict the membership function of rules-1, 2, 3, and 4 using dashed lines, dotted lines, dots with dashes lines and solid lines, respectively.
The first rule consists of,(i) the weight of the rule; 0.2991,(ii) the membership functions of the premise parts(iii) The consequent part of a rule can be found using Table 6.
Similarly other rules can also be formed.
5.2. Industrial Dryer
In this, we consider the industrial dryer data obtained from the DaISy database. Fuel is combusted to produce the hot gas, which is blown by an exhaust fan on the raw material fed into the dryer. It is desired to determine the relationship between the functional elements of the dryer with the moisture content of the raw material at the dryer output. The measurements are designated by(i)fuel flow rate, ,(ii)hot gas exhaust-fan speed, ,(iii)rate of flow of raw material, ,(iv)moisture content of the raw material after drying which is .
Sampling period of 10 seconds is considered to generate a total of 867 samples. Input and output data are normalized using (40) to maintain compatibility in the range 0 to 1. The input variables , , , and affect the current output . The input-output vectors are represented by . Out of these vectors 70% is taken as the training set. The remaining 30% is kept for prediction and evaluation of our proposed model.
The number of clusters is found to be 4 using EM clustering, which yields the parameters of GMM in Table 7, and these are also the premise model parameters of the non-additive GFM.