Advances in Fuzzy Systems

Volume 2014, Article ID 197876, 7 pages

http://dx.doi.org/10.1155/2014/197876

## Interest Measures for Fuzzy Association Rules Based on Expectations of Independence

Institute for Research and Applications of Fuzzy Modeling, Centre of Excellence IT4Innovations, University of Ostrava, 30. dubna 22, 701 03 Ostrava, Czech Republic

Received 25 August 2014; Accepted 9 September 2014; Published 7 October 2014

Academic Editor: Salvatore Sessa

Copyright © 2014 Michal Burda. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Lift, leverage, and conviction are three of the best commonly known interest measures for crisp association rules. All of them are based on a comparison of observed support and the support that is expected if the antecedent and consequent part of the rule were stochastically independent. The aim of this paper is to provide a correct definition of lift, leverage, and conviction measures for fuzzy association rules and to study some of their interesting mathematical properties.

#### 1. Introduction

Searching for association rules is a broadly discussed, developed, and accepted data mining technique [1, 2]. An association rule is an expression , where antecedent and consequent are conditions, the former usually in the form of elementary conjunction and the latter being usually atomic. Such rules are usually interpreted as the following implicational statement: “if is satisfied then is true very often too.”

Naturally, analysts are interested only in such rules that are somehow interesting, unusual, or exceptional. To assess rule interestingness objectively, there have been developed many measures of rule interest or intensity. Among the most essential,* support* and* confidence* are traditionally considered. An objective of association rules mining is to find rules with* support* and* confidence* above some user-defined thresholds.

Searching for association rules fits particularly well on binary or categorical data and many have been written on that topic [1–4]. For association analysis on numeric data, a prior discretization is proposed, for example, by Srikant and Agrawal [5]. Another alternative is to take an advantage of fuzzy logic [6].

The use of fuzzy logic in connection with association rules has been motivated by many authors (see e.g., [7] for recent overview). Fuzzy association rules are appealing also because of the use of vague linguistic terms such as “small” and “very big” [8–11].

In this paper, we focus on three measures of rule intensity that are all based on comparison between the observed support and the support that is expected under the assumption of independence of the rule’s antecedent and consequent. These measures are* lift*,* leverage*, and* conviction*. All of them were initially developed for nonfuzzy (i.e., “crisp”) association rules.

*Lift* was firstly described in [12] under its original name “interest.” It was well studied for association rules on binary data in [13, 14]. Lift is defined as a ratio of observed support to the support that is expected under the assumption of independence of and .

On the other hand,* leverage* [15] measures the difference of observed and expected supports. (Hence, it is very similar to lift.)

*Conviction* [12] is defined as a ratio of expected and observed support of . Although it is very similar to lift, its properties are more similar to confidence. Lallich et al. [16] provide a nice overview of many other crisp rule measures.

As quite many have been written about these measures for crisp association rules, not so much has been done with respect to fuzzy rules. Unfortunately, simplicity of definitions for crisp rules sometimes led to oversimplified definitions for fuzzy rules. Some authors believe the generalization of those crisp measures for fuzzy data is as trivial as substituting crisp terms with analogous fuzzy terminology inside of crisp-case definitions; see, for example, [17, 18]. Unfortunately, as discussed in this paper, such oversimplification may lead to erroneous outputs. In order to preserve some nice mathematical properties, one must take care of the type of the -norms being used.

In Section 2, a brief theoretical background for both binary and fuzzy association rules is provided. Section 2.3 discusses shortly naive definitions of lift, leverage, and conviction and shows a simple example where the rational interpretations of these measures become broken. Before providing corrected definitions in Section 4, an essential notion of* expected support* is analysed in Section 3. Finally, Section 5 concludes the paper with summarization of the achieved goals and drawings of the possible directions of future research.

#### 2. Theoretical Background

##### 2.1. Binary Association Rules

Let , , be a finite set of objects and let , , be a finite set of attributes (features). Each attribute can be considered as a logical predicate: is true (or false) accordingly to whether the th attribute applies (or does not apply) to object . For a subset of attributes, let us define a new predicate of a logical conjunction of the attributes contained in as follows:

Moreover, let us define a negated predicate as follows:

An association rule is a formula , where is an* antecedent*, is a* consequent,* and . (Typically, and .) Both and are sometimes called* itemsets*. Please consider the following rule as an example:

The* support* and* confidence* are defined as follows [2, 19]:
where . Evidently, .

If is a random sample, then observing is a random event. Then also a random event may be defined on the basis of the truth value of the predicate and support becomes an estimate of a probability . Then also would correspond to the probability , where is complementary event to . Confidence would then be an estimate of conditional probability .

*Lift* [12],* leverage* [15], and* conviction* [12] for binary data are defined as follows:

If and are stochastically independent, ; that is, the expression is an estimation of support under the assumption of and being independent.

Hence, lift is a ratio of observed support to the support that is expected under the assumption of independence, leverage is a difference of observed and expected support, and conviction is a ratio of the expected support of appearing without to the observed support .

If and are independent, , , and . The values of , , and indicate positive relationship, while , , and indicate negative relationship.

##### 2.2. Fuzzy Association Rules

For fuzzy association rules, domain of each fuzzy attribute is not binary (or “crisp”) but graded (or “fuzzy”), that is, interval . That is, for each and , . For a subset of fuzzy attributes, we define a new predicate of a logical conjunction (similarly to binary case (1)) by using a -norm as

*T-norm * is a generalized logical conjunction, that is, a function which is associative, commutative, and monotone increasing (in both places) and which satisfies the boundary conditions and for each . Some well-known examples of -norms are as follows:(i)product -norm: ;(ii)minimum -norm: ;(iii)Łukasiewicz -norm: .

Negation of fuzzy predicate is often defined for any as . Then, we can define (similarly to (2)) a negated predicate as follows:

Let , , , , , , , and . Several intensity measures may be defined as follows:

*Convention 1. *Throughout this text, we assume is an arbitrary (but fixed) -norm. Where it is important to explicitly specify a concrete -norm, say , we put in subscript and write, for example, instead of or instead of .

*Convention 2. *For the sake of simplicity, we will sometimes express a fuzzy attribute as a vector of membership degrees. For instance, suppose is a fuzzy attribute on a set of objects . Then, instead of writing , , and , we will use a much concise form: .

*Example 1. *Let , , and be fuzzy attributes. Then,

##### 2.3. Naive Definition of Lift, Leverage, and Conviction for Fuzzy Association Rules

A naive approach for introducing lift, leverage, and conviction into the fuzzy association rules framework is to use simply their definitions (7), (8), and (9) for binary rules and replace binary support (4) and (5) and confidence (6) with their fuzzy alternatives (12) as, for example, in [17, 18]. Unfortunately, that approach works well only for being the product -norm. As indicated in the following experiment, using minimum or Łukasiewicz -norms may lead to erroneous interpretations.

*Experiment 1. *Two vectors and of size were randomly generated from the uniform distribution on the interval so that they are stochastically independent. Above-described naive versions of lift, leverage, and conviction of a rule were computed with using minimum, product, and Łukasiewicz -norms as ; see Table 1 for results.

As discussed in Section 2.1, stochastically independent data are expected to result in lift and conviction being close to 1 while leverage is expected to be close to 0. As can be seen, this is the case only for product -norm.

The values of naive lift and naive leverage wrongly indicate positive (resp., negative) relationship, if the minimum (resp., Łukasiewicz) -norm is used. Paradoxically, naive conviction indicates opposite sign of relationship.

#### 3. Expected Support of a Conjunction of Fuzzy Attributes under the Assumption of Independence

As indicated in Experiment 1 presented in Section 2.3 above, naive lift, leverage, and conviction no more behave like their alternatives for crisp data: they no more represent a ratio of what is observed to what is expected under the assumption of independence. To recover their definitions, independency of fuzzy attributes must be treated correctly. Only then a proper definitions of lift, leverage, and conviction can be formulated.

Given sets and of fuzzy attributes, what support of is expected if and are independent? Moreover, what does independency of fuzzy attributes mean?

For the sake of simplicity, let us assume and are sets containing a single fuzzy attribute; that is, . For more complex cases, a new attribute can be created from the set of fuzzy attributes by using (10).

If is a set of randomly selected objects, one can consider membership values and as random variables and and treat the independence of fuzzy attributes as stochastic independence of random variables and .

Two random variables are stochastically independent, if the combined random variable has a joint probability density as If and are two independent random variables from interval , then is a random variable with probability density function .

By definition,* expected value * of a random variable is a weighted average of all possible values. More formally,
where is a probability density function of random variable .

Similarly, an expected value is a weighted average of all possible pairs, namely,

In real setting, is unknown but we can estimate its values from data (i.e., from objects and their fuzzy attributes ) by using the assumption of independence (14) as follows: where is the number of objects from that belong to with degree ; that is, .

Assuming and , we obtain, from (15), (17), and (18),

Since and are independent, and hence expected value of is

Now, we are ready to define notions of* expected support* and* expected confidence*.

*Definition 2. *Let be a -norm, and let be sets of fuzzy attributes such that and . Then, the* expected fuzzy support * and the* expected fuzzy confidence * are defined as follows:

Proposition 3. *Let be sets of fuzzy attributes. Then, *(1)*if , then ,*(2)*.*

*Proof. *(1) If , then
and hence
Therefore also .

(2) For any -norm holds that . Therefore,
and similarly for . Hence, .

*Remark 4. *Note that the reverse direction of the first implication of Proposition 3 is not generally true. For example, for and , we have but .

*Proposition 5. Let be sets of fuzzy attributes and let ; that is, the product -norm is being used as a conjunction. Then,
*

*Proof. *If , equals

*Proposition 6. Let be sets of fuzzy attributes and let ; that is, the minimum -norm is being used as a conjunction. Then,
*

*Proof. *Let . The first inequality follows from the fact that as
The second inequality follows directly from Proposition 3.

*Proposition 7. Let be sets of fuzzy attributes and let ; that is, the Łukasiewicz -norm is being used as a conjunction. Then,
*

*Proof. *Let , , and . To prove the first inequality, it suffices to prove
which is obvious for . Let us therefore assume ; then, (30) can be rewritten as
Then,
where is a sequence of numbers and , for . Since and each , we can immediately see that
hence, (30) holds.

The second inequality follows from because then we have

*4. Correct Definition of Lift, Leverage, and Conviction for Fuzzy Association Rules*

*4. Correct Definition of Lift, Leverage, and Conviction for Fuzzy Association Rules**Now, we are ready to provide a correct definition of lift, leverage, and conviction in the framework of fuzzy association rules.*

*Definition 8. *Let be sets of fuzzy attributes with . Then, lift, leverage, and conviction of a fuzzy association rule are defined as follows:

*Let us now study some interesting properties of the newly defined notions.*

*Proposition 9. Let be sets of fuzzy attributes. Then, (1);(2);(3);(4)if , then .*

*Proof. *(1) and (2) directly follow from the definitions and from the fact that -norms are commutative.

(3) Since the membership degrees are defined on interval , their sums cannot be negative either. Hence, . Next, assume to the contrary that as follows:
therefore,
which is a contradiction.

(4) If , then, from Proposition 3, we know that also . Therefore, exists and is greater than .

*Proposition 10. Let be sets of fuzzy attributes. Then, (1),(2),(3).*

*Proof. *(1) and (2) directly follow from the definitions and from the fact that -norms are commutative.

(3) Let and . Evidently, and . Then, . Obviously, and . Therefore,

*Proposition 11. Let be sets of fuzzy attributes. Then, (1);(2);(3);(4).*

*Proof. *Everything directly follows from Definition 8 and Proposition 9.

*Corollary 12. Let be sets of fuzzy attributes and let ; that is, the product -norm is being used as a conjunction. Then,
*

*Proof. *Everything directly follows from Definitions 2 and 8, from Proposition 5, and from the fact that .

*Corollary 12 copies properties that are well known for crisp variants of lift, leverage, and conviction. In practice, the use of these equations is much more convenient than the original ones from Definition 8. However, note that Corollary 12 holds only if product -norm is used. For other -norms such as minimum or Łukasiewicz , Definition 8 must not be oversimplified that way. See the subsequent corollaries for more details.*

*Corollary 13. Let be sets of fuzzy attributes and let ; that is, the minimum -norm is being used as a conjunction. Then,
where , , , , and .*

*Proof. *Everything directly follows from Definitions 2 and 8 and from Propositions 6 and 11.

*Corollary 14. Let be sets of fuzzy attributes and let ; that is, the Łukasiewicz -norm is being used as a conjunction. Then,
where , , , , and .*

*Proof. *Everything directly follows from Definitions 2 and 8 and from Propositions 7 and 11.

*Experiment 2. *The same data as in Experiment 1 were processed accordingly to correct definitions of lift, leverage, and conviction (see Definition 8). The results can be found in Table 2. Since the data are randomly generated from uniform distribution, they are stochastically independent; hence, lift and conviction should be close to 1 and lift should be close to 0 regardless of -norm being used. As one can see, the results in Table 2 are perfectly in compliance with our expectations.

*5. Conclusion*

*Lift is a ratio of observed support (resp., confidence) to the support (resp., confidence) that is expected under the assumption of independence. Leverage is similar to lift, since it is a difference between observed and expected support. Conviction is often treated as an alternative to confidence; nevertheless, it is defined on the basis of observed and expected support too.*

*In this paper, a correct definition of lift, leverage, and conviction for fuzzy data was provided. It should be stressed here that there already exist some research papers that use incorrect (a.k.a. “naive”) definition of fuzzy lift (e.g., [18]) that is also discussed here.*

*The naive definition does not preserve interpretation of positive/negative relationship. For crisp lift and conviction (resp., leverage), the stochastically independent attributes and yield , , and . As shown in Experiment 1, this is no more the case for naive definitions of these measures. On the other hand, Experiment 2 shows that correct definitions of lift, leverage, and conviction again recover that feature for fuzzy data.*

*All the lift, leverage, and conviction definitions are similar to their “crisp” alternatives (i.e., definitions for binary data) if the -norm being used is the product . For Łukasiewicz and minimum -norms, a more complicated computation takes place.*

*In [20], an algorithm was developed in for fast evaluation of fuzzy lift. A future research will therefore address improvements of association rules search algorithms by introducing fast computations of other measures, adding pruning heuristics based on boundary conditions provided by Corollaries 13 and 14. Also, other interest measures may be studied and their applicability to fuzzy rules may be considered.*

*Conflict of Interests*

*The author declares that there is no conflict of interests regarding the publication of this paper.*

*Acknowledgment*

*This work was supported by the European Regional Development Fund in the Project of IT4Innovations Centre of Excellence (CZ.1.05/1.1.00/02.0070, VP6).*

*References*

- M. Hahsler and K. Hornik, “New probabilistic interest measures for association rules,”
*Intelligent Data Analysis*, vol. 11, no. 5, pp. 437–455, 2007. View at Google Scholar · View at Scopus - R. Agrawal, T. Imielinski, and A. Swami, “Mining association rules between sets of items in large databases,” in
*Proceedings of the ACM SIGMOD International Conference on Management of Data*, pp. 207–216, Washington, DC, USA, May 1993. View at Scopus - A. Berrado and G. C. Runger, “Using metarules to organize and group discovered association rules,”
*Data Mining and Knowledge Discovery*, vol. 14, no. 3, pp. 409–431, 2007. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus - G. I. Webb, “Discovering significant patterns,”
*Machine Learning*, vol. 68, no. 1, pp. 1–33, 2007. View at Google Scholar - R. Srikant and R. Agrawal, “Mining quantitative association rules in large relational tables,”
*SIGMOD Record*, vol. 25, no. 2, pp. 1–12, 1996. View at Google Scholar · View at Scopus - V. Novák, I. Perfilieva, and J. Močkoř,
*Mathematical Principles of Fuzzy Logic*, Kluwer, Boston, Mass, USA, 1999. - H. Kalia, S. Dehuri, and A. Ghosh, “A survey on fuzzy association rule mining,”
*International Journal of Data Warehousing and Mining*, vol. 9, no. 1, pp. 1–27, 2013. View at Publisher · View at Google Scholar · View at Scopus - K. C. C. Chen and W.-H. Au,
*Mining Fuzzy Association Rules*, ACM, New York, NY, USA, 1997. - V. Novák, I. Perfilieva, A. Dvořák, G. Chen, Q. Wei, and P. Yan, “Mining pure linguistic associations from numerical data,”
*International Journal of Approximate Reasoning*, vol. 48, no. 1, pp. 4–22, 2008. View at Publisher · View at Google Scholar · View at Scopus - M. Burda, “Fast evaluation of t-norms for fuzzy association rules mining,” in
*Proceedings of the 14th IEEE International Symposium on Computational Intelligence and Informatics (CINTI '13)*, pp. 465–470, Budapest, Hungary, November 2013. View at Publisher · View at Google Scholar · View at Scopus - M. Burda, V. Pavliska, and R. Valášek, “Parallel mining of fuzzy association rules on dense data sets,” in
*Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE '14)*, pp. 2156–2162, IEEE, Beijing, China, 2014. - S. Brin, R. Motwani, J. D. Ullman, and S. Tsur, “Dynamic itemset counting and implication rules for market basket data,” in
*Proceedings ACM SIGMOD International Conference on Management of Data (SIGMOD '97)*, pp. 255–264, Tucson, Ariz, USA, 1997. - P. D. McNicholas, T. B. Murphy, and M. O'Regan, “Standardising the lift of an association rule,”
*Computational Statistics & Data Analysis*, vol. 52, no. 10, pp. 4712–4721, 2008. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus - P. Hájek, I. Havel, and M. Chytil, “The GUHA method of automatic hypotheses determination,”
*Computing*, vol. 1, no. 4, pp. 293–308, 1966. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus - G. Piatetsky-Shapiro, “Discovery, analysis, and presentation of strong rules,” in
*Knowledge Discovery in Databases*, pp. 229–248, AAAI/MIT Press, 1991. View at Google Scholar - S. Lallich, O. Teytaud, and E. Prudhomme, “Association rule interestingness: measure and statistical validation,” in
*Quality Measures in Data Mining*, vol. 43 of*Studies in Computational Intelligence*, pp. 251–275, Springer, Berlin, Germany, 2007. View at Publisher · View at Google Scholar - A. L. Buczak and C. M. Gifford, “Fuzzy association rule mining for community crime pattern discovery,” in
*Proceedings of the ACM SIGKDD Workshop on Intelligence and Security Informatics (ISI-KDD '10)*, vol. 2, pp. 1–10, ACM, New York, NY, USA, 2010. View at Publisher · View at Google Scholar - D. P. Pancho, J. M. Alonso, J. Alcalá-Fdez, and L. Magdalena, “Interpretability analysis of fuzzy association rules supported by fingrams,” in
*Proceedings of the 8th Conference of the European Society for Fuzzy Logic and Technology (EUSFLAT '13)*, pp. 469–474, September 2013. View at Scopus - R. Agrawal, “Fast discovery of association rules,” in
*Advances in Knowledge Discovery and Data Mining*, pp. 307–328, AAAI Press/MIT Press, Cambridge, Mass, USA, 1996. View at Google Scholar - M. Burda, “Lift measure for fuzzy association rules,” in
*Proceedings of the 7th International Conference on Soft Methods in Probability and Statistics (SMPS '2014)*, Springer, Warsaw, Poland, September 2014.

*
*