Abstract

Lift, leverage, and conviction are three of the best commonly known interest measures for crisp association rules. All of them are based on a comparison of observed support and the support that is expected if the antecedent and consequent part of the rule were stochastically independent. The aim of this paper is to provide a correct definition of lift, leverage, and conviction measures for fuzzy association rules and to study some of their interesting mathematical properties.

1. Introduction

Searching for association rules is a broadly discussed, developed, and accepted data mining technique [1, 2]. An association rule is an expression , where antecedent and consequent are conditions, the former usually in the form of elementary conjunction and the latter being usually atomic. Such rules are usually interpreted as the following implicational statement: “if is satisfied then is true very often too.”

Naturally, analysts are interested only in such rules that are somehow interesting, unusual, or exceptional. To assess rule interestingness objectively, there have been developed many measures of rule interest or intensity. Among the most essential, support and confidence are traditionally considered. An objective of association rules mining is to find rules with support and confidence above some user-defined thresholds.

Searching for association rules fits particularly well on binary or categorical data and many have been written on that topic [14]. For association analysis on numeric data, a prior discretization is proposed, for example, by Srikant and Agrawal [5]. Another alternative is to take an advantage of fuzzy logic [6].

The use of fuzzy logic in connection with association rules has been motivated by many authors (see e.g., [7] for recent overview). Fuzzy association rules are appealing also because of the use of vague linguistic terms such as “small” and “very big” [811].

In this paper, we focus on three measures of rule intensity that are all based on comparison between the observed support and the support that is expected under the assumption of independence of the rule’s antecedent and consequent. These measures are lift, leverage, and conviction. All of them were initially developed for nonfuzzy (i.e., “crisp”) association rules.

Lift was firstly described in [12] under its original name “interest.” It was well studied for association rules on binary data in [13, 14]. Lift is defined as a ratio of observed support to the support that is expected under the assumption of independence of and .

On the other hand, leverage [15] measures the difference of observed and expected supports. (Hence, it is very similar to lift.)

Conviction [12] is defined as a ratio of expected and observed support of . Although it is very similar to lift, its properties are more similar to confidence. Lallich et al. [16] provide a nice overview of many other crisp rule measures.

As quite many have been written about these measures for crisp association rules, not so much has been done with respect to fuzzy rules. Unfortunately, simplicity of definitions for crisp rules sometimes led to oversimplified definitions for fuzzy rules. Some authors believe the generalization of those crisp measures for fuzzy data is as trivial as substituting crisp terms with analogous fuzzy terminology inside of crisp-case definitions; see, for example, [17, 18]. Unfortunately, as discussed in this paper, such oversimplification may lead to erroneous outputs. In order to preserve some nice mathematical properties, one must take care of the type of the -norms being used.

In Section 2, a brief theoretical background for both binary and fuzzy association rules is provided. Section 2.3 discusses shortly naive definitions of lift, leverage, and conviction and shows a simple example where the rational interpretations of these measures become broken. Before providing corrected definitions in Section 4, an essential notion of expected support is analysed in Section 3. Finally, Section 5 concludes the paper with summarization of the achieved goals and drawings of the possible directions of future research.

2. Theoretical Background

2.1. Binary Association Rules

Let , , be a finite set of objects and let , , be a finite set of attributes (features). Each attribute can be considered as a logical predicate: is true (or false) accordingly to whether the th attribute applies (or does not apply) to object . For a subset of attributes, let us define a new predicate of a logical conjunction of the attributes contained in as follows:

Moreover, let us define a negated predicate as follows:

An association rule is a formula , where is an antecedent, is a consequent, and . (Typically, and .) Both and are sometimes called itemsets. Please consider the following rule as an example:

The support and confidence are defined as follows [2, 19]: where . Evidently, .

If is a random sample, then observing is a random event. Then also a random event may be defined on the basis of the truth value of the predicate and support becomes an estimate of a probability . Then also would correspond to the probability , where is complementary event to . Confidence would then be an estimate of conditional probability .

Lift [12], leverage [15], and conviction [12] for binary data are defined as follows:

If and are stochastically independent, ; that is, the expression is an estimation of support under the assumption of and being independent.

Hence, lift is a ratio of observed support to the support that is expected under the assumption of independence, leverage is a difference of observed and expected support, and conviction is a ratio of the expected support of appearing without to the observed support .

If and are independent, , , and . The values of , , and indicate positive relationship, while , , and indicate negative relationship.

2.2. Fuzzy Association Rules

For fuzzy association rules, domain of each fuzzy attribute is not binary (or “crisp”) but graded (or “fuzzy”), that is, interval . That is, for each and , . For a subset of fuzzy attributes, we define a new predicate of a logical conjunction (similarly to binary case (1)) by using a -norm as

T-norm is a generalized logical conjunction, that is, a function which is associative, commutative, and monotone increasing (in both places) and which satisfies the boundary conditions and for each . Some well-known examples of -norms are as follows:(i)product -norm: ;(ii)minimum -norm: ;(iii)Łukasiewicz -norm: .

Negation of fuzzy predicate is often defined for any as . Then, we can define (similarly to (2)) a negated predicate as follows:

Let , , , , , , , and . Several intensity measures may be defined as follows:

Convention 1. Throughout this text, we assume is an arbitrary (but fixed) -norm. Where it is important to explicitly specify a concrete -norm, say , we put in subscript and write, for example, instead of or instead of .

Convention 2. For the sake of simplicity, we will sometimes express a fuzzy attribute as a vector of membership degrees. For instance, suppose is a fuzzy attribute on a set of objects . Then, instead of writing , , and , we will use a much concise form: .

Example 1. Let , , and be fuzzy attributes. Then,

2.3. Naive Definition of Lift, Leverage, and Conviction for Fuzzy Association Rules

A naive approach for introducing lift, leverage, and conviction into the fuzzy association rules framework is to use simply their definitions (7), (8), and (9) for binary rules and replace binary support (4) and (5) and confidence (6) with their fuzzy alternatives (12) as, for example, in [17, 18]. Unfortunately, that approach works well only for being the product -norm. As indicated in the following experiment, using minimum or Łukasiewicz -norms may lead to erroneous interpretations.

Experiment 1. Two vectors and of size were randomly generated from the uniform distribution on the interval so that they are stochastically independent. Above-described naive versions of lift, leverage, and conviction of a rule were computed with using minimum, product, and Łukasiewicz -norms as ; see Table 1 for results.
As discussed in Section 2.1, stochastically independent data are expected to result in lift and conviction being close to 1 while leverage is expected to be close to 0. As can be seen, this is the case only for product -norm.

The values of naive lift and naive leverage wrongly indicate positive (resp., negative) relationship, if the minimum (resp., Łukasiewicz) -norm is used. Paradoxically, naive conviction indicates opposite sign of relationship.

3. Expected Support of a Conjunction of Fuzzy Attributes under the Assumption of Independence

As indicated in Experiment 1 presented in Section 2.3 above, naive lift, leverage, and conviction no more behave like their alternatives for crisp data: they no more represent a ratio of what is observed to what is expected under the assumption of independence. To recover their definitions, independency of fuzzy attributes must be treated correctly. Only then a proper definitions of lift, leverage, and conviction can be formulated.

Given sets and of fuzzy attributes, what support of is expected if and are independent? Moreover, what does independency of fuzzy attributes mean?

For the sake of simplicity, let us assume and are sets containing a single fuzzy attribute; that is, . For more complex cases, a new attribute can be created from the set of fuzzy attributes by using (10).

If is a set of randomly selected objects, one can consider membership values and as random variables and and treat the independence of fuzzy attributes as stochastic independence of random variables and .

Two random variables are stochastically independent, if the combined random variable has a joint probability density as If and are two independent random variables from interval , then is a random variable with probability density function .

By definition, expected value of a random variable is a weighted average of all possible values. More formally, where is a probability density function of random variable .

Similarly, an expected value is a weighted average of all possible pairs, namely,

In real setting, is unknown but we can estimate its values from data (i.e., from objects and their fuzzy attributes ) by using the assumption of independence (14) as follows: where is the number of objects from that belong to with degree ; that is, .

Assuming and , we obtain, from (15), (17), and (18),

Since and are independent, and hence expected value of is

Now, we are ready to define notions of expected support and expected confidence.

Definition 2. Let be a -norm, and let be sets of fuzzy attributes such that and . Then, the expected fuzzy support and the expected fuzzy confidence are defined as follows:

Proposition 3. Let be sets of fuzzy attributes. Then, (1)if , then ,(2).

Proof. (1) If , then and hence Therefore also .
(2) For any -norm holds that . Therefore, and similarly for . Hence, .

Remark 4. Note that the reverse direction of the first implication of Proposition 3 is not generally true. For example, for and , we have but .

Proposition 5. Let be sets of fuzzy attributes and let ; that is, the product -norm is being used as a conjunction. Then,

Proof. If , equals

Proposition 6. Let be sets of fuzzy attributes and let ; that is, the minimum -norm is being used as a conjunction. Then,

Proof. Let . The first inequality follows from the fact that as The second inequality follows directly from Proposition 3.

Proposition 7. Let be sets of fuzzy attributes and let ; that is, the Łukasiewicz -norm is being used as a conjunction. Then,

Proof. Let , , and . To prove the first inequality, it suffices to prove which is obvious for . Let us therefore assume ; then, (30) can be rewritten as Then, where is a sequence of numbers and , for . Since and each , we can immediately see that hence, (30) holds.
The second inequality follows from because then we have

4. Correct Definition of Lift, Leverage, and Conviction for Fuzzy Association Rules

Now, we are ready to provide a correct definition of lift, leverage, and conviction in the framework of fuzzy association rules.

Definition 8. Let be sets of fuzzy attributes with . Then, lift, leverage, and conviction of a fuzzy association rule are defined as follows:

Let us now study some interesting properties of the newly defined notions.

Proposition 9. Let be sets of fuzzy attributes. Then, (1);(2);(3);(4)if , then .

Proof. (1) and (2) directly follow from the definitions and from the fact that -norms are commutative.
(3) Since the membership degrees are defined on interval , their sums cannot be negative either. Hence, . Next, assume to the contrary that as follows: therefore, which is a contradiction.
(4) If , then, from Proposition 3, we know that also . Therefore, exists and is greater than .

Proposition 10. Let be sets of fuzzy attributes. Then, (1),(2),(3).

Proof. (1) and (2) directly follow from the definitions and from the fact that -norms are commutative.
(3) Let and . Evidently, and . Then, . Obviously, and . Therefore,

Proposition 11. Let be sets of fuzzy attributes. Then, (1);(2);(3);(4).

Proof. Everything directly follows from Definition 8 and Proposition 9.

Corollary 12. Let be sets of fuzzy attributes and let ; that is, the product -norm is being used as a conjunction. Then,

Proof. Everything directly follows from Definitions 2 and 8, from Proposition 5, and from the fact that .

Corollary 12 copies properties that are well known for crisp variants of lift, leverage, and conviction. In practice, the use of these equations is much more convenient than the original ones from Definition 8. However, note that Corollary 12 holds only if product -norm is used. For other -norms such as minimum or Łukasiewicz , Definition 8 must not be oversimplified that way. See the subsequent corollaries for more details.

Corollary 13. Let be sets of fuzzy attributes and let ; that is, the minimum -norm is being used as a conjunction. Then, where , , , , and .

Proof. Everything directly follows from Definitions 2 and 8 and from Propositions 6 and 11.

Corollary 14. Let be sets of fuzzy attributes and let ; that is, the Łukasiewicz -norm is being used as a conjunction. Then, where , , , , and .

Proof. Everything directly follows from Definitions 2 and 8 and from Propositions 7 and 11.

Experiment 2. The same data as in Experiment 1 were processed accordingly to correct definitions of lift, leverage, and conviction (see Definition 8). The results can be found in Table 2. Since the data are randomly generated from uniform distribution, they are stochastically independent; hence, lift and conviction should be close to 1 and lift should be close to 0 regardless of -norm being used. As one can see, the results in Table 2 are perfectly in compliance with our expectations.

5. Conclusion

Lift is a ratio of observed support (resp., confidence) to the support (resp., confidence) that is expected under the assumption of independence. Leverage is similar to lift, since it is a difference between observed and expected support. Conviction is often treated as an alternative to confidence; nevertheless, it is defined on the basis of observed and expected support too.

In this paper, a correct definition of lift, leverage, and conviction for fuzzy data was provided. It should be stressed here that there already exist some research papers that use incorrect (a.k.a. “naive”) definition of fuzzy lift (e.g., [18]) that is also discussed here.

The naive definition does not preserve interpretation of positive/negative relationship. For crisp lift and conviction (resp., leverage), the stochastically independent attributes and yield , , and . As shown in Experiment 1, this is no more the case for naive definitions of these measures. On the other hand, Experiment 2 shows that correct definitions of lift, leverage, and conviction again recover that feature for fuzzy data.

All the lift, leverage, and conviction definitions are similar to their “crisp” alternatives (i.e., definitions for binary data) if the -norm being used is the product . For Łukasiewicz and minimum   -norms, a more complicated computation takes place.

In [20], an algorithm was developed in for fast evaluation of fuzzy lift. A future research will therefore address improvements of association rules search algorithms by introducing fast computations of other measures, adding pruning heuristics based on boundary conditions provided by Corollaries 13 and 14. Also, other interest measures may be studied and their applicability to fuzzy rules may be considered.

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.

Acknowledgment

This work was supported by the European Regional Development Fund in the Project of IT4Innovations Centre of Excellence (CZ.1.05/1.1.00/02.0070, VP6).