A New Approach for Resolving Conflicts in Actionable Behavioral Rules
Knowledge is considered actionable if users can take direct actions based on such knowledge to their advantage. Among the most important and distinctive actionable knowledge are actionable behavioral rules that can directly and explicitly suggest specific actions to take to influence (restrain or encourage) the behavior in the users’ best interest. However, in mining such rules, it often occurs that different rules may suggest the same actions with different expected utilities, which we call conflicting rules. To resolve the conflicts, a previous valid method was proposed. However, inconsistency of the measure for rule evaluating may hinder its performance. To overcome this problem, we develop a new method that utilizes rule ranking procedure as the basis for selecting the rule with the highest utility prediction accuracy. More specifically, we propose an integrative measure, which combines the measures of the support and antecedent length, to evaluate the utility prediction accuracies of conflicting rules. We also introduce a tunable weight parameter to allow the flexibility of integration. We conduct several experiments to test our proposed approach and evaluate the sensitivity of the weight parameter. Empirical results indicate that our approach outperforms those from previous research.
Data mining is the process of discovering patterns in large data sets and then processing these obtained patterns into understandable form for further use [1, 2]. The primary motive behind data mining process is to provide support in decision making process by detecting useful or actionable patterns from a large volume of data . Extensive research has been conducted in data mining to discover hidden patterns from underlying data. However, the knowledge obtained through these data mining techniques is not always useful to the user, and it generally requires significant amount of expertise in postprocessing.
Tradition data mining studies concentrated primarily on predictive mining, where the cause and effect scenario is being described. But this information alone is not sufficient as it does not provide much benefit to the user. What becomes even more interesting and critical to business or organizations is to make the mined patterns or knowledge actionable . Knowledge is considered actionable if users can take direct actions based on such knowledge to their advantage. Actionability is a very important criterion measuring the interestingness of mined patterns. Among the most important and distinctive actionable knowledge are actionable behavioral rules that can directly and explicitly suggest specific actions to take to influence (restrain or encourage) the behavior in the users’ best interest . Mining such rules is also an active research area of social computing .
In mining such rules, all possible action combinations are considered in turn as a rule’s condition. Therefore, candidate actionable behavioral rules may share actions in their conditions. This may lead to several conflicting rules with the same actions, yet different consequences and expected utilities. A rule’s expected utility is the utility prediction when the actions suggested by the rule are actually taken. Hence, conflicting rules give different utility predictions for the same actions. Naturally, the rule with the most accurate utility prediction should be reserved for the user and the others should be pruned. Therefore, the key problem is how to accurately evaluate the utility prediction accuracies of conflicting rules.
To resolve conflicting rules, a pruning method was introduced in . Firstly, each general rule whose antecedent is part of another rule is pruned. Subsequently, the remaining rules sharing the same actions are consolidated into a new rule according to their supports. A new rule’s expected utility is in fact the weighted average of the remaining rules’ expected utilities with their supports as the weights. A major disadvantage of this method lies in the fact that the measure for pruning, namely, antecedent length, is not used consistently. That is to say, antecedent length is only used for some of the conflicting rules.
In this paper, to overcome the shortcomings of the previous method proposed in , a new rule ranking method is developed to resolve the conflicts in actionable behavioral rules. More specifically, an integrative measure, which linearly combines the measures of the support and antecedent length, is proposed to evaluate the utility prediction accuracies of conflicting rules. Furthermore, a tunable weight parameter will be introduced in the proposed measure to allow the flexibility of integration. We conduct an experiment to validate our proposed method and evaluate the performance of the proposed method against the previous method and random selection. The experimental results strongly suggest the validity and the superiority of our approach.
Related research on rule pruning in associative classification (AC) inspired our work. Most associative classification algorithms favor rules with large support and confidence values. Another important measure used to determine the precedence of the rules is rule antecedent length. Some AC algorithms, such as those in [7, 8], tend to prefer general rules (those with shorter antecedents) and consequently suffer from poor classification accuracy. On the contrary, other algorithms, such as those in [9, 10], tend to prefer specific rules, reducing the chance of misclassification. However, these methods may lead to more specific rules (i.e., those with longer antecedents). Thabtah argued that ranking of rules should not be limited to support and confidence parameters and proposed a rule ranking procedure that takes into account the class distribution frequency of each rule after considering confidence, support, and rule antecedent length .
Mining big data for complex decision making has been a major challenge for next generation data mining. The resolution of the conflicting rule may have significant implications. Actionable knowledge discovery (AKD) not only provides an important tool to decision makers to take appropriate actions but also delivers reliable and actionable outcomes to homeland security and to businesses, to name a few. The rest of the paper is organized as follows. Section 2 introduces the formal definition of actionable behavioral rules mining. Section 3 uses an example to illustrate the problem. Section 4 describes our proposed method. Section 5 presents our experimental study demonstrating the validity and the superiority of the proposed method. Section 6 reviews some of the related works. Finally, Section 7 summarizes our contributions and the directions for future research.
2. Problem Definitions and Elaborations
To understand the domain in actionable behavioral rule mining, we first introduce some formal definitions listed in Table 1.
Within a behavioral information system, is a set of behavior attributes describing the behaviors of the entity, and is a set of environment attributes characterizing the environment in which the entity situates and having some causal influence upon behavior attributes. An action set , also called a -action set, is defined as a finite nonempty set of actions such that for any . We say that action set holds, if every holds. We say that observation supports , if for every . The support of is defined as
We call a frequent action set, or frequent -action set, with regard to a user-specified threshold referred to as minsup, if minsup.
In effect, if the value of changes from to , ) takes place. The observation supports , where is an action set and is an effect, if supports and . An effect-probability is defined as a pair ), where is an effect and . We say that an effect-probability takes place, if takes place with probability .
We call an actionable behavioral rule an interesting actionable behavioral rule with regard to a user-specified threshold minutil, if util ( minutil.
Please note in the definition, except for the value of , that the rest of the observation pertaining to environment or behavior attributes comes from a time period of a certain interval. Based on the recent observation, the forthcoming projection is predicted. Based on the value of , the user decides whether or not to take a particular action and how it is going to influence the entity’s behavior. Consider a medical scenario where a patient is suffering from a rare disease and the right way to treat this disease is not currently available to the doctors. It is very likely that in a period of time (e.g., a year) the condition of the patient is going to remain unchanged or worsen if proper medication and treatment are not provided. This signifies that a proper action should be taken within a specified time to get a better output. The objective of actionable behavior rule mining is to identify the useful actions that may be applied to improve the projected next observation . In the above example the doctors would like to know the actions that can be taken to improve the patients’ health condition.
In this paper, we assume that all the attributes (both environmental and behavioral) are categorical, while numerical attributes have been discretized in advance. The behavior attribute values are not restricted to be binary indicating whether a certain behavior occurs or not. The values can also describe how frequent the behavior occurs, the extent of the behavior, and so on. In the next section, we use an example to illustrate some of these concepts and the motivations. For detailed information, please refer to  for more explanations and examples.
3. An Illustrative Example
Consider an example of a Palestinian terrorist organization Hamas, an Arabic acronym for Harakat al-Muqawma al-Islamiyya. It has carried out sophisticated attacks on Israeli targets and has caused serious troubles for the Israeli government. The Israeli government wants to take some action that will either stop or improve the current situation. For instance, the following prediction rule has been proposed to the government against Hamas:
The rule says “if the Israeli government changes the degree of using lethal violence against Hamas from level 1 (not using lethal violence) to level 2 (using periodic lethal violence) and the degree of being in agreement with Hamas from level 1 (negotiation) to level 3 (major concession), the degree of terrorist attacks aiming at domestic targets launched by Hamas will change from level 3 to 2 with 1/9 confidence or 1 with 3/9 confidence or remain unchanged with 5/9 confidence, and the Israeli government will get a utility of 0.84.”
Consider a hypothetical behavioral information system for Hamas organization , where , , , , , and is presented in Table 2 (e.g., , ). The values and the corresponding meanings of the attributes are shown in Table 3.
The problem of mining actionable behavioral rules is to mine all reliable and interesting actionable behavioral rules from a behavioral information system. The threshold minsup is used to assure that the rules are not found by chance. The threshold minutil is used to assure that the rules are sufficiently beneficiary to warrant deliberation by the user.
These actionable rules are important to the user in making decisions. However, rules identified sometimes tend to be conflicting with each other. For example, the following four rules are conflicting, that is,, , , .
These rules are conflicting as they suggest the same standard-action with different expected utilities, 0.10, 0.13, 0.21, and 0.17. This means that if the user takes the action , we cannot provide him/her with a definite utility estimation. Note that , , and are nonstandard actions and they cost the government nothing.
Earlier research has dealt with the issues involving conflicting rules. For example, following with the previous example, and were pruned first, as they are general rules with regard to . Subsequently, and were consolidated into the new rule whose expected utility is the weighted average of the expected utilities of and according to their supports. or is not pruned as neither is a general rule with regard to the other one.
Previous method prefers the rule with longer rule antecedent length while pruning the general rules. However, the same measure is not used for the rules which do not have general-specific relationship. The nonconsistent use of antecedent length may hurt the performance of the previous method. In addition, there may be another rule selection measure than rule antecedent length, that is, rule support.
4. Proposed Approach
The approach we propose to resolve conflicting rules is ranking rules according to their utility prediction accuracies. The highest-order rule should be reserved for the user and the others should be pruned. The key strategy is to find appropriate measure for evaluating conflicting rules.
4.1. Candidate Measures for Evaluating Conflicting Rules
There are two main measures in terms of which we can evaluate the utility prediction accuracies of conflicting rules: support and rule antecedent length (i.e., the number of actions in its antecedent). When all other conditions are the same, the higher is a rule’s support, the more accurate is its utility prediction. This is due to the fact that higher support can reduce a rule’s contingency. When all other conditions are the same, the longer is a rule’s antecedent, the more accurate is its utility prediction. The main reason for giving a rule with longer antecedent length more accurate utility prediction than a rule with shorter antecedent length is that the former provides more evidence to make predictions.
However, it is not uncommon that these two measures are conflicting or mutually exclusive. That is to say, it is very likely that a rule may have a longer or shorter antecedent length and lower or higher support than another rule at the same time. Moreover, the precedence of these two parameters is unknown. Finally, the precedence of these two measures will be different for different datasets.
4.2. The Rule Ranking Strategy
Due to the conflict between the two measures, we linearly combine them using a tunable weight parameter. In particular, we proposed an integrative measure, , which combines support and antecedent length to evaluate the utility prediction accuracies of the conflicting rules. This is presented as follows: where : antecedents of rule , : set of behavioral observations, : set of conditional attributes, and : a weight parameter tunable from 0 to 1.
When , the measure depends ultimately on rule antecedent length. When , the measure depends ultimately on support. When , the measure depends on the combination of these two parameters. The larger is, the more support contributes to the measure. The smaller is, the more rule antecedent length contributes to the measure.
When several rules have the identical highest score, we can choose one of the rules randomly, which possibly in some cases may degrade performance. When the possibility of two or more rules with the same score is relatively high, there should be other parameters to consider in favoring one rule over another in order to limit rule random selection. Thus, we propose a rule ranking procedure that takes into account the class distribution frequency of each conflicting rule after considering score.
This rule ranking procedure adds upon previous rule ranking approaches by looking at the class distribution frequencies in the behavioral information systems and prefers rules whose effects are associated with labels that occur more frequently in the behavioral information systems. The computing procedure of this measure (general) is shown in Algorithm 1.
5. Experiments and Results
In this section, we empirically validate the proposed method for resolving conflicting rules. We aim to answer the following five questions. (1) Is the method valid? (2) For which values of does the proposed method outperform the previous method? (3) Does the score-distribution method outperform the score method? (4) For which values of does the proposed method outperform random selection? (5) When increases, how will the performance change?
5.1. Experimental Design
To test the validity of our method, we conduct an experiment with the benchmark MAROB datasets (The Minorities at Risk Organizational Behavior Datasets: http://www.cidcm.umd.edu/mar/). The MAROB datasets cover several ethnopolitical organizations in the Middle East and North Africa. The datasets keep track of several attributes on a yearly basis from 1980 to 2004. We extracted three subdatasets about the Hezbollah Organization in Lebanon, the Kurdistan Democratic Party of Iran, and the Iraqi Communist Party, respectively. Utility values of the possible actions and effects from the viewpoints of the corresponding governments were elicited from human analysts and normalized into the range of .
Parameter is set to each value in which is believed to be a good representative of the set of permissible values of ().
5.2. Evaluation Criterion
We use the mean absolute error (MAE), a standard measure to assess the closeness between predictions and eventual outcomes, as the criterion to evaluate the performance of different methods. The MAE is given by where is the expected utility our method or another method estimates and is the actual utility. Typically, the MAE threshold is set by the domain experts in security informatics to a reasonable value around 0.07 for field evaluation.
5.3. Experimental Results
Tables 4 and 5 show the experimental results on the three sub-MAROB datasets with 25 actual action sets that are antecedent of some candidate actionable behavioral rules. Note that there is no other actual action set that is antecedent of any candidate rule. The table entries present the actual utilities, the absolute errors of the proposed score method (Table 4), the proposed score-distribution method (Table 5), the previous method, and random selection with minsup set to 7. For the absolute errors, the means are presented.
Note that although the utilities of actionable behavioral rules-induced actions and effects were assigned by the experts in a subjective fashion, what is actually compared in our experiment is the absolute difference between the actual utility and the expected utility of the action set induced by a rule. For the same action set, different methods will yield different estimated probability distributions over the effects or outcomes. Generally speaking, the closer the estimated distribution of a rule’s effects is to the actual realization, the closer the estimated utility is to the actual utility.
From Tables 4 and 5, we can see that the MAE of our approach is much smaller than the validity threshold 0.07. This shows the validity of the proposed method. Furthermore, the proposed method outperforms the previous approach and random selection, when is set to 0 or 0.1. In particular, the proposed method reaches the best performance when is set to 0.1. The score-distribution method outperforms the score method when is set to 0. The two methods yield the same results when is set to the other values, as for these values only one rule has the identical highest score. In general, MAE increases monotonically when becomes larger. This suggests that, in the proposed integrative measure, rule antecedent length has much effect than support. The main reason may be that the small number of instances (fewer than 30) in each dataset leads to the small supports of the rules, and hence the differences among different supports are trivial.
6. Related Work
Previous research has looked into actionable knowledge discovery. For example, in discovering actionable knowledge for customer relationship management (CRM), Liu et al. proposed an approach to suggest actions to reclassify a customer from an undesired status to a desired one while postprocessing decision trees to maximum expected net profit . However, these methods could miss some actions with higher net profit. To handle this problem, multiple trees with different subsets of “hard” attributes need to be built. To get optimal actions, the number of trees could be very large, especially when there are many “hard” attributes.
In order to increase profit of an institute to help devise a direct-marketing plan, Yang et al. proposed a lazy approach to use “role models” to generate advice and plans . However, this method failed to provide rules in advance and will incur high computational cost when generating action recommendations. To improve the profitability of a bank, Schrodt proposed action rules constructed from certain pairs of classification rules .
Ras and Tzacheva defined interesting action rules as the rules of the lowest cost . Raś et al. proposed a heuristic strategy to construct interesting action rules . Wang et al. combined the action forest algorithm to extract action rules and proposed a heuristic strategy to generate interesting action rules . Tzacheva and Tsay introduced the notion of cost and feasibility of an action rule and proposed a graph based method to search and construct feasible action rules at the lowest cost .
Previous studies on mining action rules all produced actionable rules based on a certain pair of classification rules or a single classification rule. A main drawback of these approaches is that some interesting actionable rules can be missed. To address this problem, He et al. proposed another strategy in a support-confidence-cost framework to discover action rules directly from a database . Raś et al. proposed an approach to generate association-type action rules . Subsequently, Raś and Dardzińska proposed a bottom-up strategy to discover action rules without using preexisting classification rules .
Actionable behavioral rules have wide applicability. Finding such rules and applying them are of great importance in the field of data mining [21–27].
In this paper, to resolve conflicts in actionable behavioral rules, we propose a new ranking method using an integrative measure for evaluating the utility prediction accuracies of conflicting rules. The new measure combines the support and antecedent length using a tunable weight parameter. The experimental results strongly suggest the validity and the superiority of our approach.
While we have conducted a preliminary experiment using domain datasets, more comprehensive experiments with many large datasets drawn from various domains can be conducted to validate the generalizability of our findings. For different datasets, the proposed method can tune the weight parameter for the best performance.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work has been supported by NSFC (nos. 71025001, 91024030, and 70890084) and the Scientific Research Foundation of Yunnan Provincial Education Department (no. 2014Z137), and Doctoral Science Foundation of Dali University.
U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, Advances in Knowledge Discovery and Data Mining, AAAI/MIT Press, 1996.
D. Hand, H. Mannila, and P. Smyth, Principles of Data Mining, MIT Press, Cambridge, Mass, USA, 2001.
Silberschatz and A. Tuzhilin, “What makes patterns interesting in knowledge discovery systems,” IEEE Transactions on Knowledge and Data Engineering, vol. 8, no. 6, pp. 970–974, 1996.View at: Google Scholar
L. Cao, Y. Zhao, H. Zhang, D. Luo, C. Zhang, and E. K. Park, “Flexible frameworks for actionable knowledge discovery,” IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 9, pp. 1299–1312, 2010.View at: Publisher Site | Google Scholar
P. Su, W. Mao, D. Zeng, and H. Zhao, “Mining actionable behavioral rules,” Decision Support Systems, vol. 54, pp. 142–152, 2012.View at: Google Scholar
D. Zeng, F.-Y. Wang, and K. M. Carley, “Social computing,” IEEE Intelligent Systems, vol. 22, no. 5, pp. 20–22, 2007.View at: Publisher Site | Google Scholar
W. Li, J. Han, and J. Pei, “CMAR: accurate and efficient classification based on multiple class-association rules,” in Proceedings of the 1st IEEE International Conference on Data Mining (ICDM '01), pp. 369–376, December 2001.View at: Google Scholar
K. Wang, S. Zhou, and Y. He, “Growing decision trees on support-less association rules,” in Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '01), pp. 265–269, August 2000.View at: Google Scholar
E. Baralis and P. Garza, “A lazy approach to pruning classification rules,” in Proceedings of the 2nd IEEE International Conference on Data Mining (ICDM '02), pp. 35–42, December 2002.View at: Google Scholar
E. Baralis, S. Chiusano, and P. Garza, “On support thresholds in associative classification,” in Proceedings of the ACM Symposium on Applied Computing, pp. 553–558, March 2004.View at: Google Scholar
F. Thabtah, “Pruning techniques in associative classification: survey and comparison,” Journal of Digital Information Management, vol. 4, no. 3, pp. 197–202, 2006.View at: Google Scholar
B. Liu, W. Hsu, and S. Chen, “Using general impressions to analyze discovered classification rules,” in Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining (KDD '97), pp. 31–36, 1997.View at: Google Scholar
Q. Yang, J. Yin, C. X. Ling, and T. Chen, “Postprocessing decision trees to extract actionable knowledge,” in Proceedings of the 3rd IEEE International Conference on Data Mining (ICDM '03), pp. 685–688, November 2003.View at: Google Scholar
P. Schrodt, “Forecasting conflict in the Balkans using hidden Markov models,” in ProgrammIng for Peace, T. Robert, Ed., pp. 161–184, Springer, 2006.View at: Google Scholar
Z. Ras and A. Tzacheva, “In search for action rules of the lowest cost,” in Monitoring, Security, and Rescue Techniques in Multiagent Systems, B. Dunin-Keplicz, A. Jankowski, A. Skowron, and M. Szczuka, Eds., pp. 261–272, Springer, 2005.View at: Google Scholar
Z. W. Raś, L.-S. Tsay, A. A. Tzacheva, and O. Gürdal, “Mining for interesting action rules,” in Proceedings of the IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT '05), pp. 187–193, September 2005.View at: Publisher Site | Google Scholar
A. A. Tzacheva and L.-S. Tsay, “Tree-based construction of low-cost action rules,” Fundamenta Informaticae, vol. 86, no. 1-2, pp. 213–225, 2008.View at: Google Scholar
Z. He, X. Xu, S. Deng, and R. Ma, “Mining action rules from scratch,” Expert Systems with Applications, vol. 29, no. 3, pp. 691–699, 2005.View at: Publisher Site | Google Scholar
Z. W. Raś, L.-S. Tsay, A. Dardzińska, and H. Wasyluk, “Association action rules,” in Proceedings of the IEEE International Conference on Data Mining Workshops, pp. 283–290, December 2008.View at: Publisher Site | Google Scholar
Z. W. Raś and A. Dardzińska, “Action rules discovery without pre-existing classification rules,” in Rough Sets and Current Trends in Computing, vol. 5306 of Lecture Notes in Computer Science, pp. 181–190, Springer, 2008.View at: Publisher Site | Google Scholar
X. Luo, Z. Xu, J. Yu, and X. Chen, “Building association link network for semantic link on web resources,” IEEE Transactions on Automation Science and Engineering, vol. 8, no. 3, pp. 482–494, 2011.View at: Publisher Site | Google Scholar
Z. Xu, X. Luo, and L. Wang, “Incremental building association link network,” Computer Systems Science and Engineering, vol. 26, no. 3, pp. 153–162, 2011.View at: Google Scholar
Y. Liu, Y. Zhu, L. Ni, and G. Xue, “A reliability-oriented transmission service in wireless sensor networks,” IEEE Transactions on Parallel and Distributed Systems, vol. 22, no. 12, pp. 2100–2107, 2011.View at: Publisher Site | Google Scholar
Y. Liu, Q. Zhang, and L. Ni, “Opportunity-based topology control in wireless sensor networks,” IEEE Transactions on Parallel and Distributed Systems, vol. 21, no. 3, pp. 405–416, 2010.View at: Publisher Site | Google Scholar
C. Hu, Z. Xu, Y. Liu, L. Mei, L. Chen, and X. Luo, “Semantic link network based model for organizing multimedia big data,” IEEE Transactions on Emerging Topics in Computing, 2014.View at: Publisher Site | Google Scholar
X. Liu, Y. Yang, D. Yuan, and J. Chen, “Do we need to handle every temporal violation in scientific workflow systems,” ACM Transactions on Software Engineering and Methodology, vol. 23, no. 1, pp. 5:1–5:34, 2013.View at: Publisher Site | Google Scholar
L. Wang, J. Tao, R. Ranjan et al., “G-Hadoop: MapReduce across distributed data centers for data-intensive computing,” Future Generation Computer Systems, vol. 29, no. 3, pp. 739–750, 2013.View at: Publisher Site | Google Scholar