Overcoming “Big Data” Barriers in Machine Learning Techniques for the Real-Life ApplicationsView this Special Issue
An Efficient Method for Mining Erasable Itemsets Using Multicore Processor Platform
Mining erasable itemset (EI) is an attracting field in frequent pattern mining, a wide tool used in decision support systems, which was proposed to analyze and resolve economic problem. Many approaches have been proposed recently, but the complexity of the problem is high which leads to time-consuming and requires large system resources. Therefore, this study proposes an effective method for mining EIs based on multicore processors (pMEI) to improve the performance of system in aspect of execution time to achieve the better user experiences. This method also solves some limitations of parallel computing approaches in communication, data transfers, and synchronization. A dynamic mechanism is also used to resolve the load balancing issue among processors. We compared the execution time and memory usage of pMEI to other methods for mining EIs to prove the effectiveness of the proposed algorithm. The experiments show that pMEI is better than MEI in the execution time while the memory usage of both methods is the same.
Data mining is an interesting field that has attracted many experts because of the huge amounts of data that were collected every day and the need to transfer such data into useful information to use in intelligence systems such as recommendation systems, decision making, and expert systems. Data mining has been widely used in market basket analysis, manufacturing engineering, financial banking, bioinformatics and future healthcare, and so on. The mining frequent pattern (FP) has a vital position in many data mining fields including association rule mining , clustering , and text mining . Mining FP is to find all patterns that have the frequency satisfying the user-given threshold. There are many methods [4, 5] for mining FPs in recent years. In addition, some issues related to FP mining has been proposed such as maximal frequent patterns , top- cooccurrence items with sequential pattern , weighted-based patterns , periodic-frequent patterns , and their applications [10, 11].
In 2009, the erasable itemset mining was first introduced , which comes from predicting merchandises of the production scheming as an exciting alteration of pattern mining. For an example, a factory needs to produce several new products, each of which requires some amount of raw materials to produce. However, the factory does not have enough budget to purchase all materials. Therefore, the factory managers need to determine what essential materials are needed to produce while the profit is not affected. The main problem is how to efficiently find these materials, without which the loss of the profit is less than the given threshold. These elements are also called as erasable itemsets. Based on these erasable itemsets, the consulting team can give the managers several suggestions about production plans in the near future. It has attracted a lot of research and become an ideal topic in recent years. There are many approaches, which were summarized in , to mine such patterns including META , MERIT , MEI , and EIFDD . Several related problems of mining erasable closed itemsets , top-rank- erasable itemsets , erasable itemsets with constraints , and weighted erasable itemsets [20, 21] have also been developed. Erasable closed itemset , a condensed representation of EIs without information loss, was proposed to reduce the computational cost. Top-rank- erasable itemset  is to merge the mining and ranking phases into one phase that only returns a small number of EIs to use in intelligent systems. Erasable itemset with constraints  is another approach only producing a small number of EIs, which delight a special requirement. In addition, mining weighted erasable itemset [20, 21] is a framework for mining erasable itemsets with the weight conditions for each item.
The existing algorithms for mining EIs have high computational complexity. It leads to very long execution time, especially on huge datasets and inefficiently intelligent systems. Therefore, this study proposes a parallel approach named parallel mining erasable itemsets (pMEI) using a multicore processor platform to improve the execution time for mining EIs. The major contributions of this paper are as follows: (i)A parallel method, namely, pMEI for mining EIs, splits the jobs into several duties to lessen the operating cost.(ii)Applying the difference pidset (dPidset) structure for quickly determining EIs information.(iii)A dynamic mechanism is used for load balancing the workload among cores when some processes are free.
The experiment results show that pMEI algorithm is better than MEI in execution time for the most of experimental datasets.
The remaining parts of the paper are arranged as follows: the preliminaries and related works, comprising the erasable itemset mining problem, several methods for mining EIs, as well as multicore processor architecture are presented in Section 2. Section 3 presents the pMEI algorithm proposed. The experiments on runtime and memory usage of pMEI and MEI methods for mining EIs are shown in Section 4. Finally, Section 5 reviews the outcomes and proposes some potential study topics.
2. Related Works
Let be a set of distinct items, which are the conceptual descriptions of elements of products created in a manufactory. For example, assuming that , then the included items are sugar, milk, cheese, cafe, snack, and wine. A product dataset consists of products, , where is a product shown in the pair of (items, value). In this pair, items are the items that compose product , and value is the profit value of the product . Table 1 shows the example datasets that have five items including and the set of products .
Definition 1 (itemset). An itemset , such that , is called a -itemset.
Definition 2 (profit of itemset). Given an itemset , the profit of itemset (denoted ) is calculated by
For example, let the set of products that contains , , or from Table 1 which are . Therefore, USD.
An itemset in product dataset is said to be erasable iff , where is the minimum threshold defined by user; is the total value of determined by this formula .
Definition 3 (EI mining problem). The EI mining problem is to discover all itemsets () that have not greater than .
For the example dataset in Table 1 and = 20%, we have by summing the value of all products. The itemset is an erasable itemset due to .
2.2. Erasable Itemset Mining
The erasable itemset (EI) mining problem was first introduced in 2009 , which comes from predicting merchandises of the production industry. It supports managers to decide their manufacturing strategy to guarantee the development of the company. The managers can decide which new merchandises are suitable for the factory without affecting the company’s profit. For example, a company that makes many types of goods, each good that is created will have a profit value. To create all products, the factory has to buy all essential materials. Currently, the company has no enough budget to purchase all materials. Hence, the managers of this company should deliberate their manufacturing strategies to make sure the steadiness of the company. The challenge is to obtain the itemsets that can be excluded but do not significantly change the company’s profit.
2.3. Methods for Erasable Itemset Mining
Many methods have been suggested to mine EIs such as META , MERIT , MEI , and EIFDD . Firstly, META, an Apriori-based algorithm, generates candidate itemsets using level-wise approach. Let be the set of erasable −1-itemsets. An itemset is verified with for coherency to produce applicant erasable -itemsets. However, only a small number of that have the same prefix as are joined. Later, MERIT based on the NC_sets is structured to decrease memory manipulation, which is its main improvement. The performance of MERIT is better than that of META significantly. However, storing NC_sets structure leads to high computational cost including memory usage and runtime. Therefore, MEI uses a divide-and-conquer approach associated with the difference of pidsets (dPidset) for mining EIs to improve the memory usage and runtime. It only scans the dataset one time to determine the total profit (), the index of gain (), and the erasable 1-itemsets with their pidsets. Although the runtime and memory consumption are enhancing than those of META and MERIT, however, the MEI’s performance from several dense datasets is quite weak. To resolve this drawback of MEI for dense datasets, EIFDD is proposed by using the subsume theory. This concept helps to quickly determine the information of EIs, without the generation cost. In brief, EIFDD is regularly applied to mine EIs for dense datasets, while MEI is applied to mine EIs for the other kinds of datasets.
Although existing methods improved the computation of mining EIs, however, these methods still consume more time in large datasets or large thresholds. Hence, in this paper, we develop a new technique to improve the computational cost for mining EIs.
2.4. dPidset Structure
The dPidset structure was suggested by Le and Vo  to lessen memory consumption by using an index of profit for effectively mining EIs. This structure is summarized as follows.
Definition 4. Given an itemset and an item , the pidset of itemset is denoted as where is the pidset of item , that is, the set of product identifiers (IDs) which contain item .
Definition 5. Let be the profit of itemset that is computed as follows:
Theorem 1 . Two given itemsets and have the same prefix which is . and are pidsets of and , correspondingly. is determined as follows:
Example 1. Consider the illustration datasets in Table 1, and . We have .
Definition 6. The dPidset of itemset signified by is defined as follows: where is the list of itemset IDs that only exist in .
Example 2. We have and , so .
Theorem 2 . Two given itemsets and have the dPidsets which are and . The is determined as follows:
Example 3. We have , , and . As in Definition 6, and . As in Theorem 2, .
Theorem 3 . The profit of , denoted by (), is calculated by as follows: where is the profit of and is the profit of .
Example 4. We have and , and thus, , , and . According to Example 3, and , and thus, and . , so .
2.5. Multicore Processor Platform
A multicore processor (MCP) is a physical chip including many separate cores in the same circuit . MCPs enable executed multiple missions concurrently to increase the performance of applications. In a multicore processor, each core has a distinct L1 cache and execution module and uses a public L2 cache for the whole processor. This makes the greatest use of the resources and optimizes the communication between intercores. If several tasks carry on separate cores of the same circuit and if they portion data that match in the cache, then the public last-level cache between cores will reduce the data replication. Hence, it is further effective in interaction. The major improvement of multicore processors is decreasing the temperature occurring off CPU and to extensively growing the speedup of processors while it is low cost than distributed systems, so it broadly applied in many fields such as computer vision, social network, image processing, and embedded system.
In data mining, there are many studies using this architecture to enhance the performance. Nguyen et al.  implement a technique for parallel mining class association rules on a computer that has the multicore processor platform which uses SCR-tree (single constraint rule-tree) structure and task parallel mechanism on .NET framework. Huynh et al. in  utilize multicore processors for mining frequent sequential patterns and frequent closed sequential patterns which use DBV-tree (dynamic bit vector-tree) structure and data parallel strategy based on TPL (task parallel library). Laurent et al.  proposed PGP-mc, which uses parallel gradual pattern mining used by the POSIX thread library. Flouri et al.  implement GapMis-OMP, a tool for pairwise short read alignment used by the OpenMP application programming interface, and Sánchez et al.  propose SW (Smith-Waterman), a method comparing sequence lengths based on the CellBE hardware. Recently, Kan et al.  proposed the parallelization and acceleration of the shuffled complex evolution utilizing the multicore CPU and many-core GPU. In 2018, Le et al.  suggested a parallel approach for mining intersequence patterns with constraints, which used DBV-PatternList structure and task parallel approach on multicore architecture.
3. A Parallel Method for Erasable Itemset Mining
Our proposed approach (Algorithm 1) in this study first determines the total profit of dataset, the erasable 1-itemsets () with their pidsets (line 2). Then, pMEI will sort by the size of their pidsets in not ascending order (line 3) and add them to child node of the root node (line 4). Finally, for each node in root, pMEI will start a new task (line 6) and call pMEI_Ext procedure to process this node with the created task in parallel that is a lightweight object in .NET framework for handling a parallel element of work. It can be used when we would like to perform something in parallel. The works (or jobs) are stretched across multiple processors to maximize performance of computer. Tasks are adjusted for leveraging multicore processors to improve performance.
For procedure pMEI_Ext, the algorithm will combine each node in root together and create the next level of candidates. This strategy will be used until there is no new EI to create. The entire procedure pMEI_Ext task will be performed in parallel to achieve the good performance.
An outstanding point of pMEI algorithm is that it uses a dynamic load balancing mechanism. pMEI uses a queue to store jobs (a list of work to perform), and if the queue is not empty and there exists a task that is idle, then task is assigned a job to execute. In contrast, if the queue is full, the task will perform a job until completion. After a task completes a job or is idle, then it will be immediately assigned a new job and this job will be removed from the queue, and the process continues until the queue is empty. This mechanism is effective and can help avoid both task idling and achieve balanced workloads.
In addition, one of the differences between the pMEI and parallel methods in [24, 29] is in the sorting strategy to balance the search space. For mining frequent patterns, we can sort patterns according to their supports. In mining EIs, we do not need to compute the support of itemsets, so pMEI sorts itemsets according to the size of their pidsets to balance the search space.
The illustration of the mining process of MEI and pMEI is shown in Figure 1.
(a) The mining process of MEI
(b) The mining process of pMEI
pMEI executed in depth-first search; the result (EIs) is writing to global memory. Thus, it does not need to merge or synthesize process. In addition, pMEI uses a global queue for parent task and local queue for child task; both are accessed in LIFO order. The synchronization between tasks is not necessary because of the task using local queue does not involve any shared data.
4. Experimental Results
This section presents the results of the experiments that were executed on a PC with Intel Core I5-6200U (2.30 GHz, 4 threads) with 4 GB RAM and implemented in C# in Visual Studio 2015.
The experiments have been performed on Accidents, Chess, Connect, Mushroom, Pumsb, and T10I4D100K datasets which were downloaded from UCI datasets (http://fimi.ua.ac.be/data/). We have added a new column to hold the value associated to each product because the value of products has not existed in these datasets. The value was calculated based on the normal distribution . The characteristics of these datasets are exhibited in Table 2 and are accessible at http://sdrv.ms/14eshVm.
In this part, we will evaluate the memory manipulation and running time of the suggested algorithm with MEI algorithm  to show the effectiveness of pMEI algorithm.
4.1. Running Time
We evaluate the execution times between MEI and pMEI algorithms on six experimental datasets (Figures 2–7). Note that the running times are averaged across five runs.
For dense datasets, such as Chess, Connect, and Mushroom, the execution time of pMEI is much better than MEI (Figures 3–5). In detail, for Chess dataset at , pMEI only requires 0.203 s while MEI requires 0.31 s; and at , pMEI only requires 0.389 s while MEI requires 0.654 s, respectively. Specifically, this gap will be increased as the threshold increases. Like Connect dataset, at , the time gap between the execution time of pMEI and that of MEI is 3.49 s while at , the time gap is 7.07 s.
For sparse datasets, such as Accidents, Pumsb, and T10I4D100K, the time gaps between pMEI and MEI are small (Figures 2, 6, and 7). Therefore, pMEI outperforms MEI in terms of the execution times for all experimental datasets especially for dense datasets.
4.2. Memory Usage
For all experimental datasets, pMEI and MEI have the same memory usage (see Figures 8–13). In summary, pMEI improves the execution times for mining EIs on all experimental datasets while keeping the memory usage compared with MEI algorithm.
To evaluate the effectiveness of multicore systems, we executed the proposed method with the various numbers of cores (Figure 14). The speedup increases nearly two times on 2 cores and increases nearly four times on 4 cores. The average speedup rate is 1.94, 1.95, 1.95, 1.98, 1.99, and 2.06 on 2 cores and 3.82, 3.87, 3.82, 3.80, 3.82, and 3.93 on 4 cores with Accidents, Chess, Connect, Mushroom, Pumsb, and T10I4D100K datasets, respectively. Generally, the speedup is always proportional to the number of cores of the computer.
This study proposed a proficient technique for mining EIs, namely, pMEI based on multicore computers to enhance the performance. This method overcomes these drawbacks of parallel computing approaches including the interactive expense cost, synchronization, and data duplication. A dynamic mechanism for load balancing the processor workloads was also used. Experiments show that pMEI is better than MERIT for mining EIs in execution time.
In the future, we will study mining the top-rank- erasable closed itemsets and maximal erasable itemsets. In addition, we will expand the study of EI mining associated with some kinds of item constraints. Besides, approaches for mining such patterns on distributed computing systems will be developed.
The product data used to support the findings of this study have been deposited in the Frequent Itemset Mining Dataset Repository (http://fimi.ua.ac.be/data/).
Conflicts of Interest
The authors declare that they have no conflicts of interest.
The authors wish to thank Tuong Le for his valuable comments and suggestions.
T. Mai, B. Vo, and L. T. T. Nguyen, “A lattice-based approach for mining high utility association rules,” Information Sciences, vol. 399, pp. 81–97, 2017.View at: Publisher Site | Google Scholar
D. T. Hai, L. H. Son, and T. L. Vinh, “Novel fuzzy clustering scheme for 3D wireless sensor networks,” Applied Soft Computing, vol. 54, pp. 141–149, 2017.View at: Publisher Site | Google Scholar
N. Indurkhya, “Emerging directions in predictive text mining,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 5, no. 4, pp. 155–164, 2015.View at: Publisher Site | Google Scholar
Z.-H. Deng, “Diffnodesets: an efficient structure for fast mining frequent itemsets,” Applied Soft Computing, vol. 41, pp. 214–223, 2016.View at: Publisher Site | Google Scholar
W. Gan, J. C.-W. Lin, P. Fournier-Viger, H.-C. Chao, and J. Zhan, “Mining of frequent patterns with multiple minimum supports,” Engineering Applications of Artificial Intelligence, vol. 60, pp. 83–96, 2017.View at: Publisher Site | Google Scholar
B. Vo, S. Pham, T. Le, and Z.-H. Deng, “A novel approach for mining maximal frequent patterns,” Expert Systems with Applications, vol. 73, pp. 178–186, 2017.View at: Publisher Site | Google Scholar
T. Kieu, B. Vo, T. Le, Z.-H. Deng, and B. Le, “Mining top-k co-occurrence items with sequential pattern,” Expert Systems with Applications, vol. 85, pp. 123–133, 2017.View at: Publisher Site | Google Scholar
W. Gan, J. C.-W. Lin, P. Fournier-Viger, H.-C. Chao, J. M.-T. Wu, and J. Zhan, “Extracting recent weighted-based patterns from uncertain temporal databases,” Engineering Applications of Artificial Intelligence, vol. 61, pp. 161–172, 2017.View at: Publisher Site | Google Scholar
R. U. Kiran, J. N. Venkatesh, M. Toyoda, M. Kitsuregawa, and P. K. Reddy, “Discovering partial periodic-frequent patterns in a transactional database,” Journal of Systems and Software, vol. 125, pp. 170–182, 2017.View at: Publisher Site | Google Scholar
A. Hellal and L. Ben Romdhane, “Minimal contrast frequent pattern mining for malware detection,” Computers & Security, vol. 62, pp. 19–32, 2016.View at: Publisher Site | Google Scholar
J. Wen, M. Zhong, and Z. Wang, “Activity recognition with weighted frequent patterns mining in smart environments,” Expert Systems with Applications, vol. 42, no. 17-18, pp. 6423–6432, 2015.View at: Publisher Site | Google Scholar
Z.-H. Deng, G.-D. Fang, Z.-H. Wang, and X.-R. Xu, “Mining erasable itemsets,” in 2009 International Conference on Machine Learning and Cybernetics, pp. 67–73, Hebei, China, 2009.View at: Publisher Site | Google Scholar
T. Le and B. Vo, “The lattice-based approaches for mining association rules: a review,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 6, no. 4, pp. 140–151, 2016.View at: Publisher Site | Google Scholar
Z.-H. Deng and X.-R. Xu, “Fast mining erasable itemsets using NC_sets,” Expert Systems with Applications, vol. 39, no. 4, pp. 4453–4463, 2012.View at: Publisher Site | Google Scholar
T. Le and B. Vo, “MEI: an efficient algorithm for mining erasable itemsets,” Engineering Applications of Artificial Intelligence, vol. 27, pp. 155–166, 2014.View at: Publisher Site | Google Scholar
G. Nguyen, T. Le, B. Vo, and B. Le, “EIFDD: an efficient approach for erasable itemset mining of very dense datasets,” Applied Intelligence, vol. 43, no. 1, pp. 85–94, 2015.View at: Publisher Site | Google Scholar
B. Vo, T. le, G. Nguyen, and T.-P. Hong, “Efficient algorithms for mining erasable closed patterns from product datasets,” IEEE Access, vol. 5, no. 1, pp. 3111–3120, 2017.View at: Publisher Site | Google Scholar
T. Le, B. Vo, and S. W. Baik, “Efficient algorithms for mining top-rank-k erasable patterns using pruning strategies and the subsume concept,” Engineering Applications of Artificial Intelligence, vol. 68, pp. 1–9, 2018.View at: Publisher Site | Google Scholar
B. Vo, T. le, W. Pedrycz, G. Nguyen, and S. W. Baik, “Mining erasable itemsets with subset and superset itemset constraints,” Expert Systems with Applications, vol. 69, pp. 50–61, 2017.View at: Publisher Site | Google Scholar
G. Lee, U. Yun, H. Ryang, and D. Kim, “Erasable itemset mining over incremental databases with weight conditions,” Engineering Applications of Artificial Intelligence, vol. 52, pp. 213–234, 2016.View at: Publisher Site | Google Scholar
U. Yun and G. Lee, “Sliding window based weighted erasable stream pattern mining for stream data applications,” Future Generation Computer Systems, vol. 59, pp. 1–20, 2016.View at: Publisher Site | Google Scholar
A. Vajda, “Multi-core and many-core processor architectures,” in Programming Many-Core Chips, pp. 9–43, Springer, Boston, MA, USA, 2011.View at: Publisher Site | Google Scholar
D. Nguyen, B. Vo, and B. Le, “Efficient strategies for parallel mining class association rules,” Expert Systems with Applications, vol. 41, no. 10, pp. 4716–4729, 2014.View at: Publisher Site | Google Scholar
B. Huynh, B. Vo, and V. Snasel, “An efficient method for mining frequent sequential patterns using multi-core processors,” Applied Intelligence, vol. 46, no. 3, pp. 703–716, 2017.View at: Publisher Site | Google Scholar
A. Laurent, B. Négrevergne, N. Sicard, and A. Termier, “Efficient parallel mining of gradual patterns on multicore processors,” in Advances in Knowledge Discovery and Management, vol. 398 of Studies in Computational Intelligence, pp. 137–151, Springer, Berlin, Heidelberg, 2012.View at: Publisher Site | Google Scholar
T. Flouri, C. S. Iliopoulos, K. Park, and S. P. Pissis, “GapMis-OMP: pairwise short-read alignment on multi-core architectures,” in Artificial Intelligence Applications and Innovations, vol. 382 of IFIP Advances in Information and Communication Technology, pp. 593–601, Springer, Berlin, Heidelberg, 2012.View at: Publisher Site | Google Scholar
F. Sánchez, F. Cabarcas, A. Ramirez, and M. Valero, “Long DNA sequence comparison on multicore architectures,” in Euro-Par 2010 - Parallel Processing, vol. 6272 of Lecture Notes in Computer Science, pp. 247–259, Springer, Berlin, Heidelberg, 2010.View at: Publisher Site | Google Scholar
G. Kan, T. Lei, K. Liang et al., “A multi-core CPU and many-core GPU based fast parallel shuffled complex evolution global optimization approach,” IEEE Transactions on Parallel and Distributed Systems, vol. 28, no. 2, pp. 332–344, 2016.View at: Publisher Site | Google Scholar
T. Le, A. Nguyen, B. Huynh, B. Vo, and W. Pedrycz, “Mining constrained inter-sequence patterns: a novel approach to cope with item constraints,” Applied Intelligence, vol. 48, no. 5, pp. 1327–1343, 2018.View at: Publisher Site | Google Scholar