Knowledge Reduction Based on Divide and Conquer Method in Rough Set Theory

Abstract

The divide and conquer method is a typical granular computing method using multiple levels of abstraction and granulations. So far, although some achievements based on divided and conquer method in the rough set theory have been acquired, the systematic methods for knowledge reduction based on divide and conquer method are still absent. In this paper, the knowledge reduction approaches based on divide and conquer method, under equivalence relation and under tolerance relation, are presented, respectively. After that, a systematic approach, named as the abstract process for knowledge reduction based on divide and conquer method in rough set theory, is proposed. Based on the presented approach, two algorithms for knowledge reduction, including an algorithm for attribute reduction and an algorithm for attribute value reduction, are presented. Some experimental evaluations are done to test the methods on uci data sets and KDDCUP99 data sets. The experimental results illustrate that the proposed approaches are efficient to process large data sets with good recognition rate, compared with KNN, SVM, C4.5, Naive Bayes, and CART.

1. Introduction

In the search for new paradigms of computing, there is a recent surge of interest, under the name of granular computing [1–3], in computations using multiple levels of abstraction and granulation. To a large extent, the majority of existing studies include rough sets, fuzzy sets, cluster analysis, and classical divide and conquer methods [4, 5] and aim at solving specific problems [6].

Rough set (RS) [7–10] is a valid mathematical theory for dealing with imprecise, uncertain, and vague information. It has been applied in such fields as machine learning, data mining, intelligent data analysis, and control algorithm acquiring, successfully since it was proposed by Pawlak and Skowron in 2007 [10]. Knowledge reduction is one of the most important contributions of rough set theory to machine learning, pattern recognition, and data mining. Although the problem of finding a minimal reduction of a given information system was proven to be an NP-hard problem [8], many promising heuristics have been developed that are promising. A variety of methods for knowledge reduction and their applications can be found in [2, 7–43]. Among these existing methods, one group method focuses on the indiscernibility relation in a universe that captures the equivalence of objects, while the other group considers the discernibility relation that explores the differences of objects [42]. For indiscernibility relation, one can employ it to induce a partition of the universe and thereby to construct positive regions whose objects can be undoubtedly classified into a certain class with respect to the selected attributes. Thus, knowledge reduction algorithms based on positive regions have been proposed in [8, 10, 15, 16, 21, 28, 30]. For discernibility relation, we have knowledge reduction algorithms based on a discernibility matrix and information entropy. Reduction methods based on discernibility matrix [34] have high cost of storage with space complexity 𝑂(𝑚×𝑛²) for a large decision table with 𝑛 objects and 𝑚 conditional attributes. Thus, storing and deleting the element cells in a discernibility matrix is a time-consuming process. Many researchers have studied discernibility matrix construction and contributed to a lot [10, 18, 28, 35, 38, 39, 43]. As well, knowledge reduction algorithms based on information entropy [20, 37, 42] have been developed. Although so many algorithms have been developed, it is valuable to study some new high efficient algorithms.

Divide and conquer method is a simple granular computing method. When the algorithms are designed by divide and conquer method, the decision table can be divided into many subdecision tables recursively in attribute space. That^'s to say, an original big data set can be divided into many small ones. If the small ones can be processed one by one, instead the original big one is processed on a whole, it will save a lot time. Thus, it may be an effective way to process large data set. The divide and conquer method consists of three vital stages.

Stage 1. Divide the big original problem into many independent subproblems with the same structure.

Stage 2. Conquer the sub-problems recursively.

Stage 3. Merge the solutions of sub-problems into the one of original problem.

So far, some good results for knowledge reduction based on divide and conquer method have been achieved, such as the computation of the attribute core and the computation of attribute reduction under given attribute order [12, 13]. Besides, decision tree-based methods [26, 27, 44] have been studied and are very popular. In fact, the construction of decision tree is a special case of divide and conquer method, because it can be generated from top to down recursively. In the methods based on decision tree, a tree should be constructed by using decomposition at first. It costs more in the first stage, which is convenient to the following stages, and costs less in the following two ones.

However, the systematic method for knowledge reduction based on divide and conquer method is still absent, especially “how to keep invariance between the solution of original problem and the ones of sub-problems.” It results in the difficulty to design the high efficient algorithms for knowledge reduction based on divide and conquer method. Therefore, it is urgent to discuss the knowledge reduction method based on divide and conquer methods systematically and overall.

Contributions of this work are (1) some principles for “keeping invariance between the solution of original problem and the ones of sub-problems” are concluded. Then, the abstract process for knowledge reduction based on divide and conquer method in the rough set theory is presented, which is helpful to design high efficient algorithm based on divide and conquer method. (2) Fast approaches for knowledge reduction based on divide and conquer method, including an algorithm for attribute reduction and an algorithm for attribute value reduction, are proposed. Experimental evaluations show that the presented methods are efficient.

The remainder of this paper is organized as follows. The basic theory and methods dealing with the application of rough set theory in data mining are presented in Section 2. Section 3 introduces the abstract process for knowledge reduction based on divide and conquer method in the rough set theory. A quick algorithm based on divide and conquer method for attribute reduction is presented in Section 4. In Section 5, a fast algorithm for attribute value reduction using divide and conquer method is proposed. Some simulation experimental evaluations are discussed to show the performance of the developed methods in Section 6. The paper ends with conclusion in Section 7.

2. Preliminaries

Rough set theory was introduced by Pawlak as a tool for concept approximation relative to uncertainty. Basically, the idea is to approximate a concept by three description sets, namely, the lower approximation, upper approximation, and boundary region. The approximation process begins by partitioning a given set of objects into equivalence classes called blocks, where the objects in each block are indiscernible from each other relative to their attribute values. The approximation and boundary region sets are derived from the blocks of a partition of the available objects. The boundary region is constituted by the difference between the lower approximation and upper approximation and provides a basis for measuring the “roughness” of an approximation. Central to the philosophy of the rough set approach to concept approximation is the minimization of the boundary region [28].

For the convenience of description, some basic notions of decision tables are introduced here at first.

Definition 2.1 (decision table [36]). A decision table is defined as 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, where 𝑈 is a non-empty finite set of objects, called the universe, 𝐴 is a nonempty finite set of attributes, 𝐴=𝐶∪𝐷, where 𝐶 is the set of conditional attributes, and 𝐷 is the set of decision attributes, 𝐷≠𝜙. For any attribute 𝑎∈𝐴, 𝑉_𝑎 denotes the domain of attribute 𝑎. Each attribute determines a mapping function 𝑓∶𝑈×𝐴→𝑉.

Definition 2.2 (indiscernibility relation [36]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, each subset of attribute 𝐵⊆𝐴 determines an indiscernibility relation IND(𝐵) as follows: IND(𝐵)={(𝑥,𝑦)∣(𝑥,𝑦)∈𝑈×𝑈,∀𝑏∈𝐵(𝑏(𝑥)=𝑏(𝑦))}.

Definition 2.3 (lower approximation and upper approximation [36]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for any subset 𝑋⊆𝑈 and indiscernibility relation IND(𝐵), the 𝐵 lower approximation and upper approximation of 𝑋 are defined as 𝐵₋(𝑋)=⋃_{𝑌𝑖∈𝑈/IND(𝐵)∧𝑌𝑖⊆𝑋}𝑌_𝑖, 𝐵⁻(𝑋)=⋃_{𝑌𝑖∈𝑈/IND(𝐵)∧𝑌𝑖∩𝑋≠𝜙}𝑌_𝑖.

Definition 2.4 (positive region [36]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, 𝑃⊆𝐴, and 𝑄⊆𝐴, the 𝑃 positive region of 𝑄 is defined as Pos_𝑃(𝑄)=∪_{𝑥∈𝑈/𝑄}𝑃(𝑋).

Definition 2.5 (relative core [36]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, 𝑃⊆𝐴, 𝑄⊆𝐴, 𝑟∈𝑃, 𝑟 is unnecessary in 𝑃 relative to 𝑄 if and only if Pos_𝑃(𝑄)=Pos_{𝑃−{𝑟}}(𝑄), otherwise 𝑟 is unnecessary in 𝑃 relative to 𝑄. The core of 𝑃 relative to 𝑄 is defined as CORE_𝑄(𝑃)={𝑟∣𝑟∈𝑃; 𝑟 is necessary in 𝑃 relative to 𝑄}.

Definition 2.6 (see [36]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, 𝑃⊆𝐴 and 𝑄⊆𝐴, for all 𝑟∈𝑃, if 𝑟 is relatively necessary in 𝑃 relative to 𝑄, 𝑃 is called independent relative to 𝑄.

Definition 2.7 (relative reduction [36]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, 𝑃⊆𝐴, 𝑄⊆𝐴, 𝑅∈𝑃, if 𝑅 is indispensable relative to 𝑄 and Pos_𝑅(𝑄)=Pos_𝑃(𝑄), 𝑅 is called a reduction of 𝑃 relative to 𝑄.

Definition 2.8 (see [18]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, the element of discernibility matrix 𝑀 can be defined as 𝐵^𝑠_𝑥𝑦={𝑎∈𝐶∣𝑓(𝑥,𝑎)≠𝑓(𝑦,𝑎)∧𝑤(𝑥,𝑦)=1}, where, 𝑥∈𝑈, 𝑦∈𝑈, and 𝑤𝑥,𝑦=1,𝑥∈Pos_𝐶(𝐷),𝑦∉Pos_𝐶(𝐷),1,𝑥∉Pos_𝐶(𝐷),𝑦∈Pos_𝐶(𝐷),1,𝑥,𝑦∈Pos_𝐶(𝐷),𝑑(𝑥)≠𝑑𝑦,0,others. (2.1)

3. The Abstract Process for Knowledge Reduction Based on Divide and Conquer Method in Rough Set Theory

3.1. The Knowledge Reduction Based on Divide and Conquer Method under Equivalence Relation

In the research of rough set theory, the divide and conquer method is an effective method to design high effective algorithm. It can be used to compute equivalence classes, positive region, and attribute core of decision table (see Propositions 3.1, 3.2, and 3.3) even execute some operators of discernibility matrix (see Propositions 3.4 and 3.5). In this section, the divide and conquer method under equivalence relation in rough set theory will be discussed.

Proposition 3.1 (see [13]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for all 𝑐(𝑐∈𝐶), divide 𝑆 into 𝑘(𝑘=|IND(𝑈/{𝑐})|) subdecision tables 𝑆₁,𝑆₂,…,𝑆_𝑘 by 𝑈/{𝑐}, where 𝑆_𝑖=⟨𝑈_𝑖,(𝐶⧵{𝑐})∪𝐷,𝑉_𝑖,𝑓_𝑖⟩, which satisfies ∀_{𝑥∈𝑈𝑖}∀_{𝑦∈𝑈𝑖}𝑐(𝑥)=𝑐(𝑦)(1≤𝑖≤𝑘) and ∀_{𝑥∈𝑈𝑖}∀_{𝑧∈𝑈𝑗}𝑐(𝑥)≠𝑐(𝑧)(1≤𝑖<𝑗≤𝑘). Let 𝑅=𝐶−{𝑐}. Let us denote by Pos^𝑖_𝑅(𝐷)(1≤𝑖≤𝑘) the positive region of 𝑆_𝑖 and the one of 𝑆 by Pos_𝐶(𝐷), respectively. Then, Pos_𝐶(𝐷)=⋃_{1≤𝑖≤𝑘}Pos^𝑖_𝑅(𝐷).

Proposition 3.1 presents the approach of computing equivalence classes or positive region by using divide and conquer method. Compared with decision tree-based method (without pruning), the approach allows us to generate “clear” leaves (with the same decision) for objects in the positive region and “unclear” leaves where objects are with different decisions and correspond to the boundary region. It may be an effective way to prevent overfitting, because “conquer” can play a role of pruning of tree. Furthermore, the approach needs less space because the construction of tree is not necessary.

Proposition 3.2 (see [13]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for all 𝑐(𝑐∈𝐶), divide 𝑆 into 𝑘(𝑘=|𝐼𝑁𝐷(𝑈/{𝑐})|) sub-decision tables 𝑆₁,𝑆₂,…,𝑆_𝑘 by 𝑈/{𝑐}, where 𝑆_𝑖=⟨𝑈_𝑖,(𝐶⧵{𝑐})∪𝐷,𝑉_𝑖,𝑓_𝑖⟩, which satisfies ∀_{𝑥∈𝑈𝑖}∀_{𝑦∈𝑈𝑖}𝑐(𝑥)=𝑐(𝑦)(1≤𝑖≤𝑘) and ∀_{𝑥∈𝑈𝑖}∀_{𝑧∈𝑈𝑗}𝑐(𝑥)≠𝑐(𝑧)(1≤𝑖<𝑗≤𝑘). Let us denote by Core_𝑖(1≤𝑖≤𝑘) the attribute core of 𝑆_𝑖 and the one of 𝑆 by Core. Then, ⋃_{1≤𝑖≤𝑘}Core_𝑖⊆Core⊆{𝑐}∪⋃_{1≤𝑖≤𝑘}Core_𝑖.

Proposition 3.3 (see [13]). Given a decision table 𝑆-⟨𝑈,𝐴,𝑉,𝑓⟩, for all 𝑐(𝑐∈𝐶), divide 𝑆 into 𝑘(𝑘=|𝐼𝑁𝐷(𝑈/{𝑐})|) sub-decision tables 𝑆₁,𝑆₂,…,𝑆_𝑘 by 𝑈/{𝑐}, where 𝑆_𝑖=⟨𝑈_𝑖,(𝐶⧵{𝑐})∪𝐷,𝑉_𝑖,𝑓_𝑖⟩, which satisfies ∀_{𝑥∈𝑈𝑖}∀_{𝑦∈𝑈𝑖}𝑐(𝑥)=𝑐(𝑦)(1≤𝑖≤𝑘) and ∀_{𝑥∈𝑈𝑖}∀_{𝑧∈𝑈𝑗}𝑐(𝑥)≠𝑐(𝑧)(1≤𝑖<𝑗≤𝑘). Let us denote by red_𝑖(1≤𝑖≤𝑘) the attribute reduction of 𝑆_𝑖 and the one of 𝑆 by 𝑅. Let 𝑅=⋃_{1≤𝑖≤𝑘}red_𝑖, 𝑅₁={𝑐}∪⋃_{1≤𝑖≤𝑘}red_𝑖. Then, Pos_𝑅(𝐷)⊆Pos_𝐶(𝐷)=Pos_𝑅1(𝐷).

Propositions 3.2 and 3.3 present the approach of attribute core determining based on divide and conquer method. Compared with the method of computing attribute core on original decision table, it may be more efficient and can process bigger data set, since the big data set has been divided into many small data sets.

Obviously, Propositions 3.1, 3.2, and 3.3 hold.

Discernibility matrix by Skowron is a useful method to design some algorithms in rough set theory. However, due to its high complexity of algorithms based on explicit computing of the discernibility matrices, the efficiency of algorithms based on discernibility matrix needs to be improved. In the literature some useful methods have been proposed (see [26–31] by Nguyen et al., and decomposition methods implemented in RSES). Our methods differ from the existing ones as follows.(1)“How to keep invariance between the solution of original problem and the ones of sub-problems” is a key problem. We conclude some principles for computing positive region, attribute core, attribute reduction, and value reduction (see Propositions 3.1, 3.2, 3.3, 3.4, 3.5, 3.11, and 3.12), which were not concluded before.(2)Although the decision tree-based methods and our approaches both belong to divide and conquer method, our approaches cost more on “conquer” and “merge” while they cost less on “divide,” compared with the decision tree-based methods. Furthermore, our approaches need not to construct a tree, which maybe save space.(3)The existing heuristic ones in [26–28] can improve the efficiency by measuring the discernibility degree of different objects quickly. In our approaches, the element cells of discernibility matrix can be deleted by dividing decision table fast without storing the discernibility matrix. Thus, it may be a quick one for operating discernibility matrix with small space (see Propositions 3.4 and 3.5).

Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩ and its discernibility matrix 𝑀 (Definition 2.8); for all 𝐶₁(𝐶₁⊆𝐶), let us denote by 𝑀^𝐶1 a subset of element cells in 𝑀. 𝑀^𝐶1 can be labeled as {𝛼₁,𝛼₂,…,𝛼_𝑡}(𝑡=|𝑀^𝐶1|), where(1)∀_{𝛼𝑖∈𝑀}^𝐶₁𝛼_𝑖∈𝑀(1≤𝑖≤|𝑀^𝐶1|);(2)for all 𝑐∈𝐶₁, if ∃_𝛼∈𝑀(𝑐∈𝛼), then 𝛼∈𝑀^𝐶1.

Proposition 3.4. Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for all 𝑐(𝑐∈𝐶), divide the decision table 𝑆 into 𝑘(𝑘=|IND(𝑈/{𝑐})|) sub-decision tables 𝑆₁,𝑆₂,…,𝑆_𝑘 by 𝑈/{𝑐}, where 𝑆_𝑖=⟨𝑈_𝑖,(𝐶⧵{𝑐})∪𝐷,𝑉_𝑖,𝑓_𝑖⟩. Let us denote by 𝑀 the discernibility matrix of 𝑆 and the discernibility matrix of 𝑆_𝑖(1≤𝑖≤𝑘) by 𝑀_𝑖, respectively. Then, ⋃_𝛼∈𝑀𝛼=⋃^𝑘_𝑖=1⋃_{𝛽∈𝑀𝑖}𝛽∪⋃_𝛾∈𝑀^𝐶₁𝛾.

(Note: If  ∀_{𝑥,𝑦∈𝑈𝑖}∀_𝑐1∈𝐶𝑐₁(𝑥)=𝑐₁(𝑦), then, 𝑀_𝑖=𝜙)

Proof. First, prove ⋃_𝛼∈𝑀𝛼⊆⋃^𝑘_𝑖=1⋃_{𝛽∈𝑀𝑖}𝛽∪⋃_𝛾∈𝑀^𝐶₁𝛾.
For all 𝛼∈𝑀, then, ∃_{𝑥,𝑦∈𝑈}𝛼=𝐵^𝑆_𝑥𝑦 (Definition 2.8). After the partition of 𝑈/𝐶₁, suppose 𝑥 and 𝑦 be divided into sub-decision tables 𝑆_𝑖(1≤𝑖≤|𝑈/𝐶₁|) and 𝑆_𝑗(1≤𝑗≤𝑘), respectively.
If 𝑖=𝑗, then, 𝛼∈𝑀_𝑖, that is, 𝛼∈⋃^𝑘_𝑖=1⋃_{𝛽∈𝑀𝑖}𝛽.
If 𝑖≠𝑗, then, 𝛼∈𝑀^𝐶1, that is, 𝛼∈⋃_𝛾∈𝑀^𝐶₁𝛾.
Thus, 𝛼∈⋃^𝑘_𝑖=1⋃_{𝛽∈𝑀𝑖}𝛽∪⋃_𝛾∈𝑀^𝐶₁𝛾. That is, ⋃_𝛼∈𝑀𝛼⊆⋃^𝑘_𝑖=1⋃_{𝛽∈𝑀𝑖}𝛽∪⋃_𝛾∈𝑀^𝐶₁𝛾.
Similarly, we can proof ⋃_𝛼∈𝑀𝛼⊇⋃^𝑘_𝑖=1⋃_{𝛽∈𝑀𝑖}𝛽∪⋃_𝛾∈𝑀^𝐶₁𝛾.
Therefore, Proposition 3.4 holds.

Proposition 3.5. Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for all 𝐶₁(𝐶₁⊆𝐶), divide the decision table 𝑆 into 𝑘(𝑘=|IND(𝑈/𝐶₁)|) sub-decision tables 𝑆₁,𝑆₂,…,𝑆_𝑘 by 𝑈/𝐶₁, where 𝑆_𝑖=⟨𝑈_𝑖,(𝐶⧵𝐶₁)∪𝐷,𝑉_𝑖,𝑓_𝑖⟩. Let us denote by 𝑀 the discernibility matrix of 𝑆. Then, in the viewpoints of operating discernibility matrix, divide the decision table 𝑆 by 𝑈/𝐶₁ on attribute set 𝐶₁  if and only if one deletes all the element cells of 𝑀^𝐶1from 𝑀 one by one.

According to Proposition 3.4, it is easy to find that Proposition 3.5 holds.

Propositions 3.4 and 3.5 present the approach of deleting element cells of discernibility matrix. By using the approach, the element cells of discernibility matrix can be deleted quickly without constructing or storing the discernibility matrix. It may be an effective way to operate discernibility matrix quickly within small space. Thus, Propositions 3.4 and 3.5 can be used to design some efficient algorithms without explicit computing of discernibility matrix.

3.2. The Knowledge Reduction Based on Divide and Conquer Method under Tolerance Relation

In the course of attribute value reduction, tolerance relation is often adopted due to some attribute values on condition attribute being deleted. Thus, tolerance relation in attribute value reduction may be needed. A method is introduced by Kryszkiewicz and Rybinski [17] to process incomplete information system, where “∗” is used to represent the missing values on condition attributes. Here, “∗” can be also represent the deleted values on condition attributes. According to the tolerance relation by Kryszkiewicz, the divide and conquer method under the tolerance relation will be discussed.

Definition 3.6 (see [17]). Given an incomplete decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, a tolerance relation 𝑇 is defined as ∀𝑥,𝑦_∈𝑈1∀_𝐵⊆𝑅𝑇_𝐵𝑥,𝑦⟺_{∀𝑏∈𝐵}𝑏(𝑥)=𝑏𝑦∨(𝑏(𝑥)=∗)∨𝑏𝑦=∗. (3.1)

Definition 3.7 (see [17]). The tolerance class 𝑇_𝐵(𝑥) of an object 𝑥 relative to an attribute set 𝐵 is defined as 𝑇_𝐵(𝑥)={𝑦∣𝑦∈𝑈∧𝑇_𝐵(𝑥,𝑦)}. In the tolerance relation-based extension of rough set theory, the lower approximation 𝑋^𝑇_𝐵 and upper approximation 𝑋^𝐵_𝑇 of an object set 𝑋 relative to an attribute set 𝐵(𝐵⊆𝐶) are defined as 𝑋^𝑇_𝐵={𝑥∈𝑈∣𝑇_𝐵(𝑥)⊆𝑋}, 𝑋^𝐵_𝑇={𝑥∈𝑈∣𝑇_𝐵(𝑥)∩𝑋≠∅}.

Definition 3.8 (the covering of the universe under tolerance relation). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩ and condition attribute set 𝐶₁(𝐶₁⊆𝐶), according to the tolerance relation by Kryszkiewicz, the covering of the universe of 𝑆 can be defined as 𝑈=𝑈₁∪𝑈₂∪⋯∪𝑈_𝑘, where ∀_{𝑥,𝑦∈𝑈𝑝(1≤𝑝≤𝑘)}∀_𝑐∈𝐶1(𝑐(𝑥)=𝑐(𝑦)∨𝑐(𝑥)=∗∨𝑐(𝑦)=∗), and ∀_{𝑥∈𝑈𝑟}∀_{𝑦∈𝑈𝑝(1≤𝑟<𝑝≤𝑘)}∃_𝑐∈𝐶1((𝑐(𝑥)≠∗∧𝑐(𝑦)≠∗)⇒𝑐(𝑥)≠𝑐(𝑦)).

Definition 3.9 (certain decision rule). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for all 𝑥_𝑖∈Pos_𝐶(𝐷), the object 𝑥_𝑖 can result in a certain decision rule 𝑑_𝑖∶des([𝑥_𝑖]_𝐶)⇒des([𝑥_𝑖]_𝐷), 𝑑_𝑖(𝑎)=𝑎(𝑥_𝑖), 𝑎∈𝐶∪𝐷. 𝑑_𝑖∣𝐶 and 𝑑_𝑖∣𝐷 are called the condition attribute set and decision attribute set of 𝑑_𝑖, respectively.

Definition 3.10 (see [36]). Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for arbitrary certain decision rule, there is [𝑥_𝑖]_𝐶⊂[𝑥_𝑖]_𝐷. For all 𝑎∈𝐶, if [𝑥_𝑖]_{𝐶⧵{𝑎}}⊆[𝑥_𝑖]_𝐷 does not hold, then, 𝑎 is necessary in 𝑑_𝑖; otherwise, 𝑎 is not necessary in 𝑑_𝑖.

Proposition 3.11. Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, and for all 𝐶₁⊆𝐶, given an decomposing order 𝑐₁,𝑐₂,…𝑐_𝑝(𝑝=|𝐶₁|), according to the order and tolerance relation, 𝑆 can be divided into 𝑘 sub-decision tables 𝑆₁,𝑆₂,…,𝑆_𝑘, where 𝑆_𝑖=⟨𝑈_𝑖,𝐴,𝑉,𝑓⟩. Assume 𝑆 and 𝑆₁,𝑆₂,…,𝑆_𝑘 be processed with the same way. For each sub-decision table 𝑆_𝑖(1≤𝑖≤𝑘), for all 𝑥(𝑥∈𝑆_𝑖∧𝑥∈Pos_𝐶(𝐷)), let us denote by 𝑑_𝑥 a decision rule relative to object 𝑥 in 𝑆_𝑖 and a decision rule relative to 𝑥 in 𝑆 by 𝑑¹_𝑥, respectively. There is (1)in 𝑆_𝑖, if ∃𝑐(𝑐∈𝐶), 𝑐 is necessary in 𝑑_𝑥. Then, 𝑐 is necessary in 𝑑¹_𝑥;(2)in 𝑆_𝑖, if ∃𝑐(𝑐∈𝐶), 𝑐 is not necessary in 𝑑_𝑥. Then, 𝑐 is not necessary in 𝑑¹_𝑥; (3)𝑑_𝑥=𝑑¹_𝑥.

Proposition 3.12. Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for all 𝐶₁⊆𝐶, given an decomposing order 𝑐₁,𝑐₂,…𝑐_𝑝(𝑝=|𝐶₁|), and according to the order, 𝑆 can be divided into 𝑘 sub-decision tables 𝑆₁,𝑆₂,…,𝑆_𝑘, where 𝑆_𝑖=⟨𝑈_𝑖,𝐴,𝑉,𝑓⟩. Assume 𝑆 and 𝑆₁,𝑆₂,…,𝑆_𝑘 be processed with the same. For each sub-decision table 𝑆_𝑖(1≤𝑖≤𝑘), let us denote by 𝑅𝑢𝑙𝑒 the certain decision rule set of 𝑆 and the one of 𝑆_𝑖 by Rule_𝑖, respectively. Then, Rule=⋃^𝑘_𝑖=1Rule_𝑖.

Propositions 3.11 and 3.12 present the approach of value reduction based on divide and conquer method. It can keep invariance between the solution of original decision table and the ones of sub-decision tables. By using the approach, it allows us to generate decision rules from sub-decision tables, not from original decision table. It may be a feasible way to process big data set.

3.3. The Abstract Process for Knowledge Reduction Based on Divide and Conquer Method in Rough Set Theory

According to the divide and conquer method under equivalence relation and tolerance relation, the abstract process for knowledge reduction in rough set theory based on the divide and conquer method APFKRDAC(𝑃, 𝑆) will be discussed in this section.

Algorithm 3.13 (APFKRDAC(𝑃,𝑆)). Input: The problem 𝑃 on 𝑆.
Output: The solution 𝑆𝑜𝑙𝑢 of the problem 𝑃.

Step 1 (determine a similarity relation of different objects). Determine a similarity relation between different objects, such as equivalence relation or tolerance relation. Generally, reflexivity and symmetry of different objects may be necessary.

Step 2 (determine the decomposing order). Determine the order 𝐶₁(𝐶₁⊆𝐶) for decomposing the universe of decision table. Let 𝐶₁={𝑐₁,𝑐₂,…,𝑐_𝑝},(𝑝=|𝐶₁|).

Step 3 (determine the decomposing strategy). (3.1) Design a judgment criteria 𝐶𝑎𝑛𝐵𝑒𝐷𝑖𝑣𝑖𝑑𝑒() for judging whether the universe can be decomposed.(3.2) Design a decompose function 𝐷𝑒𝑐𝑜𝑚𝑝𝑜𝑠𝑖𝑛𝑔𝐹𝑢𝑛𝑐(), which can be used to decompose the universe recursively.(3.3) Design a boolean function 𝐼𝑠𝐸𝑛𝑜𝑢𝑔ℎ𝑆𝑚𝑎𝑙𝑙(), which can be used to judge if the size of problem is small enough to be processed easily.(3.4) Design a computation function 𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑆𝑚𝑎𝑙𝑙𝑃𝑟𝑜𝑏𝑙𝑒𝑚(), which can be used to process small problems directly.(3.5) Design a computation function 𝑀𝑒𝑟𝑔𝑖𝑛𝑔𝑆𝑜𝑙𝑢𝑡𝑖𝑜𝑛(), which can be used to merge the solutions of sub-problems.

Step 4 (process the problem based on divide and conquer method).   (4.1) IF 𝐼𝑠𝐸𝑛𝑜𝑢𝑔ℎ𝑆𝑚𝑎𝑙𝑙(𝑆) THEN 𝑆𝑜𝑙𝑢=𝑆𝑜𝑙𝑢𝑡𝑖𝑜𝑛𝑆𝑢𝑏𝑃𝑟𝑜𝑏𝑙𝑒𝑚𝑠(𝑆),    goto Step  5. (4.2) (Divide)     According to the decomposing order, divide 𝑆 into 𝑘 sub-decision tables     𝑆₁,𝑆₂,…,𝑆_𝑘 on 𝐶₁. (4.3) (Conquer sub-problems recursively)    FOR 𝑖=1 TO 𝑘 DO      𝑆𝑜𝑙𝑢_𝑖=APFKRDAC(𝑃_𝑖,𝑆_𝑖).    END FOR.    Where, 𝑆_𝑖=⟨𝑈_𝑖,𝐶_𝑖∪𝐷,𝑉,𝑓⟩, 𝐶_𝑖⊆𝐶, 𝑃_𝑖 is the sub-problem of 𝑆_𝑖.  (4.4) (Merge the solutions of sub-problems)    𝑆𝑜𝑙𝑢=𝑀𝑒𝑟𝑔𝑖𝑛𝑔𝑆𝑜𝑙𝑢𝑡𝑖𝑜𝑛(𝑆𝑜𝑙𝑢₁,𝑆𝑜𝑙𝑢₂,…,𝑆𝑜𝑙𝑢_𝑘).

Step 5 (optimize the solution). If necessary, optimize the solution 𝑆𝑜𝑙𝑢.

Step 6 (return the solution). RETURN 𝑆𝑜𝑙𝑢.

Now, let us give an example for computing the positive region of decision table to explain Algorithm 3.13 (see Algorithm 3.14: the algorithm for computing positive region based on divide and conquer method).

Algorithm 3.14 (CPRDAC(𝑃,𝑆)).  Input: The problem 𝑃 on 𝑆: compute positive region. Output: The positive region Pos_𝐶(𝐷) of 𝑆.

Step 1 (Determine the similar relation). equivalence relation.

Step 2 (Determine the decomposing order). 𝑐₁,𝑐₂,…,𝑐_𝑚(𝑚=|𝐶|).

Step 3 (Determine the decomposing strategy).   (3.1) Design a judgment criteria 𝐶𝑎𝑛𝐵𝑒𝐷𝑖𝑣𝑖𝑑𝑒():     On attribute 𝑐_𝑖, for all 𝑈_𝑖(𝑈_𝑖⊆𝑈),     IF ∃_{𝑥∈𝑈𝑖}∃_{𝑦∈𝑈𝑖}(𝑐_𝑖(𝑥)≠𝑐_𝑖(𝑦))∧∃_{𝑧∈𝑈𝑖}∃_{𝑤∈𝑈𝑖}(𝐷(𝑧)≠𝐷(𝑤)) THEN      𝐶𝑎𝑛𝑏𝑒𝐷𝑖𝑣𝑖𝑑𝑒(𝑐_𝑖)=true;    ELSE 𝐶𝑎𝑛𝐵𝑒𝐷𝑖𝑣𝑖𝑑𝑒(𝑐_𝑖)=false;     END IF  (3.2) Design a decompose function 𝐷𝑒𝑐𝑜𝑚𝑝𝑜𝑠𝑖𝑛𝑔𝐹𝑢𝑛𝑐():    According to 𝑈/𝐶, 𝑆 can be divided into 𝑘=|𝑈/𝐶| sub-decision tables    𝑆₁,𝑆₂,…,𝑆_𝑘 on attributes 𝑐₁,𝑐₂,…,𝑐_𝑚 recursively. (3.3) Design a boolean function 𝐼𝑠𝐸𝑛𝑜𝑢𝑔ℎ𝑆𝑚𝑎𝑙𝑙():     Let 𝑐_𝑟 be the attribute on which the universe is being decomposed.     IF (|𝑈_𝑖|==1) or (𝑟>𝑚) or ∀_{𝑥∈𝑈𝑖}∀_{𝑦∈𝑈𝑖}(𝐷(𝑥)==𝐷(𝑦)) THEN      𝐼𝑠𝐸𝑛𝑜𝑢𝑔ℎ𝑆𝑚𝑎𝑙𝑙(𝑆_𝑖)=𝑡𝑟𝑢𝑒.    END IF  (3.4) Design a computation function 𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑆𝑚𝑎𝑙𝑙𝑃𝑟𝑜𝑏𝑙𝑒𝑚():    For arbitrary sub-decision table 𝑆_𝑖 and its universe 𝑈_𝑖,      IF ∀_{𝑥∈𝑈𝑖}∀_{𝑦∈𝑈𝑖}(𝐷(𝑥)==𝐷(𝑦)) THEN 𝑈_𝑖⊆Pos_𝐶(𝐷)∧𝑆𝑜𝑙𝑢_𝑖=𝑈_𝑖;     ELSE 𝑆𝑜𝑙𝑢_𝑖=𝜙;     END IF  (3.5) Design a computation function 𝑀𝑒𝑟𝑔𝑖𝑛𝑔𝑆𝑜𝑙𝑢𝑡𝑖𝑜𝑛():     𝑆𝑜𝑙𝑢=𝑆𝑜𝑙𝑢₁∪𝑆𝑜𝑙𝑢₂∪⋯∪𝑆𝑜𝑙𝑢_𝑘;

Step 4 (Process the problem based on the divide and conquer method).  (4.1) IF 𝐼𝑠𝐸𝑛𝑜𝑢𝑔ℎ𝑆𝑚𝑎𝑙𝑙(𝑆) THEN 𝑆𝑜𝑙𝑢=𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑆𝑚𝑎𝑙𝑙𝑃𝑟𝑜𝑏𝑙𝑒𝑚(𝑆); goto Step  5. (4.2) (Divide)     According to the order {𝑐₁,𝑐₂,…,𝑐_𝑚}, 𝑆 can be divided into 𝑘 sub-    decision tables 𝑆₁,𝑆₂,…,𝑆_𝑘 on 𝑐₁ by using 𝑈/IND({𝑐₁}). (4.3) (Conquer sub-problems recursively)    FOR 𝑖=1 TO 𝑘 DO      𝑆𝑜𝑙𝑢_𝑖= CPRDAC(𝑃_𝑖,𝑆_𝑖).     END FOR    Where, 𝑆_𝑖=⟨𝑈_𝑖,(𝐶−{𝑐₁})∪𝐷,𝑉,𝑓⟩, 𝑃_𝑖 denotes computing the positive region of     𝑆_𝑖.  (4.4) (Merge the solutions of sub-problems)    𝑆𝑜𝑙𝑢=𝑆𝑜𝑙𝑢₁∪𝑆𝑜𝑙𝑢₂∪⋯∪𝑆𝑜𝑙𝑢_𝑘.

Step 5 (Optimize the solution). 𝑆𝑜𝑙𝑢 is an optimized result.

Step 6 (Return the solution). RETURN 𝑆𝑜𝑙𝑢.

Example 3.15. Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, (𝑈={𝑥₁,𝑥₂,𝑥₃, 𝑥₄,𝑥₅,𝑥₆},𝐶={𝑐₁,𝑐₂,𝑐₃}), now compute the positive region of 𝑆 according to Algorithm 3.14. The whole process can be found in Figure 1.
Let us denote by Pos_𝐶(𝐷),Pos¹_𝐶(𝐷),Pos²_𝐶(𝐷),…,Pos⁷_𝐶(𝐷) the positive region of 𝑆,𝑆₁,𝑆₂,…,𝑆₇, respectively.
Divide 𝑆 into 𝑆₁,𝑆₂,𝑆₃ on 𝑐₁.
Conquer 𝑆₁.
 Divide 𝑆₁ into 𝑆₄,𝑆₅ on 𝑐₂.
 Conquer 𝑆₄. Pos⁴_𝐶(𝐷)={𝑥₃,𝑥₅}.
 Conquer 𝑆₅. Pos⁵_𝐶(𝐷)={𝑥₆}.
 Merge the solutions of 𝑆₄ and 𝑆₅. Pos¹_𝐶(𝐷)=Pos⁴_𝐶(𝐷)∪Pos⁵_𝐶(𝐷)={𝑥₃,𝑥₅,𝑥₆}.
Similarly, we can conquer 𝑆₂ and 𝑆₃.
Conquer 𝑆₂. Pos²_𝐶(𝐷)={𝑥₄}.
Conquer 𝑆₃. Pos³_𝐶(𝐷)=𝜙.
Merge the solutions of 𝑆₁, 𝑆₂, and 𝑆₃. Pos_𝐶(𝐷)=Pos¹_𝐶(𝐷)∪Pos²_𝐶(𝐷)∪Pos³_𝐶(𝐷)=𝑥₃,𝑥₄,𝑥₅,𝑥₆. (3.2)

Figure 1 

The course of computing positive region of 𝑆.

4. A Fast Algorithm for Attribute Reduction Based on Divide and Conquer Method

Knowledge reduction is the key problem in rough set theory. When the divide and conquer method is used to design the algorithm for knowledge reduction, some good results may be obtained. However, implementing the knowledge reduction based on the divide and conquer method is very complex, though it is only a simple granular computing method. Here, we will discuss the quick algorithm for knowledge reduction based on divide and conquer method.

In the course of attribute reduction, the divide and conquer method is used to compute the equivalence class, the positive region, and the non-empty label attribute set and delete the elements of discernibility matrix. Due to the complexity of attribute reduction, the following algorithm is not presented as Algorithm 3.14 in details.

According to Step  2 of Algorithm 3.13, the attribute set and the order must be determined, on which the universe of decision table will be partitioned in turns. Generally speaking, the decomposing order depends on the problem which needs to be solved. Furthermore, if the order is not given by field experts, it can be computed by the weights in [10, 15, 23, 25, 26, 28, 33, 36, 37, 41]. Of course, if the attribute order is given, it will be more suitable for Algorithm 3.13. Most techniques discussed below are based on a given attribute order and divide and conquer method. In this section, a quick attribute reduction algorithm based on a given attribute order and divide and conquer method is proposed.

4.1. Attribute Order

In 2001, an algorithm for attribute reduction based on the given attribute order is proposed by Jue Wang and Ju Wang [38]. For the convenience of illustration, some basic notions about attribute order are introduced here.

Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, an attribute order relation over 𝐶(𝑆𝑂∶𝑐₁≺𝑐₂≺⋯≺𝑐_|𝐶|) can be defined. Let us denote by 𝑀 the discernibility matrix of 𝑆. For any 𝛿∈𝑀, the attributes of discernibility matrix 𝛿 inherit the order relation of 𝑆𝑂 from left to right, that is, 𝛿=𝑐_𝑗𝐵, where 𝑐_𝑗∈𝐶 and 𝐵⊂(𝐶−{𝑎}), and 𝑐_𝑗 is the first element of 𝛿 by 𝑆𝑂, called the non-empty label attribute of 𝛿 [33].

For 𝑐_𝑗, a set 𝐿(𝑆𝑂)={𝛿∣𝛿=𝑐_𝑗𝐵, 𝛿 inherit the order relation of 𝑆𝑂 from left to right and 𝛿∈𝑀} is defined. Hence, 𝑀 can be divided into equivalence classes by label attributes defining a partition {[1],[2],…,[|𝐶|]} of 𝑀 denoted by 𝑀/𝐿(𝑆𝑂) [33]. Supposing 𝑁=max{𝑘∣[𝑘]∈𝑀/𝐿(𝑆𝑂)∧[𝑘]≠𝜙}, its maximum non-empty label attribute is 𝑎_𝑁.

4.2. Attribute Reduction Based on the Divide and Conquer Method

Lemma 4.1. ∀_𝐵^𝑆_{𝑥𝑦∈𝑀}(𝐵^𝑆_𝑥𝑦≠𝜙) (Definition 2.8) if and only if (𝑥∈Pos_𝐶(𝐷)∧𝑦∈Pos_𝐶(𝐷)∧(𝑑(𝑥)≠𝑑(𝑦))∨(𝑥∈Pos_𝐶(𝐷)∧𝑦∉Pos_𝐶(𝐷))∨(𝑥∉Pos_𝐶(𝐷)∧𝑦∈Pos_𝐶(𝐷)).

According to Definition 2.8, obviously Lemma 4.1 holds.

Proposition 4.2. Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for all 𝑐∈𝐶, let 𝑀 be the discernibility matrix of 𝑆. Then, ∃_𝛼∈𝑀(𝑐∈𝛼)   if and only if the following conditions hold: (1) |𝑈|>1; (2) ∃_𝑥∈𝑈(𝑥∈Pos_𝐶(𝐷)); (3) |𝑉_𝐷|>1; (4) |𝑉_{𝑐}|>1.

Proof. (Necessity) according to Lemma 4.1, obviously Proposition 4.2 holds.
(Sufficiency):
 for all 𝑥(𝑥∈Pos_𝐶(𝐷)∧𝑥∈𝑈), according to (|𝑈|>1)∧(|𝑉_𝐷|>1), then,
 ∃_{𝑦(𝑦≠𝑥)}(𝑦∈𝑈∧𝐷(𝑥)≠𝐷(𝑦)).
 If 𝑐(𝑥)≠𝑐(𝑦), then 𝑐∈𝐵^𝑆_𝑥𝑦, that is, ∃_{𝛼(𝛼∈𝑀)}𝑐∈𝛼. The proposition holds.
 If 𝑐(𝑥)=𝑐(𝑦), then, there are two cases.Case 1 (𝑦∈Pos_𝐶(𝐷)). According to |𝑉_{𝑐}|>1, ∃_{𝑧(𝑧≠𝑥∧𝑧≠𝑦)}(𝑐(𝑧)≠𝑐(𝑥)), thus (𝑐∈𝐵^𝑆_𝑥𝑧)∨(𝑐∈𝐵^𝑆_𝑦𝑧)=1.
That is, ∃_{𝛼(𝛼∈𝑀)}𝑐∈𝛼. The proposition holds.
Case 2 (𝑦∉Pos_𝐶(𝐷)). According to |𝑉_{𝑐}|>1, ∃_{𝑧(𝑧≠𝑥∧𝑧≠𝑦)}(𝑐(𝑧)≠𝑐(𝑥)). If 𝑧∈Pos_𝐶(𝐷), then 𝑐∈𝐵^𝑆_𝑦𝑧;
if 𝑧∉Pos_𝐶(𝐷), then 𝑐∈𝐵^𝑆_𝑦𝑧. Thus, the proposition holds.
Therefore, Proposition 4.2 holds.

According to the algorithm in [38], in order to compute the attribute reduction of a decision table, its non-empty label attribute set 𝐿(𝑆𝑂) should be first calculated. Using the divide and conquer method, an efficient algorithm for computing the non-empty label attribute set 𝐿(𝑆𝑂) is developed. A recursive function for computing the non-empty label attribute set is used in the algorithm.

Function 1
NonEmptyLabelAttr(𝑆,𝑟)
//𝑆 is decision table. 𝑟 is the number of attributes (1≤𝑟≤|𝐶|).

Step 1 (Ending Condition). According to Propositions 4.2 and 5.1,
IF 𝑟=|𝐶|+1 or |𝑈|==1 or Pos_𝐶(𝐷)==𝜙 or |𝑉_𝐷|==1 THEN
 return;
END IF

Step 2 (Compute non-empty labels based on divide and conquer method). Let NonEmptyLabel[] be an array used to store the solution.
Step 2.1. IF |𝑉_{𝑐𝑟}|>1 THEN
 Denote non-empty label attribute: NonEmptyLabel[𝑟] = 1;
END IF
Step 2.2 (Divide). Divide 𝑆 into 𝑆₁,𝑆₂,…,𝑆_𝑘 by 𝑈/IND({𝑐_𝑟});
Step 2.3 (Conquer sub-problems). FOR 𝑖 = 1 TO 𝑘 DO
 NonEmptyLabelAttr(𝑆_𝑖,𝑟+1);
END FOR.
Step 2.4 (Merge the solutions). Here, there is no operation because the solutions are stored in the array NonEmptyLabel[].
END Function 1.

Using the above recursive function, an algorithm for computing the non-empty label attribute set of a decision table is developed.

Algorithm 4.3. Computation of The Non-empty Label Attribute Set 𝐿(𝑆𝑂)
Input: A decision table 𝑆 and an attribute order 𝑆𝑂∶𝑐₁≺𝑐₂≺⋅⋅⋅≺𝑐_|𝐶|
Output: The non-empty label attribute set 𝑅₁ of 𝑆.

Step 1. 𝑅₁=𝜙; 𝑟=1;
FOR 𝑗=1 TO |𝐶| DO
 𝑁𝑜𝑛𝐸𝑚𝑝𝑡𝑦𝐿𝑎𝑏𝑒𝑙[𝑗]=0;
END FOR

Step 2. NonEmptyLabelAttr(𝑆,1);

Step 3. FOR 𝑗=1 TO |𝐶| DO
 IF 𝑁𝑜𝑛𝐸𝑚𝑝𝑡𝑦𝐿𝑎𝑏𝑒𝑙[𝑗]==1 THEN 𝑅₁=𝑅₁∪{𝑐_𝑗};
END FOR

Step 4. RETURN 𝑅₁.

Suppose 𝑛=|𝑈|, 𝑚=|𝐶|. According to the conclusion of literature [45], the average time and space complexities of the Algorithm 4.3 are 𝑇=𝑂(𝑛×(𝑚+log𝑛)) and 𝑆=𝑂(𝑚+𝑛).

Obviously, Algorithm 4.3 is an instance of Algorithm 3.13. Given an attribute order of the conditional attributes in a decision table, using the Algorithm 4.3 and divide and conquer method, an efficient attribute reduction algorithm is developed.

Algorithm 4.4. Computation of Attribute Reduction Based on Divide and Conquer Method
Input: A decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩ and an attribute order 𝑆𝑂∶𝑐₁≺𝑐₂≺⋅⋅⋅≺𝑐_|𝐶|
Output: Attribute reduction 𝑅 of 𝑆.

Step 1. 𝑈¹₁=𝑈, 𝑅=𝜙.

Step 2. Compute the positive region Pos_𝐶(𝐷), according to Algorithm 3.14.

Step 3. Compute the non-empty label attribute set 𝑅₁ by Algorithm 4.3.

Step 4. //Suppose 𝑐_𝑁 be the maximum label attribute of 𝑅₁.
𝑅=𝑅∪{𝑐_𝑁′};𝑅₁=𝑅₁−𝑅;
IF 𝑅1==𝜙 THEN RETURN 𝑅;
ELSE
 Generate a new attribute order:
 𝑐¹₁≺𝑐¹₂≺⋯≺𝑐¹_|𝑅|≺𝑐²₁≺𝑐²₂≺⋯≺𝑐²_|𝑅1|;
 Compute new non-empty label attribute set 𝑅₁ of 𝑆 by Algorithm 4.3;
 GOTO Step  4.
END IF

Suppose 𝑛=|𝑈| and 𝑚=|𝐶|, the average time complexity of the Algorithm 4.4 is 𝑂(𝑛×𝑚×(𝑚+log𝑛)). Its space complexity is 𝑂(𝑚+𝑛).

In Algorithm 4.4, Step  1 is the initialization. In Step  2 of Algorithm 4.4, divide and conquer method is used to compute equivalence classes and positive region, thus Step  2 is an instance of Algorithm 3.13. In Step 3, Algorithm 4.3 is used to compute non-empty label attribute set (Algorithm 4.3 is also an instance of Algorithm 3.13). Step  4 is responding to Step  5 of Algorithm 3.13. In Step  4, Algorithm 4.3 is called repeatedly to reduce the redundant attribute set. That is, Algorithm 4.4 is composed of instances of Algorithm 4.3, which illustrates that Algorithm 4.4 is implemented by divide and conquer method. Basically, divide and conquer method is used primarily to compute equivalence classes, positive region, and non-empty label attribute set and delete element cells in discernibility matrix in Algorithm 4.4.

5. A Fast Algorithm for Value Reduction Based on Divide and Conquer Method

Proposition 5.1. Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, for all 𝑥∈Pos_𝐶(𝐷), a certain rule can be induced by object 𝑥.

Proposition 5.2. Given a decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩, let us denote by 𝐷𝑅 a certain rule set of 𝑆. ∀_{𝑑𝑖∈𝐷𝑅(1≤𝑖≤|𝐷𝑅|)}𝑑_𝑖, 𝑑_𝑖 must be induced by 𝑥(𝑥∈Pos_𝐶(𝐷)).

Proposition 5.3. Given a decision table 𝑆⟨𝑈,𝐴,𝑉,𝑓⟩, let us denote by 𝐷𝑅 certain rule set of 𝑆. For all 𝑥∈Pos_𝐶(𝐷), let us denote by 𝑑₁ the certain rule by 𝑥. Let 𝑀₁={𝐵^𝑠_𝑥𝑦∣𝑦∈𝑈}. Then, ∀_𝛼∈𝑀1(𝑀₁≠𝜙)⇒∃_{𝑐∈(𝑑1∣𝐶)}(𝑐∈𝛼).

According to Algorithm 3.13, Propositions 5.1, 5.2, and 5.3, a recursive function and an algorithm for value reduction based on divide and conquer method are developed as follows.

Function 2
DRAVDAC(𝑆,𝑐_𝑟)
//Denote by array CoreValueAttribute[] the result of value reduction of 𝑐_𝑟.
//The values of array CoreValueAttribute[] are all 0 initially.

Step 1 (Ending Condition). IF there is contradiction on 𝑆, THEN
 return;
END IF

Step 2 (Value reduction on 𝑐_𝑟 based on divide and conquer method). Step 2.1 (Divide).
Divide 𝑆 into 𝑘 sub-decision tables 𝑆₁,𝑆₂,…,𝑆_𝑘 on attribute 𝑐_𝑟 by using tolerance relation.
Step 2.2 (Conquer sub-problems recursively).
Denote by array 𝑆𝑜𝑙𝑢_𝑖[|𝑈_𝑖|] the solution of 𝑆_𝑖.
Where, 𝑆_𝑖=⟨𝑈_𝑖,(𝐶−{𝑐_𝑟})∪𝐷,𝑉,𝑓⟩.
FOR 𝑖 = 1 TO 𝑘 DO
 𝑆𝑜𝑙𝑢_𝑖 = DRAVDAC(𝑆_𝑖);
END FOR
Step 2.3 (Merge the solutions).
FOR 𝑗 = 1 TO |𝑈| DO
 FOR 𝑖 =1 TO 𝑘 DO
  IF 𝑆𝑜𝑙𝑢_𝑖[𝑗] == 1 THEN break; END IF
 END FOR
 IF 𝑖==𝑘+1 THEN CoreValueAttribute[𝑗] =1 END IF
END FOR
END Function 2.

Using Function 2, we present an algorithm for value reduction based on divide and conquer method (see Algorithm 5.4).

Algorithm 5.4. An Algorithm for Value Reduction Based on Divide and Conquer Method
Input: A decision table 𝑆=⟨𝑈,𝐴,𝑉,𝑓⟩
Output: The certain rule set 𝐷𝑅 of 𝑆.

Step 1 (Initiation). 𝐴𝑆𝑒𝑡=𝜙, 𝐷𝑅=𝜙.

Step 2 (Compute the positive region). According to Algorithm 3.14, compute the positive region Pos_𝐶(𝐷) of 𝑆.

Step 3 (Compute the non-empty label attribute). Assume the order for dividing decision table be 𝑐₁,𝑐₂,…,𝑐_𝑚(𝑚=|𝐶|).
Compute the non-empty label attribute set 𝐴𝑆𝑒𝑡 by using Function 1.

Step 4 (Value reduction on attribute set 𝐴𝑆𝑒𝑡). Let 𝐴𝑆𝑒𝑡={𝑐^′₁,𝑐^′₂,…,𝑐^′_{|𝐴𝑆𝑒𝑡|}} and the divide order be 𝑐^′₁,𝑐^′₂,…,𝑐^′_{|𝐴𝑆𝑒𝑡|}.
FOR i = |𝐴𝑆𝑒𝑡| TO 1 DO
 FOR 𝑗 = 1 TO |𝑈| DO
  CoreValueAttribute[𝑗] = 0.
 END FOR.
  𝐶=𝐴𝑆𝑒𝑡.
  Invoke Function 2: DRAVDAC(𝑆,{𝑐^′_𝑖).
 Update 𝑆, according to the array CoreValueAttribute[].
END FOR.

Step 5 (Get the rule set). FOR 𝑖= 1 TO |𝑈| DO
 IF 𝑥_𝑖∈Pos_𝐶(𝐷) THEN
  Construct a rule 𝑑_𝑖 in terms with 𝑥_𝑖; 𝐷𝑅=𝐷𝑅∪{𝑑_𝑖};
 ENE IF
END FOR

Step 6. RETURN 𝐷𝑅.

Suppose |𝐶|=𝑚,|𝑈|=𝑛. The time complexity of Step  1 is 𝑂(𝑚+𝑛). The average time complexities of Steps  2 and 3 are 𝑂(𝑛×(𝑚+log𝑛)) [45]. The time complexities of Steps  5 and 6 are both 𝑂(𝑚×𝑛). Now, let us analyze the time complexity of Step  4.

In Step  4, let the number of non-empty label attribute set be 𝑢. Then, the time complexity of Step  4 is 𝑂(𝑢×(2𝑛+𝑇(1,𝑛))), where 𝑇(1,𝑛) is an instance of 𝑇(𝑟,𝑛) which can be expressed by the following recursive equation: 𝑇(𝑟,𝑛)=()1.𝑛==12𝑛.𝑟≥𝑢2𝑛+𝑇𝑟+1,𝑛.∀_{𝑥,𝑦∈𝑈}𝑇_𝐵𝑥,𝑦,𝐵=𝑐₁,𝑐₂,…,𝑐_𝑟𝑝𝑛+𝑇𝑟,𝑛₁+𝑛^′+𝑇𝑟,𝑛₂+𝑛^′+⋯+𝑇𝑟,𝑛_𝑝+𝑛^′.𝑛₁+𝑛₂+⋯+𝑛_𝑝+𝑛^′=𝑛,𝑝=𝑐_𝑟(𝑥)∣𝑐_𝑟(𝑥)≠∗,𝑥∈𝑈. (5.1)𝑇(1,𝑛) is between 𝑂(𝑢×𝑛) and 𝑇(𝑝₁×𝑝₂×⋯×𝑝_𝑢×𝑛). So the time complexity of Step  4 is between 𝑂(𝑢²×𝑛) and 𝑇(𝑢×𝑝₁×𝑝₂×⋯×𝑝_𝑢×𝑛). Thus, the time complexity of Algorithm 5.4 is between max(𝑂(𝑛×(𝑚+log𝑛)),𝑂(𝑢²×𝑛)) and 𝑂(𝑢×𝑛×𝑝₁×𝑝₂×⋯×𝑝_𝑢).

Suppose the data obey the uniform distribution. The time complexity of Algorithm 5.4 is 𝑂(𝑛^{log((𝑝+1)/2)𝑝}×𝑚)+𝑂(𝑛×𝑚)=𝑂(𝑛^{log((𝑝+1)/2)𝑝}×𝑚). When 𝑝≥2, the time complexity of Algorithm 5.4 is less than 𝑂(𝑛²×𝑚). When 𝑝≥5, the time complexity of Algorithm is less than 𝑂(𝑛^1.5×𝑚).

The space complexity of Algorithm 5.4 is 𝑂(𝑚×𝑛).

6. Experimental Evaluations

In order to test the efficiency of knowledge reduction based on divide and conquer method, some experiments have been performed on a personal computer. The experiments are shown as follows.

6.1. The Experimental Evaluations on UCI Data Sets

In this experiment, some experimental evaluations are done to present the efficiency and recognition results of Algorithms 4.4 and 5.4. In the mean time, some famous approaches for data mining are used to compare with our methods.

The test course is as follows. First, 11 uci data sets (Zoo, Iris, Wine, Machine, Glass, Voting, Wdbc, Balance-scale, Breast, Crx, and Tic-tac-toe) are used. Second, our methods: the algorithm for discretization [14] (it is an improved one based on the discretization method in [28]), the algorithm for attribute reduction (Algorithm 4.4), and the algorithm for attribute value reduction (Algorithm 5.4) are used to test the 11 uci data sets. Third, 5 methods (KNN, SVM, C4.5, Naive Bayes, and CART) are also used to test the data sets, which belong to the “top 10 algorithms in data mining” [44], and their source codes are afforded by Weka software. Weka (http://en.wikipedia.org/wiki/Weka_(machine_l earning)) is used as the experimental plat and “Java Eclipse Weka” as the developing tool. The test method is LOOCV (Leave One Out Cross Validation). The specifications of the experimental computer are an Intel(R) Core(TM2) Quad CPU Q8200 @2.33 GHz CPU, 2 GB RAM, and Microsoft Windows 7. The specifications of 11 data sets and the experimental results are as follows.

From Table 1 and Figure 2, it can be found that the recognition results of our methods on the 11 uci data sets are closed to the ones of KNN and CART, which are better than Naive Bayes, C4.5, and SVM.

Table 1 

Specifications of 11 uci data sets.

Figure 2 

Recognition result on uci data sets.

6.2. The Experimental Results on KDDCUP99 Data Sets

In order to test the efficiency of our methods for processing large data sets, some experiments are done on KDDCUP99 data sets with 4898432 records, 41 condition attributes, and 23 decision classifications (http://kdd.ics.uci.edu/databases//kddcup99/kddcup99.html). Our methods consist of the discretization algorithm [14], Algorithms 4.4, and 5.4 still. Weka is used as the experimental plat and “Java Eclipse Weka” as the developing tool (Table 2). The test method is 10 cross-validation. The specifications of the experimentation computer are an Intel(R) Core(TM2) Quad CPU Q8200 @2.33 GHz CPU, 4 GB RAM, and Microsoft Windows Server 2008. The experimental results are as follows.

Table 2 

The recognition of 6 algorithms on uci data sets.

First, the experiments are done on 10 data sets (≤10⁴ records) from the original KDDCUP99 data sets. The experimental evaluations are showed in Tables 3, 4, and 5, where the time unit is “ms” in Tables 4 and 5.

Table 3 

The recognition of 6 algorithms on KDDCUP99 data sets (≤10⁴ records).

Table 4 

The training time of 6 algorithms on KDDCUP99 data sets (≤10⁴ records).

Table 5 

The test time of 6 algorithms on KDDCUP99 data sets (≤10⁴ records).

From Tables 3, 4, and 5, it can be found that it will cost much time to train by SVM, which shows that SVM is not a good way to process KDDCUP99 data sets with large records. Thus, in the following experiments, SVM will be not tested.

Second, the experiments are done on 10 data sets (≤10⁵ records) from the original KDDCUP99 data sets. The experimental evaluations are showed in Tables 6 and 7, where “Tr” is the training time, “Te” is the test time, and the time unit is “ms” in Table 7.

Table 6 

The recognition of 5 algorithms on KDDCUP99 data sets (≤10⁵ records).

Table 7 

The running time of 5 algorithms on KDDCUP99 data sets (≤10⁵ records).

From Tables 6 and 7, it can be found that the recognition is lower than others by Naive Bayes and much test time is needed for KNN. Thus, Naive Bayes and KNN will be not tested in the following experiments.

Third, the experiments are done on 10 data sets (≤10⁶ records) from the original KDDCUP99 data sets. The experimental evaluations are showed in Table 8, where “RRate” is the recognition rate, “Tr” is the training time, “Te” is the testing time, and the time unit is “ms” in Table 8.

Table 8 

The experimental results of 3 algorithms on KDDCUP99 data sets (≤10⁶ records).

Fourth, the experiments are done on 10 data sets (<5×10⁶ records) from the original KDDCUP99 data sets. The experimental results are showed in Table 9, where “RRate” is the recognition rate, “Tr” is the training time, “Te” is the testing time, “-” is the overflow of memory, and the time unit is “second” in Table 9.

Table 9 

The experimental results of 3 algorithms on KDDCUP99 data sets (<5 × 10⁶ records).

From Tables 8 and 9, it can be found that C4.5 is the best one and our method is the second best one among C4.5, CART, and our method. Due to the high complexity for discretization, our method can not complete the knowledge reduction of 4898432 records in this experiment.

6.3. The Conclusions of Experimental Evaluations

Now, we will give some conclusions for our approaches compared with KNN, SVM, C4.5, Naive Bayes, and CART, according to the LOOCV experimental results on the uci data sets and 10 cross-validation experimental results on KDDCUP99 data sets.(1)Compared with KNN, SVM, and Naive Bayes, the LOOCV recognition results by our methods on uci data sets are better than KNN, SVM, and Bayes. Furthermore, our methods on KDDCUP99 data sets have higher efficiency than KNN, SVM, and Naive Bayes, while they also have good recognitions results.(2)Compared with CART, the LOOCV recognition results by our methods on uci data sets are closed to CART. But our methods can process larger data sets than CART on KDDCUP99 data sets, while they both have good recognition results.(3)Compared with C4.5, the LOOCV recognition results by our methods on uci data sets are better than C4.5. Furthermore, the test time by our methods on KDDCUP99 data sets is less than C4.5, while C4.5 can process larger data sets than our methods. After these two methods are analyzed, we find that our methods are more complex than C4.5, due to the complex discretization (C4.5 can process the decision table with continuous values directly, while the discretization should be necessary for our methods). As a coin has two sides, enough learning contributes to the better rule sets, thus less test time is needed by our methods than C4.5.

Therefore, the knowledge reduction approaches based on divide and conquer method are efficient to process large data set, although they need to be improved further in the future.

7. Conclusions

In this paper, the abstract process of knowledge reduction based on divide and conquer method is concluded, which is original from the approaches under the equivalence relation and the one under the tolerance relation. Furthermore, an example for computing positive region of the decision table is introduced. After that, two algorithms for knowledge reduction based on divide and conquer method, including an algorithm for attribute reduction and an algorithm for attribute value reduction, are presented, respectively. The proposed algorithms are efficient to process the knowledge reduction on uci data sets and KDDCUP99 data set, according to the experimental evaluations. Therefore, the divide and conquer method is an efficient and, therefore, suitable method to be used to knowledge reduction algorithms in rough set theory. With this efficiency, widespread industrial application of rough set theory may become possible.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (NSFC) under Grants no. 61073146, no. 61272060, no. 61203308, and no. 41201378, Scientific and Technological Cooperation Projects between China and Poland Government, under Grant no. 34-5, Natural Science Foundation Project of CQ CSTC under Grant no. cstc2012jjA1649, and Doctor Foundation of Chongqing University of Posts and Telecommunications under Grant no. A2012-08.