An Optimal Classification Method for Biological and Medical Data

Abstract

This paper proposes a union of hyperspheres by the mixed-integer nonlinear program to classify biological and medical datasets. A classifying program with nonlinear terms uses piecewise linearization technique to obtain a global optimum. The numerical examples illustrate that the proposed method can obtain the global optimum more effectively than current methods.

1. Introduction

Classification techniques have been widely applied in the biological and medical research domains [1–5]. Either objects classification or patterns recognition for biological and medical datasets necessarily demands an optimum accuracy for saving patients’ lives. However, cancer identification with the supervised learning technique does not take a global view in identifying species or predicting survivals. The improvement should cover the whole scope to give implications instead of only considering the efficiency for diagnosis. This research aims to extract features from whole datasets in terms of induction rules.

In the given dataset with several objects, in which each object has some attributes and belongs to a specific class, classification techniques are used to find a rule of attributes that appropriately describes the features of a specified class. The techniques have been studied over the last four decades, including decision tree-based methods [6–11], hyperplane-based methods [12–14], and machine learning-based methods [14–17].

To assess the effects of these classifying techniques, three criteria are used for evaluating the quality of inducing rules based on the study of Li and Chen [3].(i)Accuracy. The rule fitting a class should not cover the objects of other classes. The accuracy of a rule should be the higher the better.(ii)Support. A good rule of fitting a class should be supported by most of the objects of the same class.(iii)Compact. A good rule should be expressed in a compact way. That is, the fewer the number of rules, the better the rules are.

This study proposes a novel method to induce rules with high rates of accuracy, support, and compactness based on global optimization techniques, which have become more and more useful in biological and medical researches.

The rest of this paper is organized as follows. Section 2 gives an overview of the related literatures. Two types of mathematical models and a classification algorithm are proposed in Section 3. The numerical examples demonstrate the effectiveness of the proposed method in Section 4. Finally, the main conclusions of this study and future work are drawn in Section 5.

2. Literature Review

Currently, two well-known methods are used to induce classification rules. The first method is the decision tree-based method, which has been developed in the last few decades [6–10]. It is widely applied to fault isolation of an induction motor [18] to classify normal or tumor tissues [19], skeletal maturity assessment [20], proteomic mass spectra classification [21], and other cases [22, 23]. Although the decision tree-based method assumes that all classes can be separated by linear operations, the inducing rules will suffer if the boundaries between the classes are nonlinear. In fact, the linearity assumption prohibits practical applications because many biological and medical datasets have complicated nonlinear interactions between attributes and predicted classes.

Consider the classification problem with two attributes as shown in Figure 1, where “○” represents a first-class object, and “●” represents a second-class object. Figure 1 depicts a situation in which a nonlinear relationship exists between the objects of two classes. Decision tree method focuses on inducing classification rules for the objects, as shown in Figure 1(b), in which the decision tree method requires four rectangular regions to classify the objects.

(a) Nonlinear dataset

(b) Classify by decision tree method

Figure 1 

Classifying the objects of two classes.

The second is the support vector hyperplane method, which conducts feature selection and rule extraction from the gene expression data of cancer tissue [24]; it is also applied in other applications [12–14, 25]. The technique separates observations of different classes by multiple hyperplanes. As the number of decision variables is required to express the relationship between each training datum and hyperplane, and the separating hyperplane is assumed a nonlinear programming problem, the training speed becomes slow for a large number of training data. Additionally, similar hypersphere support vector methods have been developed by Lin et al. [26], Wang et al. [27], Gu and Wu [28], and Hifi and M’Hallah [29] for classifying objects. In classification algorithms, they partition the sample space using the sphere-structured support vector machine [14, 30]. However, these methods need to form a classification problem as a nonlinear nonconvex program, which makes reaching an optimal solution difficult. Taking Figure 1 as an example, a hyperplane-based method requires four hyperplanes to discriminate the objects, as shown in Figure 2.

Figure 2 

Classify by hyperplane method method.

As previously mentioned, many biological and medical datasets have complicated boundaries between attributes and classes. Both decision tree-based methods and hyperplane-based methods find only the rules with high accuracy, which either cover only a narrow part of the objects or require numerous attributes to explain a classification rule. Although these methods are computationally effective for deducing the classifications rules, they have two limitations as follows.(i)Decision tree-based methods are heuristic approaches that can only induce feasible rules. Moreover, decision tree-based methods split the data into hyperrectangular regions using a single variable, which may generate a large number of branches (i.e., low rates of compactness).(ii)Hyperplane-based methods use numerous hyperplanes to separate objects of different classes and divide the objects in a dataset into indistinct groups. The method may generate a large number of hyperplanes and associated rules with low rates of compactness.

Therefore, this study proposes a novel hypersphere method to induce classification rules based on a piecewise linearization technique. The technique reformulates the original hypersphere model by a piecewise linearization approach using a number of binary variables and constraints in the number of piecewise line segments. As the number of break points used in the linearization process increases, the error in linear approximation decreases, and an approximately global optimal solution of the hypersphere model can be obtained. That is, the proposed method is an optimization approach that can find the optimal rules with a high rate of accuracy, support, and compactness. The concept of the hypersphere method is depicted in Figure 3, in which only one circle is required to classify the objects. All objects of class “●” are covered by a circle, and those not covered by this circle belong to class “○.”

Figure 3 

Classify by hypersphere.

3. The Proposed Models and Algorithm

As the classification rules directly affect the rates of accuracy, support, and compactness, we formulate two models to determine the highest accuracy rate and support rate, respectively. To facilitate the discussion, the related notations are introduced first:𝑎_𝑖,𝑗:𝑗^′th attribute value of the 𝑖^′th object,ℎ_{𝑡,𝑘,𝑗}:𝑗^′th center value of the 𝑘^′th hypersphere for class 𝑡,𝑟_𝑡,𝑘:radius of the 𝑘^′th hypersphere for class 𝑡,𝑛(𝑡):number of objects for class 𝑡,𝑐_𝑖:𝑖^′th object belonging to class 𝑐_𝑖∈{1,2,…,𝑔},𝑚:number of attributes,𝑅_𝑡:a rule describing class 𝑡.

Based on these notations, we propose two types of classification models as follow.

3.1. Two Types of Classification Models

Considering the object 𝑥_𝑖 and hypersphere 𝑆_𝑡,𝑘, and normalizing 𝑎_𝑖,𝑗 (i.e., to express its scale easily), we then have the following three notations.

Notation 1. Normalization rescales all 𝑎_𝑖,𝑗 as 𝑎^′_𝑖,𝑗. The following is a normalizing formula: 𝑎^′_𝑖,𝑗=𝑎_𝑖,𝑗−𝑎_𝑗𝑎_𝑗−𝑎_𝑗, (3.1) where 0≤𝑎^′_𝑖,𝑗≤1, 𝑎_𝑗 is the largest value of attribute 𝑗, and 𝑎_𝑗 is the smallest value of attribute 𝑗.

Notation 2. A general form for expressing an object 𝑥_𝑖 is written as 𝑥_𝑖=𝑎^′_𝑖,1,𝑎^′_𝑖,2,…,𝑎^′_𝑖,𝑚;𝑐_𝑖, (3.2) where 𝑐_𝑖 is the class index of object 𝑥_𝑖.

Notation 3. A general form for expressing a hypersphere 𝑆_𝑡,𝑘 is written as 𝑆_𝑡,𝑘=ℎ_𝑡,𝑘,1,ℎ_𝑡,𝑘,2,…,ℎ_{𝑡,𝑘,𝑚};𝑟_𝑡,𝑘, (3.3) where 𝑆_𝑡,𝑘 is the k’th hypersphere for class 𝑡.
We use two and three dimensions (i.e., two attributes and three attributes) as visualizations to depict clearly a circle and a sphere, respectively (Figure 4). Figure 4(a) denotes the centroid of the circle as (ℎ_𝑡,𝑘,1,ℎ_𝑡,𝑘,2) and the radius of the circle as 𝑟_𝑡,𝑘. They are extended to three dimensions called sphere (Figure 4(b)); in 𝑚 dimensions (i.e., 𝑚 attributes), 𝑚>3, which are then called hyperspheres.
To find each center and the radius of the hypersphere, the following two nonlinear models are considered. The first model looks for a support rate as high as possible while the accuracy rate is fixed to 1, as shown in Model 1.

(a) Two dimensions

(b) Three dimensions

Figure 4 

The concept of hypersphere method.

Model 1. One has the following: Maximize_𝑖∈𝐼+𝑢_{𝑡,𝑖,𝑘}subjectto^𝑚_𝑗=1𝑎^′_𝑖,𝑗−ℎ_{𝑡,𝑘,𝑗}²≤1−𝑢_{𝑡,𝑖,𝑘}𝑀+𝑟²_𝑡,𝑘,∀𝑖∈𝐼⁺,_𝑚𝑗=1𝑎^′_𝑖′,𝑗−ℎ_{𝑡,𝑘,𝑗}²>𝑟²_𝑡,𝑘,∀𝑖^′∈𝐼⁻,𝑢_{𝑡,𝑖,𝑘}∈{0,1}∀𝑖∈𝐼⁺,𝑟_𝑡,𝑘≥0,and𝑀isbigenoughconstant, (3.4) where 𝐼⁺ and 𝐼⁻ are the two sets for all objects expressed, respectively, by 𝐼⁺=𝑖∣𝑖=1,2,…,𝑛,whereobject𝑖∈class𝑡, (3.5)𝐼⁻=𝑖^′∣𝑖^′=1,2,…,𝑛,whereobject𝑖^′∉class𝑡. (3.6)
Referring to Li and Chen [3], the rates of accuracy and support of 𝑅_𝑡 in Model 1 can be specified by the following definitions.

Definition 3.1. The accuracy rate of a rule 𝑅_𝑡 for Model 1 is 𝐴𝑅(𝑅_𝑡)=1.

Definition 3.2. The support rate of a rule 𝑅_𝑡 for Model 1 is specified as follows.(i)If ∑_𝑘∈𝐾𝑢_{𝑡,𝑖,𝑘}≥1 for all 𝑖 belonging to class 𝑡, then 𝑈_𝑡,𝑖=1; otherwise 𝑈_𝑡,𝑖=0, where 𝐾 indicates the hypersphere set for class 𝑡.(ii)𝑆𝑅𝑅_𝑡=∑_{𝑖∈class𝑡}𝑈_𝑡,𝑖𝑛(𝑡), (3.7) where 𝑛(𝑡) indicates the number of objects belonging to class 𝑡.
The second model looks for an accuracy rate as high as possible while the support rate is fixed to 1, as shown in Model 2.

Model 2. One has the following: Maximize_{𝑖′∈𝐼−}𝑣_{𝑡,𝑖′,𝑘}subjectto^𝑚_𝑗=1𝑎^′_𝑖,𝑗−ℎ_{𝑡,𝑘,𝑗}²≤𝑟²_𝑡,𝑘,∀𝑖∈𝐼⁺,_𝑚𝑗=1𝑎^′_𝑖′,𝑗−ℎ_{𝑡,𝑘,𝑗}²>𝑣_{𝑡,𝑖′,𝑘}−1𝑀+𝑟²_𝑡,𝑘,∀𝑖^′∈𝐼⁻,𝑣_{𝑡,𝑖′,𝑘}∈{0,1},∀𝑖^′∈𝐼⁻,𝑟_𝑡,𝑘≥0, (3.8) where 𝐼⁺ and 𝐼⁻ are the two sets expressed by (3.5) and (3.6), respectively.
Similarly, the rates of accuracy and support of 𝑅_𝑡 in Model 2 can be considered as follows.

Definition 3.3. The accuracy rate of a rule 𝑅_𝑡 of Model 2 is denoted as 𝐴𝑅(𝑅_𝑡) and is specified as follows.(i)If ∑_𝑘∈𝐾𝑣_{𝑡,𝑖′,𝑘}=0 belongs to class 𝑡, then 𝑉_𝑡,𝑖′=1 for all 𝑖^′; otherwise, 𝑉_𝑡,𝑖′=0, where 𝐾 represents the hypersphere set for class 𝑡.(ii)𝐴𝑅𝑅_𝑡=𝑅_𝑡−∑_{𝑖′∈class𝑡}𝑉_𝑡,𝑖′𝑅_𝑡, (3.9) where ‖𝑅_𝑡‖ represents the number of total objects covered by 𝑅_𝑡.

Definition 3.4. The support rate of a rule 𝑅_𝑡 of Model 2 is denoted as 𝑆𝑅(𝑅_𝑡), and 𝑆𝑅(𝑅_𝑡)=1.

Definition 3.5. The compactness rate of a set of rules 𝑅₁,…,𝑅_𝑔, denoted as 𝐶𝑅(𝑅₁,…,𝑅_𝑔), is expressed as follows: 𝐶𝑅𝑅₁,…,𝑅_𝑔=𝑔∑^𝑔_𝑡=1𝑈𝑆_𝑡, (3.10) where 𝑈𝑆_𝑡 means the number of hyperspheres and unions of hyperspheres for class 𝑡.A union of hyperspheres indicates that the object is covered by different hyperspheres, as shown in Figure 5. Take Figure 5 for an example, in which there are two classes. The objects of class “○” are covered by two unions of the circles (i.e., 𝑆_1,1∪𝑆_1,2∪𝑆_1,3 and 𝑆_1,4∪𝑆_1,5), and the objects of class “●” are covered by one circle (i.e., 𝑆_2,1). Therefore, 𝑈𝑆₁=2, 𝑈𝑆₂=1, and 𝐶𝑅(𝑅₁,𝑅₂)=2/3.
Moreover, Models 1 and 2 are separable nonlinear programs solvable to find an optimal solution by linearizing the quadratic terms ℎ²_{𝑡,𝑘,𝑗}. The piecewise linearization technique is discussed as follows.

Figure 5 

Classify by hypersphere method.

Proposition 3.6 (referring to Beale and Forrest [31]). Denote approximate function 𝐿(𝑓(𝑥)) as a piecewise linear function (i.e., linear convex combination) of 𝑓(𝑥), where 𝑏_𝑙, 𝑙=1,2,…,𝑞 represents the break points of 𝐿(𝑓(𝑥)). 𝐿(𝑓(𝑥)) is expressed as follows: 𝑓(𝑥)≅𝐿𝑓(𝑥)=∑^𝑞_𝑙=1𝑓𝑏_𝑙𝑤_𝑙,𝑥=^𝑞_𝑙=1𝑤_𝑙𝑏_𝑙,_𝑞𝑙=1𝑤_𝑙=1, (3.11)(3.12)(3.13) where 𝑤_𝑙≥0, and (3.13) is a special-ordered set of type 2 (SOS2) constraint (reference to Beale and Forrest [31]).
Note that the SOS2 constraint is a set of variables in which at most two variables may be nonzero. If two variables are nonzero, they must be adjacent in the set.

Notation 4. According to Proposition 3.6, let 𝑓(𝑥)=ℎ²_{𝑡,𝑘,𝑗}. 𝑓(𝑥) is linearized by the Proposition 3.6 and is expressed as 𝐿(ℎ²_{𝑡,𝑘,𝑗}).

3.2. Solution Algorithm

A proposed algorithm is also presented to seek the highest accuracy rate or the highest support rate, as described as follows.

Algorithm 3.7. Step 1. Normalize all attributes (i.e., rescale 𝑎^′_𝑖,𝑗=(𝑎_𝑖,𝑗−𝑎_𝑗)/(𝑎_𝑗−𝑎_𝑗) to be 0≤𝑎^′_𝑖,𝑗≤1).Step 2. Initialization: 𝑡=1 and 𝑘=1.Step 3. Solve Model 1 (or Model 2) to obtain the k′th hypersphere of class 𝑡. Remove the objects covered by 𝑆_𝑡,𝑘 from the dataset temporarily.Step 4. Let 𝑘=𝑘+1, and resolve Model 1 (or Model 2) until all objects in class 𝑡 are assigned to the hyperspheres of same class.Step 5. Let k = 1 and t = t + 1, and reiterate Step 3 until all classes are processed.Step 6. Check the independent hyperspheres and unions of hyperspheres 𝑆_𝑡,𝑘 in the same class t.Step 7. Calculate and record the number of independent hyperspheres and unions of hyperspheres in 𝑈𝑆_𝑡, and iterate 𝑡 until all classes are done.According to this algorithm, we can obtain the optimal rules to classify objects most efficiently. The process of the algorithm is depicted in Figure 6.

Figure 6 

Flowchart of the proposed algorithm.

3.3. Operation of a Simple Example

Consider a dataset 𝑇 in Table 1 as an example, which has object 𝑖, two attributes (𝑎_𝑖,1,𝑎_𝑖,2), and an index of classes (𝑐_𝑖) for 𝑖=1,2,…,15. The dataset 𝑇 is expressed as 𝑇={𝑥_𝑖∣(𝑎_𝑖,1,𝑎_𝑖,2;𝑐_𝑖)∀𝑖=1,2,…,15}. There are the domain values of 𝑐_𝑖∈{1,2,3}. As there are only two attributes, these 15 objects can be plotted on a two-dimensional space after normalizing them, as shown in Figure 7(a).

Table 1 

Dataset of Example 1.

(a) Normalized data for Example 1

(b) Classified by the hypersphere method for Example 1

Figure 7 

Visualization of Example 1.

This example can be solved by the proposed algorithm as follows.

Step 1. Normalize all attributes (i.e., 𝑎^′_𝑖,1=(𝑎_𝑖,1−6)/(33.5−6) and 𝑎^′_𝑖,2=(𝑎_𝑖,2−3.5)/(30−3.5)).

Step 2. Initialization: t = 1 and k = 1.

Step 3. The classification model (i.e., Model 1) is linearly formulated as follows: Maximize_𝑖∈𝐼+𝑢_{𝑡,𝑖,𝑘}subjectto^𝑚_𝑗=1𝑎^′_𝑖,j²−2𝑎^′_𝑖,jℎ_{𝑡,𝑘,𝑗}+𝐿ℎ²_{𝑡,𝑘,𝑗}≤1−𝑢_{𝑡,𝑖,𝑘}𝑀+𝑟²_𝑡,𝑘,∀𝑖∈𝐼⁺,_𝑚𝑗=1𝑎^′_𝑖′²_,𝑗−2𝑎^′_𝑖′,𝑗ℎ_{𝑡,𝑘,𝑗}+𝐿ℎ²_{𝑡,𝑘,𝑗}>𝑟²_𝑡,𝑘,∀𝑖^′∈𝐼⁻,𝑢_{𝑡,𝑖,𝑘}∈{0,1},𝑟²_𝑡,𝑘≥0, (3.14)where 𝐼⁺={𝑥₁,𝑥₂,…,𝑥₆} and 𝐼⁻={𝑥₇,𝑥₈,…,𝑥₁₅}. The optimal solution of the (ℎ_𝑡,𝑘,1,ℎ_𝑡,𝑘,2,𝑟_𝑡,1)= (0.047,0.265,0.15749) for 𝑆_1,1, where 𝑆_1,1 covers objects 1–4. We then temporarily remove these objects covered by 𝑆_1,1.

Step 4. k = k + 1: the optimal solution of the (ℎ_𝑡,𝑘,1,ℎ_𝑡,𝑘,2,𝑟_𝑡,𝑘) = (0.736,0.5,0.0138) for 𝑆_1,2, where 𝑆_1,2 covers objects 5-6. Class 1 is then done.

Step 5. As 𝑡=𝑡+1, 𝑘=1, and Steps 3 and 4 are iterated, we then, respectively, have optimal solutions for 𝑆_𝑡,𝑘 as follows. The results are shown in Figure 7(b).(i)(ℎ_2,1,1,ℎ_2,1,2,𝑟_2,1)=(0.514,0.469,0.0127), where 𝑆_2,1 covers objects 7–9.(ii)(ℎ_2,2,1,ℎ_2,2,2,𝑟_2,2)=(0.929,0.210,0.0251), where 𝑆_2,2 covers objects 10-11.(iii)(ℎ_3,1,1,ℎ_3,1,2,𝑟_3,1)=(0.583,0.188,0.0436), where 𝑆_3,1 covers objects 12–14.

Step 6. Check and calculate the unions of hypersphere 𝑆_𝑡,𝑘 for all 𝑘 in class 𝑡 (i.e., Initial 𝑡=1).

Step 7. As 𝑡=𝑡+1, mark the number of unions of class 𝑡 into 𝑈𝑆_𝑡 and iterate Step 6 until 𝑡=𝑔.

4. Numerical Examples

This study shows how the experimental results evaluate the performance, including accuracy, support, and compactness rates, and compares the proposed model with different methods using CPLEX [32]. All tests were run on a PC equipped with an Intel Pentium D 2.8 GHz CPU and 2 GMB RAM. Three datasets were tested in our experiments as follows:(i)Iris Flower dataset introduced by Sir Ronald Aylmer Fisher (1936),(ii)European barn swallow (Hirundo rustica) dataset obtained by trapping individual swallows in Stirlingshire, Scotland, between May and July 1997 [1, 3],(iii)the highly selective vagotomy (HSV) patient dataset of F. Raszeja Memorial Hospital in Poland [3, 33, 34].

4.1. Iris Flower Dataset

The Iris Flower dataset contains 150 objects. Each object is described by four attributes (i.e., sepal length, sepal width, petal length, and petal width) and is classified by one of three classes (i.e., setosa, versicolor, and virginica). By solving the proposed method, we induced six hyperspheres (i.e., 𝑆_1,1∈Class1, 𝑆_2,1,𝑆_2,2,𝑆_2,3∈Class2, and 𝑆_3,1,𝑆_3,2∈Class3). The induced classification rules are reported in Table 2. Table 2 also lists a hypersphere and two unions (i.e., 𝑆_1,1, 𝑆_2,1∪𝑆_2,2∪𝑆_2,3, and 𝑆_3,1∪𝑆_3,2) of hyperspheres with centroid points and radii.

Table 2 

Centroid points for the Iris data set by the proposed method.

Rule 𝑅₁ in Table 2 contains a hypersphere 𝑆_1,1, which implies that(i)“if (𝑎^′_𝑖,1−0.366)²+(𝑎^′_𝑖,2−0.807)²+(𝑎^′_𝑖,3−0)²+(𝑎^′_𝑖,4−0)²≤0.6205, then object 𝑥_𝑖 belongs to class 1.”

Rule 𝑅₂ in Table 2 contains a union of three hyperspheres (i.e., 𝑆_2,1∪𝑆_2,2∪𝑆_2,3) which implies that(i)“if (𝑎^′_𝑖,1−0.557)²+(𝑎^′_𝑖,2−0.32)²+(𝑎^′_𝑖,3−0.205)²+(𝑎^′_𝑖,4−0.46)²≤0.254, then object 𝑥_𝑖 belongs to class 2,” or(ii)“if (𝑎^′_𝑖,1−0.575)²+(𝑎^′_𝑖,2−0.581)²+(𝑎^′_𝑖,3−0.626)²+(𝑎^′_𝑖,4−0.515)²≤0.0612, then object 𝑥_𝑖 belongs to class 2,” or(iii)“if (𝑎^′_𝑖,1−0.423)²+(𝑎^′_𝑖,2−0.261)²+(𝑎^′_𝑖,3−0.352)²+(𝑎^′_𝑖,4−0.49)²≤0.1388, then object 𝑥_𝑖 belongs to class 2.”

Rule 𝑅₃ in Table 2 contains a union of two hyperspheres (i.e., 𝑆_3,1∪𝑆_3,2), which implies that(i)“if (𝑎^′_𝑖,1−0.248)²+(𝑎^′_𝑖,2−0)²+(𝑎^′_𝑖,3−2.226)²+(𝑎^′_𝑖,4−2.151)²≤4.8087, then object 𝑥_𝑖 belongs to class 3,” or(ii)“if (𝑎^′_𝑖,1−0.329)²+(𝑎^′_𝑖,2−0.187)²+(𝑎^′_𝑖,3−0.65)²+(𝑎^′_𝑖,4−0.613)²≤0.033, then object 𝑥_𝑖 belongs to class 3.”

Comparing the proposed method with both decision tree [3] and hyperplane methods [35] in deducing the classification rules for the Iris Flower dataset, Table 3 lists the experimental result.

Table 3 

Comparing results for the Iris flower data set (R₁,R₂,R₃).

The accuracy rates of (𝑅₁,𝑅₂,𝑅₃) in the proposed method are (1,1,1), as Model 1 has been solved. This finding indicates that none of objects in class 2 or class 3 are covered by 𝑆_1,1, none of objects in classes 1 or 3 are covered by 𝑆_2,1∪𝑆_2,2∪𝑆_2,3, and none of the objects in classes 1 or 2 are covered by 𝑆_3,1∪𝑆_3,2. The support rate of (𝑅₁,𝑅₂,𝑅₃) in the proposed method is (1,0.98,0.98), indicating that all objects in class 1 are covered by 𝑆_1,1, 98% of the objects in class 2 are covered by 𝑆_2,1, 𝑆_2,2, and 𝑆_2,3, and 98% of the objects in class 3 are covered by 𝑆_3,1 and 𝑆_3,2. The compactness rate of rules 𝑅₁, 𝑅₂, and 𝑅₃ is computed as 𝐶𝑅(𝑅₁,𝑅₂,𝑅₃)=3/3=1. Finally, we determine the following.(i)Although all three methods perform very well in the rates of accuracy and support, the proposed method has the best performance for the accuracy of classes 2 and 3 (i.e., 𝑅₂ and 𝑅₃).(ii)The proposed method has the best compactness rate.

4.2. Swallow Dataset

The European barn swallow (Hirundo rustica) dataset was obtained by trapping individual swallows in Stirlingshire, Scotland, between May and July 1997. This dataset contains 69 swallows. Each object is described by eight attributes, and it belongs to one of two classes (i.e., the birds are classified by the gender of individual birds).

Here, we also used Model 1 to induce the classification rules. Table 4 lists the optimal solutions (i.e., centroid and radius) for both rules 𝑅₁ and 𝑅₂.

Table 4 

Centroid points for the Swallow data set by the proposed method.

The result of the decision tree method, which is referred to in Li and Chen [3], is listed in Table 5, where 𝐴𝑅(𝑅₁,𝑅₂)=(0.97,1), 𝑆𝑅(𝑅₁,𝑅₂)=(0.97,1), and 𝐶𝑅=0.3.

Table 5 

Comparing results for the Swallow data set (𝑅₁,𝑅₂).

The result of the hyperplane method, referred to in Chang and Lin [35], is also listed in Table 5, where𝐴𝑅(𝑅₁,𝑅₂)=(0.97,1), 𝑆𝑅(𝑅₁,𝑅₂)=(0.97,1), and 𝐶𝑅=0.1.

We compared the three methods in Table 5 to show that the proposed method can induce rules with better or equivalent values of 𝐴𝑅 and 𝑆𝑅. In fact, the proposed method also has the best compactness rate.

4.3. HSV Dataset

The HSV dataset contains 122 patients classified into four classes, with each patient having 11 preoperating attributes. To maximize the support rate with respect to the proposed method (i.e., Model 1), the proposed method generated seven hyperspheres and three unions of hyperspheres. The centroids and radii of the hyperspheres are reported in Table 6, and a comparison with other methods is reported in Table 7.

Table 6 

Centroid points for the HSV data by the proposed method.

Table 7 

Comparing results for the HSV data set (R₁,R₂,R₃,R₄).

Using the decision tree method in the HSV dataset generates 24 rules. In addition, the hyperplane method deduces 45 hyperplanes for the HSV dataset. Table 7 also shows that the proposed method can find rules with the highest rates (i.e., 𝐴𝑅, 𝑆𝑅, and 𝐶𝑅) compared with the other two methods.

4.4. Limitation of the Proposed Method

The hypersphere models are solved by one of the most powerful mixed-integer program software CPLEX [32] running in a PC. Based on optimization technique, the results of the numerical examples illustrate that the usefulness of the proposed method is better than that of the current methods, including the decision tree method and the hyperplane support vector method. As the solving time of the hypersphere model, which is linearized, mainly depends on the number of binary variables and constraints, solving the reformulated hypersphere model from the proposed algorithm takes about one minute for each dataset (i.e., in Sections 4.1 and 4.3), in which using eight piecewise line segments linearizes the nonlinear nonconvex term (i.e., 𝐿(ℎ²_{𝑡,𝑘,𝑗})) of Model 1.

The computing time for solving a linearized hypersphere program grows rapidly as the numbers of binary variables and constraints increase. Also, the computing time of the proposed method is slower than that of the decision tree method and hyperplane method, especially for large datasets or a great number of piecewise line segments. In the further study, utilizing a mainframe-version optimization software [36–38], integrating meta-heuristic algorithms, or using distributed computing techniques can enhance solving speed to conquer this problem.

5. Conclusions and Future Work

This study proposes a novel method for deducing classification rules, which can find the optimal solution based on a hypersphere domain. The optimization technique for finding classification rules is approached to optimal. Results of the numerical examples illustrate that the usefulness of the proposed method is better than that of the current methods, including the decision tree method and the hyperplane method. The proposed method is guaranteed to find an optimal rule, but the computational complexity grows rapidly by increasing the problem size. More investigation and research are required to enhance further the computational efficiency of globally solving large-scale classification problems, such as running mainframe-version optimization software, integrating meta-heuristic algorithms, or using distributed computing techniques.

Acknowledgments

The authors wish to thank the editor and the anonymous referees for providing insightful comments and suggestions, which have helped them improving the quality of the paper. This work was supported by the National Science Council of Taiwan under Grants NSC 100-2811-E-009-040-, NSC 99-2221-E-030-005-, and NSC 100-2221-E-030-009-.