Abstract
Recent trend of research is to hybridize two and more metaheuristics algorithms to obtain superior solution in the field of optimization problems. This paper proposes a newly developed wrapper-based feature selection method based on the hybridization of Biogeography Based Optimization (BBO) and Sine Cosine Algorithm (SCA) for handling feature selection problems. The position update mechanism of SCA algorithm is introduced into the BBO algorithm to enhance the diversity among the habitats. In BBO, the mutation operator is got rid of and instead of it, a position update mechanism of SCA algorithm is applied after the migration operator, to enhance the global search ability of Basic BBO. This mechanism tends to produce the highly fit solutions in the upcoming iterations, which results in the improved diversity of habitats. The performance of this Improved BBO (IBBO) algorithm is investigated using fourteen benchmark datasets. Experimental results of IBBO are compared with eight other search algorithms. The results show that IBBO is able to outperform the other algorithms in majority of the datasets. Furthermore, the strength of IBBO is proved through various numerical experiments like statistical analysis, convergence curves, ranking methods, and test functions. The results of the simulation have revealed that IBBO has produced very competitive and promising results, compared to the other search algorithms.
1. Introduction
In recent times, there has been a growing interest in developing and utilizing metaheuristic population-based optimization algorithms to solve combinatorial optimization problems. This is mainly due to the simplicity, inexpensive computational cost, gradient-free mechanism, and flexibility of them. These optimization algorithms have been applied in various research areas as they have gained a lot of success in bringing out the best solutions. The problem of optimization grows bigger when handling large volume datasets, as there would be a large feature space with wide number of classes. These datasets cause problems to machine learning and make the task of classification difficult to solve. This is due to the presence of more than thousands and thousands of features, among which most of the features do not contribute to the classification process. As a result, choosing the discriminative features demand an extreme importance towards the construction of efficient classifiers with high predictive accuracy. To overcome this problem, one efficient way is to select a small subset of information-rich features from these large volume datasets (using an optimization algorithm) that best describes the target concept. This technique is known as feature selection (FS) and it helps in solving the data overfitting problem by getting rid of noisy features, reducing the computational load, and increasing the overall classification performance of the learning models.
Curse of dimensionality may affect the performance of pattern recognition or classification systems. Hence, FS methods are profoundly used to select parsimonious features and to remove irrelevant/redundant features without affecting the prediction/classification accuracy [1]. FS methods can be categorized into wrappers, filters, and embedded methods. Each method has its own advantages and disadvantages. Principle criteria for best attribute selection in filter-based methods are ranking techniques and it is independent of the classifier performance. Wrapper-based methods use the performance of the classifier to evaluate the quality of selected subset of features. Though wrapper-based methods are computationally expensive and their solution lacks generality, they are more popular than the filter-based methods as they generally achieve higher classification or recognition rates [2].
Many metaheuristic search algorithms have been employed for FS to search for (near) optimal subset of features from these large volume datasets, as they prove their superiority in bringing out a better performance. Some of the most popular metaheuristic algorithms are Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC) and Ant Colony Optimization (ACO), algorithms inspired by fish schools [3], Gravity search [4], different aspects of the behaviour of bees [5], Fireflies [6], Bats [7], Cuckoo birds [8], etc. Newly proposed modifications in search heuristics like chaotic maps [9], Sine Cosine Algorithm [10], Evolutionary methods [11], Local searches [12], and Biogeography Based Optimization [13] have also improved the performance of the search heuristics internally. Nowadays, hybridizing one or more metaheuristics is the latest research trend for solving high dimensionality problems. These hybridization algorithms are highly potential enough to overcome the poor exploration ability of one algorithm and poor exploitation ability of the other algorithm.
In recent times, Nature Inspired Algorithms (NIAs) have been introduced to perform efficient feature selection in various applications. For instance, Fister et al. have conducted detailed review about swarm-intelligence-based, bioinspired, physics-based and chemistry-based search algorithms [14]. Zhang et al. have developed a hybridized algorithm based upon BBO and Fireworks Algorithm (FWA). This combination has accelerated the solution diversity without harming the exploitation ability of FWA [15]. Jehad Ababneh has introduced combined scheme of greedy Particle Swarm Optimization (GPSO) and BBO to solve global optimization problems and further proved the efficiency of this algorithm by utilizing upon standard benchmark test functions [16]. In 2017, a Memetic Based BBO (MBBO) algorithm has been proposed by Wang et al., which uses the strategy of Affinity Propagation (AP) to modify the migration operation of the BBO algorithm and based on the memetic frame, with SA as the local search strategy [17].
The modified NIAs, i.e., hybridized NIAs, are becoming attractive alternatives to standard (or the basic version of) NIAs. The hybrid NIAs tend to perform efficiently using the multiple search algorithms (which are present within them) and this substantiates the strength of the hybrid NIAs. Generally, hybrid NIAs can outperform their basic algorithms as these algorithms interact very well among them. This has led to a strong motivation for the study of hybrid NIAs. Researchers are continuously developing more promising and refined NIAs by acquiring the different search techniques within one specific optimization framework. However, these NIAs need to be explored and fine-tuned within their basic algorithm, in order to increase their exploration and exploitation ability during the optimization. This emphasizes that there are many criteria lying within these NIAs that have to be determined and investigated yet.
Firstly, the method of combining or hybridizing one or more NIAs has to be done, i.e., determining the hybridization technique. Secondly, method of finalising the number of NIAs that has to be infused within the search procedure has to be done. Thirdly, method of determining the application area upon which the NIAs have to be dealt with has to be done. Finally, method of applying these hybridized search algorithms into the specific application has to be done [18]. Thus, the objective of this paper is to address the first, second, and third areas, i.e., to investigate and highlight the optimization performance achieved through the hybridization of BBO and SCA algorithm in solving feature selection problems. Being inspired by this ideology, in this paper, an Improved BBO (IBBO) has been proposed to perform the task of pattern classification. The organization of this paper is as follows: Section 2 discusses the existing literature in the related field. Section 3 details about the feature selection approaches and working of the proposed IBBO algorithm. Section 4 presents the experimental setup and Section 5 discusses the experimental results and their discussion. Section 6 concludes this research and its future works.
2. Literature Reviews
2.1. Review of Previous Works on SCA Algorithm
Sine Cosine Algorithm (SCA) algorithm is a wrapper-based FS algorithm. It makes use of the mathematical formulations of sine and cosine functions. It transits from exploration to exploitation using adaptive range in the sine and cosine functions. This algorithm helps the search procedure to avoid the local optima, which further accelerates the search ability in a fast manner [10]. It has strong exploration ability than the exploitation ability, when compared with any other NIAs [13, 19–21]. Recently, much advancement has been made within SCA algorithm and with the combination of several other search algorithms to solve various global optimization and engineering problems. For instance, in 2017, a newly hybridized FS method has been developed and tested upon 8 UCI datasets. This hybrid approach utilized the Differential Evolution (DE) operators as local search method to improve the SCA. The experimental results of the proposed approach were compared with three well-known algorithms (SCA, SSO, and ABC) and they proved to be highly noteworthy [22].
A new optimization approach called Sine Cosine Crow Search Algorithm (SCCSA) has been utilized and this combination of concept and operators of the two individual algorithms (of CSA and SCA) has facilitated to make an appropriate trade-off between exploration and exploitation abilities of this efficient SCCSA algorithm [23]. A newly introduced method called Opposition Based SCA (OBSCA) has improved the SCA algorithm through the opposition based learning approach. This method has solved optimization problems like pressure vessel design, tension/compression spring design, and welded beam design [24]. A newly developed hybrid self-adaptive SCA algorithm with opposition based learning has been proposed by Gupta and Deep. This modified SCA improves its search procedure using self-adaptation and opposition based learning approach [25].
2.2. Review of Previous Works on BBO Algorithm
Biogeography Based Optimization (BBO) was developed based on biogeography theory and it mimics relationships between different species located in different habitats in terms of migration and mutation. Migration and mutation are the two important operators of BBO, in which migration is responsible for sharing the information within the population. The solution quality highly depends upon this migration operator. Mutation operator is responsible for maintaining diversity among the population. In BBO algorithm, there have been lots of advancements done by improving the existing operators and by merging new operators. Some advancements in this BBO algorithm have been proposed in many of the previous researches like modified migration [26–28], modified mutation [29–31], and introducing new operators [32].
For instance, in 2010, Gong et al. have applied perturbation in the form of various mutation operators, namely, Gaussian mutation, Cauchy mutation, and Levy mutation [29]. Lohakare et al. have proposed a memetic algorithm named as accelerated BBO embedded with a modified Differential Evolution as a neighborhood search operator (aBBOmDE). In this study, the convergence speed is improved by modifying mutation operator and exploitation is maintained by keeping original migration [30]. In some of the previous studies like [32, 33], BBO has proved itself as a good search algorithm and it has effectively solved complex problems with multiple targets and constraints. Thus the Basic BBO has been applied in handling several optimization problems.
2.3. Review of Hybridization of BBO with Other Search Algorithms
Hybridization of algorithms to improve the quality of solution is a new insight of research. Hybridizing one or more search algorithms not only increases their search ability, but also helps to improve their optimization performance to certain degree and resolve more combinatorial problems. In this regard, BBO has been hybridized with various search algorithms like PSO, GWO, FWA, etc. to prove their excellence in solving many global optimization problems. For example, Ling-Ling et al. have hybridized BBO with population competition algorithm (BBOPC) method in the field of location planning of electrical substations. Upon implementing a modified BBO variation operator, this hybrid BBOPC method has proved to attain a better convergence characteristics and robustness compared to the other approaches [34].
Pushpa Farswan and Jagdish Chand Bansal have proposed a hybrid algorithm obtained by incorporating fireworks explosion concept of Fireworks Algorithm (FWA) into Biogeography Based Optimization, which is named as Fireworks inspired Biogeography Based Optimization (FBBO). The key feature of this algorithm is the hybridization of two different searching skills of (FWA and BBO) algorithms to improve solution quality. FBBO provides a better balance between solution diversification and intensification. The FBBO method uses migration and mutation operator of BBO algorithm and explosion operator of Fireworks Algorithm (FWA) and its efficiency has been proved in solving CEC benchmark problems successfully [18]. Apart from these previous works in the literature, BBO has been employed in solving many optimization based engineering applications like aircraft maintenance sensor selection [35], Yuga-Uda’s antenna design [36], parameter estimation of chaotic systems [37], optimal operation of Reservoir systems [38], and solving nonproductive time during hole-making process [39]. Table 1 tabulates about the recent research studies conducted through the hybridization of BBO with various optimization algorithms.
Hence, based on the inferences from the recent literature, it can be concluded that most of the researchers have combined the Basic BBO algorithm along with various search algorithms to perform efficient feature selection and utilized these algorithms upon various benchmark test functions to prove their efficiency. Furthermore, it is evident that these modifications have been done to prove its strength during the process of optimization. However, still, there is a room for improving this algorithm by combining one or more NIAs into it or by modifying the existing operators or by infusing new operators within this algorithm. Being inspired by the previous works, in this article, SCA is hybridized with BBO to choose the information-rich discriminative features that would achieve higher classification accuracy. The inclusion of position update mechanism (instead of mutation operation) in the proposed IBBO approach has enhanced the exploration of unvisited regions of the search space. Local optima stagnation is efficiently avoided in this IBBO approach. The classification results have revealed that this approach has produced better classification performance than the other approaches in majority of the datasets.
3. Optimization Based Feature Selection
Metaheuristic optimization based FS methods are becoming more popular among the researchers to select the most significant attributes/features [21, 52]. By mimicking biological or physical phenomena, nature-inspired metaheuristics optimization algorithms solve many real-world problems [13, 16, 20, 36, 52–54]. Feature selection using optimization techniques (prior to classification) will reduce the curse of dimensionality problem, since higher dimensional feature set can cause data overfitting and the performance of learning/classification method may degrade. Further, it reduces the data acquisition process, increases the comprehensibility of classification method, and maximises the recognition rates. Rather than applying a single FS algorithm for the optimization purpose, hybridizing one or more FS algorithms will ease the optimization process at faster rate. Such hybridization of NIAs can be implemented to perform optimization based feature selection either at the algorithm level or at iteration level [55].
Studies from the literature reveal that Basic BBO algorithm is good at exploiting current population information with its migration operator, but it is slow in exploring the global search space [19]. To intensify the exploration ability of the BBO algorithm, a combined scheme of BBO and SCA is developed to select the most relevant feature set from original features. In order to enhance its exploring capacity furthermore, the mutation operation is got rid of, and it is replaced by the position updating mechanism of SCA algorithm, which is applied to the worst half of the population. In this way, the proposed IBBO enhances the Basic BBO and helps in increasing the diversity among the population.
3.1. SCA Algorithm
The Basic SCA algorithm initializes the optimization process with a random set of search agents (individuals). The quality (fitness) of the search agents is evaluated using a fitness function. After the fitness evaluation, the algorithm then saves the best search agents obtained so far, assigns it as the destination point, and updates other solutions concerning it. Eventually, SCA reaches the expected solution by undergoing several numbers of iterations. Finally, the algorithm stops itself on satisfying the termination criteria. The pseudocode of this Basic SCA approach is explained in Algorithm 1.
| ||||||||||||||||
The ranges of the sine and cosine functions will be updated as the number of iterations increases and this position update is defined for both the phases using the following two equations:The above two equations are combined to be used as the following equation: According to (3), there are four main parameters in SCA: r1, r2, r3, and r4 for determining the position of the new solution. The parameter r1 dictates the next position’s region (or movement direction) which could be either in the space between the solution and destination or outside it. The parameter r2 describes how far the movement should be towards or outwards the destination. The parameter r3 brings a random weight for the destination in order to stochastically emphasize () or deemphasize () the effect of destination in defining the distance. Finally, the parameter r4 equally switches between the sine and cosine components in (3). The value of r4 will be a random number between 0,1. In order to balance the exploration and exploitation phases, the range of sine and cosine in (1) to (3) is changed adaptively using the following equation:where t is the current iteration, T is the maximum number of iterations, and a is a constant and its value is 2. As mentioned earlier, the initial population produces a better solution after the fitness evaluation and its position will be updated. The periodic behaviour of the sine and cosine functions facilitates the solution to be generated around the other and this feature ensures better exploitation capability. On the other hand, if the new solution is generated outside of the space between the current solution and the best solution that is obtained so far, then the global search capability is generated during optimization.
3.2. BBO Algorithm
The Biogeography Based Optimization (BBO) algorithm was developed based on biogeography theory, and it mimics relationships between different species located in different habitats regarding migration and mutation [56]. In BBO, the fitness value of habitat is called Habitat Suitability Index (HSI), and the factors contributing to each solution are called Suitability Index Variable (SIV). Habitats which are more favourable and suitable for species to reside are said to have high HSI. Similarly, habitats which are less suitable for species to reside are said to have low HSI. In this way, high HSI habitats house a relatively larger number of species. The characterization of habitability is called Suitability Index Variables. Rainfall, vegetation, temperature, etc. are various examples of SIVs. These variables decide or characterize the fitness or HSI of a solution. In BBO model, two parameters, immigration rate (λ) and emigration rate (µ), govern the migration of species from one habitat to another habitat. The migration operator is the same as the crossover operator of the evolutionary algorithm and is responsible for sharing the features among candidate solutions for modifying fitness. Each solution depends on the calculation of immigration rate (λk) and emigration rate (μk), which are the major elements of HSI and they can be manipulated using the following equations:where I is the maximum possible immigration rate; E is the maximum possible emigration rate; i is the number of species of the ith island; and n is the maximum habitat number of each island. The final HSI is recalculated based on the probability of migration and mutation. The pseudocode of the BBO algorithm is given in Algorithm 2.
| ||||||||||||||||||||||
3.3. Improved BBO Algorithm
The working of the proposed IBBO is described in Algorithm 3. To begin with, an initial set of habitats is generated. After the random initialization, the initial set of habitats is subjected to fitness evaluation through its objective function (using (6)). A minimization fitness function has been used for a fitness evaluation, which means that if the fitness value is low, then the quality of the solutions will be high. The resulting population obtained after fitness evaluation is subjected to undergo migration and mutation. Migration and the mutation procedures make it possible to evolve newly generated solutions. This procedure of governing the habitats to the migration procedure, followed by the mutation procedure, is continued to the next generation until the termination criteria are satisfied.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Migration operator is used for information sharing within the candidates using their migration rates (immigration rate, λi, and emigration rate, µi). In the migration procedure, immigrating habitat is selected according to the probability of immigration rate and emigrating habitat is selected according to the probability of emigration rate of habitats. After undergoing migration, the habitats are subjected to mutation. Mutation operator is responsible for maintaining the diversity of the population. Generally, in Basic BBO, mutation is performed on the worst half of the habitats which have low HSI solutions. Usually, the mutation rates are fixed using species count probabilities. In the proposed IBBO method, mutation is replaced by the position updating mechanism of the SCA algorithm, which is applied to the worst half of the population. The habitats which are obtained after this position update mechanism are subjected to undergo several iterations, to achieve the desired solution. Algorithm 3 shows the working of the IBBO approach. In this algorithm, N represents the number of habitats and Island represents the population resulting after the migration operation. The migration operation (Step (10)–(17)) is the Basic BBO which helps in modifying the habitats within the population. The proposed method (Step (18)–(29)) is carried out after the basic migration operation. The updated SCA equation is defined in Step (20). Finally, the algorithm stops by itself on satisfying the termination criteria (which is the maximum number of iterations).
4. Materials and Methods
4.1. Datasets for Experimentation
To perform the pattern classification, 14 benchmark datasets are taken from the UCI machine learning repository and the website: www.gems-sys tem.org. They are 14_Tumors, 11_Tumors, 9_Tumors, Prostrate_Tumors, Leukemia2, Sonar, LSVT, Land Cover, Heart, Lymphography, Meter A, Meter B, Meter C, and Meter D datasets, respectively. Some of these datasets have a small number of samples and a large number of features. These datasets vary from each other regarding feature dimension, a number of instances, and number of classes as well. The number of dimension in the optimization is equal to the original feature count of the dataset. The underlying reason for choosing these large volume data sets is that these contain a large number of features and instances, which represent a variety of issues on which the proposed IBBO can be tested. The proposed IBBO method is treated upon 14 datasets to validate its performance and applicability. Table 2 depicts the details of the selected datasets such as a number of features and instances in every dataset.
4.2. Experimental Setup
For experimentation, the IBBO method is tested with 14 benchmark datasets. These datasets were of different size (low and high-dimensional) and a number of classes. ELM kernel was used as a learning algorithm in this work. The experimental results are compared with 8 other optimization algorithms like Basic Sine Cosine Algorithm (SCA), Biogeography Based Optimization Algorithm (Basic BBO), Salp Swarm Algorithm (SSA), Grasshopper Optimization Algorithm (GOA), Whale Optimization Algorithm (WOA) [57], Grey Wolf Optimization (GWO) [40], Moth Flame Optimization (MFO) [58], and Particle Swarm Optimization (PSO) [59], thus proving the strength of the proposed method. Furthermore, the efficiency of this proposed method is evaluated through statistical analysis (of Wilcoxon Rank-sum test). Numerical experiments like convergence curves, ranking methods, and test functions demonstrate the effectiveness and potency of the proposed IBBO method.
For all algorithms, the population size is set to 20 search agents and the number of iterations is set to 100, which is the termination criteria. The population size has been fixed through experimental studies. The value of this parameter has been tested and adjusted (i.e., by increasing or decreasing) according to the performance and the population size has been set to 20 [40, 60]. In the RBF kernel of ELM classifier, regularization coefficient and kernel parameter were varied respectively. The parameters of the ELM classifier are fixed through empirical methods. A common fitness function (of minimization function) was used for fitness evaluation in all the search algorithms. As mentioned earlier, during the manipulation of position update in SCA algorithm, the value of r1 is fixed using (4). The value of r1 linearly decreases from 2 to 0. The random location either inside or outside is achieved by defining a random number for r2 in (0, 2π) in (3). The value of r3 and r4 will be a random number between (0, 1).
The fitness function is given below aswhere is equalization factor. On combining error rate and a number of features into a single fitness function, represents the relative importance of the number of features and () represents the relative importance of the error rate. Through empirical methods, is set to be 0.9, as its range is good beyond this value. All the algorithms are run for 20 independent runs. After every run, the best feature subset was given to the ELM classifier, and the classifier experiment (10-fold cross validation) was run for 10 times. As a result, four performance measures, namely, classification accuracy, number of selected features, fitness values, and reduction rate of features, were recorded. Further, each algorithm was implemented using MATLAB 2013 and was run on Intel core i5 machine, 2.8 GHz, and 8GB of RAM. Table 3 gives a brief description of the specification and parameters of the three optimization algorithms.
5. Results
In this section, a comparative study is shown carefully to examine the exploratory and exploitative behaviour of the proposed IBBO algorithm. To intuitively verify the efficiency of this IBBO algorithm and its optimization performance, the results have been compared with eight other optimization algorithms and analysed in detail. A set of seven benchmark test functions are used to evaluate the proposed system, and the results are tabulated in Tables 4 and 5. The classification results are investigated in detail through comparative analysis, graphical illustrations, statistical tests, test functions, and ranking methods as well. The comparison of the experimental results of the IBBO algorithm and other algorithms is presented regarding average classification accuracy, an average fitness value, the average number of selected features, and reduction rate of the features. Tables 4, 5, and 6 show the comparison of these experimental results. Figures 1–4 depict the comparative analysis of IBBO with other search algorithms regarding convergence curves. To make a reliable conclusion and to show the superiority of the proposed hybrid algorithm over state-of-the-art algorithms, a nonparametric statistical test called Wilcoxon rank-sum test is also conducted. The statistical results produced for p-values, h-values, and z-values by this test for the pairwise comparison of eight groups are given in Tables 7 and 8. Table 11 presents the comparative study of overall ranking results of the IBBO approach.

(a) 14_Tumors dataset

(b) 11_Tumors dataset

(c) 9_Tumors dataset

(d) Prostrate_Tumors dataset

(a) Leukemia2 dataset

(b) Sonar dataset

(c) LSVT dataset

(d) Land Cover dataset

(a) Heart dataset

(b) Lymphography dataset

(c) Meter A dataset

(d) Meter B dataset

(a) Meter C dataset

(b) Meter D dataset
5.1. Discussion
5.1.1. Benchmark Functions
In order to verify the optimization performance of the IBBO, seven benchmark functions are used for testing. For this purpose, seven test functions are employed to evaluate the solution quality, convergence speed and the ability of the proposed IBBO algorithm. This problem contains three unimodal functions (which are named as F1, F2, and F3) and four multimodal functions (which are named as F4, F5, F6, and F7). Among these test functions, six functions are scalable (F1, F2, F3, F4, F5, and F6) and one function is nonscalable (F7). Table 4 gives a detailed description about the benchmark functions utilized for the experimentation purpose.
The results presented in Table 5 clearly explain about the enhanced performance of the IBBO method when compared with six algorithms like Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Differential Evolution (DE), Artificial Bee Colony Optimization (ABC), Basic SCA, and Basic BBO. In functions F1, F2, and F3, only one optimum point is present, and these problems are known as unimodal test problems. These unimodal test problems evaluate the strength of the exploitation or local search. In functions F4-F7, more than one optimum point is present. These multimodal test problems are used to assess the exploration or global search and local optima avoidance ability of metaheuristic optimization algorithm. Inferences from Table 5 unfolds that the proposed IBBO algorithm performs much better than other search algorithms and reaches optimal solutions for all the seven test functions. The small standard deviation values for the IBBO algorithm seems to be robust as it attains optimal solutions for all runs when compared to the other six algorithms.
5.1.2. Investigation of Classification Results
The experimental results obtained through the feature classification carried out by IBBO have been explained in Tables 6 and 7. In this section, the efficiency, convergence, and the quality of the proposed approach are deeply analysed and compared with the other optimizing algorithms to distinguish the individual performance. For instance, in 14_Tumors datasets, 81.01% of best average classification accuracy has been achieved through the IBBO approach with a minimum of 2633 features (out of 15, 099 original features). This accuracy has been yielded with best average fitness of 0.17 along with the best reduction rate of 82.45%. This reduction rate interprets that the maximum utility of the original feature count has been reduced to 82.45%; i.e., the remaining 17.55% of the features are sufficient enough to classify the data.
For the 11_Tumors dataset, the best average classification accuracy of 99.02% has been obtained by IBBO using a minimum of 822.20 features with a reduction rate of 93.44% and best average fitness of 0.01. For the 9_Tumors dataset, an average classification of 86.83% is attained by IBBO, with a best reduction rate of 87.18% and best fitness of 0.11 along with minimum feature count of 733 features. For the Prostrate_Tumors dataset, the best average classification accuracy of 98.73% has been achieved using a minimum of 502 features with an average fitness value of 0.01. For the Leukemia2 dataset, the highest classification accuracy of 100% is reached through IBBO with 204 discriminative features and an average reduction rate of 98.18%. For the Sonar dataset, the best average accuracy of 92.60% is yielded through IBBO using an average of 16 features with the best fitness value of 0.06. For the LSVT dataset, an average accuracy of 94% is achieved by IBBO with a minimum of 38 features and best reduction rate of 87.70%.
For the Land Cover dataset, an average accuracy of 87.79% has been obtained by IBBO through 30 features with best average fitness value of 0.12 and feature reduction of 79.25%. For the Heart dataset, the best average classification accuracy of 84.41% is attained by IBBO through 4 features with the best average fitness values of 0.16 and feature reduction of 70.77%. For the Lymphography dataset, the best average accuracy of 85% is achieved by IBBO with best fitness value of 0.16 and data reduction rate of 51.67%. For the Meter A dataset, an average accuracy of roughly 99% is reached by IBBO using 4 features with best average fitness of 0.01 and a reduction rate of 88%. For the Meter B dataset, an average accuracy of 99.8% is yielded by IBBO using 2 features with feature reduction of roughly 95.77%. For the Meter C dataset, 90.99% of best accuracy is obtained by IBBO using 16 features with best average fitness value of 0.09 and feature reduction rate of 61.82%. For the Meter D dataset, the best average accuracy of 92% was attained by IBBO with best fitness value of 0.08 using 10 features and a reduction rate of 78%.
On the whole, the classification results unfold that IBBO approach has outperformed all the other algorithms in providing best average classification accuracy, number of features in the majority of the datasets. Alongside achieving best accuracy, it has strived to attain the best optimal feature count and best average fitness values, which shows its fast convergence speed and strong global search ability. Regarding the reduction rate, the IBBO approach has attempted to achieve higher reduction rates by removing the redundant, irrelevant, and noisy features for most of the datasets. These higher reduction rates have contributed towards attaining the higher classification accuracies as well. Based on the classification results, the proposed method proves to be the best algorithm among other search algorithms.
5.1.3. Convergence Behaviour of the Feature Selection Methods
The performance of the IBBO is measured by observing the convergence behaviour, and it is displayed in Figures 1–4. These graphs portray the convergence behaviour (of all the 9 search algorithms) during the search procedure throughout the 100 iterations and their corresponding fitness values. As mentioned earlier, a minimization function has been used for fitness evaluation and this fitness value must be of low range during the search procedure. This interprets that if the fitness is low, the quality of the candidate solutions will be high. Figure 1 plots the convergence graphs for 14_Tumors, 11_Tumors, 9_Tumors, and Prostrate_Tumors datasets. Figure 2 plots the convergence graphs of Leukemia2, Sonar, LSVT, and Land Cover datasets. Figure 3 plots the convergence graphs of Heart, Lymphography, Meter A, and Meter B datasets. Figure 4 plots the convergence graphs for Meter C and Meter D datasets.
It can be seen by observing these figures that IBBO initiates the convergence starting from the earlier generations for each and every dataset and finally, strives to achieve the global optima for most of the datasets. This convergence behaviour shows that the inclusion of the position update mechanism of the SCA algorithm of the IBBO approach promises to explore more search regions starting from the initial generations. Hence, it can be decided that the presence of many local optima in a multimodal problem may be responsible for stagnating in the local optima. However, the random jumps in BBO algorithm ensure to visit the unexplored regions of the search space and try to avoid the local optima. This concept is highly instrumental in reaching the global optima with very high rate of convergence, especially due to the combined scheme of BBO and SCA algorithm.
5.1.4. Statistical Analysis
The potency of the proposed IBBO algorithm is verified through a nonparametric test called the Wilcoxon rank-sum test with a significance level of 5%. The results produced by this test for the pairwise comparison of eight groups are given in Tables 8 and 9. Observations from these tables show that experimental results obtained are statistically significant for most of the datasets. However, for some datasets, the results seem to be statistically insignificant during the search procedure using other search algorithms. This uncovers the fact that the performance of the proposed method is highly commendable when compared with other 8 search algorithms. So, it can be concluded that the overall statistical results were highly significant from the results of other state-of-the-art algorithms on the majority of the datasets.
5.1.5. Ranking Methods
Tables 10 and 11 reveal the detailed representation of ranking of all the 9 search algorithms and their comparative analysis. In this section, the ranking has been conducted based on the average accuracy (AA), average fitness (AF), average selected (AS), and reduction rate (RR). While discussing the performance measures, the proposed IBBO method has obtained the best values in all these measures for most of the datasets. Based on the final ranks, it can be noticed that IBBO has shown a noteworthy performance with an overall rank of 107. This reveals the fact that IBBO has surpassed all the other algorithms (like Basic SCA, Basic BBO, and GWO) in obtaining best ranking results. The reason underlying behind this is that IBBO has an advanced potential to retain and recover a stable balance between the exploration and exploitation inclinations. This fact has enabled the IBBO to achieve speed convergence, avoid local optima, and discover better results.
5.1.6. Analysis of Computational Complexity
The time complexity of the two major algorithms (SCA and BBO) has been discussed in this section. The overall time complexity of the proposed IBBO algorithm is manipulated through the combined equations of both SCA and BBO [61]. The time complexity in performing sine and cosine operations in SCA algorithm is O(M N d) where M is the number of generations, N is the number of search agents, and d is dimension (or number of features) of the dataset. The time complexity of BBO algorithm is O(N2 d M + N M O(f)) where N is the number of habitats in the island (or search agents), M is the number of iterations, O(f) is the is the time complexity for computing the fitness function, and f and d is the dimension (or number of features) of the dataset [50].
Generally, in Basic BBO, mutation is performed on the worst half of the habitats which have low HSI solutions. Usually, the mutation rates are fixed using species count probabilities. In the proposed IBBO method, mutation is replaced by the position updating mechanism of the SCA algorithm, which is applied to the worst half of the population. The habitats which are obtained after this position update mechanism are subjected to undergo several iterations, to achieve the desired solution. So, the time complexity of sine cosine operation for the proposed IBBO algorithm is O(M N/2 d), in which the number of search agents (N) is replaced with N/2. Hence, the overall time complexity of the proposed IBBO algorithm is O(N + M N/2 d + N2 d M + N M O(f)).
6. Conclusion and Future Works
Generally, hybrid algorithms are highly efficient from the basic versions, as they benefit from all the advantages of their basic algorithms. In our research work, we have chosen BBO algorithm combined with SCA for pattern classification of benchmark datasets. BBO is highly competitive optimization algorithm, which has various merits and demerits. The merits of BBO algorithm are as follows:(1)BBO is an efficient algorithm for optimization and it does not take unnecessary computational time.(2)It is good in exploring the solutions.(3)In BBO algorithm, the solutions do not die at the end of each generation like other optimization algorithms.
However, BBO has certain demerits as well, which are given below:(1)BBO is poor in exploiting the solutions.(2)There is no provision for selecting the best members from each generation.(3)A habitat does not consider its resultant fitness while immigrating the features, as a result so many infeasible solutions are generated.
These demerits have paved the way for some modifications and advancements that can be made within the BBO algorithm or by hybridizing one or more search algorithms along with BBO [62]. In this regard, we have hybridized SCA along with BBO algorithm, to accelerate the performance of the Basic BBO algorithm. As SCA is good in exploitation, this ability is utilized in the proposed IBBO algorithm. This IBBO algorithm does not affect the configuration of the search procedure of Basic BBO, instead, it improves the BBO algorithm using SCA algorithm. The salient features of the proposed IBBO algorithm are as follows:(1)The inclusion of position update mechanism (instead of mutation operation) in the proposed IBBO approach has enhanced the exploration of unvisited regions of the search space.(2)Fourteen well-known datasets are taken from the UCI repository to evaluate the potency of the IBBO approach.(3)The experimental results have shown that the combined scheme of BBO and SCA has significantly enhanced the performance of BBO in terms of classification accuracy, fitness values, number of selected features, and reduction rate of features. The overall classification performance is considerably better than the results reported in the previous works.(4)The potency and superior performance of IBBO are further exhibited through convergence graphs, statistical results, ranking methods, and test functions.
For future direction of research, the proposed IBBO method can be applied to various other public datasets and to solve real-world problems. Finally, IBBO can be applied as a preprocessing step of many pattern recognition, machine learning, and feature selection tasks as well. IBBO can also be customized for constrained and multiobjective optimization problems to extend the current approach.
Data Availability
We have used the public datasets taken from UCI machine learning database. This link is here: https://archive.ics.uci.edu/ml/datasets.html.
Conflicts of Interest
The authors of the paper declare that they have no conflicts of interest.
Authors’ Contributions
All authors have contributed equally in all the areas such as implementation, paper writing, and experimentations.