Investigation of Effectiveness of Shuffled Frog-Leaping Optimizer in Training a Convolution Neural Network

Baseri Saadi, Soroush; Tataei Sarshar, Nazanin; Sadeghi, Soroush; Ranjbarzadeh, Ramin; Kooshki Forooshani, Mersedeh; Bendechache, Malika

doi:https://doi.org/10.1155/2022/4703682

Journal of Healthcare Engineering

On this page

Abstract Introduction Experimental Results and Discussion Conclusion Data Availability Disclosure Conflicts of Interest References Copyright Related Articles

Special Issue

Complexity Systems for Scheduling in Healthcare

View this Special Issue

Research Article | Open Access

Volume 2022 | Article ID 4703682 | https://doi.org/10.1155/2022/4703682

Investigation of Effectiveness of Shuffled Frog-Leaping Optimizer in Training a Convolution Neural Network

Soroush Baseri Saadi,¹Nazanin Tataei Sarshar,²Soroush Sadeghi,³Ramin Ranjbarzadeh,⁴Mersedeh Kooshki Forooshani,⁵and Malika Bendechache⁴

Academic Editor: Agostino Forestiero

Received29 Dec 2021

Revised04 Feb 2022

Accepted07 Mar 2022

Published23 Mar 2022

Abstract

One of the leading algorithms and architectures in deep learning is Convolution Neural Network (CNN). It represents a unique method for image processing, object detection, and classification. CNN has shown to be an efficient approach in the machine learning and computer vision fields. CNN is composed of several filters accompanied by nonlinear functions and pooling layers. It enforces limitations on the weights and interconnections of the neural network to create a good structure for processing spatial and temporal distributed data. A CNN can restrain the numbering of free parameters of the network through its weight-sharing property. However, the training of CNNs is a challenging approach. Some optimization techniques have been recently employed to optimize CNN’s weight and biases such as Ant Colony Optimization, Genetic, Harmony Search, and Simulated Annealing. This paper employs the well-known nature-inspired algorithm called Shuffled Frog-Leaping Algorithm (SFLA) for training a classical CNN structure (LeNet-5), which has not been experienced before. The training method is investigated by employing four different datasets. To verify the study, the results are compared with some of the most famous evolutionary trainers: Whale Optimization Algorithm (WO), Bacteria Swarm Foraging Optimization (BFSO), and Ant Colony Optimization (ACO). The outcomes demonstrate that the SFL technique considerably improves the performance of the original LeNet-5 although using this algorithm slightly increases the training computation time. The results also demonstrate that the suggested algorithm presents high accuracy in classification and approximation in its mechanism.

1. Introduction

Currently, Deep Learning (DL) is the base for many cutting-edge artificial intelligence (AI) applications [1–3]. DL can learn features at a high-level state with more complexity and abstraction than shallower neural networks. It presents hierarchical features providing various methods, taking for instance, probabilistic models, and supervised and unsupervised methods [4, 5]. The most notorious feature of DL is its ability in reducing computer hardware and software manipulation, making advancements in computational capabilities, machine learning, and signal processing. Furthermore, it is proved to be a highly applicable solution in objects recognition [6–8], speech recognition [9–12], SAR image processing [13–16], and a highly viable method in medical image processing for the detection of potential drug molecules activities [17, 18], liver and lung tumor segmentation [19, 20].

DL principle can be employed for the design of a variety of neural networks, among which Deep Neural Network (DNN), Recurrent Neural Network (RNN), and Convolution Neural Network (CNN) are the most popular. There are also generative and hybrid models of DL. For the former, some examples are Deep Belief Network (DBN) and Boltzmann Machine (DBM), and for the latter referring to a combination of discriminative and generative models, a well-known example is pretrained deep CNN using DBN. Between the various models of DL, the focus of this study is on CNN [4, 21].

DL has a great ability for resolving learning problems. However, this method is challenging to be trained for producing the optimal results. A CNN model learns many patterns through many weights and biases inside the convolutional layers. These weights and biases obtain their best possible values through a learning process with a large number of data. Actually, in a CNN model, the number of the training samples plays a crucial role in the obtaining best possible solution [22–24]. To achieve this goal, many optimization methods have been proposed for manipulating the value of these weights and biases. The most well-known algorithms are Adaptive Gradient methods. Basically, these methods modify the learning rate by a backpropagation strategy. These approaches reduce the learning rate if the gradient of parameters is large or vice versa. Stochastic gradient descent (SGD) is the most preferred technique among adaptive gradient methods [25]. Nevertheless, they proved to have poor performance especially when the network is large like CNN since the learning rate needs to be manually tuned in SGD. This significantly increases the training time for large-scale datasets [26, 27]. To overcome this obstacle and improve the efficiency of adaptive, new variants of adaptive gradient methods are proposed such as Nostalgic Adam [28], which place bigger weights on the past gradient compared to the recent gradient, or YOGI [29], which increases in effective learning rate to achieve better convergence. However, they have not gained popularity for image processing applications with CNN, which gives rise to the use of Metaheuristic algorithms as alternatives.

Recently, metaheuristic methods have been employed to resolve complex problems such as scheduling [30] and detection problems [31]. The performance of metaheuristic algorithms in solving efficiently these problems makes them a good alternative to train neural networks with large parameters as they are simple to use, are independent form gradient, and avoid local optima [32, 33]. Metaheuristic algorithms could prove to be highly efficient in optimizing CNN parameters with large image datasets specifically in the field of image analysis. In a study carried out by Zhang et al. [34], a metaheuristic optimizer is employed to pretrain the CNN for the classification of skin cancer images. To achieve this goal, the Whale Optimization Algorithm (WOA), one of the subgroups of metaheuristic methods, was applied to reduce the error rate during the learning process. Their WOA method considers half value precision as the cost function during skin cancer validation steps that contains the simplified measured error between the output of the system and the reference. The results of this study demonstrated the accuracy prominence of this algorithm compared to the other popular classification methods used in this study. In another study performed by da Silva et al. [35], the hyperparameters of a convolutional neural network were trained with Particle Swarm Optimization (PSO), another subgroup of metaheuristic algorithm, for the classification of lung cancers (classify nodule candidates, benign or malignant tumors, into nonnodules and nodules). They used two preprocessing steps including (1) employing each CT slice as a separate sample and (2) resizing of all the samples into 28 × 28. The PSO method optimizes the size of trainable filters, number of batches in the training, type of pooling, number of neurons in the hidden layer, and number of kernels in the convolutional layers. Although, in this study, a large dataset of CT images was used, PSO demonstrated the accuracy of 97.62%, sensitivity of 92.20%, and specificity of 98.64%. Hoseini et al. [36] proposed an AdaptAhead optimization technique to learn a Deep CNN with robust architecture in relation to the high volume data. They utilized several MR images of BRATS 2015 and BRATS 2016 data sets to validate the proposed method. Their model fails to utilize the technique of Nesterov and the adaptive learning rate in computing the gradient leading to failure to reach the optimal convergence point.

Metaheuristic algorithms can be categorized into the following subgroups [37–39]:(i)Swarm based methods act based on animal social behavior like PSO [40, 41], WOA [42](ii)Evolutionary methods act based on a natural evolutionary process like Genetic algorithm (GA) [43](iii)Biological based optimizers like Satin Bowerbird Optimizer (SBO) [44](iv)Human-based algorithms inspired from human behavior such as Life Choice Based Optimizer [45](v)System-based algorithms inspired by natural ecosystems such as Artificial Ecosystem-based Optimizer (AEO) [46](vi)Physics-based methods that mimic the physical phenomenon in nature like Equilibrium optimizer [47] and Simulated Annealing [48]

In this paper, to overcome the problem of overfitting and convergence, we optimize the values of weight and biases of a LeNet-5 by an optimization algorithm. From the various types of metaheuristic algorithms, we used the Shuffled frog-leaping algorithm (SFLA) to optimize the performance of the LeNet-5 CNN [49]. This is conducted by changing the values of weight and biases inside the model to reach a high accuracy of the model. This optimizer belongs to the Swarm-based algorithms of metaheuristic algorithms inspired by the natural behavior of frogs in searching for food. The reason for using this optimizer is that, at the time of working on this research work, based on our knowledge, there is no research to employ SFLA in training CNN and to make a good comparison with other optimizers to show the ability of this optimizer for image classification. Moreover, in order to verify our results, we applied the other well-known optimizers WOA, Ant Colony Optimizer (ACO), and Bacteria Swarm Foraging Optimization (BFSO) for comparison.

This paper is organized as follows: Section 1 is an introduction, Section 2 presents a literature review of convolution neural networks, Section 3 explains the SFL Algorithm mechanism, Section 4 describes the design of proposed methods, Section 5 presents the result and discussion of the experiment, and lastly, Section 6 is the conclusion.

2. Convolutional Neural Network

Neural networks or artificial neural networks are computing systems that have been inspired by the human brain. In other words, these networks mimic the biological functions that are transmitted the signal to other neurons such as synapses in a biological brain [11, 50–52]. A neural network contains mathematical functions, which compute the weighted sum of multiple inputs, output, and activation functions. Specifically, these functions are layers of interconnected nodes, which are known as artificial neurons. The convolutional neural network is one of the various classes of neural networks, which is often applied to analyze and process the vision dataset. Nowadays, a CNN model has a critical role since fast growth in deep learning and artificial intelligence. It is necessary to mention that deep learning is a neural network that is composed of more than three layers and, as well as CNN, is used multiple layers, such as convolution, pooling, and fully connected layers to learn features and detect patterns of image [4, 53–55].

The convolutional layer is a critical component of CNN that uses the information of adjacent pixels as a linear operation to extract features. Each location of the tensor is calculated through an element-wise product between input tensor, which is an array of numbers and kernel or filter, while its output is summed up in order to obtain a single value in the corresponding location of output tensor, which is known as a feature map. Several various kernels should be applied to represent a different characteristic of the input tensor in order to achieve variant feature extractors such as a horizontal edge detector or a vertical edge detector [4, 56]. The convolution operation reduces the size of the feature maps in comparison to the input tensor. Typically, the padding technique should be used to increase the dimension of the image and lose less information by adding zeros around the image. The stride is considered as a distance between two kernels in convolution operation, that is, commonly one, while sometimes usage of the values more than one is to obtain downsampling of feature map for a specified purpose [57, 58].

A pooling layer provides alternative and more robust downsampling, as well as avoiding overfitting and a lot of computation by representing abstracted feature maps. This layer is operated on each feature map independently to make a new set of the same number of pooled feature maps and reduce the number of subsequent learnable parameters. Specifically, filter size, stride, and padding are also applied as hyperparameters in pooling operations. There are two common functions in pooling operation, average pooling, which is computed average value for each patch of the feature map, and max pooling, which is calculated the maximum value. In all cases, polling supports that the value of pooled features is remained almost invariant by translating the small amount of input [4, 59, 60].

The linear output of convolution operation is passed through nonlinear activation function, which is considered as a fundamental component in order to learn the complex patterns and the ability to add nonlinearity into the network. There are various types of activation functions such as sigmoid or hyperbolic tangent function, which are taken into account as smooth nonlinear functions and rectified linear unit (ReLU) function, which is recently the most widely used activation function. This is due to the fact that sigmoid and tanh activation functions are commonly saturated and really sensitive to modify around their mid-point of their input [55, 61].

Typically, the output of the final convolution and pooling layer is transformed to a one-dimension array of numbers, which is known as flattening. The output of flattening is considered as the input of one or more fully connected layers, in which every neuron in one layer is connected to every neuron in the other layer. In other words, the nonlinear combination of high-level features, which is the output of the convolution layer, is learned by a fully connected layer in order to map the final output of the network such as the probability of each class in the classification task [4, 62]. It is important to mention that each fully connected layer is followed by an activation function such as ReLU except the last nonlinear function that is usually different from the others. The last activation function, which is applied in the classification task, is Softmax to obtain a probability of the input being in the specified class [58, 63, 64].

The cost function is another essential component in Neural Networks. Actually, cost and loss functions are synonymous; the only difference is that the single training batch uses the loss function, while the cost function is referred to apply the loss function over the entire training set. The loss function is evaluated by the compatibility between the predicted value and the ground truth label, in which the higher output of loss means the incapable performance of the model. Another hyperparameter that is required for assigning is selecting an appropriate loss function with respect to the performed task. Since the problem is an optimization problem, Gradient descent is usually applied as an optimization algorithm to minimize the loss function [4, 65, 66].

The type of CNN employed in this study is LeNet-5, which is one of the earliest CNN models [49] (Figure 1). It is a classical CNN developed originally for recognizing characters. The architecture of LeNet-5 is composed of seven layers, in which, except the input layer, the rest can be trained (weights). As shown in the Figure 1, the LeNet-5 network possesses three convolutional layers C1, C3, and C5 among its processing layers. These convolutional layers are composed of two pooling layers S2 and S4, and the output layer is F6. The arrangement of convolutional layers and subsampling layers is in the form of plans and form feature maps. Each neuron in convolutional layers is linked locally to the local receptive field in the previous layer. Neurons that have the same feature maps obtain data from different local receptive fields. This process continues until the entire input plane is skimmed, and similar weights are employed together. Feature maps are spatially downsampled in the subsampling layer, and their size is reduced by a factor of 2. There is a similar kernel size of 5 5 for the three convolution layers C1, C3, and C5. However, the numbers of feature maps and parameters for each layer are different from each other (Table 1). The last convolution layer C5 is fully connected to the S4, and it has the feature maps size of 1 1. F6 is the last layer that performs the classification task. This layer is composed of 84 units and fully connected to the last convolutional layer C5.

Essentially, convolution layers are connected to several feature maps, kernels, and correlated to the prior layers. Each extracted feature matrix (feature map) is generated as a consequence of a sum of convolution from extracted feature matrices of the earlier layer, their corresponding mask (kernel), and a linear filter. Moreover, a bias value is summed to the extracted feature matrix and subsequently; it is applied to a nonlinear function. The tanh function is employed for this purpose. The kth feature map with the weights and is achieved by applying the tanh function as follows:

Through a subsampling layer, the size of each extracted feature matrix is reduced in relation to one of the extracted feature matrices of the former layer. This pooling strategy decreases the resolution of the extracted feature matrix. The pooling layers summarize the features present in an area of the feature map created by convolution layers. Also, a pooling layer diminishes the number of parameters for learning and the amount of computation performed in the network.

The classification task is carried out through the classification layer. This layer is placed after all the convolution and subsampling layers. In the classification layer, the output of each neuron is given to a single class label, and in the case of Oxford17, Oxford 102, Caltech/UCSD birds, and Caltech 101 airplanes dataset, this layer is composed often neurons corresponding to their labels.

3. Shuffled Frog-Leaping Optimizer

Shuffled Frog-Leaping Algorithm (SFLA) is a memetic metaheuristic approach that is employed to find a global solution through an informed heuristic search by employing a heuristic function [67]. This algorithm is a population-based technique that is occasioned by natural memetic. The term memetic is coming from “meme” considered as the unit of cultural evolution. Theoretically, the SFLA is similar to the particle swarm optimization (PSO). However, the values of weights and biases can be exchanged among local searches through a shuffling technique, thereby obtaining global optimum. The genuine aims of genes and memes are different from each other since they apply different mechanisms for data distribution from one population member to another [68, 69]. Gene’s transmission is only possible from parents to offspring and only occurs between generations. However, memes can be transmitted between any two individuals, and instead of waiting for a full generation of genes to be replicated, it can cooperate with other memes immediately once an improved idea is found. Moreover, the replication of the genes is limited to the slight number of offspring that can belong to a single parent. On the other hand, a meme can be taken over by an unlimited number of individuals [70].

SFL algorithm is a population-based approach composed of frogs of the same attributes. Each frog can be considered as a solution. The total population of frogs is divided into numerous subgroups known as memeplexes. Diverse subgroups can be appreciated as dissimilar frog memes. Each memeplex is responsible for a limited exploration. At each memeplex, other frogs might affect the behavior of each frog, and the evolution will take place through the process of memetic evolution [70, 71]. After a number of memetic evolution periods, the memeplexes are forced to join together leading to the generation of novel memeplexes through a shuffling method. Shuffling will completely make unbiased the cultural evolution in the direction of any specific interest. The stopping criteria are satisfied once the local search and the shuffling procedure alternate [68]. The flowchart of SFLA is shown in Figure 2. The different steps are described as follows [67, 72, 73]:(1)The algorithm contains a population “p” of the potential number of solutions, controlled by a set of virtual frogs (n).(2)The population is split into subsets denoted as memeplexes (m). The memeplexes can be considered as a set of parallel frog cultures trying to achieve some goals.(3)Frog i is shown by X_i = (X_i1, X_i2, …, X_is) in which S indicates the number of variables.(4)Within each memeplex, each frog culture searches the space in different directions and exchanges ideas independently. The frogs with best and worst fitness are denoted as X_b and .(5)Frog with global best fitness is identified as .(6)The frog with the worst fitness is modified based on the following equation:

In which the rand function generates an arbitrary number between the range [0, 1], D_i is the size of the leaping step of the frog_i and D_max is the maximum value permitted to adjust frog position. If the value of fitness is better than the current value of , it will be accepted. If the fitness value is not modified, then the calculation is repeated by replacing X_b with . If there is no potential for enhancement, a novel will be created arbitrarily. This shuffling process and the local search continue until defined convergence criteria provide satisfactory results [68, 70, 71].

4. Design of Proposed Method

Problem representation is the foremost step in training a CNN employing metaheuristics algorithms. To train a CNN, the problem should be formulated in a suitable way for metaheuristics. The most important variables to training this type of network are weights and biases. In the trainer, the best biases and weights values are found to provide the highest classification, approximation, and prediction accuracy for the network. Thus, biases and weights are trainable variables. It means by changing the values of biases and weights of all neurons the output results of the network can be varied. So, controlling the process of applying new weights and biases by an optimizer approach leads to reaching higher accuracy. As the SFL procedure takes the variables in the format of a vector, the variables of a CNN denoted for this technique are as follows:

In which n is the size of the input, denotes the connection weight between layers ith and jth, and is the bias (threshold) of the jth hidden node.

Once the variables are defined, defining the objective function for the SFL technique is the next goal. Mean Square Error (MSE) is a common metric for the evaluation of networks. Once a set of training samples are given to the CNN, this equation measures the difference between the obtained output values and the desirable values through the following equation:where m indicates the output numbers, the preferred output of the ith input unit when the kth training data samples are utilized is denoted by , and the actual output of the ith input unit when the kth training data appear in the input is .

To design an effective CNN, the network should be adapted to the whole set of training samples. As a result, CNN performance is assessed according to the average of MSE with respect to the training samples as the following equation:

As, at each iteration, the weights and biases move towards having the best CNN, the probability of an improved CNN increases gradually. However, due to stochastic nature of SFL algorithm, there is no guarantee that the optimal CNN is obtained. On the other hand, with sufficient number of iteration, the SFL algorithm finally reaches to a solution that works more efficient than random preliminary solutions. The following section assesses the advantages of the SFL algorithm in training a CNN practically.

5. Experimental Results and Discussion

In this part, the suggested SFL-based CNN is investigated by employing four standard classification datasets from [74–77], OxFord Flowers 17 (Figure 3(a)), OxFord Flowers 102 (Figure 3(b)), Caltech/UCSD Birds (Figure 3(c)), and Caltech 101 Airplanes (Figure 3(d)). The specifications of the datasets are presented in Table 2.

(a)

(b)

(c)

(d)

The classification datasets were intentionally selected with the diverse test/training data and difficulty levels to efficiently evaluate the performance of our SFL-based CNN. We employed a Hewlett-Packard (hp) computer with the processor of Intel (R) Core (TM) i7-6500U CPU @ 2.50 GHz 2.60 GHz, installed memory (RAM) of 8.00 GB, System type of 64-bit Operating System, and Windows 10 Home. For data processing, we used MATLAB and Statistics Toolbox Release 2019a. The SFL assumptions and other techniques are shown in Table 3. The accuracy of classification for each dataset using different optimization algorithms is demonstrated in Figure 4. It is considered that the optimization method begins by creating random biases and weights in the range of [−10, 10] for all four datasets.

(a)

(b)

(c)

(d)

To generate the results, the datasets are solved 50 times by applying each technique. As illustrated in Figure 4, after the last iteration, the error of classification for all the datasets employing different optimizers decreases to approximately less than 10%, in which the SFL algorithm appears to be the most efficient optimizer. The average (AVE) and standard deviation (STD) shown in Tables 4–7 are actually the best MSEs obtained in the last iteration. Clearly, the lowest average and standard deviation of MSE in the final iteration illustrate the best result. The statistical outcomes are shown in the form of AVE ± STD. It should be noted that the best rates of classification attained by each method during 50 iterations are reported as another metric of comparison. Statically analysis of the results shows that training CNN with SFL algorithm provides the best accuracy of classification in all the mentioned datasets, as well as superior local optima avoidance, which is the reason for the improved MSE. Moreover, the results of this study demonstrate that, unlike other swarm-based algorithms, SFL algorithm has a better performance since it does not have a mechanism for substantial sudden movements to search the space. It is also demonstrated that ACO and WO optimizers reach a minimum time for training the model in comparison of BFSO and SFL methods. Also, the SFL method takes more time than ACO and WO approaches, but it obtains higher classification rates and minimum MSE scores.

6. Conclusion

This study presented Shuffled Frog-Leaping Algorithm (SFLA) as one model of metaheuristic algorithms to optimize one type of Convolutional Neural Network. At first, the training problem of a CNN was formulated for the SFL technique. This method was then applied to define the optimum values for biases and weights. Theoretically, the SFLA is similar to the particle swarm optimization (PSO). However, the values of weights and biases can be exchanged among local searches through a shuffling technique, thereby obtaining global optimum. The proposed SFLA was employed to train four standard classification datasets (OxFord Flowers 17, OxFord Flowers 102, Caltech/UCSD Birds, and Caltech 101 Airplanes). To verify the performance of SFLA, the results were compared to three other stochastic optimization trainers: WO, BFSO, and ACO. The resulting outcomes demonstrated that the suggested technique can effectively train the CNN. It improves the probability of finding optimal values for biases and weights for CNNs. Our optimization strategy can obtain noticeable accuracy for classifying objects in four standard classification datasets.

For future study, finding the proper parameters for SFL algorithm needs to be investigated. Moreover, by exploring the optimal values for number of batches in the training, type of pooling, number of neurons in the hidden layer, and number of kernels in the convolutional layers, we can obtain more noticeable results. Further optimal tuning of this method is worth further research using different datasets such as CKP and facial expression datasets, as well as ImageNet and ORI.

Data Availability

These datasets are public datasets and are available at the following links: https://www.robots.ox.ac.uk/~vgg/data/flowers/17/http://www.vision.caltech.edu/Image_Datasets/Caltech101/http://www.vision.caltech.edu/visipedia/CUB-200.html https://www.robots.ox.ac.uk/~vgg/data/flowers/102/.

Disclosure

The funding sources had no involvement in the study design, collection, analysis or interpretation of data, writing of the manuscript, or in the decision to submit the manuscript for publication.

Conflicts of Interest

The authors declare that there are no conflicts of interest.

References

H. Cecotti, A. Rivera, M. Farhadloo, and M. A. Pedroza, “Grape detection with convolutional neural networks,” Expert Systems with Applications, vol. 159, Article ID 113588, 2020.
View at: Publisher Site | Google Scholar
N. Razmjooy and S. Razmjooy, “Skin melanoma segmentation using neural networks optimized by quantum invasive weed optimization algorithm,” Lecture Notes in Electrical Engineering, vol. 696, pp. 233–250, 2021.
View at: Publisher Site | Google Scholar
S. B. Saadi, R. Ranjbarzadeh, and K. Ozeir, “Osteolysis: a literature review of basic science and potential computer-based image processing detection methods,” Computational Intelligence and Neuroscience, vol. 2021, p. 21, 2021.
View at: Publisher Site | Google Scholar
R. Ranjbarzadeh, S. Jafarzadeh Ghoushchi, M. Bendechache et al., “Lung infection segmentation for COVID-19 pneumonia based on a cascade convolutional network from CT images,” BioMed Research International, vol. 2021, Article ID 5544742, 16 pages, 2021.
View at: Publisher Site | Google Scholar
W. A. Ali, M. Aljunid, M. Bendechache, and P. Sandhya, “Review of current machine learning approaches for anomaly detection in network traffic,” Journal of Telecommunications and the Digital Economy, vol. 8, no. 4, pp. 64–95, 2020.
View at: Publisher Site | Google Scholar
K. Nguyen, C. Fookes, A. Ross, and S. Sridharan, “Iris recognition with off-the-shelf CNN features: a deep learning perspective,” IEEE Access, vol. 6, pp. 18848–18855, 2018.
View at: Publisher Site | Google Scholar
Y. Cai, W. Wang, Y. Chen, and Q. Ye, “IOS-Net: an inside-to-outside supervision network for scale robust text detection in the wild,” Pattern Recognition, vol. 103, Article ID 107304, 2020.
View at: Publisher Site | Google Scholar
U. Albalawi, S. Manimurugan, and R. Varatharajan, “Classification of breast cancer mammogram images using convolution neural network,” Concurrency and Computation: Practice and Experience, Article ID e5803, 2020.
View at: Publisher Site | Google Scholar
T. Mikolov, M. Karafiat, L. Burget, J. Cernocky, and S. Khudanpur, “Recurrent neural network based language model,” Process. INTERSPEECH, vol. 2, pp. 1045–1048, 2010.
View at: Publisher Site | Google Scholar
A. Graves, N. Jaitly, and A.-R. Mohamed, “Hybrid speech recognition with deep bidirectional lstm,” in Proceedings of the IEEE Work. Au- tomatic Speech Recognit Underst, pp. 273–278, Olomouc, Czech Republic, 2013.
View at: Publisher Site | Google Scholar
T. Kooi, G. Litjens, B. van Ginneken et al., “Large scale deep learning for computer aided detection of mammographic lesions,” Medical Image Analysis, vol. 35, pp. 303–312, 2017.
View at: Publisher Site | Google Scholar
H. M. Fayek, M. Lech, and L. Cavedon, “Evaluating deep learning architectures for speech emotion recognition,” Neural Networks, vol. 92, pp. 60–68, 2017.
View at: Publisher Site | Google Scholar
F. Gao, T. Huang, J. Sun, J. Wang, A. Hussain, and E. Yang, “A new algorithm for SAR image target recognition based on an improved deep convolutional neural network,” Cognitive Computation, vol. 11, no. 6, pp. 809–824, 2019.
View at: Publisher Site | Google Scholar
F. Lattari, B. Gonzalez Leon, F. Asaro, A. Rucci, C. Prati, and M. Matteucci, “Deep learning for SAR image despeckling,” Remote Sensing, vol. 11, no. 13, p. 1532, 2019.
View at: Publisher Site | Google Scholar
R. Ranjbarzadeh, S. B. Saadi, and A. Amirabadi, “LNPSS: SAR image despeckling based on local and non-local features using patch shape selection and edges linking,” Measurement, vol. 164, Article ID 107989, 2020.
View at: Publisher Site | Google Scholar
A. Hojatimalekshah, Z. Uhlmann, N. F. Glenn et al., “Tree canopy and snow depth relationships at fine scales with terrestrial laser scanning,” The Cryosphere, vol. 15, no. 5, pp. 2187–2209, 2021.
View at: Publisher Site | Google Scholar
J. Y. Ryu, H. U. Kim, and S. Y. Lee, “Deep learning improves prediction of drug-drug and drug-food interactions,” Proceedings of the National Academy of Sciences, vol. 115, no. 18, pp. E4304–E4311, 2018.
View at: Publisher Site | Google Scholar
L. Zhang, J. Tan, D. Han, and H. Zhu, “From machine learning to deep learning: progress in machine intelligence for rational drug discovery,” Drug Discovery Today, vol. 22, no. 11, pp. 1680–1685, 2017.
View at: Publisher Site | Google Scholar
T. Zhou, H. Lu, Z. Yang, S. Qiu, B. Huo, and Y. Dong, “The ensemble deep learning model for novel COVID-19 on CT images,” Applied Soft Computing, vol. 98, Article ID 106885, 2021.
View at: Publisher Site | Google Scholar
N. Coudray, P. S. Ocampo, T. Sakellaropoulos et al., “Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning,” Nature Medicine, vol. 24, no. 10, pp. 1559–1567, 2018.
View at: Publisher Site | Google Scholar
A. Valizadeh, S. Jafarzadeh Ghoushchi, R. Ranjbarzadeh, and Y. Pourasad, “Presentation of a segmentation method for a diabetic retinopathy patient’s fundus region detection using a convolutional neural network,” Computational Intelligence and Neuroscience, vol. 2021, pp. 1–14, 2021.
View at: Publisher Site | Google Scholar
S. Aleem, T. Kumar, S. Little, M. Bendechache, R. Brennan, and K. McGuinness, “Random Data Augmentation based Enhancement: AGeneralized Enhancement Approach for Medical Datasets,” 2022.
View at: Google Scholar
N. Razmjooy, S. Razmjooy, Z. Vahedi, V. V. Estrela, and G. G. de Oliveira, “Skin color segmentation based on artificial neural network improved by a modified grasshopper optimization algorithm,” Lecture Notes in Electrical Engineering, vol. 696, pp. 169–185, 2021.
View at: Publisher Site | Google Scholar
S. Jafarzadeh Ghoushchi, R. Ranjbarzadeh, S. A. Najafabadi, E. Osgooei, and E. B. Tirkolaee, “An extended approach to the diagnosis of tumour location in breast cancer using deep learning,” Journal of Ambient Intelligence and Humanized Computing, pp. 1–11, 2021.
View at: Publisher Site | Google Scholar
D. Soydaner, “A comparison of optimization algorithms for deep learning,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 34, no. 13, Article ID 2052013, 2020.
View at: Publisher Site | Google Scholar
A. Chandio, Y. Shen, M. Bendechache, I. Inayat, and T. Kumar, “AUDD: audio Urdu digits dataset for automatic audio Urdu digit recognition,” Applied Sciences, vol. 11, no. 19, p. 8842, 2021.
View at: Publisher Site | Google Scholar
F. Zhan, H. Zhu, and S. Lu, “Scene text synthesis for efficient and effective deep network training,” 2020, https://arxiv.org/abs/1901.09193v1.
View at: Google Scholar
H. Huang, C. Wang, and B. Dong, “Nostalgic Adam: weighting more of the past gradients when designing the adaptive learning rate,” Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, vol. 2019, pp. 2556–2562, 2019.
View at: Publisher Site | Google Scholar
M. Zaheer, S. J. Reddi, D. Sachan, S. Kale, G. Research, and S. Kumar, “Adaptive methods for nonconvex optimization,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal Canada, December 2018.
View at: Publisher Site | Google Scholar
K. N. Reid, J. Li, A. Brownlee et al., “A hybrid metaheuristic approach to a real world employee scheduling problem,” in Proceedings of the 2019 Genetic and Evolutionary Computation Conference, pp. 1311–1318, July 2019.
View at: Publisher Site | Google Scholar
F. Altunbey Ozbay and B. Alatas, “A novel approach for detection of fake news on social media using metaheuristic optimization algorithms,” Elektronika ir Elektrotechnika, vol. 25, no. 4, pp. 62–67, 2019.
View at: Publisher Site | Google Scholar
K. Hussain, M. N. Mohd Salleh, S. Cheng, and Y. Shi, “Metaheuristic research: a comprehensive survey,” Artificial Intelligence Review, vol. 52, no. 4, pp. 2191–2233, 2019.
View at: Publisher Site | Google Scholar
S. M. Mousavi, A. Asgharzadeh-Bonab, and R. Ranjbarzadeh, “Time-frequency analysis of EEG signals and GLCM features for depth of anesthesia monitoring,” Computational Intelligence and Neuroscience, vol. 2021, Article ID 8430565, pp. 1–14, 2021.
View at: Publisher Site | Google Scholar
L. Zhang, H. J. Gao, J. Zhang, and B. Badami, “Optimization of the convolutional neural networks for automatic detection of skin cancer,” Open Medicine, vol. 15, no. 1, pp. 27–37, 2020.
View at: Publisher Site | Google Scholar
G. L. F. da Silva, T. L. A. Valente, A. C. Silva, A. C. de Paiva, and M. Gattass, “Convolutional neural network-based PSO for lung nodule false positive reduction on CT images,” Computer Methods and Programs in Biomedicine, vol. 162, pp. 109–118, 2018.
View at: Publisher Site | Google Scholar
F. Hoseini, A. Shahbahrami, and P. Bayat, “AdaptAhead optimization algorithm for learning deep CNN applied to MRI segmentation,” Journal of Digital Imaging, vol. 32, no. 1, pp. 105–115, 2018.
View at: Publisher Site | Google Scholar
R. P. França, A. C. B. Monteiro, V. V. Estrela, and N. Razmjooy, “Using metaheuristics in discrete-event simulation,” Lecture Notes in Electrical Engineering, vol. 696, pp. 275–292, 2021.
View at: Google Scholar
A. C. B. Monteiro, R. P. França, V. V. Estrela, N. Razmjooy, Y. Iano, and P. D. M. Negrete, “Metaheuristics applied to blood image analysis,” Lecture Notes in Electrical Engineering, vol. 696, pp. 117–135, 2021.
View at: Publisher Site | Google Scholar
S. Mahdinia, M. Rezaie, M. Elveny, N. Ghadimi, and N. Razmjooy, “Optimization of PEMFC model parameters using meta-heuristics,” Sustainability, vol. 13, no. 22, p. 12771, 2021.
View at: Publisher Site | Google Scholar
S. Sengupta, S. Basak, and R. Peters, “Particle Swarm Optimization: a survey of historical and recent developments with hybridization perspectives,” Machine Learning and Knowledge Extraction, vol. 1, no. 1, pp. 157–191, 2018.
View at: Publisher Site | Google Scholar
F. Cicirelli, A. Forestiero, A. Giordano, and C. Mastroianni, “Transparent and efficient parallelization of swarm algorithms,” ACM Transactions on Autonomous and Adaptive Systems, vol. 11, no. 2, p. 26, 2016.
View at: Publisher Site | Google Scholar
S. Mirjalili and A. Lewis, “The Whale optimization algorithm,” Advances in Engineering Software, vol. 95, pp. 51–67, 2016.
View at: Publisher Site | Google Scholar
B. Doerr, H. P. Le, R. Makhmara, and T. D. Nguyen, “Fast genetic algorithms,” in Proceedings of the 2017 Genetic and Evolutionary Computation Conference, pp. 777–784, Berlin, Germany, July 2017.
View at: Publisher Site | Google Scholar
B. Duan, Q. Cao, and N. Afshar, “Optimal parameter identification for the proton exchange membrane fuel cell using Satin Bowerbird optimizer,” International Journal of Energy Research, vol. 43, no. 14, pp. 8623–8632, 2019.
View at: Publisher Site | Google Scholar
A. Khatri, A. Gaba, K. P. S. Rana, and V. Kumar, “A novel life choice-based optimizer,” Soft Computing, vol. 24, no. 12, pp. 9121–9141, 2020.
View at: Publisher Site | Google Scholar
W. Zhao, L. Wang, and Z. Zhang, “Artificial ecosystem-based optimization: a novel nature-inspired meta-heuristic algorithm,” Neural Computing & Applications, vol. 32, no. 13, pp. 9383–9425, 2020.
View at: Publisher Site | Google Scholar
A. Faramarzi, M. Heidarinejad, B. Stephens, and S. Mirjalili, “Equilibrium optimizer: a novel optimization algorithm,” Knowledge-Based Systems, vol. 191, Article ID 105190, 2020.
View at: Publisher Site | Google Scholar
A. Franzin and T. Stützle, “Revisiting simulated annealing: a component-based analysis,” Computers & Operations Research, vol. 104, pp. 191–206, 2019.
View at: Publisher Site | Google Scholar
“MNIST Demos on Yann LeCun’s Website,” 2021, http://yann.lecun.com/exdb/lenet/.
View at: Google Scholar
J. Zhong, Z. Liu, Z. Han, Y. Han, and W. Zhang, “A CNN-based defect inspection method for catenary split pins in high-speed railway,” IEEE Transactions on Instrumentation and Measurement, vol. 68, no. 8, pp. 2849–2860, 2019.
View at: Publisher Site | Google Scholar
S. R. de Assis Neto, G. L. Santos, E. da Silva Rocha et al., “Detecting human activities based on a multimodal sensor data set using a bidirectional long short-term memory model: a case study,” Challenges and Trends in Multimodal Fall Detection for Healthcare, vol. 273, pp. 31–51, 2020.
View at: Publisher Site | Google Scholar
Y. Pourasad, R. Ranjbarzadeh, and A. Mardani, “A new algorithm for digital image encryption based on chaos theory,” Entropy, vol. 23, no. 3, p. 341, 2021.
View at: Publisher Site | Google Scholar
A. Waleed Salehi, P. Baglat, and G. Gupta, “Review on machine and deep learning models for the detection and prediction of coronavirus,” Materials Today Proceedings, vol. 33, pp. 3896–3901, 2020.
View at: Publisher Site | Google Scholar
M. Gour, S. Jain, and T. Sunil Kumar, “Residual learning based CNN for breast cancer histopathological image classification,” International Journal of Imaging Systems and Technology, vol. 30, no. 3, pp. 621–635, 2020.
View at: Publisher Site | Google Scholar
R. Ranjbarzadeh, S. Dorosti, S. Jafarzadeh Ghoushchi et al., “Nerve optic segmentation in CT images using a deep learning model and a texture descriptor,” Complex & Intelligent Systems, pp. 1–15, 2022.
View at: Publisher Site | Google Scholar
S. Iqbal, M. U. Ghani Khan, T. Saba et al., “Deep learning model integrating features and novel classifiers fusion for brain tumor segmentation,” Microscopy Research and Technique, vol. 82, no. 8, pp. 1302–1315, 2019.
View at: Publisher Site | Google Scholar
G. Murtaza, L. Shuib, A. W. Abdul Wahab et al., “Deep learning-based breast cancer classification through medical imaging modalities: state of the art and research challenges,” Artificial Intelligence Review, vol. 53, no. 3, pp. 1655–1720, 2020.
View at: Publisher Site | Google Scholar
A. Khatami, M. Babaie, H. R. Tizhoosh, A. Khosravi, T. Nguyen, and S. Nahavandi, “A sequential search-space shrinking using CNN transfer learning and a Radon projection pool for medical image retrieval,” Expert Systems with Applications, vol. 100, pp. 224–233, 2018.
View at: Publisher Site | Google Scholar
A. Doğantekin, F. Özyurt, E. Avcı, and M. Koç, “A novel approach for liver image classification: PH-C-elm,” Measurement, vol. 137, pp. 332–338, 2019.
View at: Publisher Site | Google Scholar
X. Yu, N. Zeng, S. Liu, and Y.-D. Zhang, “Utilization of DenseNet201 for diagnosis of breast abnormality,” Machine Vision and Applications, vol. 30, no. 7-8, pp. 1135–1144, 2019.
View at: Publisher Site | Google Scholar
R. Ranjbarzadeh, A. Bagherian Kasgari, S. Jafarzadeh Ghoushchi, S. Anari, M. Naseri, and M. Bendechache, “Brain tumor segmentation based on deep learning and an attention mechanism using MRI multi-modalities brain images,” Scientific Reports, vol. 11, no. 1, p. 10930, 2021.
View at: Publisher Site | Google Scholar
A. Rasmus, A. Peimankar, and S. Puthusserypady, “A deep learning approach for real-time detection of atrial fibrillation,” Expert Systems with Applications, vol. 115, pp. 465–473, 2019.
View at: Google Scholar
S. J. Ghoushchi and M. N. A. Rahman, “Performance study of artificial neural network modelling to predict carried weight in the transportation system,” International Journal of Logistics Systems and Management, vol. 24, no. 2, pp. 200–212, 2016.
View at: Publisher Site | Google Scholar
A. Aghamohammadi, R. Ranjbarzadeh, F. Naiemi, M. Mogharrebi, S. Dorosti, and M. Bendechache, “TPCNN: two-path convolutional neural network for tumor and liver segmentation in CT images using a novel encoding approach,” Expert Systems with Applications, vol. 183, Article ID 115406, 2021.
View at: Publisher Site | Google Scholar
M. Frid-Adar, I. Diamant, E. Klang, M. Amitai, J. Goldberger, and H. Greenspan, “GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification,” Neurocomputing, vol. 321, pp. 321–331, 2018.
View at: Publisher Site | Google Scholar
Q. T. Ain, M. Ali, A. Riaz et al., “Sentiment analysis using deep learning techniques: a review,” IJACSA Int. J. Adv. Comput. Sci. Appl, vol. 8, no. 6, 2017.
View at: Google Scholar
K. Lu, L. Ting, W. Keming, Z. Hanbing, T. Makoto, and Y. Bin, “An improved shuffled frog-leaping algorithm for flexible job shop scheduling problem,” Algorithms, vol. 8, no. 1, pp. 19–31, 2015.
View at: Publisher Site | Google Scholar
M.-L. Pérez-Delgado, “Color image quantization using the shuffled-frog leaping algorithm,” Engineering Applications of Artificial Intelligence, vol. 79, pp. 142–158, 2019.
View at: Publisher Site | Google Scholar
J. Cai, R. Zhou, and D. Lei, “Dynamic shuffled frog-leaping algorithm for distributed hybrid flow shop scheduling with multiprocessor tasks,” Engineering Applications of Artificial Intelligence, vol. 90, Article ID 103540, 2020.
View at: Publisher Site | Google Scholar
M. Eusuff, K. Lansey, and F. Pasha, “Shuffled frog-leaping algorithm: a memetic meta-heuristic for discrete optimization,” Engineering Optimization, vol. 38, no. 2, pp. 129–154, 2006.
View at: Publisher Site | Google Scholar
J. Tang, R. Zhang, P. Wang, Z. Zhao, L. Fan, and X. Liu, “A discrete shuffled frog-leaping algorithm to identify influential nodes for influence maximization in social networks,” Knowledge-Based Systems, vol. 187, Article ID 104833, 2020.
View at: Publisher Site | Google Scholar
J. Siahbalaee, N. Rezanejad, and G. B. Gharehpetian, “Reconfiguration and DG sizing and placement using improved shuffled frog leaping algorithm,” Electric Power Components and Systems, vol. 47, no. 16-17, pp. 1475–1488, 2019.
View at: Publisher Site | Google Scholar
R. Dash, “Performance analysis of a higher order neural network with an improved shuffled frog leaping algorithm for currency exchange rate prediction,” Applied Soft Computing, vol. 67, pp. 215–231, 2018.
View at: Publisher Site | Google Scholar
“Visual Geometry Group, University of Oxford,” 2022, https://www.robots.ox.ac.uk/∼vgg/data/flowers/17/.
View at: Google Scholar
“Caltech101,” 2022, http://www.vision.caltech.edu/Image_Datasets/Caltech101/.
“Caltech-UCSD Birds 200,” 2022, http://www.vision.caltech.edu/visipedia/CUB-200.html.
“Visual Geometry Group, University of Oxford,” 2022, https://www.robots.ox.ac.uk/∼vgg/data/flowers/102/.
View at: Google Scholar

Copyright

Copyright © 2022 Soroush Baseri Saadi et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

569

Downloads

332

Citations