Abstract
This paper experimentally investigates the effect of nine chaotic maps on the performance of two Particle Swarm Optimization (PSO) variants, namely, Random Inertia Weight PSO (RIW-PSO) and Linear Decreasing Inertia Weight PSO (LDIW-PSO) algorithms. The applications of logistic chaotic map by researchers to these variants have led to Chaotic Random Inertia Weight PSO (CRIW-PSO) and Chaotic Linear Decreasing Inertia Weight PSO (CDIW-PSO) with improved optimizing capability due to better global search mobility. However, there are many other chaotic maps in literature which could perhaps enhance the performances of RIW-PSO and LDIW-PSO more than logistic map. Some benchmark mathematical problems well-studied in literature were used to verify the performances of RIW-PSO and LDIW-PSO variants using the nine chaotic maps in comparison with logistic chaotic map. Results show that the performances of these two variants were improved more by many of the chaotic maps than by logistic map in many of the test problems. The best performance, in terms of function evaluations, was obtained by the two variants using Intermittency chaotic map. Results in this paper provide a platform for informative decision making when selecting chaotic maps to be used in the inertia weight formula of LDIW-PSO and RIW-PSO.
1. Introduction
PSO algorithm is one of the many algorithms that have been proposed over the years for global optimization. When it was proposed in 1995 [1], swarm size, particle velocity, acceleration coefficients, and random coefficients were the associated parameters that controlled its operations. A close look at the algorithm shows that randomness plays very useful role in making the algorithm effectively solve optimization problems. Randomness comes into play at the point of initializing the particles in the solution space and in updating the velocities of particles at each iteration of the algorithm. This random feature has contributed immensely to the performance of PSO [1–3]. To further enhance the performance of PSO, inertia weight strategy (IWS) was introduced into it by [4] to facilitate the intensification and diversification characteristics of the algorithm. Intensification searches around the current best solutions and selects the best candidate, while diversification makes the algorithm explore the search space more efficiently, mostly by means of randomization. As a result, randomness has been brought into the IWS by different researchers [5–8]. The important role of randomization can also be played by using chaos theory. Chaos is mathematically defined as randomness generated by simple deterministic system [2]. It is generally characterised by three dynamic properties, namely, ergodicity, stochastic, and sensitivity, to its initial conditions [2, 9]. These characteristics can enhance the search ability of PSO. This seems to be the motivation behind the introduction of chaos feature into IWS in [10], which led to improved optimizing capabilities of CDIW-PSO and CRIW-PSO due to better global search mobility compared with LDIW-PSO and RIW-PSO, respectively. Chaos optimizations have been applied to different aspects of PSO by various researchers over the years [9, 11–14]. In order to increase the diversity of the swarm and prevent premature convergence to local optimal, chaos mutation operator based on logistic map was used in [13] and another based on zaslavskii was used in [11]. But in [14], PSO was hybridized with chaotic local search procedure based on logistic map. The Logistic and Tent chaotic maps were, respectively, used as inertial weight by [9] in binary PSO to handle feature selection problem. In [12], twelve different chaos maps were implemented to tune the attraction parameter of accelerated PSO algorithm.
The aim of this paper is to further investigate the performances of two PSO variants, LDIW-PSO and RIW-PSO algorithms, with various chaotic maps incorporated into their IWSs. For this purpose, 9 additional chaotic maps along with logistic map are introduced in this paper and used with the two variants at different times. Some well-studied benchmark mathematical problems in the literature were used to test the algorithms using these maps. The outcome of the experiments should help ascertain the chaotic maps that contribute better to the performances of the algorithms in comparison to logistic chaotic map in order to provide some useful information regarding the usage of these maps in the IWSs of the PSO variants.
In the sections that follow, the inertia weight PSO and its variants considered in this paper are introduced in Section 2, chaotic maps used in the experiments are described in Section 3, setting of the experiments is given in Section 4, and experimental results and discussions are presented in Section 5, while Section 6 concludes the paper.
2. Inertia Weight PSO
PSO algorithm is a population-based evolutionary stochastic technique made up of a swarm of particles which coexist and evolve simultaneously based on knowledge shared with neighbouring particles. The PSO process is initialized with a swarm of random particles in the search space and the algorithm is allowed to execute a number of times in order to carry out a search for optimal solutions in the search space. In inertia weight PSO, each particle is assumed to have position and velocity in a physical n-dimensional search space; the position and velocity of a particle in each iteration is represented as the vectors and , respectively. When the particles move in the search space searching for the optimum solution for a particular optimization problem, other particles follow the current optimum particle by adjusting their velocities and positions using (1). The positions and velocities of the particles are confined within and , respectively, as follows:
A particle’s position is taken as possible solution for the problem being optimized, while the fitness of this possible solution is determined by evaluating the problem’s objective function. The best position searched by the particle itself so far () and the optimization position searched by the whole particle swarms so far () are -dimensional vectors representing personal best position of particle at iteration and global best positions selected from the personal best positions of all the particles in the swarm at iteration . whereas and are two -dimensional vectors of random numbers between 0 and 1, which introduces randomness to the searching strategy, and the two positive constants and are cognitive and social scaling parameters that determine the magnitude of the random forces in the direction of and .
The inertia weight () strikes a balance between exploration and exploitation characteristics of PSO and it determines the level of contribution of previous particle velocity to the present velocity.
2.1. Linear Decreasing Inertia Weight PSO (LDIW-PSO)
This variant implements the linear decreasing IWS which has greatly improved the algorithm [15, 16]. In this variant, the inertia weight starts with a large initial value and then linearly decreases to a smaller final value with the belief that a large inertia weight facilitates a global search, while a small inertia weight facilitates a local search. The commonly used initial and final values are 0.9 and 0.4 [10, 17, 18]; other values have also been used [19–21]. The inertia weight creates a means of flexibility for the movements of particles in the search space. According to [22], relatively high inertia weight value (e.g., 0.9) creates a medium of low viscosity for the particles to facilitate extensive exploration while gradually reducing it to a much lower value (e.g., 0.4) creates a high viscosity medium to facilitate exploitation. The experimental results in [15] showed that using the linearly decreasing inertia weight can make PSO suffer from premature convergence due to lack of search ability towards the end of run to jump out of the local minimum in some cases. Because of this challenge, employing adapting strategy for adjusting the inertia weight was suggested to improve the performance PSO near the optima [15]. Many researchers have made efforts to achieve this through various improvements on LDIW-PSO by introducing new parameters into its IWS or proposing new IWS to generally improve the performance of PSO technique [5, 10, 17, 19, 23, 24]. Equation (2) represents the commonly used LDIW strategy: where and are the initial and final values of inertia weight, is the current iteration number, is the maximum iteration number, and is the inertia weight value in the th iteration.
2.2. Random Inertia Weight PSO (RIW-PSO)
There are different IWSs with random features [5–8], but the one in [5] is adopted in this paper for the experiments. With the aim of using particle swarms to track and optimize dynamic systems, a new way of calculating the inertia weight value was proposed in [5] as shown in (3). The formula makes the inertia weight change randomly and produces a number randomly varying between 0.5 and 1.0, with a mean value of 0.75. Consider
In (3), is a uniformly distributed random number within the range . As a result of the difficulty in predicting whether exploration (a larger inertia weight value) or exploitation (a smaller inertia weight) will be better at any given time in tracking a nonlinear dynamic system, the strategy in (3) was introduced by [5] to address the inefficiency of linearly decreasing inertia weight, which decreases from 0.9 to 0.4 during a run, in handling such a problem.
2.3. Chaotic Inertia Weight PSO
The chaotic inertia weights PSO that were proposed in [10] are Chaotic Decreasing Inertia Weight PSO (CDIW-PSO) and Chaotic Random Inertia Weight PSO (CRIW-PSO) shown in (5) and (6), respectively. The aim was to improve LDIW-PSO and RIW-PSO using logistic map in order to avoid getting into local optimum in searching process by utilizing the merits of chaotic optimization. Logistic map, represented in (4), is one of the simplest maps that appear in nonlinear dynamics of biological population evidencing chaotic behavior. Consider where is the iteration number, is the th chaotic number, and . This map generates values between 0 and 1, provided that the initial value and that . where and are the initial and final values of inertia weight, is a uniform random number in , is the current iteration, and is the maximum iteration. The results in [10] show that the PSO had preferable convergence precision, quick convergence velocity, and better global search ability. This is because, due to nonrepetition of chaos, the algorithm could carry out overall searches at higher speed, diversify the particles, and improve the algorithm’s performance in preventing premature convergence too quickly to local minima compared with RIW-PSO and LDIW-PSO which have no chaos characteristics. No other chaotic maps were implemented with the linear decreasing and random IWSs to see if they could make the algorithms perform better in comparison to using logistic map. Besides, the results in [10] leave much room for further improvement on the performance of RIW-PSO and LDIW-PSO.
3. Chaotic Maps
Chaos is a deterministic dynamic system that is very sensitive to its initial conditions and parameters. The application of chaotic maps instead of random sequence in PSO is a powerful strategy to diversify the swarm and improve its performance. There are various chaotic maps that exist in the literature which could also be used with RIW-PSO and LDIW-PSO apart from logistic map used in [10]. Introduced below are some of these maps, adopted from [2, 12], and they are described as used in the experiments conducted in this paper.
3.1. Circle
This map has two parameters a (which can be interpreted as the strength of nonlinearity) and (which can be interpreted as externally applied frequency). It is a one-dimensional map which maps a Circle into itself and it is represented by (7):
In this paper, and .
3.2. Cubic
This map is somehow similar to Sine map, but it generates values in the interval . It is defined as
In this paper, the values generated were normalized between 0 and 1.
3.3. Gaussian
This map is also known as Gauss or mouse map. It is defined as where and denotes the largest integer less than which acts as a shift on the continued fraction representation of numbers.
3.4. Intermittency
This map is the extension of the Bernoulli Shift in which one of the piecewise linear segments is replaced by a nonlinear segment as shown below where , and .
3.5. Iterative Chaotic Map with Infinite Collapses (ICMIC)
This map is represented by where and the results were normalized between 0 and 1.
3.6. Piecewise
This map consists of line segments and is defined as where is the control parameter between 0 and 0.5.
3.7. Sinusoidal
This map is represented by the following
This map is similar to logistic map in shape with and generates values in . The values were normalized between 0 and 1 for .
3.8. Skew Tent
This map is a Tent map skewed to either left or right controlled by the parameter . The map is defined by where .
3.9. Tent
The tent map is like the logistic map but has a “” shape unlike logistic map which has a dome-like shape. It is defined by where is the iteration number, is the th chaotic number and . This map also generates values between 0 and 1, provided that the initial value .
4. Experimental Setup
To investigate the performance of RIW-PSO and LDIW-PSO with the chaotic maps incorporated into them at different times, five well-studied benchmark problems described below were used to validate them. The swarm size was set to 20 particles, while the number of variables (dimensions) for all problems was set to 30 and 50 with respective maximum number of iterations of 2000 and 3000. The algorithm was allowed to run the maximum number of iterations and number of successful runs recorded. A run is successful if the mean fitness value obtained by an algorithm is less than the success criterion after the maximum iteration. Values for and were set to 0.9 and 0.4 and and were set to 2.0 as used in [10] and was clamped to be 15% of the search space [25]. For fairness, the same random seeds were used in all the experiments with 50 independent runs for the test problems. The performance criteria were the mean best solution, standard deviation, success rate, and number of function evaluations of the algorithms. The simulation program was developed in Microsoft Visual programming language.
The success rate (SR) was computed according to (16) and expected number of function evaluations (NFE) according to (17): where is the total number of times the set goal was reached over 50 independent runs and is the total number of independent runs (i.e., 50, in our experiment): where is the swarm size and is the average number of iterations the set goal was reached over 50 independent runs.
4.1. Test Problems
The test problems used in the experiments are well studied in the literature [26–29]. They are Ackley, Griewank, Rastrigin, Rosenbrock, and Sphere. All the problems are continuous, scalable, and multimodal except the last two problems, which are unimodal. Each of these problems is described below.
The Ackley problem is nonseparable. It is a widely used test problem and it is defined in (18). The global minimum is obtainable at and the number of local minima is not known. Consider
The Griewank problem is non-separable with many widespread local minima regularly distributed. The complexity of the problem decreases as the dimensionality increases. Its global minimum is obtainable at and the number of local minima in a two-dimensional case is about 500. This problem is represented by
The Rastrigin problem represented in (20) is separable and has many local minima arranged in a lattice-like configuration. It is based on the Sphere problem with the addition of cosine modulation so as to produce frequent local minima. There are about 50 local minima for two-dimensional case and its global minimum is obtainable at . Consider
Shown in (21) is the Rosenbrock problem; it is non-separable. It is a classic optimization problem also known as banana function, the second function of De Jong, or extended Rosenbrock function. Its global minimum obtainable at , lies inside a long narrow, parabolic’s shaped valley. Though it look-simple to solve, yet due to a saddle points it is very difficult to converge to the global optimum. Consider
The Sphere problem also known as the first De Jong function is separable. It is one of the simplest test benchmark problems. Its global minimum is obtainable at and the problem is represented by
4.2. Properties of the Test Problems
Shown in Table 1 are the properties of the test problems. The success criteria are stated here as used in [10].
5. Results and Discussions
Many researchers have measured the performance of PSO algorithms using mean (average) fitness value and standard deviation [17, 19, 30–35]. The number of iterations or evaluations of the objective function that takes the algorithm to find optimum solution with specified accuracy to an optimization problem has also been used. However, the number of function evaluations appears to be more informative and popularly used to measure the performance of optimization algorithms [3, 25]. This is because it reflects the time or computational complexity of optimization algorithms and takes into account the swarm size, average number of algorithm iterations to reach the set goal (e.g., success threshold), and the success rate of the algorithm. In this paper, average fitness value, standard deviation of fitness value, success rate, and number of function evaluations were used for the performance measurement of the two PSO variants.
The mean best fitness (mean) is a measure of the precision that an algorithm can get within a given number of iterations; standard deviation (SD) is a measure of the algorithm’s stability and robustness, while success rate (SR) is the number of times an algorithm is able to meet success criterion out of a specified number of independent runs, which is a reflection of global search ability of the algorithm.
Tables 2–11 show the numerical results obtained in the experiments when the various chaotic maps were incorporated into RIW-PSO and LDIW-PSO algorithms. In the tables, “None” means that the algorithms were implemented without any chaotic map. In Tables 5 and 10, “—” indicates that no trial run satisfied the success criterion. The values in bold are the best results obtained by the algorithms using the corresponding chaotic map incorporated into it compared with others. The values with asterik (*) are the better results obtained by the algorithms relative to the corresponding chaotic maps, compared to Logistic map.
5.1. Results for LDIW-PSO Using the Various Chaotic Maps
Presented in Tables 2–5 are the results with respect to mean fitness value, standard deviation, success rate, and number of function evaluations as obtained in the experiments.
Table 2 shows the mean fitness values obtained by LDIW-PSO using the various chaotic maps. None of the chaotic maps used with the variants could enable it to perform best in all test problems compared with other maps. Apart from Sine, Intermittency and Cubic maps, the algorithm was able to obtain best optimal fitness for different test problems using other maps. Circle and Gaussian maps look more robust than others, because with them the algorithm was able to obtain the best optimal fitness for two test problems in the two problem dimensions which shows better characteristics for convergence precision. Besides , logistic map was outperformed by other maps as shown by the shaded portions in obtaining better optimal fitness.
Table 3 shows the standard deviation (stability measure) of LDIW-PSO using the various chaotic maps. As shown by the results, none of the chaotic maps when used with the variant could enable the algorithm to have the best stability across the test problems. However, the algorithm achieved the best stabilities with the various maps for different test problems, except with Sine, Intermittency, and Cubic maps. The algorithm looks more stable using Piecewise and Gaussian maps than others, because with them it was able to obtain the best stability in two test problems with the two problem dimensions. Other maps were able to facilitate better stability than logistic map as shown by the shaded portions in the table.
Table 4 shows the global search ability of LDIW-PSO using the various chaotic maps. The algorithm had the same search ability using all the maps for in both dimensions and performed poorly in under dimension 50. From the results, the algorithm seems to have better global search ability using Tent map than other maps, because with the map it had the best search ability in both problem dimensions for . However, the algorithm could not demonstrate the best global search ability across the test problems and dimensions using any of the chaotic maps. Sine and Cubic maps had least positive influence on the algorithm in terms of search ability. Besides , other maps had better influence on the algorithm in achieving better global search ability than logistic map as shown by the shaded portions in the table.
Table 5 shows the number of function evaluations by the algorithm, using the various chaos maps. When Intermittency map was used and the problem dimension was set to 30, the algorithm had the lowest number of function evaluations in all the test problems except in two of the test problems under dimension 50. As indicated by the shaded portions, the algorithm executed lesser number of function evaluations using other chaotic maps than logistic map.
Table 6 shows the average ranking of the performance of LDIW-PSO when each of the chaotic maps was incorporated into it to solve all the test problems. In other words, each value in the table represents the average rank of the corresponding map in comparison to others across the test problems for each problem dimension. The least value indicates that the associated map performed best, while the largest value indicates that the associated map performed worst. Generally, when the problem dimension was set to 30, the algorithm obtained the best convergence precision using Sine map and was more stable using Intermittency map. But it demonstrated the best global search ability with Logistic, Skew Tent, Sine, and cubic maps. Less computational effort was needed using Intermittency map compared with others. When the problem dimension was set to 50, the algorithm obtained the best convergence precision and was more stable when Gaussian map was used. But it demonstrated the best global search ability using Circle and Piecewise maps. Less computational effort was needed using Gaussian and Intermittency maps in comparison with others. On the average, Intermittency map performed best when the problem dimension was 30, while Gaussian map performed best when the problem dimension was 50.
5.2. Results for RIW-PSO Using the Various Chaotic Maps
Presented in Tables 7–10 are the results with respect to mean fitness value, standard deviation, success rate, and number of function evaluations as obtained in the experiments by RIW-PSO.
Table 7 shows the mean fitness values obtained by RIW-PSO using the various chaotic maps. None of the chaotic maps could make the algorithm perform the best across the test problems in both dimensions. Apart from Skew Tent, Sine, and Gaussian maps, the algorithm was able to obtain best optimal fitness for different test problems using other maps. Intermittency map looks more effective than others, because with it the algorithm was able to obtain best optimal fitness for three test problems in the two problem dimensions showing better characteristics for convergence precision. Apart from , logistic map was less effective compared with other maps as shown by the shaded portions.
Presented in Table 8 is the standard deviation (stability measure) obtained by RIW-PSO using the various chaotic maps. From the results, none of the chaotic maps could make the algorithm have the best stability across the test problems. However, the algorithm achieved the best stabilities with the various maps for different test problems across the problem dimensions, except with ICMIC, Gaussian, and Cubic maps. The algorithm looks more stable using Logistic, Tent and intermittency maps than others, because with them it was able to obtain the best stability in two test problems. Besides and with dimension 30, other maps were able to facilitate better stability than logistic map as shown by the shaded portions in the table.
Table 9 shows the global search ability of RIW-PSO using the various chaotic maps. The algorithm had the same search ability using all the maps for in both dimensions. As shown in the results, the algorithm seems to have better global search ability using Logistic, Tent, Sine and Cubic maps than other maps, because with the maps, the algorithm had the best search ability in two or more test problems. However, the it could not demonstrate the best global search ability across the test problems and dimensions using any of the chaotic maps. Apart from , , and (for dimension 30) and and (for dimension 50), other maps had better influence on the algorithm in achieving better global search ability than logistic map as shown by the shaded portions in the table.
Table 10 shows the number of function evaluations by the algorithm, using the various chaos maps. When Intermittency map was used and the problem dimension was set to 30, the algorithm had the lowest number of function evaluations in four of the test problems but in three of the test problems under dimension 50, thereby making the algorithm be more robust in terms of search ability than other maps. The shaded portions indicate lower number of function evaluations executed by the algorithm when the corresponding chaotic maps were used compared with logistic map.
Table 11 shows the average ranking of the performance of RIW-PSO when each of the chaotic maps was incorporated into it to solve all the test problems. Each value in the table represents the average rank of the corresponding map in comparison to others across the test problems for each problem dimension. The least value indicates that the associated map performed best, while the largest value indicates that the associated map is the least in performance. Generally, when the problem dimension was set to 30, the algorithm obtained the best convergence precision, demonstrated the best global search ability and stability when logistic map was used. But using Circle and Intermittency maps, less computational effort was needed compared with others. When problem dimension was set to 50, the algorithm obtained the best convergence precision using ICMIC map and it was more stable using Tent map; it did better global search using Piecewise map and required less computational time using Circle map. On the average, Intermittency map performed best when the problem dimension was 30, while Piecewise map performed best when the problem dimension = 50.
6. Conclusions
Chaotic features cause the values of inertia weight to fluctuate between 0 and 1. It affects the velocities and positions of each particle in each iteration to facilitate its local and global search ability as they move to new search regions in the search space.
In this paper, two PSO variants, LDIW-PSO and RIW-PSO algorithms, were implemented with different chaotic maps incorporated into their inertia weight strategies at different times. Their performances were investigated based on the results obtained from numerical simulations, using some well-studied benchmark problems. Mean best solution, standard deviation, success rate, and function evaluations of the algorithms were the instruments of measurement. Results show that, though logistic map could enhance the performance of LDIW-PSO and RIW-PSO, there are other chaotic maps that can make the variants perform better in terms of convergence speed, accuracy, stability, and global search ability.
In terms of average performance, LDIW-PSO performed best using the Intermittency map when the problem dimension is 30. But it performed best using Gaussian map when the problem dimension is 50. On the other hand, RIW-PSO also performed best using the Intermittency map when the problem dimension is 30. But it performed best using Piecewise map when the problem dimension is 50. This is an indication that the Intermittency chaotic map could enhance the performance of the two PSO variants compared with other maps, when the dimensionality of the problem is in the neighbourhood of 30.
However, it should be noted that due to the different search pattern of the chaotic maps, they could be problem dependent. The results presented in this paper can also serve as a platform for informative decision making by practitioners in the process of selecting chaotic maps to be used in the inertia weight formula of LDIW-PSO and RIW-PSO. Further work will be done in applying these variants to real-world problems to test their effectiveness relative to the chaotic maps.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.