Convergence Time Analysis of Particle Swarm Optimization Based on Particle Interaction

Abstract

We analyze the convergence time of particle swarm optimization (PSO) on the facet of particle interaction. We firstly introduce a statistical interpretation of social-only PSO in order to capture the essence of particle interaction, which is one of the key mechanisms of PSO. We then use the statistical model to obtain theoretical results on the convergence time. Since the theoretical analysis is conducted on the social-only model of PSO, instead of on common models in practice, to verify the validity of our results, numerical experiments are executed on benchmark functions with a regular PSO program.

1. Introduction

Particle swarm optimizer (PSO), introduced by [1, 2], is a stochastic population-based algorithm for solving continuous optimization problems. As shown by [3] and by lots of real-world applications, PSO is an efficient and effective optimization framework. Although PSO has been widely applied in many fields [4–7], understanding of PSO from the theoretical point of view is still quite limited. Most of previous theoretical results [8–18] are derived under the system that assumes a fixed attractor or a swarm consisting of a single particle.

Due to the lack of theoretical analysis on PSO particle interaction, in this paper, we will make an attempt to analyze the convergence time for PSO on the facet of particle interaction. In particular, we will firstly introduce a statistical interpretation of PSO, proposed by [19], to capture the essence of particle interaction. We will then analyze the convergence time based on the statistical model. Finally, numerical experiments will be conducted to confirm the validity of our theoretical results obtained on simplified PSO, the social-only model, in a normal PSO configuration.

In the next section, we will briefly introduce the algorithm of PSO and the statistical interpretation of social-only PSO. In Section 3, we will analyze the convergence time of PSO based on the statistical model. The experimental results are presented in Section 4, followed by Section 5 which concludes this work.

2. Particle Swarm Optimization and the Statistical Interpretation

The social-only model of PSO can be described as pseudocode shown in Algorithm 1. In this paper, we will use boldface for vectors, for example, 𝐗_𝐢, 𝐕_𝐢. Without loss of generality, we assume that the goal is to minimize the objective function.

procedure SOCIAL-ONLY PSO (Objective function ℱ∶ℝ𝑛→ℝ)    Initialize 𝑚 particles    while the stopping criterion is not satisfied do     for 𝑖=1,2,…,𝑚  do     if  ℱ(𝐗𝐢)<ℱ(𝐏𝐛𝐢)  then       𝐏𝐛𝐢←𝐗𝐢        if  ℱ(𝐏𝐛𝐢)<ℱ(𝐍𝐛)  then         𝐍𝐛←𝐏𝐛𝐢        end if     end if    end for    for  𝑖=1,2,…,𝑚  do     for  𝑗=1,2,…,𝑛  do      𝐕𝐢𝑗←𝑤𝐕𝐢𝑗+𝐶𝑛(𝐍𝐛𝑗−𝐗𝐢𝑗)        𝐗𝐢𝑗←𝐗𝐢𝑗+𝐕𝐢𝑗       end for    end for   end while end procedure

Algorithm 1 

Social-only model of PSO.

According to Algorithm 1, in the beginning, 𝑚 particles are initialized, where 𝑚 is the swarm size, an algorithmic parameter of PSO. Each particle contains three types of information: its location (𝐗_𝐢), velocity (𝐕_𝐢), and personal best position (𝐏𝐛_𝐢). At each generation, each particle updates its personal best position (𝐏𝐛_𝐢) and neighborhood best position (𝐍𝐛) according to its objective value. After updating the personal and neighborhood best positions, each particle updates the velocity according to 𝐏𝐛_𝐢 and 𝐍𝐛. In the velocity update formula, 𝑤 is the weight of inertia which is usually a constant. 𝐶_𝑝 and 𝐶_𝑛 are random values sampled from uniform distributions 𝑈(0,𝑐_𝑝) and 𝑈(0,𝑐_𝑛), where 𝑐_𝑝 and 𝑐_𝑛 are called acceleration coefficients. Finally, each particle updates its position according to the velocity and then goes to next generation.

From the aforementioned brief description, we can already see that particle interaction is a crucial mechanism in the design of PSO. Although there have been previous studies on particle interaction and the PSO behavior, most of these studies were totally based on the assumption of fixed attractors, a false condition for PSO in action. In order to take particle interaction into consideration, we use an alternative view of PSO that regards the whole swarm as a unity. Instead of tracking the movement of each particle, we consider the overall swarm behavior by transforming the state of entire swarm into a statistical abstraction. Furthermore, in order to concentrate on particle interaction, we adopt the social-only model of PSO [20], which does not consider personal best positions.

The statistical interpretation of PSO we use in this paper is modified from [19], summarized in Algorithm 2. In the statistical model, the exact particle locations are not traced but modeled as a distribution 𝜃 over ℝ^𝑛. Velocities are viewed as random vectors 𝒱∈ℝ^𝑛. The swarm size 𝑚 is considered as the number of samples from distribution 𝜃, since the geographic knowledge is embodied in the distribution, the neighborhood attractor can be viewed as the best of the 𝑚 samples.

procedure STATISTICAL INTERPRETATION OF SOCIAL-ONLY PSO (Objective Function ℱ∶ℝ𝑛→ℝ)    Initialize: 𝜎←𝜎0,𝜇←𝜇0    while the stopping criterion is not satisfied do     for  𝑖=1,2,…,𝑚  do      for  𝑗=1,2,…,𝑛  do        𝐏𝐢𝑗∼𝑁(𝜇𝑗,𝜎2𝑗)      end for     end for      𝐏𝐚=min𝐏𝐢{ℱ(𝐏𝐢)}      for  𝑖=1,2,…,𝑚  do      for  𝑗=1,2,…,𝑚  do        𝐏′𝐢𝑗←𝐏𝐢𝑗+𝐶(𝐏𝐚𝑗−𝐏𝐢𝑗)      end for      end for    𝜇←(∑𝑚𝑖=1𝐏′𝐢)/𝑚    𝜎2←MLE(𝐏′𝟏,𝐏′𝟐,…,𝐏′𝐦)   end while end procedure

Algorithm 2 

Statistical model of social-only PSO. Distribution 𝜃 is represented by 𝜇=(𝜇₁,𝜇₂,…,𝜇_𝑛) and 𝜎=(𝜎₁,𝜎₂,…,𝜎_𝑛). Acceleration coefficient 𝐶∼𝑈(0,𝑐).

Each particle 𝐏_𝐢 is considered as a random vector sampled from 𝜃, and the velocity 𝐕_𝐢 is sampled from 𝒱. The neighborhood attractor can then be defined as 𝐏_𝐚∶=min_𝐏𝐢{ℱ(𝑃₁),ℱ(𝑃₂),…,ℱ(𝑃_𝑚)}. At each generation, 𝐏_𝐢𝑗 is updated as 𝐏_𝐢𝑗+𝑤𝐕_𝐢𝑗+𝐶(𝐏_𝐚𝑗−𝐏_𝐢𝑗). The next distribution is thus the statistical characterization denoted by functions of the observed values:𝜃⟵𝒯_𝑝𝐏_𝟏,…,𝐏_𝐦,𝒱⟵𝒯_𝑣𝐏_𝟏,…,𝐏_𝐦,𝐕_𝟏,…,𝐕_𝐦. (1) Since 𝑤 is a constant, distribution 𝒱 can be removed because given two random vectors 𝐗∼𝜃 and 𝐕∼𝒱, we can simply let 𝜃′ be the distribution of  𝐗′∶=𝐗+𝑤𝐕.

For simplicity, in this paper, we consider the positions of each dimension of a particle is independently sampled from distribution 𝜃_𝑖. Consider the random variable 𝑋∼𝜃_𝑖 and let 𝐸[𝑋]=𝜇. If we divide the support of 𝜃_𝑖 into 𝑠 disjoint regions 𝑅₁,…,𝑅_𝑠, such that Prob[𝑋∈𝑅_𝑖]=1/𝑠 for 𝑖=1,2,…,𝑠, and each region is associated with a random variable of velocity 𝑉_𝑖∼𝒱_𝑖. By picking 𝑥_𝑖∈𝑅_𝑖 for each region, when 𝑠 is sufficiently large, the swarm can be characterized as^𝑠_𝑖=11𝑠𝑥_𝑖+𝑉_𝑖=^𝑠_𝑖=1𝑥_𝑖𝑠+^𝑠_𝑖=1𝑉_𝑖𝑠≈𝜇+^𝑠_𝑖=1𝑉_𝑖𝑠. (2) Each component of ∑^𝑠_𝑖=1𝑉_𝑖/𝑠 can be approximated with a normal distribution by the central limit theorem. As a consequence, normal distributions are a reasonable choice for describing the behavior of the entire swarm. We let the distribution of 𝑖th dimension, 𝜃_𝑖, be 𝑁(𝜇_𝑖,𝜎²_𝑖), where 𝑁(𝜇_𝑖,𝜎²_𝑖) is the normal distribution with mean 𝜇_𝑖 and variance 𝜎²_𝑖. The update of distribution becomes simply by calculating the mean and the variance.

The mean can be calculated by taking the average of updated positions, and the variance is calculated by using the maximum likelihood estimation (MLE). Let 𝜎_𝐭²_𝑖 and 𝜇_𝐭²_𝑖 be the variance and mean of the 𝑖th dimension at the 𝑡th generation. Let 𝑦_𝑗=𝐏_𝐣𝑖 and 𝑦^′_𝑗=𝐏^′_𝐣𝑖 for 𝑗=1,2,…,𝑚 and let 𝑦_𝑎=𝐏_𝐚𝑖, 𝑦=(1/𝑚)∑^𝑚_𝑗=1𝑦^′_𝑗. To estimate the variance of the 𝑖th dimension at the (𝑡+1)th generation, we use the maximum likelihood estimation (MLE). The likelihood function 𝐿(𝜎_𝐭²_𝑖) is defined as the joint probability:𝐿𝜎_𝐭²_𝑖∶=^𝑚_𝑗=11√2𝜋𝜎²_{𝑖𝐭+𝟏}−𝑦exp^′𝑗−𝑦²2𝜎²_{𝑖𝐭+𝟏}=1√2𝜋𝜎²_{𝑖𝐭+𝟏}^𝑚exp−∑^𝑚_𝑗=1𝑦^′_𝑗−𝑦²2𝜎²_{𝑖𝐭+𝟏}. (3) To find 𝜎²_{𝑖𝐭+𝟏} that maximizes 𝐿(𝜎_𝐭²_𝑖), we differentiate 𝐿(𝜎_𝐭²_𝑖) with respect to 𝜎²_{𝑖𝐭+𝟏}:𝐿^′𝜎_𝐭²_𝑖𝑚=−21√2𝜋^𝑚𝜎_{𝑖𝐭+𝟏}^−𝑚−2−∑×exp^𝑚𝑗=1𝑦^′_𝑗−𝑦²2𝜎²_{𝑖𝐭+𝟏}+1√2𝜋^𝑚∑^𝑚_𝑗=1𝑦^′_𝑗−𝑦²2𝜎_{𝑖𝐭+𝟏}^−𝑚−4×exp−∑^𝑚_𝑗=1𝑦^′_𝑗−𝑦²2𝜎²_{𝑖𝐭+𝟏}, (4) the value of 𝜎²_{𝑖𝐭+𝟏} that maximizes 𝐿(𝜎_𝐭²_𝑖) is ∑^𝑚_𝑗=1(𝑦^′_𝑗−𝑦)²/𝑚. As a result, in our model of PSO, the results of MLE is 𝜎²_{𝑖𝐭+𝟏}=∑^𝑚_𝑗=1(𝑦^′_𝑗−𝑦)²/𝑚 for 𝑖=1,2,…,𝑛.

3. Convergence Time Analysis

In this section, we will analyze the PSO convergence time based on the aforementioned statistical interpretation of the social-only model. As the first step, we must define the state of convergence. Since, in this work, we regard the entire swarm as a distribution, the state of convergence is then referred to as the variance of the distribution. We define the state of convergence as the variance for every dimension is less than a given value 𝜖>0. By using this definition, we can now start our analysis of PSO convergence time. To estimate the variance after distribution update, we need the following lemma from [21].

Lemma 1. Let 𝑋₁,𝑋₂,…,𝑋_𝑚∼𝑁(𝜇,𝜎²). Define 𝑆=∑^𝑚_𝑖=1(𝑋_𝑖−𝑋)²/(𝑚−1), where 𝑋=∑^𝑚_𝑖=1𝑋_𝑖/𝑚. One has (𝑚−1)𝑆∼𝜎²𝜒²_𝑚−1, where 𝜒²_𝑚−1 is the chi-square distribution with 𝑚−1 degrees of freedom.

With this lemma, we can obtain the following.

Lemma 2. Given the swarm size 𝑚, acceleration coefficient 𝑐, and variance of the 𝑖th dimension at the 𝑡th generation 𝜎_𝐭²_𝑖, one has 𝐸[𝜎²_{𝑖𝐭+𝟏}]=[(1/3)𝑐²−𝑐+1][(𝑚−1)/𝑚]𝜎_𝐭²_𝑖.

Proof. We know 𝜎²_{𝑖𝐭+𝟏}=∑^𝑚_𝑗=1(𝑦^′_𝑗−𝑦)²/𝑚. The expected value is 𝐸1𝑚^𝑚_𝑗=1𝑦^′_𝑗−∑^𝑚_𝑘=1𝑦^′_𝑘𝑚²=1𝑚𝐸^𝑚_𝑗=1𝑦_𝑗+𝐶𝑦_𝑎−𝑦_𝑗−∑^𝑚_𝑘=1𝑦_𝑘+𝐶𝑦_𝑎−𝑦_𝑘𝑚²=1𝑚𝐸^𝑚_𝑗=1𝑚(1−𝐶)𝑦_𝑗−(1−𝐶)∑^𝑚_𝑘=1𝑦_𝑘𝑚²=1𝑚³𝐸(1−𝐶)^2𝑚_𝑗=1𝑚𝑦_𝑗−^𝑚_𝑘=1𝑦_𝑘²=1𝑚𝐸(1−𝐶)²𝐸^𝑚_𝑗=1𝑦_𝑗−1𝑚^𝑚_𝑘=1𝑦_𝑘². (5) Let 𝑆=∑^𝑚_𝑗=1(𝑦_𝑗−(1/𝑚)∑^𝑚_𝑘=1𝑦_𝑘)². Since 𝑦_𝑗∼𝑁(𝜇_{𝑖𝐭+𝟏},𝜎²_{𝑖𝐭+𝟏}) for 𝑗=1,2,…,𝑚 and 𝑦₁,𝑦₂,…,𝑦_𝑚 are i.i.d., by Lemma 1, 𝑆∼𝜎_𝐭²_𝑖𝜒²_𝑚−1, and 𝐸[𝑆]=(𝑚−1)𝜎_𝐭²_𝑖. Then, we can obtain 𝐸𝜎²_{𝑖𝐭+𝟏}=1𝑚𝐸(1−𝐶)²𝐸^𝑚_𝑗=1𝑦_𝑗−1𝑚^𝑚_𝑘=1𝑦_𝑘²=1𝑚𝐸1−2𝐶+𝐶²𝐸[𝑆]=1𝑚13𝑐²−𝑐+1(𝑚−1)𝜎_𝐭²_𝑖=13𝑐²−𝑐+1𝑚−1𝑚𝜎_𝐭²_𝑖. (6)

Lemma 2 is derived under the condition that 𝜎_𝐭²_𝑖 is given. The following lemma will derive the relationship between 𝐸[𝜎_𝐭²_𝑖] and 𝐸[𝜎²_{𝑖𝐭+𝟏}].

Lemma 3. 𝐸[𝜎²_{𝑖𝐭+𝟏}]=((1/3)𝑐²−𝑐+1)((𝑚−1)/𝑚)𝐸[𝜎_𝐭²_𝑖].

Proof. 𝐸𝜎_𝐭²_𝑖=_𝜎2∈ℝ+𝐸𝜎²_{𝑖𝐭+𝟏}∣𝜎_𝐭²_𝑖=𝜎²Prob𝜎_𝐭²_𝑖=𝜎²𝑑𝜎²=_𝜎2∈ℝ+13𝑐²−𝑐+1𝑚−1𝑚𝜎²Prob𝜎_𝐭²_𝑖=𝜎²𝑑𝜎²=1byLemma23𝑐²−𝑐+1𝑚−1𝑚_𝜎²_∈ℝ⁺𝜎²Prob𝜎_𝐭²_𝑖=𝜎²𝑑𝜎²=13𝑐²−𝑐+1𝑚−1𝑚𝐸𝜎_𝐭²_𝑖. (7)

Now, we can obtain the relationship of convergence time and algorithmic parameters of PSO.

Theorem 4. Given swarm size 𝑚, acceleration coefficient 𝑐, 𝜖, and 𝜎_𝟎. Let ℎ=max_𝑖{𝜎_𝟎²_𝑖}. One has 𝐸[𝜎_𝐭²_𝑖]<𝜖 for 𝑖=1,2,…,𝑛 when [(1/3)𝑐²−𝑐+1][(𝑚−1)/𝑚]<1 and 𝑡>𝑙𝑜𝑔(𝜖/𝜎₀²_ℎ)/𝑙𝑜𝑔([(1/3)𝑐²−𝑐+1][(𝑚−1)/𝑚]).

Proof. From Lemma 3, we know 𝐸𝜎_𝐭²_𝑖=13𝑐²−𝑐+1^𝑡𝑚−1𝑚^𝑡𝐸𝜎_𝟎²_𝑖=13𝑐²−𝑐+1^𝑡𝑚−1𝑚^𝑡𝜎_𝟎²_𝑖. (8) Since [(1/3)𝑐²−𝑐+1][(𝑚−1)/𝑚]<1, we have 𝐸𝜎_𝐭²_𝑖=13𝑐²−𝑐+1^𝑡𝑚−1𝑚^𝑡𝜎_𝟎²_𝑖<13𝑐²−𝑐+1^𝑡𝑚−1𝑚^𝑡𝜎_𝟎²_ℎ<𝜖. (9) The last inequality holds because 1log3𝑐²−𝑐+1^𝑡𝑚−1𝑚^𝑡𝜎_𝟎²_ℎ1=𝑡log3𝑐²−𝑐+1𝑚−1𝑚+log𝜎_𝟎²_ℎ𝜖<𝑙𝑜𝑔𝜎_𝟎²_ℎ+log𝜎_𝟎²_ℎ=log𝜖. (10)

We have two corollaries immediately from Theorem 4.

Corollary 5. Given swarm size 𝑚, acceleration coefficient 𝑐, and level of convergence 𝜖 such that [(1/3)𝑐²−𝑐+1][(𝑚−1)/𝑚]<1 and 𝜖<1, one has 𝐸[𝜎_𝐭²_𝑖]<𝜖 for 𝑖=1,2,…,𝑛 for 𝑡=𝑂(−log𝜖).

Corollary 6. Given swarm size, 𝑚, 𝑐, and 𝜖 such that [(1/3)𝑐²−𝑐+1][(𝑚−1)/𝑚]<1 and 𝜖<1, there exists a constant 𝑐^′<1 such that for 𝑡=𝑂(−1/log𝑐^′(1−1/𝑚)), one has 𝐸[𝜎_𝐭²_𝑖]<𝜖 for 𝑖=1,2,…,𝑛.

Corollary 5 reveals the linear relationship between the level of convergence and the convergence time, and the interpretation of Corollary 6 is that when the swarm size is sufficiently large, the effect of enlarging swarm size on the convergence time is not important. In the next section, we will empirically examine the two corollaries with a common practical PSO configuration.

4. Experiments

In this section, we verify the validity of Corollaries 5 and 6 by running standard PSO 2006 downloaded from Particle Swarm Central. We use two objective functions in our experiments:(i)sphere function [22]:𝑓₁(𝐱)=^𝐷_𝑖=1𝑥²_𝑖,𝐱∈[−100,100]^𝐷, (11)(ii)schwefel’s problem 1.2 [22]:𝑓₂(𝐱)=^𝐷_𝑖=1^𝑖_𝑗=1𝑥²_𝑗,𝐱∈[−100,100]^𝐷. (12) We have 𝐷=10 for both 𝑓₁(𝐱) and 𝑓₂(𝐱) in the following experiments.

We firstly examine Corollary 5. The PSO algorithmic parameters are given as 𝑐_𝑝=1, 𝑐_𝑛=1, 𝑤=1/(2ln2), and swarm size = 50. The value of 𝜖 is varied from 10⁻¹ to 10⁻¹⁰. For each value of 𝜖, we perform 100 independent runs. For each run, we count the number of generations from initialization to the state in which variances for all dimensions are smaller than 𝜖, and we calculate the mean number of generations for the 100 runs.

The comparison of these experimental results and our theoretical results is shown in Figures 1 and 2. From Figure 1, we can see that the experimental results of 𝑓₁(𝐱) are very close to −4.6log𝜖+43=𝑂(−log𝜖), and from Figure 2, the experimental results of 𝑓₂(𝐱) are very close to −4.7log𝜖+43.5=𝑂(−log𝜖). The experimental results agree with our estimation in Corollary 5, in which the value of −ln𝜖 and the PSO convergence time are linearly related.

Figure 1 

Comparison of experimental results and theoretical results from Corollary 5 of 𝑓1(𝐱). The 𝑥-axis represents the value of 𝜖, and 𝑦-axis represents the mean number of generation. The experimental results are very close to 𝒪(−log𝜖).

Figure 2 

Comparison of experimental results and theoretical results from Corollary 5 of 𝑓2(𝐱). The 𝑥-axis represents the value of 𝜖, and 𝑦-axis represents the mean number of generation. The experimental results are very close to 𝒪(−log𝜖).

After Corollary 5 is empirically verified with the standard PSO, we now examine Corollary 6. The parameters we used in PSO are given as 𝑐_𝑝=1, 𝑐_𝑛=1, 𝑤=1/(2ln2), and 𝜖=10⁻⁶. The swarm size ranges from 50 to 1000 with step 5. For each swarm size, we perform 100 independent runs and record the mean as we did in last experiment.

The comparison of experimental and theoretical results is shown in Figures 3, 4, 5, and 6. From Figures 3 and 4, we can see that the convergence time is close to −64.9/log0.555(1−1/𝑚)=𝒪(−1/log𝑐′(1−1/𝑚)), where 𝑐′=0.555, and in Figures 5 and 6, the convergence time is close to −99.75/log0.405(1−1/𝑚)=𝒪(−1/log𝑐′(1−1/𝑚)), where 𝑐′=0.405. As we can observe from these figures, when the swarm size becomes large, the increase of convergence time is insignificant, confirming our estimation in Corollary 6.

Figure 3 

Comparison of experimental results and theoretical results from Corollary 6 of 𝑓1(𝐱). 𝑥-axis represents the swarm size ranging from 50 to 200, and 𝑦-axis represents the mean number of generation. The experimental results are very close to 𝒪(−1/log𝑐′(1−1/𝑚)) with 𝑐′<1.

Figure 4 

Comparison of experimental results and theoretical results from Corollary 6 of 𝑓1(𝐱). 𝑥-axis represents the swarm size ranging from 50 to 1000, and 𝑦-axis represents the mean number of generation. The experimental results are very close to 𝒪(−1/log𝑐′(1−1/𝑚)) with 𝑐′<1.

Figure 5 

Comparison of experimental results and theoretical results from Corollary 6 of 𝑓2(𝐱). 𝑥-axis represents the swarm size ranging from 50 to 200, and 𝑦-axis represents the mean number of generation. The experimental results are very close to 𝒪(−1/log𝑐′(1−1/𝑚)) with 𝑐′<1.

Figure 6 

Comparison of experimental results and theoretical results from Corollary 6 of 𝑓2(𝐱). 𝑥-axis represents the swarm size ranging from 50 to 1000, and 𝑦-axis represents the mean number of generation. The experimental results are very close to 𝒪(−1/log𝑐′(1−1/𝑚)) with 𝑐′<1.

5. Conclusions

In this paper, a statistical interpretation of a simplified model of PSO was adopted to analyze the PSO convergence time. In order to capture the essence of particle interaction, the statistical model adopted in this paper assumed no fixed attractors. The effect of particle interaction was included in our analysis. Our theoretical results revealed the relationship between the convergence time and the level of convergence as well as the relationship between the convergence time and the swarm size. Numerical results, in the standard settings of PSO, were obtained to empirically verify our theoretical results derived with a simplified PSO configuration. The agreement between the experimental and theoretical results indicated the importance of particle interaction in PSO. Consequently, more research effort should be invested into analyzing the working of particle interaction in order to better understand particle swarm optimization.

Some future extensions of this study are now ready to be explored. First of all, the relationship between PSO and the number of dimensions, that is, in the adopted model, the relationship between 𝑡 and 𝐸[𝜎_𝐭²_(𝑛)], where 𝜎_𝐭²_(𝑛)=max{𝜎_𝐭²₁,𝜎_𝐭²₂,…,𝜎_𝐭²_𝑛}. Second, the theoretical analysis conducted in this study is independent of objective functions. In Section 4, we verify the analysis with only two objective functions. More functions of various features and properties, which do not violate the settings of the adopted statistical model, should be used to examine the estimation. Third, the distribution which we used in this paper is the normal distribution. However, there should exist some objective functions that enforce the swarm to distribute according to different distributions. The analysis presented in this paper will fail on those objective functions. As a result, more sophisticated models should be adopted to provide good descriptions of the PSO macrobehavior and to enable researchers to derive more accurate PSO estimations. Finally, the social-only PSO model we adopted in this paper does not take the personal experience into consideration. We also need more sophisticated models to analyze the PSO macrobehavior influenced by the personal experience.

Acknowledgment

The work was supported in part by the National Science Council of Taiwan under Grant NSC 99-2221-E-009-123-MY2.