Convergence Analysis of a Class of Computational Intelligence Approaches

Abstract

Computational intelligence approaches is a relatively new interdisciplinary field of research with many promising application areas. Although the computational intelligence approaches have gained huge popularity, it is difficult to analyze the convergence. In this paper, a computational model is built up for a class of computational intelligence approaches represented by the canonical forms of generic algorithms, ant colony optimization, and particle swarm optimization in order to describe the common features of these algorithms. And then, two quantification indices, that is, the variation rate and the progress rate, are defined, respectively, to indicate the variety and the optimality of the solution sets generated in the search process of the model. Moreover, we give four types of probabilistic convergence for the solution set updating sequences, and their relations are discussed. Finally, the sufficient conditions are derived for the almost sure weak convergence and the almost sure strong convergence of the model by introducing the martingale theory into the Markov chain analysis.

1. Introduction

Computational intelligence approaches (CIAs) are a set of nature-inspired computational methodologies to interpret and construct the processes of optimization problems solving from different biological points of view, based on which a large number of derived approaches have been developed [1]. An example of particularly successful research direction in CIAs is genetic algorithm (GA) [2] which generates solutions to optimization problems using techniques inspired by natural evolution, such as selection, crossover, and mutation. Differential evolution (DE) [3, 4], proposed by Storn and Price, is similar to GA in the sense that it uses same evolutionary operators like mutation, crossover, and selection for guiding the population towards the optimum solution. Ant colony optimization (ACO) [5] is another very popular CIA, which was initially proposed by Marco Dorigo in his Ph.D. thesis. The first ACO was aiming to search for an optimal path in a graph, based on the behavior of ants seeking a path between their colony and a source of food. The original idea has since diversified to solve a wider class of numerical problems, and as a result, several problems have emerged, drawing on various aspects of the behavior of ants. Particle swarm optimization (PSO) [6], introduced in 1995, was intended for simulating birds flocking or fishes schooling. PSO optimizes a problem by having a population of particles and moving these particles around in the search space according to mathematical formulae over the particle's position and velocity. Innumerable variants and hybrids of these techniques have been proposed, and many more applications of CIAs to specific problems have been reported [7–9]. In the basic methodology, however, these algorithms have some features in common: (i) looking for global solutions of optimization problems by means of random search with heuristic information; (ii) promoting problem solving efficiency by multipoint search strategies; and (iii) synthesizing the information involved in a number of currently obtained candidate solutions to produce subsequent search decision. Due to the similarity, these algorithms possess several common advantages such as the robustness against the faint mathematical descriptions and the odd mathematical properties of optimization problems, as well as the capability to attain global optima. But they also share some shortcomings. One of the most well-known disadvantages is that the algorithm parameters are determined by experience rather than theoretical guidance. Therefore, when the algorithm parameters are improperly set up, the search process is apt to sink into a stagnant status; that is, no any better candidate solutions that are superior to their parents will be yielded (or the candidate solutions converged too early, resulting in being suboptimal).

A number of theoretical studies have been conducted in order to seek the seeds of the stagnation problems. Rudolph [10] analyzed the convergence properties of the canonical GA (CGA) and proved that in the sense of probability, CGA does not converge to global optima for any selection of initial population, crossover operator, and objective function, but some variants of CGA with an elite reserve policy are shown to converge in probability. Xu et al. [11] considered a class of GA with the strategy in which parental populations are put into competition with their offspring. It is shown that a variant of CGA with such a strategy can reach global optima after a finite number of iterations in probability one. Rastegar [12] gave the lower boundaries of the probability in which the compact genetic algorithm (cGA) converges to global optima and proved that the algorithm converges almost surely to the global optima of a special class of functions as the learning rate is sufficiently small. In the studies on ACO, Gütjahr [13, 14] first proved the convergence of the graph-based ant system by means of the graph theory. It is shown that under certain conditions, the candidate solutions generated in each iteration converge to the global solution of a given problem instance in a probability that can be made arbitrarily close to one. Stützle and Dorigo [15] and Badr and Fahmy [16] studied the convergence properties of ACO with the aid of the algebraic theory, but their work was incomplete due to too many hypotheses made in their proofs. Duan et al. [17] regarded the pheromone trail vector in ACO as a Markov chain and presented a new approach for analyzing the almost sure convergence properties of ACO based on the martingale theory. However, the pheromone trail vector is not self-contained if the path information is unknown. Thus further discussions are needed for Duan’s conclusions. The convergence of PSO is also an issue of concern in the work on CIAs. Clerc and Kennedy [18] analyzed a sole particle’s trajectory in simplified PSO systems and led their analysis to a generalized algorithm model in which a set of coefficients are used to control the system convergence. Based on Clerc’s analysis, the dynamic behavior and the convergence of the simplified (deterministic) PSO algorithm were analyzed by Trelea [19] with the aid of tools from the discrete-time dynamic system theory. Van den Bergh and Engelbrecht [20] investigated the influence of the inertia factor on particle trajectories in PSO and provided a formal proof that each particle converges to a stable point. Poli [21] presented a method that allowed one to exactly determine the sampling distribution of PSO and explained how it changes over time during stagnation for a large class of PSO variants. Liu et al. [22] proved that PSO converges to global optima in probability one based on an iterative function model and the probabilistic theory. Chen and Jiang [23] analyzed the behavior of PSO on the facet of particle interaction and proposed a statistical interpretation of PSO in order to capture the stochastic behavior of the entire swarm. Based on a social-only model, they derived the upper and lower boundaries of the expected particle norm.

The basic methodology in the aforementioned studies is that the convergence properties are first investigated from a specific implementation of GA, ACO, or PSO and then to be extended to a wider area of the corresponding algorithm. In this paper, the authors attempt to analyze the general convergence properties of these algorithms on the basis of their common features in methodology. The main attention is paid to the characteristics of heuristic search strategies which play crucial roles in problem solving. The purpose of this paper is to have a deeper understanding on the essence of the convergence of the CIAs and, based on which, to develop more efficient algorithms for complex problems optimization.

The rest of this paper is organized as follows. In Section 2, a computational model of a class of CIAs is built up on the basis of several common features of the heuristic search strategies adopted by GA, ACO, and PSO, and a set of definitions are given to describe the external and internal changes of a solution set updating sequence in CIAs. In Section 3, the concepts of convergence in probability and almost sure convergence are defined for the solution set updating sequences generated in the search process of the computational model, and their relations are discussed. In Section 4, the conditions under which the computational model converges almost surely to a space of weakly optimal solution sets are studied based on the Markov chain properties of the solution set updating sequences. Then the almost sure convergence to a strongly optimal solution space is proved for the computational model with the aid of the martingale theory. Several conclusive remarks are given in Section 5.

2. The CIA Model

In this section, the characteristics of solution sets representation and solution sets update in CIAs are analyzed, and based on that, the computational model of a class of the CIAs is established. Moreover, variation rate of set component and progress rate of solution set are defined, respectively, to describe the variety and the optimality of the solution sets generated in the search process of the model.

Without loss of generality, consider the optimization problem of the following type: max_𝑋𝑓(𝑋)∣𝑋∈𝐒, (1) where 𝐒 is usually called a search (or solution) space, and 𝑓(𝑋) is an evaluation function satisfying 0<𝑓(𝑋)<∞.

A problem-solving procedure observing the following principles is stipulated as the computational model of a class of the CIAs (“the CIA model” for short) under study in this paper.

Step 1. Transform the feasible space 𝐒 into the coded space of variables.

Step 2. Generate a certain number of candidate solutions to composite an initial solution set.

Step 3. Produce a new generation of solution set based on the heuristic information involved in the current solution set and corresponding evaluation values, and set the size of the solution set fixed.

Step 4. Stop the iterations and output the best candidate solution within the current solution set if a predefined termination criterion is satisfied; otherwise go back to Step 3.

Obviously, the canonical forms of GA, ACO, and PSO and most of their derived algorithms obey the aforementioned computational rules, thus can be regarded as the realization instances of the CIA model.

We consider a set of candidate solutions (“solution set” for short) 𝐗={𝑋₁,𝑋₂,…,𝑋_𝑁}∈𝐒^𝑁 composed by 𝑁 strings of length 𝐿 in CIAs, where 𝐒 is the search space of candidate solutions, and 𝐒^𝑁=𝐒×𝐒×⋯×𝐒_𝑁 is the search space of solution sets of 𝑁th order. A candidate solution is composed of 𝐿 bit numbers as its components (“solution components” for short). The candidate solution and its component have the special meaning in different CIAs. Table 1 illustrates the expressions of solution candidate and solution component in GA, ACO, and PSO. Therefore, a solution set 𝐗 generated by CIAs can be written in the following vector and matrix forms: 𝑋𝐗=¹𝑋₂⋮𝑋_𝑁=𝑥₁₁𝑥₁₂⋯𝑥_1𝐿𝑥₂₁𝑥₂₂⋯𝑥_2𝐿𝑥⋮⋮⋮⋮_𝑁1𝑥_𝑁2⋯𝑥_𝑁𝐿=𝐶₁,𝐶₂,…,𝐶_𝐿, (2) where 𝑋_𝑖=(𝑥_𝑖1,𝑥_𝑖2,…,𝑥_𝑖𝐿)∈𝐒 is the 𝑖th candidate solution in 𝐗, and 𝑥_𝑖𝑗 is the 𝑗th solution component of the 𝑖th candidate solution, 𝑖=1,2,…,𝑁,  𝑗=1,2,…,𝐿. For the sake of convenience, the 𝑗th vector which consists of the 𝑗th components of all the candidate solutions in 𝐗 is called the 𝑗th set component, written as 𝐶_𝑗=(𝑥_1𝑗,𝑥_2𝑗,…,𝑥_𝑁𝑗)^𝑇.

Table 1 

The expressions of solution candidate and solution component.

As we know, a new generation of solution set in the CIA model generally relies on the heuristic information involved in the current solution set rather than the previous solution sets. Thereby, a state transition sequence of solution set can be expressed as a Markov chain, denoted by {𝐗(𝑡),𝑡≥0} [24]. In essence, the search procedure of the CIA model can be regarded as the updating process of solution sets expressed by 𝐗(𝑡+1)=𝐻(𝐗(𝑡)), (3) where 𝐻(⋅) is the solution set updating operator or in other words, the heuristic search strategy adopted by the CIA model. It is instantiated by the evolutionary operations composed of selection, crossover, and mutation in GA; by the path search guided by pheromone updating and path selection in ACO; and by the particle motion adjusted by velocity and position updating in PSO.

It is well known that “elite reserve” is a commonly used strategy in many realisation instances of the CIA model. In this strategy, the best candidate solution of the previous generation is always retained in the current generation, so the best candidate solution obtained so far will not be missed in the future search process. We assume that an elitist strategy is always adopted by the CIA model.

It is reasonable to estimate the evolution trend of a solution set by the change of its solution components. In order to quantify the variation of solution components, the solution set 𝐗 is first pretreated as a canonical form in the following way (see Figure 1). First of all, according to the element values of set component 𝐶₁, solution set 𝐗 can be divided into two parts: 𝑋⁰₁=𝑋_𝑖satisfying𝑥_𝑖1>∑^𝑁_𝑖=1𝑥_𝑖1𝑁{,𝑋,∀𝑖∈1,2,…𝑁}₁₁=𝑋_𝑖satisfying𝑥_𝑖1≤∑^𝑁_𝑖=1𝑥_𝑖1𝑁,∀𝑖∈{1,2,…𝑁}, (4) where 𝑋⁰₁∪𝑋¹₁=𝐗 and 𝑋⁰₁∩𝑋¹₁=∅.

Figure 1 

The sorting process of the solution candidates in 𝐗.

Then, depending upon the element values of set component 𝐶₂, 𝑋⁰₁ and 𝑋¹₁ can be further decomposed into two parts, respectively, denoted by 𝑋⁰₁𝑋⁰₂=𝑋_𝑖satisfying𝑥_𝑖1>∑^𝑁_𝑖=1𝑥_𝑖1𝑁,𝑥_𝑖2>∑^𝑁_𝑖=1𝑥_𝑖2𝑁{,𝑋,∀𝑖∈1,2,…𝑁}₀₁𝑋¹₂=𝑋_𝑖satisfying𝑥_𝑖1>∑^𝑁_𝑖=1𝑥_𝑖1𝑁,𝑥_𝑖2≤∑^𝑁_𝑖=1𝑥_𝑖2𝑁{,𝑋,∀𝑖∈1,2,…𝑁}₁₁𝑋⁰₂=𝑋_𝑖satisfying𝑥_𝑖1≤∑^𝑁_𝑖=1𝑥_𝑖1𝑁,𝑥_𝑖2>∑^𝑁_𝑖=1𝑥_𝑖2𝑁{,𝑋,∀𝑖∈1,2,…𝑁}₁₁𝑋¹₂=𝑋_𝑖satisfying𝑥_𝑖1≤∑^𝑁_𝑖=1𝑥_𝑖1𝑁,𝑥_𝑖2≤∑^𝑁_𝑖=1𝑥_𝑖2𝑁,∀𝑖∈{1,2,…𝑁}, (5) where 𝑋⁰₁𝑋⁰₂∪𝑋⁰₁𝑋¹₂=𝑋⁰₁,  𝑋⁰₁𝑋⁰₂∩𝑋⁰₁𝑋¹₂=∅,  𝑋¹₁𝑋⁰₂∪𝑋¹₁𝑋¹₂=𝑋¹₁, and 𝑋¹₁𝑋⁰₂∩𝑋¹₁𝑋¹₂=∅. The process continues until the last set component 𝐶_𝐿 is treated.

Let 𝑇(𝑡) be a descriptive function of the set component variation from the (𝑡-1)th generation to the 𝑡th generation of solution set based on the canonical form, defined by 𝑇(𝑡)=𝑇₁(𝑡),𝑇₂(𝑡),…,𝑇_𝐿(𝑡)=𝐗(𝑡)⊕𝐗(𝑡−1), (6) where the 𝑗th element of 𝑇(𝑡), 𝑇_𝑗(𝑡)=[𝑥_1𝑗(𝑡)⊕𝑥_1𝑗(𝑡−1),…,𝑥_𝑁𝑗(𝑡)⊕𝑥_𝑁𝑗(𝑡−1)]^𝑇. The sign “⊕” expresses the operator which is very similar to the logical exclusive-OR; that is, the output of two compared values is 0 only if both 𝑥_𝑖𝑗(𝑡−1) and 𝑥_𝑖𝑗(𝑡) are the same. Equation (6) implies that an element of 𝑇(𝑡) is 1 if the corresponding components of 𝐗(𝑡) and 𝐗(𝑡−1) have different values; otherwise the element is 0.

Based on the descriptive function 𝑇(𝑡) we can define an index to measure the evolution trend of a solution set quantitatively.

Definition 1 (variation rate of set component). Let {𝐗(𝑡),𝑡≥0} be the solution set updating sequence generated by the CIA model, and let ∑𝑇_𝑗 be the sum of all the element values in 𝑇_𝑗. We call 𝑝_𝑗(𝑡)=∑𝑇_𝑗(𝑡)/𝑁 the variation rate of the 𝑗th set component of solution set 𝐗 evolving from 𝐗(𝑡−1) to 𝐗(𝑡).

From Definition 1 we know that in view of the fact that 0≤∑𝑇_𝑗(𝑡)≤𝑁,  0≤𝑝_𝑗(𝑡)≤1, for all 𝑡. The variation rate of a set component measures the variation of the set component by calculating the percentage of its elements which change their values from the previous generation to the current generation. Thus, if 𝑝_𝑗(𝑡)=0, then all the elements in set component 𝐶_𝑗 keep unchanged. Otherwise, if 𝑝_𝑗(𝑡)=1, it implies that all the values in 𝐶_𝑗 have changed from 𝐗(𝑡−1) to 𝐗(𝑡).

Without loss of generality, it is assumed that the objective function 𝑓(⋅) of the optimization problem (1) is used as the evaluation function in the CIA model. In order to quantitatively estimate the optimality of the candidate solutions in a solution set, it is useful to introduce the concept of progress rate of solution set as follows.

Definition 2 (progress rate of solution set). Let {𝐗(𝑡),𝑡≥0} be the solution set updating sequence generated by the CIA model. Let 𝑓(𝐗(𝑡)) denote the mean evaluation value of the candidate solutions in solution set 𝐗(𝑡), calculated by 𝑓(𝐗(𝑡))=∑^𝑁_𝑖=1𝑓(𝑋_𝑖(𝑡))/𝑁; and 𝑓(𝐗(𝑡)) the maximal evaluation value of the candidate solutions in 𝐗(𝑡), calculated by 𝑓(𝐗(𝑡))=max_{𝑖=1,2,…,𝑁}𝑓(𝑋_𝑖(𝑡)). Then 𝛼(𝑡)=𝑓(𝐗(𝑡))/𝑓(𝐗(𝑡)) is called the progress rate of solution set 𝐗 at the 𝑡th generation.

From Definition 2, we know that 0<𝛼(𝑡)≤1. 𝛼(𝑡) approaches 0 if most of the candidate solutions are far worse than the best candidate solution in 𝐗(𝑡). On the other hand, 𝛼(𝑡) approaches 1 if the evaluation values of most of the candidate solutions in 𝐗(𝑡) are very close to the evaluation value of the best candidate solution.

3. The Basic Concepts on the Convergence of the CIA Model

In this section, we first present two different notions of convergence of random variables in probability theory. Then, these concepts are introduced into the solution set updating sequence in the CIA model. Finally, the relationships among the different types of convergence are discussed.

There exist several different notions of convergence of random variables, and the convergence of sequences of random variables to some limit random variable is an important concept in probability theory. We assume that {𝛿_𝑛,𝑛=1,2,…} is a sequence of random variables, and 𝜹 is a random variable.

Definition 3 (convergence in probability [25]). A sequence {𝛿_𝑛,𝑛=1,2,…}of random variables converges in probability towards 𝜹if for all 𝜀>0, lim_𝑛→∞𝑃𝛿_𝑛−𝜹≥𝜀=0. (7)

Definition 4 (almost sure convergence [25]). A sequence {𝛿_𝑛,𝑛=1,2,…} of random variables converges almost surely or almost everywhere towards 𝜹 means that 𝑃lim_𝑛→∞𝛿_𝑛=𝜹=1. (8)
This means that the values of 𝛿_𝑛 approach the value of 𝜹, in the sense that events for which 𝛿_𝑛 does not converge to 𝜹 have probability 0.
Almost sure convergence implies convergence in probability. Convergence in probability is the type of convergence established by the weak law of large numbers, while almost sure convergence is the type of convergence established by the strong law of large numbers.
The definition of convergence may be extended from random variables to more complex random elements in CIAs. Before giving the definitions of the convergence of the CIAs, we explain two different concepts of the space of optimal solution sets.
Let the global optimal value of the evaluation function 𝑓(𝑋) be denoted by 𝑓_max=max_𝑋∈𝐒𝑓(𝑋), and let the set of the global optimal solutions of problem (1) be represented as 𝑂𝐶={𝑋∈𝐒∣forall𝑌∈𝐒,𝑓(𝑋)≥𝑓(𝑌)}. Obviously, for all 𝑋∈𝑂𝐶, we have 𝑓(𝑋)=𝑓_max.
The solution set 𝐗∈𝐒^𝑁 satisfying 𝑓(𝐗)=𝑓_max is called the weakly optimal solution set. All the weakly optimal solution sets produced by the CIA model in its search process constitute a space of weakly optimal solution set, denoted by 𝐖𝐎𝐒={𝐗∈𝐒^𝑁∣𝑓(𝐗)=𝑓_max}. The solution set 𝐗∈𝐒^𝑁 satisfying 𝑓(𝐗)=𝑓_max is called the strongly optimal solution set. All the strongly optimal solution sets constitute a space of strongly optimal solution set, denoted by 𝐒𝐎𝐒={𝐗∈𝐒^𝑁∣𝑓(𝐗)=𝑓_max}.
From the aforementioned definitions, we know that ∃𝑋_𝑖∈𝐗∈𝐖𝐎𝐒, the equality 𝑓(𝑋_𝑖)=𝑓_max holds. Otherwise, if for all 𝑋_𝑖∈𝐗∉𝐖𝐎𝐒, the inequality 𝑓(𝑋_𝑖)<𝑓_max can be deduced. Similarly, if for all 𝑋_𝑖∈𝐗∈𝐒𝐎𝐒, we have 𝑓(𝑋_𝑖)=𝑓_max, and if ∃𝑋_𝑖∈𝐗∉𝐒𝐎𝐒, we can conclude that 𝑓(𝑋_𝑖)<𝑓_max. So it is easy to deduce the relationship among the space of strongly optimal solution sets, the space of weak optimal solution sets, and the coded space of solution sets, that is, 𝐒𝐎𝐒⊂𝐖𝐎𝐒⊂𝐒^𝑁.
In view of the aforementioned notions, there are four kinds of convergence related to a solution set updating sequence in the CIA model.
The CIA model is said to converge weakly in probability if its solution set updating sequence {𝐗(𝑡),𝑡≥0} satisfies lim_𝑡→∞𝑃{𝐗(𝑡)∈𝐖𝐎𝐒}=1, for all 𝐗(0)∈𝐒^𝑁. The CIA model is said to converge strongly in probability if the sequence {𝐗(𝑡),𝑡≥0} satisfies lim_𝑡→∞𝑃{𝐗(𝑡)∈𝐒𝐎𝐒}=1, for all 𝐗(0)∈𝐒^𝑁.
The CIA model is said to weakly converge almost surely if its solution set updating sequence {𝐗(𝑡),𝑡≥0} satisfies 𝑃{lim_𝑡→∞[𝐗(𝑡)∈𝐖𝐎𝐒]}=1, forall𝐗(0)∈𝐒^𝑁. The CIA model is said to strongly converge almost surely if the sequence {𝐗(𝑡),𝑡≥0} satisfies 𝑃{lim_𝑡→∞[𝐗(𝑡)∈𝐒𝐎𝐒]}=1, for all 𝐗(0)∈𝐒^𝑁.

Based on the definitions, the almost surely weak convergence of the CIA model can also be defined by 𝑃{lim_𝑡→∞𝑓(𝐗(𝑡))=𝑓_max}=1, and the almost surely strong convergence of the CIA model can also be defined by 𝑃{lim_𝑡→∞𝑓(𝐗(𝑡))=𝑓_max}=1.

According to the above definitions, we can deduce the relationship among the four types of convergence.

Theorem 5. There are the following relations among the four types of convergence of the CIA model. (i)Almost surely strong convergence ⇒almost surely weak convergence ⇒weak convergence in probability.(ii)Almost surely strong convergence ⇒strong convergence in probability ⇒weak convergence in probability.

Proof. Based on the definition of the space of optimal solution set, we have [𝐗(𝑡)∈𝐒𝐎𝐒]⟹[𝐗(𝑡)∈𝐖𝐎𝐒]. (9) From (9) we can derive that 𝑃{[𝐗(𝑡)∈𝐒𝐎𝐒]}≤𝑃{[𝐗(𝑡)∈𝐖𝐎𝐒]}. (10) Hence we have lim_𝑡→∞𝑃{[𝐗(𝑡)∈𝐒𝐎𝐒]}⟹lim_𝑡→∞𝑃𝐗(𝑡),𝑃{[∈𝐖𝐎𝐒]}lim_𝑡→∞[𝐗(𝑡)∈𝐒𝐎𝐒]⟹𝑃lim_𝑡→∞[𝐗(𝑡)∈𝐖𝐎𝐒]. (11) Owing to the fact that lim_𝑡→∞[𝐗(𝑡)∈𝐖𝐎𝐒]=^∞∞_{𝑡=1𝑘=𝑡}[𝐗(𝑡)∈𝐖𝐎𝐒], (12) according to the monotonicity of probability, we obtain 𝑃lim_𝑡→∞[𝐗(𝑡)∈𝐖𝐎𝐒]≤lim_𝑡→∞𝑃^∞_𝑘=𝑡[𝐗(𝑡)∈𝐖𝐎𝐒]≤lim_𝑡→∞𝑃{[𝐗(𝑡)∈𝐖𝐎𝐒]}. (13) Thus, we get 𝑃lim_𝑡→∞[𝐗(𝑡)∈𝐖𝐎𝐒]⟹lim_𝑡→∞𝑃𝐗(𝑡),𝑃{[∈𝐖𝐎𝐒]}lim_𝑡→∞[𝐗(𝑡)∈𝐒𝐎𝐒]⟹lim_𝑡→∞𝑃{[𝐗(𝑡)∈𝐒𝐎𝐒]}. (14)

Based on Theorem 5, we know that almost surely strong convergence implies almost surely weak convergence and strong convergence in probability, and hence implies weak convergence in probability. However, the almost surely weak convergence and strong convergence in probability are not interdeducible.

4. Convergence Analysis of the CIA Model

In this section, the conditions under which the CIA model converges almost surely to the space of weakly optimal solution set are studied according to the Borel-Cantelli Lemma. And then the conditions under which the CIA model converges almost surely to the space of strongly optimal solution sets are deduced based on the submartingale convergence theorem.

Lemma 6 (Borel-Cantelli Lemma [26]). Let {𝜉_𝑖,𝑖=1,2,…} be a sequence of events in a probability space and limsup𝜉_𝑛 the set of outcomes that occur infinitely many times within the infinite sequence of events (𝜉_𝑛), then limsup_𝑛→∞𝜉_𝑛=^∞∞_{𝑛=1𝑘=𝑛}𝜉_𝑘. (15) The Borel-Cantelli Lemma states the facts that:(i)if the sum of the probabilities of the events 𝜉_𝑖, 𝑖=1,2,…, is finite, then the probability in which infinitely many of the events occur is zero. Namely, if ∑^∞_𝑛=1𝑃{𝜉_𝑛}<∞, then 𝑃{limsup_𝑛→∞𝜉_𝑛}=0;(ii)if the events 𝜉_𝑖, 𝑖=1,2,…, are independent and the sum of the probabilities of the events diverges to infinity, then the probability in which infinitely many of the events occur is one. Namely, if ∑^∞_𝑛=1𝑃{𝜉_𝑛}=∞ and the events 𝜉_𝑖, 𝑖=1,2,…, are independent, then 𝑃{limsup_𝑛→∞𝜉_𝑛}=1.

Lemma 7. Let {𝐗(𝑡),𝑡≥0} be a solution set updating sequence generated by the CIA model and 𝑃{𝐗(𝑡+1)∣𝐗(𝑡)} the one-step transition probability of 𝐗(𝑡). If the mutation rate of set components satisfies 𝑝_𝑗(𝑡)>0, for all 𝑗∈{1,2,…,𝐿}, then ∃𝑐∈(0,1), satisfying(i)𝑃{𝐗(𝑡+1)∈𝐖𝐎𝐒∣𝐗(𝑡)∈𝐖𝐎𝐒}=1; (ii)𝑃{𝐗(𝑡+1)∈𝐖𝐎𝐒∣𝐗(𝑡)∉𝐖𝐎𝐒}≥𝑐.

Proof. (1) Consider the case of 𝐗(𝑡)∈𝐖𝐎𝐒. By the definition in Section 3, we know 𝑓(𝐗(𝑡))=𝑓_max. (16) Since an elite reserve strategy is adopted in the CIA model, from (16) we get 𝑓(𝐗(𝑡+1))≥𝑓(𝐗(𝑡))=𝑓_max. (17) Thus 𝑃{𝐗(𝑡+1)∈𝐖𝐎𝐒∣𝐗(𝑡)∈𝐖𝐎𝐒}=1. The proof of the first part of Lemma 7 is completed.
(2) Consider the case of 𝐗(𝑡)∉𝐖𝐎𝐒. Also by the definition in Section 3, we know 𝑓(𝐗(𝑡))<𝑓_max. (18) If 𝐗(𝑡+1)∈𝐖𝐎𝐒, then ∃𝑋_𝑖(𝑡+1)∈𝐗(𝑡+1),  𝑖∈{1,2,…,N}, satisfying 𝑓(𝐗(𝑡+1))=𝑓𝑋_𝑖(𝑡+1)=𝑓_max. (19) When 𝑓(𝑋_𝑖(𝑡+1))=𝑓_max, we have 𝑃𝑋_𝑖(𝑡+1)∣𝐗(𝑡)∉𝐖𝐎𝐒≥^𝐿_𝑗=1𝑝_𝑗(𝑡)>0. (20) Thus ∃𝑐∈(0,1), where 𝑐=min𝑃{𝑋_𝑖(𝑡+1)∣𝐗(𝑡)∉𝐖𝐎𝐒}, such that 𝑃{𝐗()(𝑡)}𝑋𝑡+1∈𝐖𝐎𝐒∣𝐗∉𝐖𝐎𝐒=max𝑃_𝑖(𝑡+1)∈𝑂𝐶∣𝐗(𝑡)∉𝐖𝐎𝐒≥𝑐. (21) The proof of the second part of Lemma 7 is completed.

Theorem 8. Let {𝐗(𝑡),𝑡≥0} be a solution set updating sequence generated by the CIA model and 𝐗(0) an initial solution set. {𝐗(𝑡),𝑡≥0} is weakly convergent almost surely if for all 𝑗∈{1,2,…,L} and 𝑡>0, the inequality 𝑝_𝑗(𝑡)>0 holds.

Proof. Consider the case of  𝐗(0)∈𝐖𝐎𝐒. By the definition in Section 3, we know 𝑓(𝐗(0))=𝑓_max. (22) Since the CIA model is assumed to adopt an elite reserve strategy, we get 𝑓(𝐗(𝑡))≥𝑓(𝐗(𝑡−1))≥⋯≥𝑓(𝐗(0)). (23) Combining (23) with (22) yields 𝑓(𝐗(𝑡))=𝑓(𝐗(𝑡−1))=⋯=𝑓(𝐗(0))=𝑓_max. (24) Thus, the theorem is valid for 𝐗(0)∈𝐖𝐎𝐒.
Consider the case of  𝐗(0)∉𝐖𝐎𝐒. Let 𝑝_𝑇=𝑃{𝐗(𝑇)∉𝐖𝐎𝐒}, then we have 𝑝_𝑇={}(𝑡)𝑝0,if∃𝑡∈0,1,2,…𝑇,𝐗∈𝐖𝐎𝐒,_′𝑇,if∀𝑡∈{0,1,2,…𝑇},𝐗(𝑡)∉𝐖𝐎𝐒. (25) According to Lemma 7, we obtain 𝑝^′_𝑇{𝐗(𝑡)()}{𝐗(𝑇)()}{𝐗(1)(0)}≤()=𝑃∉𝐖𝐎𝐒∣𝐗𝑡−1∉𝐖𝐎𝐒,∀𝑡=,1,2…𝑇=𝑃∉𝐖𝐎𝐒∣𝐗𝑇−1∉𝐖𝐎𝐒⋯𝑃∉𝐖𝐎𝐒∣𝐗∉𝐖𝐎𝐒1−𝑐^𝑇. (26) From (25) and (26) and considering the fact that 0<𝑐<1, we have ^∞_𝑇=1𝑝_𝑇≤^∞_𝑇=1(1−𝑐)^𝑇=1−𝑐𝑐<∞. (27) Based on (27), the first part of Lemma 6 holds. Then we have 𝑃^∞∞_{𝑡=1𝑇=𝑡}𝐗(𝑇)∉𝐖𝐎𝐒=0. (28) Thus 𝑃{lim_𝑡→∞𝑓(𝐗(𝑡))=𝑓_max}=1.

The sequence of the maximal evaluation values and the sequence of the average evaluation values of the solution sets generated by the CIA model will be used to describe the evolution process of the solution sets for further discussion. The martingale theory is introduced into the Markov chain analysis of the solution set updating sequence instead of the traditional ergodic theory. The conditions under which the CIA model converges almost surely to the space of strongly optimal solution sets are deduced based on the submartingale convergence theorem and the related conclusion in Section 2.

The concept of submartingale on a random sequence {𝛽(𝑘),𝛾(𝑘),𝑘≥0} can be found in [26] in the effort to construct the convergence criteria for random sequences; that is, a sequence {𝛽(𝑘),𝑘≥0} is said to be a submartingale with respect to another sequence {𝛾(𝑘),𝑘≥0} if  for {𝛽(𝑘),𝛾(𝑘),𝑘≥0} the following conditions are satisfied: (i) 𝐸|𝛽(𝑘)|<∞; (ii) 𝛽(𝑘) is the function of  𝛾(0),𝛾(1),…,𝛾(𝑘); and (iii) 𝐸[𝛽(𝑘+1)∣𝛾(0),𝛾(1),…,𝛾(𝑘)]≥𝛽(𝑘). In particular, a sequence {𝛽(𝑘),𝑘≥0} is called the sequence of submartingale if for all 𝑘≥0,  𝛽(𝑘) is an integrable random variable satisfying 𝐸[𝛽(𝑘+1)∣𝛽(0),𝛽(1),…,𝛽(𝑘)]≥𝛽(𝑘).

Based on the concept of submartingale and Doob’s upcrossing inequality, the following submartingale convergence theorem was deduced for random sequences.

Lemma 9 (submartingale convergence theorem [26]). If {𝛽(𝑘),𝑘≥0} is a sequence of submartingale with sup_𝑘𝐸|𝛽(𝑘)|<∞, then 𝛽(∞)=lim_𝑘𝛽(𝑘) almost surely, and 𝛽(∞) is integrable; that is, 𝑃(lim_𝑘→∞𝛽(𝑘)=𝛽(∞))=1 and 𝐸|𝛽(∞)|<∞.
The search process of the CIA model can be regarded as the progress of the maximal evaluation values of the solution sets generated in the process, expressed by the sequence {𝑓(𝐗(𝑡)),𝑡≥0}; or the progress of the average evaluation values of the solution sets, expressed by the sequence {𝑓(𝐗(𝑡)),𝑡≥0}. Both of the sequences are Markov chains. Thus, the convergence analysis of the CIA model can be firstly transferred to the submartingale analysis of the maximal evaluation value sequence {𝑓(𝐗(𝑡)),𝑡≥0} and then be further transferred to the submartingale analysis of the average evaluation value sequence {𝑓(𝐗(𝑡)),𝑡≥0}.

Lemma 10. 𝐖𝐎𝐒 is a closed set.

Proof. Let {𝐗(𝑡),𝑡≥0} be a solution set updating sequence generated by the CIA model. Since the elite reserve strategy is adopted in the CIA model, we have 𝑓(𝐗(𝑡+1))≥𝑓(𝐗(𝑡)). On the other hand, according to the definition in Section 3, for all 𝐘∈𝐖𝐎𝐒 and forall𝐙∉𝐖𝐎𝐒, we have 𝑓(𝐘)>𝑓(𝐙). Thus 𝐗(𝑡+1)≠𝐙 if 𝐗(𝑡)=𝐘, which results in the conclusion that 𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘)=0. Then, for all 𝐘∈𝐖𝐎𝐒, we have _{𝐙∉𝐖𝐎𝐒}𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘)=0, (29) or equivalently _{𝐙∈𝐖𝐎𝐒}𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘)=1. (30) Therefore, 𝐖𝐎𝐒 is a closed set.

According to Lemma 10, we can prove that the maximal evaluation value sequence of solution set is a submartingale with respect to the corresponding solution set updating sequence.

Lemma 11. The maximal evaluation value sequence {𝑓(𝐗(𝑡)),𝑡≥0} generated by the CIA model is a submartingale with respect to the solution set updating sequence {𝐗(𝑡),𝑡≥0}.

Proof. According to the non-after-effect property of Markov chain, we can conclude that 𝐸𝑓(𝐗((0)(1)(𝑡𝑓(𝐗((𝑡,𝑡+1))∣𝐗,𝐗,…,𝐗)=𝐸𝑡+1))∣𝐗) (31) based on which we only need to prove that for all 𝐘∈𝐒^𝑁 the following inequality is true: 𝐸𝑓(𝐗(𝑡+1))∣𝐗(𝑡)=𝐘≥𝑓(𝐗(𝑡)),∀𝑡≥0. (32) According to the theory of random process [26], we have 𝐸𝑓(𝐗(𝑡+1))∣𝐗(𝑡)=𝐘=_{𝐙∈𝐒𝑁}𝑓(𝐙)𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘)=_{𝐙∈𝐖𝐎𝐒}𝑓(𝐙)𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘)+_{𝐙∉𝐖𝐎𝐒}𝑓(𝐙)𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘)≥_{𝐙∈𝐖𝐎𝐒}𝑓(𝐙)𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘). (33) For the maximal evaluation value sequence {𝑓(𝐗(𝑡)),𝑡≥0}, we have 𝑓(𝐗(𝑡+1)=𝐙)≥𝑓(𝐗(𝑡)=𝐘),∀𝐙∈𝐖𝐎𝐒. (34) Combining (34) with (33) yields _{𝐙∈𝐖𝐎𝐒}𝑓(𝐙)𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘)≥_{𝐙∈𝐖𝐎𝐒}𝑓(𝐘)𝑃(𝐗()(𝑡))=𝑓(𝐘)𝑡+1=𝐙∣𝐗=𝐘_{𝐙∈𝐖𝐎𝐒}𝑃(𝐗(𝑡+1)=𝑍∣𝐗(𝑡)=𝐘). (35) By Lemma 10, we obtain _{𝐙∈𝐖𝐎𝐒}𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘)=1. (36) From (35) and (36) it results in _{𝐙∈𝐖𝐎𝐒}𝑓(𝐙)𝑃(𝐗(𝑡+1)=𝐙∣𝐗(𝑡)=𝐘)≥𝑓(𝐘). (37) Equation (37) implies that 𝐸[𝑓(𝐗(𝑡+1))∣𝐗(𝑡)=𝐘]≥𝑓(𝐗(𝑡)).

Based on Lemma 11, we can associate the average evaluation value sequence of solution set with the progress rate of solution set defined in Section 2 to obtain the following corollary.

Corollary 12. The average evaluation value sequence {𝑓(𝐗(𝑡)),𝑡≥0} is a submartingale with respect to the solution set updating sequence {𝐗(𝑡),𝑡≥0} if the corresponding progress rate of the solution set satisfies 𝛼(t+1)≥𝛼(t), for all t≥0.

Proof. Based on Lemma 11, it can be obtained that 𝐸𝑓(𝐗(𝑡+1))∣𝐗(0),𝐗(1),…,𝐗(𝑡)≥𝑓(𝐗(𝑡)),∀𝑡≥0. (38) It follows that if 𝛼(t+1)≥𝛼(t)>0, for all t≥0, we have 𝛼()𝐸𝑓(𝐗((0)(1)(𝑡(𝑡)𝑓(𝐗(𝑡.𝑡+1𝑡+1))∣𝐗,𝐗,…,𝐗)≥𝛼)) (39) By Definition 2, (39) implies 𝐸𝑓(𝐗(𝑡+1))∣𝐗(0),𝐗(1),…,𝐗(𝑡)≥𝑓(𝐗(𝑡)). (40) Thus {𝑓(𝐗(𝑡)),𝑡≥0} is a submartingale with respect to {𝐗(𝑡),𝑡≥0}.

Based on Lemma 11 and its corollary, the sufficient conditions can be obtained for the strong convergence of the solution set updating sequence generated by the CIA model.

Theorem 13. Let {𝐗(𝑡),𝑡≥0} be a solution set updating sequence generated by the CIA model and 𝑝_𝑗(𝑡) the variation rate of set component 𝐶_𝑗(𝑡) in solution set 𝐗(𝑡). {𝐗(𝑡),𝑡≥0} is strongly convergent almost surely if Corollary 12 holds and ∑^∞_𝑡=1∏^𝐿_𝑗=1𝑝_𝑗(𝑡)=∞.

Proof. Let the complement of the set of the globally optimal solutions be denoted by 𝑂𝐶^𝐶 and {𝑀_𝑡,𝑡≥0} a sequence of random events that at least a part of the candidate solutions in solution set 𝐗(𝑡) belongs to 𝑂𝐶^𝐶; that is, 𝑀_𝑡={𝐗(𝑡)∩𝑂𝐶^𝐶≠∅}={𝑓(𝐗(𝑡))≠𝑓_max}. Similarly, let 𝑀^𝐶_𝑡 be the exclusive event sequence of 𝑀_𝑡; that is, there is no any candidate solution in solution set 𝐗(𝑡) belonging to 𝑂𝐶^𝐶, expressed by 𝑀^𝐶_𝑡={𝐗(𝑡)∩𝑂𝐶^𝐶=∅}={𝑓(𝐗(𝑡))=𝑓_max}. Let 𝑃{𝑀_𝑡} and 𝑃{𝑀^𝐶_𝑡} be the probability measures on 𝑀_𝑡 and 𝑀^𝐶_𝑡, respectively.
From Corollary 12, we know that {𝑓(𝐗(𝑡)),𝑡≥0} is a submartingale with respect to {𝐗(t),t≥0}. Then according to the properties of submartingale, it is known that 𝑓_max≥𝐸𝑓(𝐗(𝑡+1))≥𝐸𝑓(𝐗(𝑡)). (41) According to (41) and the properties of expectation, we get _𝑀𝐶𝑡𝑓(𝐗(𝑡+1))𝑑𝑃=𝑓_max⋅𝑃𝑀^𝐶_𝑡. (42) In addition, in view of the fact that 0≤𝑓(𝐗(𝑡))≤𝑓_max and according to the properties of expectation, we have sup_𝑡𝐸𝑓(𝐗(𝑡))≤sup_𝑡𝑓_max<∞. (43) Then according to Lemma 9, (43) results in the conclusion that ∃𝐗(∞)=lim_𝑡𝐗(𝑡), 𝑃{lim_𝑡→∞𝐗(𝑡)=𝐗(∞)}=1. In other words, {𝐗(𝑡),𝑡≥0} is almost surely convergent.
In the following we further prove that the average evaluation value sequence of {𝐗(𝑡),𝑡≥0} converges to 𝑓_max almost surely. From (42) it can be seen that 𝑓(𝐗(𝑡+1))=𝑓_max if the event 𝑀^𝐶_𝑡 occurs. Namely, 𝑀^𝐶_𝑡 surely triggers off the event 𝑀^𝐶_𝑡+1. Thus the following conclusion holds for the event sequence {𝑀^𝐶_𝑡,𝑡≥0}: 𝑀^𝐶_𝑡⊂𝑀^𝐶_𝑡+1. (44)
On the other hand, for any solution sets 𝐘 and 𝐙, if 𝐘∩𝑂𝐶^𝐶≠∅, then ∃𝑌_𝑘∈𝐘, 𝑓(𝑌_𝑘)<𝑓_max; and if 𝐙∩𝑂𝐶^𝐶=∅, then for all 𝑍_𝑘∈𝐙, 𝑓(𝑍_𝑘)=𝑓_max. Hence the following inequality holds: 𝑓(𝐙)≥𝑓(𝐘)+𝛿𝑁, (45) where 𝛿=min(𝑓_max−𝑓(𝑌_𝑘)) and 𝑁 is the size of solution set.
In addition, by the properties of conditional expectation we have 𝐸𝑓(𝐗(𝑡+1))−𝐸𝑓(𝐗(𝑡𝐸))=𝐸𝑓(𝐗(𝑡+1))∣𝐗(𝑡)−𝐸𝑓(𝐗(𝑡))=_𝑌𝑃{𝐗(𝑡)=𝐘}⋅_𝑍𝑃{𝐘,𝐙}⋅𝑓(𝐙)−𝑓(𝐘)≥_{𝐘∩𝑂𝐶𝐶≠∅}𝑃{𝐗(𝑡)=𝐘}⋅_𝐙𝑃{𝐘,𝐙}⋅𝑓(𝐙)−𝑓(𝐘) (46) and from the definition of  𝑀_𝑡, we have _{𝐘∩𝑂𝐶𝐶≠∅}𝑃{𝐗(𝑡)=𝐘}=𝑃𝑀_𝑡. (47) Moreover, by the definition in Section 3, we know that _𝑍𝑃{𝐘,𝐙}≥^𝐿_𝑗=1𝑝_𝑗(𝑡). (48) Substituting (45), (47), and (48) into (46), we obtain 𝐸𝑓(𝐗(𝑡+1))−𝐸𝑓(𝐗(𝑡≥𝛿))𝑁𝑃𝑀_𝑡⋅^𝐿_𝑗=1𝑝_𝑗(𝑡). (49) Now calculating the sum for both sides of the inequality in (49) from 𝑡=1 to 𝑡, it yields 𝑓_max≥𝐸𝑓(𝐗(𝑡+1))−𝐸𝑓(𝐗(1≥𝛿))𝑁^𝑡_𝑘=1𝑃𝑀_𝑘⋅^𝐿_𝑗=1𝑝_𝑗(𝑘). (50) Note that (50) holds for any 𝑡. Then let 𝑡→∞ in (50); we have ^∞_𝑡=1𝑃𝑀_𝑡⋅^𝐿_𝑗=1𝑝_𝑗(𝑡)≤𝑁𝛿𝑓_max<∞. (51) It follows that 𝑃{𝑀_𝑡}→0 and 𝑃{𝑀^𝐶_𝑡}→1 if ∑^∞_𝑡=1∏^𝐿_𝑗=1𝑝_𝑗(𝑡)=∞, which further leads to 𝑃{𝑀^𝐶_𝑡+1}→1 according to (44). Thus we have 𝑃lim_𝑡→∞𝑀^𝐶_𝑡=𝑃lim_𝑡→∞𝑓(𝐗(𝑡))=𝑓_max=1. (52)

Remark 14. The method shown in this paper has a certain similarity to methods reported in [11] for GA and in [17] for ACO. More specifically, the elitist strategy has the same benefits as the strategy of the parents population competing with their offsprings, and the condition of variation rates of set component satisfying ∑^∞_𝑡=1∏^𝐿_𝑗=1𝑝_𝑗(𝑡)=∞ in Theorem 13 is expected to act in a similar manner as the condition of minimum mutation probability of population space satisfying ∑^∞_𝑛=1𝑟_𝑛=∞ in [11]. However, the method reported in [11] only applies to GA rather than a class of CIAs. Furthermore, the variation rates of set component are measurable while minimum mutation probability of population space is not. Duan’s convergence proof of ant colony algorithm is also based on the Markov chain and the martingale theory, but they ignore important information [17]. Namely, the vector of pheromone values {𝜏(𝑛),𝑛≥1}, without the best-found path 𝜔(𝑛−1), is not a Markov process and cannot be defined in a self-contained way. However, the stochastic process with states {𝑋_𝑛=(𝜏(𝑛),𝜔(𝑛−1)),𝑛≥1} is an inhomogeneous Markov process in discrete time [14].

5. Conclusions

CIAs is a relatively new interdisciplinary field of research, which has gained huge popularity in these days. One of the most important problems obsessing the researchers in the area of CIAs is the premature convergence or the stagnation of an algorithm. The solution to this problem is not only helpful to the setup of the parameters in an existing algorithm for fast convergence to global optima but also related to how to construct a more effective and efficient algorithm for complex problems optimization. In this paper, we first analyze the common characteristics shared by GA, ACO, and PSO, and based on that, a computational model is constructed for a class of CIAs. Two quantification indices, that is, the variation rate and the progress rate, are subsequently defined to estimate the evolution trend of the solution sets generated by CIAs. Secondly, we define four different notions of convergence of a solution set updating sequence generated by the CIA model, and their relations are discussed. Thirdly, a weak convergence criterion is presented by associating the almost surely weak convergence with the positive definiteness of the variation rate of set component. Finally, the conditions for strong convergence of the CIAs are derived based on submartingale convergence theorem. Two sufficient conditions are obtained for the almost surely strong convergence: (i) the progress rate of solution set does not diminish in the search process; and (ii) the summation of the Cartesian products of the variation rates of set component for all solution sets is infinite.

It is obvious that the conditions described in Theorems 8 and 13 are sufficient but not necessary; thus, there exists the possibility that these theorems are too conservative to be used in practice. Moreover, a fast convergence rate is absolutely critical for a wide range of applications of CIAs, but it cannot be estimated by using the results presented in this paper. Further research work will be conducted by the authors in an effort to solve these interesting and important problems.

Acknowledgments

This paper was supported in part by the National Natural Science Foundation of China (Grant no. 61203365), the Jiangsu Province Natural Science Foundation (Grant no. BK2012149), and the Open Issues Foundation of Jiangsu Key Laboratory of Power Transmission and Distribution Equipment Technology (Grant no. 2010JSSPD06).